thedwick

By default, CMP Entity Beans on JBoss are set to require a transaction. Also by default, any time you touch any session or entity bean, your request thread takes out a lock on that entire object, even if you are only reading it and not updating it. Lastly, also by default, JBoss will make sure that for any given entity, there is only one instance of that entity in memory at a time.

All of these defaults have serious implications. For one, it implies that anything other than a toy application will likely become a de-facto, single-threaded application. Imagine, for example, that you have an earthquake tracking application. Your application might have an Entity Bean called Earthquake. After getting under way with the application, you realize there are different kinds of earthquake: tectonic, volcanic, and man-made. These don’t merit having a full-on Earthquake subclass of their own, but maybe you want to model the types as a new Entity called EarthquakeType so that the application can be data-driven and new types can be added later without changing code. The vast majority (~90%) of earthquakes are tectonic, so most of what you ever display to a user will be “tectonic”.

So, you might have a web page that displays the last 40 earthquakes in descending chronological order in a table and also a count of how many different types. This could lead to innocent code like, say:

foreach (Earthquake earthquake : earthquakes){
 typeSum[earthquake.getType().getId()]++;
}

The moment you call earthquake.getType() for the first earthquake in the list, you will lock the “tectonic” instance of the EarthquakeType Entity bean. This means that every other thread executing in the same JVM (if configured the default JBoss way) will most likely block (who doesn’t need to know what the earthquake type is, after all?) until this thread is done displaying its page. Even worse, if this thread is holding a lock that some other thread needs, and that other thread is holding a lock that this thread needs, then you have a deadlock. All of this in spite of the fact that actually updating an EarthquakeType is extremely rare because they are read-mostly.

A telltale sign that you are having this problem is seeing stack traces like this one:

org.jboss.util.deadlock.ApplicationDeadlockException: Application deadlock detected, resource=org.jboss.ejb.plugins.lock.QueuedPessimisticEJBLock@290df5c3, bean=

…snip…

at org.jboss.util.deadlock.DeadlockDetector.deadlockDetection(DeadlockDetector.java:69)
at org.jboss.ejb.plugins.lock.QueuedPessimisticEJBLock.waitForTx(QueuedPessimisticEJBLock.java:292)
at org.jboss.ejb.plugins.lock.QueuedPessimisticEJBLock.doSchedule(QueuedPessimisticEJBLock.java:230)

…snip.

At first, it’s tempting to fume at JBoss for having such conservative default settings. I know I did this morning as I was learning more about the details. But the fact is that they really have no choice. The application container has no idea that EarthquakeType is read-mostly. It doesn’t know if you will read it at the beginning of the request and then modify it 300 milliseconds later at the end of the request. So, it is forced to loop absolutely everything you touch into a giant transaction unless you tell it otherwise.

Now, the “telling it otherwise” is where things start to get tricky. Here, I really do think that JBoss hasn’t done us any favors. It’s a multi-step process to making sure you maximize your throughput and minimize deadlocks. If you do some steps but don’t do others, then nothing will change and you won’t know why.

So, here are the steps…

This is just a quick post because it’s been a while and I wanted to save others from the pain I experienced yesterday.

If you want to parse a Java stack trace with a regular expression and pull out the class name, method name, and line number, then you can use this code below:

Pattern pattern = Pattern.compile("([a-zA-Z0-9_\\.]*)\\.([a-zA-Z0-9_\\.]*)\\([a-zA-Z0-9_\\.]*:([\\d]*)\\)");
Matcher matcher = pattern.matcher(traceString);
while (matcher.find()){
    String className = matcher.group(1);
    String methodName = matcher.group(2);
    int lineNumber = Integer.parseInt(matcher.group(3) == null ? "0" : matcher.group(3));
}


Note that because you are passing a Java string into a regular expression, you have to double-escape many of those characters. For example, if you want to say “any decimal” the usual regular expression is “\d” but because you are using a Java string to define the regular expression you have to double escape it to say “\\d” instead.

I’d like to give some props to David Matuszek whose nifty online Regular Expression Test Applet made debugging this hairy thing much easier.