Blog

Hibernate and Multi-Threading

07 Feb, 2009
Xebia Background Header Wave

When you use Hibernate for ORM and come across some functionality that requires multi threading, there are many pitfalls that might make life difficult for you. This blog will focus on those problems. Conclusion is: don’t use hibernate managed objects in multiple threads.

Let’s look at a very simple domain: clients with payments. Each client has a set of payments

@Entity public class Client {
    ...
    @OneToMany(cascade=ALL)
    private Set payments = new HashSet();
    public void addPayment(double amount) {
        payments.add(new Payment(amount));
    }
    public double getTotal() {
        int total = 0;
        for (Payment payment : payments) {
            total += payment.getAmount();
        }
        return total;
    }
}
@Entity public class Payment {
    ...
    public Payment(double amount) {
        this.amount = amount;
    }
    ...
}

Now a very simple test is to save create a client with some payments, then reload it and test the total number of the payments again.

   public void testSimpleClientWithPayment() throws Exception {
        Client client = new Client("me");
        client.addPayment(12.0);
        client.addPayment(5.0);
        assertEquals(17.0, client.getTotal());
        Serializable id = dao.save(client);
        dao.flushAndClear();
        client = dao.get(id);
        assertEquals(17.0, client.getTotal());
    }

Of course this all works like a charm and there are no problems. The project drifts easy along its ever lasting path and you keep on coding.

Introduce threads: Lazy Loading problems

At some point one of the wise-ass architects comes up with a scheme where reports of the totals of all clients have to be printed on a regular basis in separate threads. The reports are sent by mail or some other lame excuse for introducing this fancy new threading thing. So to keep him happy you create a nice service that can do that.

public class ReportingService {
    private final class Reporter implements Runnable {
        private final Client client;
        private Reporter(Client client) {
            this.client = client;
        }
        public void run() {
            System.out.println(client + ": " + client.getTotal());
        }
    }
    public void reportTotals() {
        for (final Client client : dao.findAllClients()) {
            executorService.submit(new Reporter(client));
        }
    }
    ...
}

Of course you test the service with a number of payments, and everything works out fine… until the feature is brought into production and the logs start filling up with these exceptions:

21:21:11,562 ERROR [LazyInitializationException, LazyInitializationException.] failed to lazily initialize a collection of role: domain.Client.payments, no session or session was closed
org.hibernate.LazyInitializationException: failed to lazily initialize a collection of role: domain.Client.payments, no session or session was closed
    at org.hibernate.collection.AbstractPersistentCollection.throwLazyInitializationException(AbstractPersistentCollection.java:358)
    at org.hibernate.collection.AbstractPersistentCollection.throwLazyInitializationExceptionIfNotConnected(AbstractPersistentCollection.java:350)
    at org.hibernate.collection.AbstractPersistentCollection.initialize(AbstractPersistentCollection.java:343)
    at org.hibernate.collection.AbstractPersistentCollection.read(AbstractPersistentCollection.java:86)
    at org.hibernate.collection.PersistentSet.iterator(PersistentSet.java:163)
    at domain.Client.getTotal(Client.java:43)
    at service.ReportingService$Reporter.run(ReportingService.java:18)
    at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:417)
    at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:269)
    at java.util.concurrent.FutureTask.run(FutureTask.java:123)
    at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:650)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:675)
    at java.lang.Thread.run(Thread.java:595)

But you tested that, right? What could possibly be wrong? Well, the collection of payments from a few of the clients are not yet initialized (=loaded from the database) when the client is handed to the executor for reporting. This is no problem: Hibernate can lazily load the collection whenever this is needed. For that purpose, it replaces the collection with a special implementation, that keeps a reference to the Session that was used to load the containing entity (the client). When we call payments.iterator(), the collection is loaded from the database through the Session. As long as this Session is open, the code will work.
Sessions, however, are going to be closed at some point. Using springs transaction management, for example, they are often associated with a transaction and closed after completion of that transaction. Now if the transaction that loads a client with a lot of payments shoots of all these tasks and then completes before the tasks are executed, the session will close and the tasks are left without the option to lazily load the payments. Hence the lazy loading problems. In a test environment it is often hard to simulate this, because there is less data and your testing infrastructure might keep transactions and sessions open until everything is executed. You will mostly get burned in production.
So, the solution is simple, right? Just load the payments before you execute the task:

       ...
        for (final Client client : dao.findAll()) {
            // --> Solution:
            Hibernate.initialize(client.getPayments());
            executorService.submit(new Reporter(client));
        }
        ...

That seems simple enough. You can also initialize the set by executing a method, for example .size(), or by fetching it eagerly when loading the client.
So, no more stacktraces in the production log, you saved the company!

Missing Updates

Now the business analysts have come up with a new idea: The printing of the totals is too simple, the clients want a record in the database of the total per client every week and they want to know when the last total was generated for each client. So you extend your ReportingService with a new method and a new task:

   ...
    public class ReportGenerator implements Runnable {
        private final Client client;
        public ReportGenerator(Client client) {
            this.client = client;
        }
        public void run() {
            dao.save(new Report(client, client.getTotal()));
            client.setLastCreated(new Date());
        }
    }
    ...

Now if you execute this task in your test environment, you notice something strange: The reports are added to the database, but the "lastCreated" field is not updated. What can be the problem? Hibernate should perform transactional write-behind with dirty checking: When the transaction is completing, it should check that the client object has changed and persist the changed values. This works normally, why is this different?
The dirty checking is only performed in the session in which an object was loaded. The client object was loaded in a different session, on which the transaction has already been completed. When the report is created, this is saved in a separate session. The infrastructure of the dao takes care associating the session with the current thread and of instantiating a new session when necessary.

Locking with no mode

So how do you solve this? To be able to do updates on an object loaded from a different session, you’ll have to reconnect it to a new open session. The .lock(.., LockMode.NONE) operation can be used for that. You have to make sure that the session remains open until the updates are done. To ensure this, we use a new transaction for the new report action:

       ...
        public void run() {
            new TransactionTemplate(transactionManager)
                    .execute(new TransactionCallbackWithoutResult() {
                        @Override
                        public void doInTransactionWithoutResult(TransactionStatus status) {
                            dao.lock(client, NONE);
                            dao.save(new Report(client, client.getTotal()));
                            client.setLastCreated(new Date());
                        }
                    });
        }
        ...

Using this solution in combination with the previous solution, however, will result in new exceptions:

Caused by: org.hibernate.HibernateException: Illegal attempt to associate a collection with two open sessions
    at org.hibernate.collection.AbstractPersistentCollection.setCurrentSession(AbstractPersistentCollection.java:410)
    at org.hibernate.event.def.OnLockVisitor.processCollection(OnLockVisitor.java:38)
    at org.hibernate.event.def.AbstractVisitor.processValue(AbstractVisitor.java:101)
    at org.hibernate.event.def.AbstractVisitor.processValue(AbstractVisitor.java:61)
    at org.hibernate.event.def.AbstractVisitor.processEntityPropertyValues(AbstractVisitor.java:55)
    at org.hibernate.event.def.AbstractVisitor.process(AbstractVisitor.java:123)
    at org.hibernate.event.def.AbstractReassociateEventListener.reassociate(AbstractReassociateEventListener.java:79)
    at org.hibernate.event.def.DefaultLockEventListener.onLock(DefaultLockEventListener.java:59)
    at org.hibernate.impl.SessionImpl.fireLock(SessionImpl.java:584)
    at org.hibernate.impl.SessionImpl.lock(SessionImpl.java:576)
    ...

This is because we are attempting to connect the collection, that we so cunningly initialized before, to the new session as well. This collection however, will attempt to keep a reference to one open session, but not two.
On the bright side, we do not have to initialize the session anymore if we connect the client to a new session: lazy loading will work again. In some situations however, it is very hard to determine which objects are already initialized and which aren’t and when to connect which objects to the new session and so on… Using the lock operation is often a tedious job!

Just pass the ID… silly!

After battling the problem for a while, you give up and settle for a much easier scheme: You store the object in the database and pass the ID-value to the worker thread. That thread can now reload the data from the database and safely work on it.

Conclusion

Passing Hibernate managed objects (like collections or proxies) to other threads might result in some hard to trace problems, like:

  • Lazy Initialization problems
  • Missing Updates
  • Locking exceptions

It is therefore often better not to pass Hibernate managed objects to other threads, but to save the objects and reload them by ID from the database in the other tread.

Questions?

Get in touch with us to learn more about the subject and related solutions

Explore related posts