The Author Online Book Forums are Moving

The Author Online Book Forums will soon redirect to Manning's liveBook and liveVideo. All book forum content will migrate to liveBook's discussion forum and all video forum content will migrate to liveVideo. Log in to liveBook or liveVideo with your Manning credentials to join the discussion!

Thank you for your engagement in the AoF over the years! We look forward to offering you a more enhanced forum experience.

alvinyue (1) [Avatar] Offline
#1
Hello everyone,

I just learned about this problem at work as we were doing XML processing and trying to store information with DAOs, Spring, and Hibernate using the HibernateTemplate. If the code was applied to a small XML file (Parsing with Sax) then it will extract data and write to table. If we use a bigger XML file then It will run out of memory (May be Hibernate didn't flush or clear the cache). Also, even when it doesn't run out of memory, it might not do bulk save/insert to the database. I searched the web and some people were saying bulk with Hibernate does not work in Spring (at least with the HibernateTemplate) and the JDBC or JDBCTemplate were recommended. It seems like with a higher level abstraction is taking away transaction from us. I know a lot of people loved Spring and Hibernate but I have not seen any good technical explanation why this is broken or is it fixed in latest version of Spring?

Thank you!

Alvin
simbo1905 (30) [Avatar] Offline
#2
Re: HibernateTemplate bulk insert/save
As discussed here:

http://www.manning-sandbox.com/thread.jspa?messageID=92028#92028

hibernate does not flush until the transaction commits. This is not a bug but a feature. It means that hibernate can use a single phase database transaction which contains the least amount of database work written in the shortest possible time: all done only when the commit is certain. When you are not doing batch work and you have a complex use case which makes a few updates to the same object you only write the object out to the db once with all of the changes made over the duration of the session.

To do bulk work you can consider calling session.flush() periodically to see if that clears out the pending work to the db and removes it from the session to clear down the memory. If you call flush() a hibernate session transaction as described at the link above cannot span the whole of the database work. In which case you would need to use a JTA global transaction that spans the whole batch so that you can roll back any previously flushed work if something goes wrong late in the batch. JTA sometimes has a 120s timeout so for very large batch jobs you may have to configure the transaction manager to have a longer timeout.