[InterMine Dev] web app restarts
jwcarlson at lbl.gov
Fri May 8 01:28:06 BST 2015
Hi Julie and gang
We have just deployed our latest phytozome build based on intermine 1.4.
This is our first public release using Hikari.
Our hopes were that going to hikari would solve some of the problem
we've been seeing about tomcat restarts. We've traded emails about this
in the past where we see that we have to restart every couple of hours
when under load. (We run a 'are you alive' cron job every 3 minuts and
force a restart if we don't get a response.)
At the time I think we had attributed it to the postgres connection and
we looking forward to the hikari pooling. It behaved well in internal
use, but now that we're public we continue to see the restarts.
I'm trying to do a little forensics to see what might be causing them.
I'm seeing "OutOfMemoryError: Java heap space", typically after a call
to retrieving the genomic sequence of a region
(service/regions/sequence). I had noticed that you had removed this
service in 1.4. I restored it since we're making use of it to deliver
sequence to our main web portal. Did you remove this because you had
seen it as being problematic?
At this point, I'm not absolutely sure this is the source of the
restarts but I'm very suspicious of
org.intermine.bio.web.export.GenomicRegionSequenceExporter. There is a
static map of entire chromosomes that is being stored. The substring is
retrieved by calling substring on elements of this map. This may work
for smaller mines but we have enough sequence in our database that I
suspect this is part of our problem.
Was this web service removed deliberately? Is there something to replace
it? As I recall, the other sequence retrieval services I found only
retrieved the sequence for specific features and not chromosome slices.
More information about the dev