[InterMine Dev] reducing source loading times

JD Wong jdmswong at gmail.com
Thu Feb 2 15:11:01 GMT 2012


I haven't given ant and postgres enough to consume all the memory when
running simultaneously.  Also there is plenty of free memory during loading.

-JD

On Wed, Feb 1, 2012 at 10:35 PM, Josh Goodman <jogoodma at indiana.edu> wrote:

> You need to be careful of the various memory settings here.  If you
> set ant really high (>25% of total memory) and you are also setting Pg
> high you could be suffering from the two of them fighting over system
> resources and causing the swap to get thrashed.  I would run the unix
> "free" command while you are running a load to see what is going on
> with memory.
>
> e.g.
>
> free -m -s 5
>
> If you have other processes running on this machine (tomcat instances)
> you also need to adjust ant and Pg to take that into account.
>
> Josh
>
> On Wed, Feb 1, 2012 at 5:20 PM, JD Wong <jdmswong at gmail.com> wrote:
> > Hi all,
> > Loading my Go source takes on average 2500 seconds.  I have tuned the
> > postgres configuration paramaters to the desired values and gave ant high
> > heap memory to no avail.  Is there a way to speed this up?
> >
> > -JD
> >
> > _______________________________________________
> > dev mailing list
> > dev at intermine.org
> > http://mail.intermine.org/cgi-bin/mailman/listinfo/dev
> >
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.intermine.org/pipermail/dev/attachments/20120202/5939bbb0/attachment.html>


More information about the dev mailing list