[InterMine Dev] reducing source loading times

Richard Smith richard at flymine.org
Tue Feb 7 13:34:58 GMT 2012


JD,
Could you send us the intermine.log from your integrate directory after
running a build.  This is the most helpful thing for us to investigate
performance.

Thanks,
Richard.




On 06/02/2012 19:01, JD Wong wrote:
> I was wondering how the other mods speed up their builds.  I have
> configured ant, java, and postgres accordingly without effect.  I was
> hoping to get the community's advice on this aspect.
>
> Cheers,
> -JD
>
> On Thu, Feb 2, 2012 at 10:11 AM, JD Wong <jdmswong at gmail.com
> <mailto:jdmswong at gmail.com>> wrote:
>
>     I haven't given ant and postgres enough to consume all the memory
>     when running simultaneously.  Also there is plenty of free memory
>     during loading.
>
>     -JD
>
>
>     On Wed, Feb 1, 2012 at 10:35 PM, Josh Goodman <jogoodma at indiana.edu
>     <mailto:jogoodma at indiana.edu>> wrote:
>
>         You need to be careful of the various memory settings here.  If you
>         set ant really high (>25% of total memory) and you are also
>         setting Pg
>         high you could be suffering from the two of them fighting over
>         system
>         resources and causing the swap to get thrashed.  I would run the
>         unix
>         "free" command while you are running a load to see what is going on
>         with memory.
>
>         e.g.
>
>         free -m -s 5
>
>         If you have other processes running on this machine (tomcat
>         instances)
>         you also need to adjust ant and Pg to take that into account.
>
>         Josh
>
>         On Wed, Feb 1, 2012 at 5:20 PM, JD Wong <jdmswong at gmail.com
>         <mailto:jdmswong at gmail.com>> wrote:
>          > Hi all,
>          > Loading my Go source takes on average 2500 seconds.  I have
>         tuned the
>          > postgres configuration paramaters to the desired values and
>         gave ant high
>          > heap memory to no avail.  Is there a way to speed this up?
>          >
>          > -JD
>          >
>          > _______________________________________________
>          > dev mailing list
>          > dev at intermine.org <mailto:dev at intermine.org>
>          > http://mail.intermine.org/cgi-bin/mailman/listinfo/dev
>          >
>
>
>
>
>
> _______________________________________________
> dev mailing list
> dev at intermine.org
> http://mail.intermine.org/cgi-bin/mailman/listinfo/dev




More information about the dev mailing list