[InterMine Dev] reducing source loading times

JD Wong jdmswong at gmail.com
Mon Feb 6 19:01:38 GMT 2012


I was wondering how the other mods speed up their builds.  I have
configured ant, java, and postgres accordingly without effect.  I was
hoping to get the community's advice on this aspect.

Cheers,
-JD

On Thu, Feb 2, 2012 at 10:11 AM, JD Wong <jdmswong at gmail.com> wrote:

> I haven't given ant and postgres enough to consume all the memory when
> running simultaneously.  Also there is plenty of free memory during loading.
>
> -JD
>
>
> On Wed, Feb 1, 2012 at 10:35 PM, Josh Goodman <jogoodma at indiana.edu>wrote:
>
>> You need to be careful of the various memory settings here.  If you
>> set ant really high (>25% of total memory) and you are also setting Pg
>> high you could be suffering from the two of them fighting over system
>> resources and causing the swap to get thrashed.  I would run the unix
>> "free" command while you are running a load to see what is going on
>> with memory.
>>
>> e.g.
>>
>> free -m -s 5
>>
>> If you have other processes running on this machine (tomcat instances)
>> you also need to adjust ant and Pg to take that into account.
>>
>> Josh
>>
>> On Wed, Feb 1, 2012 at 5:20 PM, JD Wong <jdmswong at gmail.com> wrote:
>> > Hi all,
>> > Loading my Go source takes on average 2500 seconds.  I have tuned the
>> > postgres configuration paramaters to the desired values and gave ant
>> high
>> > heap memory to no avail.  Is there a way to speed this up?
>> >
>> > -JD
>> >
>> > _______________________________________________
>> > dev mailing list
>> > dev at intermine.org
>> > http://mail.intermine.org/cgi-bin/mailman/listinfo/dev
>> >
>>
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.intermine.org/pipermail/dev/attachments/20120206/db8bfdbe/attachment.html>


More information about the dev mailing list