[InterMine Dev] reducing source loading times

JD Wong jdmswong at gmail.com
Tue Feb 7 16:30:03 GMT 2012


Sure

On Tue, Feb 7, 2012 at 8:34 AM, Richard Smith <richard at flymine.org> wrote:

> JD,
> Could you send us the intermine.log from your integrate directory after
> running a build.  This is the most helpful thing for us to investigate
> performance.
>
> Thanks,
> Richard.
>
>
>
>
>
> On 06/02/2012 19:01, JD Wong wrote:
>
>> I was wondering how the other mods speed up their builds.  I have
>> configured ant, java, and postgres accordingly without effect.  I was
>> hoping to get the community's advice on this aspect.
>>
>> Cheers,
>> -JD
>>
>> On Thu, Feb 2, 2012 at 10:11 AM, JD Wong <jdmswong at gmail.com
>> <mailto:jdmswong at gmail.com>> wrote:
>>
>>    I haven't given ant and postgres enough to consume all the memory
>>    when running simultaneously.  Also there is plenty of free memory
>>    during loading.
>>
>>    -JD
>>
>>
>>    On Wed, Feb 1, 2012 at 10:35 PM, Josh Goodman <jogoodma at indiana.edu
>>    <mailto:jogoodma at indiana.edu>> wrote:
>>
>>        You need to be careful of the various memory settings here.  If you
>>        set ant really high (>25% of total memory) and you are also
>>        setting Pg
>>        high you could be suffering from the two of them fighting over
>>        system
>>        resources and causing the swap to get thrashed.  I would run the
>>        unix
>>        "free" command while you are running a load to see what is going on
>>        with memory.
>>
>>        e.g.
>>
>>        free -m -s 5
>>
>>        If you have other processes running on this machine (tomcat
>>        instances)
>>        you also need to adjust ant and Pg to take that into account.
>>
>>        Josh
>>
>>        On Wed, Feb 1, 2012 at 5:20 PM, JD Wong <jdmswong at gmail.com
>>        <mailto:jdmswong at gmail.com>> wrote:
>>         > Hi all,
>>         > Loading my Go source takes on average 2500 seconds.  I have
>>        tuned the
>>         > postgres configuration paramaters to the desired values and
>>        gave ant high
>>         > heap memory to no avail.  Is there a way to speed this up?
>>         >
>>         > -JD
>>         >
>>         > ______________________________**_________________
>>         > dev mailing list
>>         > dev at intermine.org <mailto:dev at intermine.org>
>>         > http://mail.intermine.org/cgi-**bin/mailman/listinfo/dev<http://mail.intermine.org/cgi-bin/mailman/listinfo/dev>
>>
>>         >
>>
>>
>>
>>
>>
>> ______________________________**_________________
>> dev mailing list
>> dev at intermine.org
>> http://mail.intermine.org/cgi-**bin/mailman/listinfo/dev<http://mail.intermine.org/cgi-bin/mailman/listinfo/dev>
>>
>
>
> ______________________________**_________________
> dev mailing list
> dev at intermine.org
> http://mail.intermine.org/cgi-**bin/mailman/listinfo/dev<http://mail.intermine.org/cgi-bin/mailman/listinfo/dev>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.intermine.org/pipermail/dev/attachments/20120207/4cee7d09/attachment-0001.html>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: intermine.log
Type: application/octet-stream
Size: 790845 bytes
Desc: not available
URL: <http://mail.intermine.org/pipermail/dev/attachments/20120207/4cee7d09/attachment-0001.obj>


More information about the dev mailing list