[InterMine Dev] Why MouseMine was getting sluggish (Or: Range queries cause tomcat to lose its mind)

Julie Sullivan julie at flymine.org
Fri Aug 2 10:31:15 BST 2013


Joel

Thanks for the email, I think that perhaps the range queries are not 
quite ready quite yet!

We'll test some more and let you know when we have fixed the problem.

Thanks,
Julie

On 31/07/13 21:38, Joel Richardson wrote:
>
> Hi devs,
>
> Earlier today I emailed about MouseMine getting sluggish. I think I now
> know the cause.
>
> I have been experimenting with using the OVERLAPS operator in path
> queries as documented here:
> http://intermine.readthedocs.org/en/latest/api/pathquery/
>
> The issue: beyond some threshold in the number of region values in the
> constraint, tomcat goes wacko. Below the threshold, the results are
> returned OK. Tomcat's CPU and memory usage go up, but then drop back to
> near zero. Above the threshold, the query never returns. Top shows CPU
> usage near 100%, and it just stays that way. However, pg_stat_activity
> shows nothing. Whatever is happening is happening in tomcat. It is *not*
> related to the number of results. Even when I tweaked the query to
> guarantee zero results (changed the organism id to something we don't
> have), tomcat still went crazy.
>
> Initially I was doing everything though the JavaScript client (im.js),
> but I confirmed the same behavior via the web interface by loading the
> XML queries. When run via the query builder, the "running your query"
> window goes away quickly. The results window starts to render. Then it
> hangs at "Building table". (BTW, you can upload/execute OVERLAPS queries
> in the QB, but you can't edit the constraints. Another ticket?)
>
> Here is an example query that runs OK:
>
> <query model="genomic" view="SequenceFeature.organism.shortName
> SequenceFeature.primaryIdentifier SequenceFeature.symbol
> SequenceFeature.chromosomeLocation.locatedOn.primaryIdentifier
> SequenceFeature.chromosomeLocation.start
> SequenceFeature.chromosomeLocation.end"
> sortOrder="SequenceFeature.organism.shortName ASC
> SequenceFeature.chromosomeLocation.locatedOn.symbol ASC
> SequenceFeature.chromosomeLocation.start ASC" constraintLogic="A and ((B
> and C) or (D and E))" ><constraint
> path="SequenceFeature.primaryIdentifier" op="IS NOT NULL" /><constraint
> path="SequenceFeature.organism.taxonId" op="=" value="9606"
> /><constraint path="SequenceFeature.chromosomeLocation" op="OVERLAPS"
>  ><value>13:105535739..111044111</value></constraint><constraint
> path="SequenceFeature.organism.taxonId" op="=" value="10090"
> /><constraint path="SequenceFeature.chromosomeLocation" op="OVERLAPS"
>  ><value>8:6505302..10756776</value></constraint></query>
>
>
> Here is one that doesn't (Please don't execute this on MouseMine!)
>
> <query model="genomic" view="SequenceFeature.organism.shortName
> SequenceFeature.primaryIdentifier SequenceFeature.symbol
> SequenceFeature.chromosomeLocation.locatedOn.primaryIdentifier
> SequenceFeature.chromosomeLocation.start
> SequenceFeature.chromosomeLocation.end"
> sortOrder="SequenceFeature.organism.shortName ASC
> SequenceFeature.chromosomeLocation.locatedOn.symbol ASC
> SequenceFeature.chromosomeLocation.start ASC" constraintLogic="A and ((B
> and C) or (D and E))" ><constraint
> path="SequenceFeature.primaryIdentifier" op="IS NOT NULL" /><constraint
> path="SequenceFeature.organism.taxonId" op="=" value="9606"
> /><constraint path="SequenceFeature.chromosomeLocation" op="OVERLAPS"
>  ><value>X:94248091..143371935</value></constraint><constraint
> path="SequenceFeature.organism.taxonId" op="=" value="10090"
> /><constraint path="SequenceFeature.chromosomeLocation" op="OVERLAPS"
>  ><value>1:31222839..31224287</value><value>5:114828573..114829215</value><value>12:62687822..62688244</value><value>X:37126795..37192467</value><value>X:38189812..45092791</value><value>X:46329007..46330345</value><value>X:47809368..49886401</value><value>X:50555744..53370502</value><value>X:53608032..53608432</value><value>X:53669177..53670408</value><value>X:53724826..53738441</value><value>X:56346400..61185558</value><value>X:61709497..61710955</value><value>X:64264885..64876551</value><value>X:75785654..75875182</value><value>X:129749742..134362639</value><value>X:134404551..134809221</value><value>X:134944526..134964754</value><value>X:134971372..135009185</value><value>X:135742733..135891081</value><value>X:136066564..136068236</value><value>X:136072099..136203876</value><value>X:136213972..136215513</value><value>X:136245079..136271978</value><value>X:136590842..140062712</value><value>X:140367494..141725263</value><value>X:142081160..148521446</value><value>X:21484544..3711
0322</value></constraint></query>
>
>
> Thanks in advance,
> Joel
>
> --
> Joel E. Richardson, Ph.D.
> Sr. Research Scientist
> Mouse Genome Informatics
> The Jackson Laboratory
> 600 Main Street
> Bar Harbor, Maine 04609
> 207-288-6435
> joel.richardson at jax.org
>
> The information in this email, including attachments, may be
> confidential and is intended solely for the addressee(s). If you believe
> you received this email by mistake, please notify the sender by return
> email as soon as possible.
>
>
>
> _______________________________________________
> dev mailing list
> dev at intermine.org
> http://mail.intermine.org/cgi-bin/mailman/listinfo/dev
>



More information about the dev mailing list