[InterMine Dev] solr issues (again)

daniela at intermine.org daniela at intermine.org
Thu Jun 24 09:18:46 BST 2021


Dear Joe,

about the fact that the gradle process reports ‘BUILD SUCCESSFUL’, even
with an issue on creating the index, has been already discussed here 
[1].
If you prefer a BUILD FAIL instead, you can change the code here[2] 
adding
the following:
throw new org.apache.tools.ant.BuildException(e)

About the size of the indexed documents, flymine has 10583097 docs.
I'm not sure how long the create-search-index process takes, Sergio 
usually runs the build and he is on holidays,
but if it helps, he never mentions issues related to solr.

Best wishes,
Daniela

[1] https://github.com/intermine/intermine/issues/2120
[2] 
https://github.com/intermine/intermine/blob/dev/plugin/src/main/groovy/org/intermine/plugin/postprocess/PostProcessPlugin.groovy#L139




> Hello,
> 
> A year or so ago I was struggling with building my solr index. It was
> a bit frustrating since the gradle process reports ‘BUILD
> SUCCESSFUL’ even though the create-search-index task itself reports
> 
> POSTPROCESS create-search-index FAILED.
> org.apache.solr.client.solrj.SolrServerException: IOException occured
> when talking to server at: https://phytozome-next.jgi.doe.gov/….
> 
> The log files report this as an IOException but I’ve not been able
> to see more details than that. It generally happens after about an
> hour.
> 
> Last time I looked into this I had thought that the problem was that
> our solr server is sitting behind an nginx reverse proxy and nginx was
> limiting the body size of a post. The max size of a post body is
> configurable, and when I upped that to 100M, I was able to build. But,
> I should add, I’m indexing much fewer fields than the default. (It
> would just take too long to index all the collections and references.)
> Previously I was able to make my abbreviated index - and have a store
> of 11.6M documents. Now I have trouble even making an index this
> large; the IO exception occurs after ~7M documents.
> 
> I’m curious if anyone else has see IOExceptions when building solr.
> Or, from other groups, how large of an index they routinely build (and
> how long it takes to build it.)
> 
> Thanks,
> 
> Joe
> _______________________________________________
> dev mailing list
> dev at lists.intermine.org
> https://lists.intermine.org/mailman/listinfo/dev



More information about the dev mailing list