[InterMine Dev] Source loading order, does it matter?

Richard Smith richard at flymine.org
Wed Oct 26 09:04:50 BST 2011


Hi JD,
Firstly, it looks like your items file sets the genus and species for
Dmel, all our sources uses the taxon id (7227) as there are fewer
opportunities to get it wrong.  I would switch your fb-genes source to set
the taxon id.

I suspect the error is that you don't have integration keys set up for
your fb-genes source, so when it's being loaded second it isn't attempting
to merge the genes.  Do you have an fb-genes_keys.properties file in
bio/sources/fb-genes/resources?

If keys are set up correctly then load order of sources shouldn't matter.

You should specify keys for the classes you load, probably something like
this:

DataSet.key_title=name
DataSource.key_name=name
Gene.key_primaryidentifier=primaryIdentifier
Organism.key_taxon = taxonId
OntologyTerm.key_name_ontology=name, ontology
Ontology.key_title=name
Chromosome.key = primaryIdentifier

See here for more information:  http://intermine.org/wiki/PrimaryKeys


Cheers,
Richard.

> Hi dev,
>
> A little background, I generate the items xml for my fb-genes source
> directly.
>
> When I load my genes source before go-annotation, I get this stack trace.
>
> BUILD FAILED
> /home/jdmswong/intermine/imbuild/integrate.xml:54: The following error
> occurred while executing this line:
> /home/jdmswong/intermine/imbuild/source.xml:330:
> java.lang.RuntimeException:
> Exception while dataloading - to allow multiple errors, set the property
> "dataLoader.allowMultipleErrors" to true
> Problem while loading item identifier 3_1 because
> Conflicting values for field Gene.organism between fb-genes (value
> "Organism
> [commonName="null", genus="Drosophila", id="73000005", name="null",
> shortName="Dmel", species="melanogaster", taxonId="null"]" in database
> with
> ID 73122557) and go-annotation (value "Organism [commonName="null",
> genus="null", id="74000002", name="null", shortName="null",
> species="null",
> taxonId="7227"]" being stored). This field needs configuring in the
> genomic_priorities.properties file
>  at
> org.intermine.dataloader.ObjectStoreDataLoader.process(ObjectStoreDataLoader.java:164)
> at
> org.intermine.dataloader.ObjectStoreDataLoader.process(ObjectStoreDataLoader.java:60)
>  at
> org.intermine.dataloader.ObjectStoreDataLoaderTask.execute(ObjectStoreDataLoaderTask.java:128)
> at org.apache.tools.ant.UnknownElement.execute(UnknownElement.java:291)
>  at sun.reflect.GeneratedMethodAccessor4.invoke(Unknown Source)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
>  at java.lang.reflect.Method.invoke(Method.java:597)
> at
> org.apache.tools.ant.dispatch.DispatchUtils.execute(DispatchUtils.java:106)
>  at org.apache.tools.ant.Task.perform(Task.java:348)
> at org.apache.tools.ant.Target.execute(Target.java:390)
>  at org.apache.tools.ant.Target.performTasks(Target.java:411)
> at org.apache.tools.ant.Project.executeSortedTargets(Project.java:1399)
>  at
> org.apache.tools.ant.helper.SingleCheckExecutor.executeTargets(SingleCheckExecutor.java:38)
> at org.apache.tools.ant.Project.executeTargets(Project.java:1251)
>  at org.apache.tools.ant.taskdefs.Ant.execute(Ant.java:442)
> at org.intermine.task.Integrate.performAction(Integrate.java:223)
>  at org.intermine.task.Integrate.performAction(Integrate.java:136)
> at org.intermine.task.Integrate.execute(Integrate.java:127)
>  at org.apache.tools.ant.UnknownElement.execute(UnknownElement.java:291)
> at sun.reflect.GeneratedMethodAccessor4.invoke(Unknown Source)
>  at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
> at java.lang.reflect.Method.invoke(Method.java:597)
>  at
> org.apache.tools.ant.dispatch.DispatchUtils.execute(DispatchUtils.java:106)
> at org.apache.tools.ant.Task.perform(Task.java:348)
>  at org.apache.tools.ant.Target.execute(Target.java:390)
> at org.apache.tools.ant.Target.performTasks(Target.java:411)
>  at org.apache.tools.ant.Project.executeSortedTargets(Project.java:1399)
> at org.apache.tools.ant.Project.executeTarget(Project.java:1368)
>  at
> org.apache.tools.ant.helper.DefaultExecutor.executeTargets(DefaultExecutor.java:41)
> at org.apache.tools.ant.Project.executeTargets(Project.java:1251)
>  at org.apache.tools.ant.Main.runBuild(Main.java:809)
> at org.apache.tools.ant.Main.startAnt(Main.java:217)
>  at org.apache.tools.ant.launch.Launcher.run(Launcher.java:280)
> at org.apache.tools.ant.launch.Launcher.main(Launcher.java:109)
> Caused by: java.lang.IllegalArgumentException: Conflicting values for
> field
> Gene.organism between fb-genes (value "Organism [commonName="null",
> genus="Drosophila", id="73000005", name="null", shortName="Dmel",
> species="melanogaster", taxonId="null"]" in database with ID 73122557) and
> go-annotation (value "Organism [commonName="null", genus="null",
> id="74000002", name="null", shortName="null", species="null",
> taxonId="7227"]" being stored). This field needs configuring in the
> genomic_priorities.properties file
>  at
> org.intermine.dataloader.SourcePriorityComparator.compare(SourcePriorityComparator.java:268)
> at
> org.intermine.dataloader.SourcePriorityComparator.compare(SourcePriorityComparator.java:33)
>  at java.util.TreeMap.put(TreeMap.java:530)
> at java.util.TreeSet.add(TreeSet.java:238)
>  at
> org.intermine.dataloader.IntegrationWriterDataTrackingImpl.store(IntegrationWriterDataTrackingImpl.java:385)
> at
> org.intermine.dataloader.IntegrationWriterAbstractImpl.store(IntegrationWriterAbstractImpl.java:171)
>  at
> org.intermine.dataloader.ObjectStoreDataLoader.process(ObjectStoreDataLoader.java:146)
> ... 33 more
>
>
> It looks like the value for organism is conflicting, but not when the load
> orders are reversed.
>
> Can a developer shed some light on this?
>
> Thanks!
> -JD
> _______________________________________________
> dev mailing list
> dev at intermine.org
> http://mail.intermine.org/cgi-bin/mailman/listinfo/dev
>




More information about the dev mailing list