[InterMine Dev] data loading errors

Kalpana Karra kkarra at stanford.edu
Wed Oct 28 18:19:08 GMT 2020


Thanks, Sam! trying with increased -Xmx..
Interestingly enough, project_build goes on to load the next data source and data is getting into the database for the source on which this error occurs. It happens on one of the longest data source.



> On Oct 27, 2020, at 8:53 AM, Sam Hokin <shokin at ncgr.org> wrote:
> 
> I get that when the database/connections are heavily loaded, like if I'm running post-processors and sources at the same time. Sometimes it recovers with just the WARNs, and sometimes it'll drop the connection with a socket timeout as you see. You'll want to boost your resources, likely on the database server, but perhaps also on the machine running the IM code.
> 
> On 10/26/20 11:28 PM, Kalpana Karra wrote:
>> Hello,
>> If loading a data source that takes a while, I see this type of warning and eventually connection timeout, wondering how to get around this.
>> [dataLoad] [db.production housekeeper] WARN com.zaxxer.hikari.pool.HikariPool - db.production - Thread starvation or clock leap detected (housekeeper delta=58s402ms).
>>  [dataLoad] [db.production housekeeper] WARN com.zaxxer.hikari.pool.HikariPool - db.production - Thread starvation or clock leap detected (housekeeper delta=46s330ms).
>> Connection error:
>> org.postgresql.util.PSQLException: The connection attempt failed.
>> 	at org.postgresql.Driver$ConnectThread.getResult(Driver.java:403)
>> 	at org.postgresql.Driver.connect(Driver.java:261)
>> 	at java.sql.DriverManager.getConnection(DriverManager.java:664)
>> 	at java.sql.DriverManager.getConnection(DriverManager.java:247)
>> 	at org.postgresql.ds.common.BaseDataSource.getConnection(BaseDataSource.java:94)
>> 	at org.postgresql.ds.common.BaseDataSource.getConnection(BaseDataSource.java:79)
>> 	at com.zaxxer.hikari.pool.PoolBase.newConnection(PoolBase.java:369)
>> 	at com.zaxxer.hikari.pool.PoolBase.newPoolEntry(PoolBase.java:198)
>> 	at com.zaxxer.hikari.pool.HikariPool.createPoolEntry(HikariPool.java:467)
>> 	at com.zaxxer.hikari.pool.HikariPool.access$100(HikariPool.java:71)
>> 	at com.zaxxer.hikari.pool.HikariPool$PoolEntryCreator.call(HikariPool.java:706)
>> 	at com.zaxxer.hikari.pool.HikariPool$PoolEntryCreator.call(HikariPool.java:692)
>> 	at java.util.concurrent.FutureTask.run(FutureTask.java:266)
>> 	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
>> 	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
>> 	at java.lang.Thread.run(Thread.java:748)
>> Caused by: java.net.SocketTimeoutException
>> 	at java.net.SocksSocketImpl.remainingMillis(SocksSocketImpl.java:111)
>> 	at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392)
>> 	at java.net.Socket.connect(Socket.java:607)
>> 	at org.postgresql.core.PGStream.<init>(PGStream.java:69)
>> 	at org.postgresql.core.v3.ConnectionFactoryImpl.openConnectionImpl(ConnectionFactoryImpl.java:156)
>> 	at org.postgresql.core.ConnectionFactory.openConnection(ConnectionFactory.java:49)
>> 	at org.postgresql.jdbc.PgConnection.<init>(PgConnection.java:195)
>> 	at org.postgresql.Driver.makeConnection(Driver.java:452)
>> 	at org.postgresql.Driver.access$100(Driver.java:61)
>> 	at org.postgresql.Driver$ConnectThread.run(Driver.java:362)
>> 	... 1 more
>> Failed to create a Non-Pooling DataSource from PostgreSQL JDBC Driver 42.2.2 for postgres at jdbc:postgresql
>> Thanks,
>> -Kalpana
>> _______________________________________________
>> dev mailing list
>> dev at lists.intermine.org
>> https://lists.intermine.org/mailman/listinfo/dev



More information about the dev mailing list