[InterMine Dev] Using perl client library with https

Joe Carlson jwcarlson at lbl.gov
Wed Feb 17 20:57:08 GMT 2016

Ah! I had added the class_type to Types.pm and I tried isa=>[NetHTTP,NetHTTPS]. That didn’t work. I didnt know about the pipe symbol.

But your syntax does not work for me with the quotes in there. I have more success if I write it as

isa => NetHTTP|NetHTTPS,

and it even work when I query flymine!

Are you folks the maintainers of the module in CPAN? Can you push out a update? 



> On Feb 17, 2016, at 12:12 PM, Josh Goodman <jogoodma at indiana.edu> wrote:
> Hi Joe,
> You can use type unions for this.  Given how this has been implemented here, I would do the following.
> Add a new type for NetHTTPS in Webservice/InterMine/Types.pm:
> class_type NetHTTPS, { class => 'Net::HTTPS', };
> Then modify ResultIterator to import the new type and use a type union on the connection attribute.
> use Webservice::InterMine::Types qw(Uri HTTPCode NetHTTP NetHTTPS RowFormat JsonFormat RequestFormat RowParser);
> ...
> has connection => (
>     is        => 'ro',
>     isa       => 'NetHTTP|NetHTTPS',
>     writer    => 'set_connection',
>     predicate => 'has_connection',
> );
> Josh
> On Wed, Feb 17, 2016 at 2:10 PM Joe Carlson <jwcarlson at lbl.gov <mailto:jwcarlson at lbl.gov>> wrote:
> For what it’s worth, I’ve verified that if I add NetHTTPS as a type in Webservice::InterMine::Types, change the ‘isa’ in connection to NetHTTPS and open the connection with Net::HTTPS, then I can use the perl API over https.
> But I don’t know enough about Moose to adapt the code to work with either Net::HTTP or Net::HTTPS. Line 520 in ResultIterator has a call $self->set_connection($connection) which checks that this is a specific class Net::HTTP. Is there a way to get Moose to accept alternative types in the type checking?
> Joe
>> On Feb 17, 2016, at 10:15 AM, Joe Carlson <jwcarlson at lbl.gov <mailto:jwcarlson at lbl.gov>> wrote:
>> New problem.
>> We have a mandate to use https exclusively for our web services. One of our users has seen that the perl client libraries do not work at our site. We have confirmed that this is a problem. There are some warning messages about deep recursion on subroutine "Webservice::InterMine::ResultIterator::set_headers” and the script never returns results.
>> I discovered the trick of setting the DEBUG environment variable to get some verbosity in the output. The first few fetches seem to work fine. There are some calls to fetch the service version and model. That goes well.Here is the output:
>>> FETCHING https://phytozome.jgi.doe.gov/phytomine/service/version <https://phytozome.jgi.doe.gov/phytomine/service/version> 1455730008.88403 at /opt/local/lib/perl5/site_perl/5.22/Webservice/InterMine/Service.pm line 830.
>>> FINISHED FETCHING https://phytozome.jgi.doe.gov/phytomine/service/version <https://phytozome.jgi.doe.gov/phytomine/service/version> 1455730009.32112 at /opt/local/lib/perl5/site_perl/5.22/Webservice/InterMine/Service.pm line 839.
>>> FETCHING https://phytozome.jgi.doe.gov/phytomine/service/model?format=xml <https://phytozome.jgi.doe.gov/phytomine/service/model?format=xml> 1455730009.32364 at /opt/local/lib/perl5/site_perl/5.22/Webservice/InterMine/Service.pm line 830.
>>> FINISHED FETCHING https://phytozome.jgi.doe.gov/phytomine/service/model?format=xml <https://phytozome.jgi.doe.gov/phytomine/service/model?format=xml> 1455730011.81447 at /opt/local/lib/perl5/site_perl/5.22/Webservice/InterMine/Service.pm line 839.
>>> string=><model name="genomic" package="org.intermine.model.bio">
>>> <class name="DatabaseReference" is-interface="true">
>>>         <attribute name="identifier" type="java.lang.String"/>
>>>         <reference name="subject" referenced-type="Publication" reverse-reference="crossReferences"/>
>>>         <reference name="source" referenced-type="DataSource”/>
>> ….
>> This went well and it appears to use the https connection retrieve the contents.
>>> CONNECTING TO phytozome.jgi.doe.gov <http://phytozome.jgi.doe.gov/>:443 at /opt/local/lib/perl5/site_perl/5.22/Webservice/InterMine/ResultIterator.pm line 493.
>>> UNENCODED-PARAM: constraint1=Gene.primaryIdentifier
>>> UNENCODED-PARAM: format=json
>>> UNENCODED-PARAM: name=GO_terms_for_genes
>>> UNENCODED-PARAM: value1=Potri.014G066100
>>> SENDING REQUESTPOST /phytomine/service/template/results HTTP/1.1
>>> Connection: close
>>> Host: phytozome.jgi.doe.gov <http://phytozome.jgi.doe.gov/>:443
>>> Content-Type: application/x-www-form-urlencoded
>>> User-Agent: Webservice::InterMine-1.0405/Perl client library
>>> Content-Length: 110
>>> op1=%3D&constraint1=Gene.primaryIdentifier&format=json&name=GO_terms_for_genes&value1=Potri.014G066100&code1=A
>>> SETTING HEADERS at /opt/local/lib/perl5/site_perl/5.22/Webservice/InterMine/ResultIterator.pm line 26.
>>> RETURNING LINE: <!DOCTYPE HTML PUBLIC "-//IETF//DTD HTML 2.0//EN"> at /opt/local/lib/perl5/site_perl/5.22/Webservice/InterMine/ResultIterator.pm line 468, <GEN3> line 1.
>>> RETURNING LINE: <html><head> at /opt/local/lib/perl5/site_perl/5.22/Webservice/InterMine/ResultIterator.pm line 468, <GEN3> line 2.
>>> RETURNING LINE: <title>400 Bad Request</title> at /opt/local/lib/perl5/site_perl/5.22/Webservice/InterMine/ResultIterator.pm line 468, <GEN3> line 3.
>>> RETURNING LINE: </head><body> at /opt/local/lib/perl5/site_perl/5.22/Webservice/InterMine/ResultIterator.pm line 468, <GEN3> line 4.
>>> RETURNING LINE: <h1>Bad Request</h1> at /opt/local/lib/perl5/site_perl/5.22/Webservice/InterMine/ResultIterator.pm line 468, <GEN3> line 5.
>>> RETURNING LINE: <p>Your browser sent a request that this server could not understand.<br /> at /opt/local/lib/perl5/site_perl/5.22/Webservice/InterMine/ResultIterator.pm line 468, <GEN3> line 6.
>>> RETURNING LINE: Reason: You're speaking plain HTTP to an SSL-enabled server port.<br /> at /opt/local/lib/perl5/site_perl/5.22/Webservice/InterMine/ResultIterator.pm line 468, <GEN3> line 7.
>>> HEADER Reason = You're speaking plain HTTP to an SSL-enabled server port.<br /> at /opt/local/lib/perl5/site_perl/5.22/Webservice/InterMine/ResultIterator.pm line 308, <GEN3> line 7.
>>> RETURNING LINE: Instead use the HTTPS scheme to access this URL, please.<br /> at /opt/local/lib/perl5/site_perl/5.22/Webservice/InterMine/ResultIterator.pm line 468, <GEN3> line 8.
>>> RETURNING LINE: <blockquote>Hint: <a href="https://gnweb01.nersc.gov/ <https://gnweb01.nersc.gov/>"><b>https://gnweb01.nersc.gov/</b></a></blockquote></p> <https://gnweb01.nersc.gov/%3C/b%3E%3C/a%3E%3C/blockquote%3E%3C/p%3E> at /opt/local/lib/perl5/site_perl/5.22/Webservice/InterMine/ResultIterator.pm line 468, <GEN3> line 9.
>>> HEADER <blockquote>Hint = <a href="https://gnweb01.nersc.gov/ <https://gnweb01.nersc.gov/>"><b>https://gnweb01.nersc.gov/</b></a></blockquote></p> <https://gnweb01.nersc.gov/%3C/b%3E%3C/a%3E%3C/blockquote%3E%3C/p%3E> at /opt/local/lib/perl5/site_perl/5.22/Webservice/InterMine/ResultIterator.pm line 308, <GEN3> line 9.
>>> RETURNING LINE: </body></html> at /opt/local/lib/perl5/site_perl/5.22/Webservice/InterMine/ResultIterator.pm line 468, <GEN3> line 10.
>>> RETURNING LINE:  at /opt/local/lib/perl5/site_perl/5.22/Webservice/InterMine/ResultIterator.pm line 468, <GEN3> line 10.
>>> FINISHED READING HEADERS at /opt/local/lib/perl5/site_perl/5.22/Webservice/InterMine/ResultIterator.pm line 312, <GEN3> line 10.
>>> RETURNING LINE:  at /opt/local/lib/perl5/site_perl/5.22/Webservice/InterMine/ResultIterator.pm line 468, <GEN3> line 10.
>>> FINISHED READING HEADERS at /opt/local/lib/perl5/site_perl/5.22/Webservice/InterMine/ResultIterator.pm line 312, <GEN3> line 10.
>>> RETURNING LINE:  at /opt/local/lib/perl5/site_perl/5.22/Webservice/InterMine/ResultIterator.pm line 468, <GEN3> line 10.
>>> FINISHED READING HEADERS at /opt/local/lib/perl5/site_perl/5.22/Webservice/InterMine/ResultIterator.pm line 312, <GEN3> line 10.
>>> RETURNING LINE:  at /opt/local/lib/perl5/site_perl/5.22/Webservice/InterMine/ResultIterator.pm line 468, <GEN3> line 10.
>>> FINISHED READING HEADERS at /opt/local/lib/perl5/site_perl/5.22/Webservice/InterMine/ResultIterator.pm line 312, <GEN3> line 10.
>> The repeated ‘RETURNING LINE: ‘… is the start of the recursive descent into oblivion.
>> The issue is that ResultIterator is attempting to open a Net::HTTP connection on port 443 in Webservice::InterMine::ResultIterator::connect and that fails. I’ve been seeing if I can replace this with Net::HTTPS but I’m not succeeding right away; it seems that I’m running into the Moose type checking. Being a Moose newbie, I need to see how to incorporate the netHTTPS type into the code.
>> Is this something that has come up before? Did you have any suggestions of other code changes or workarounds to enable the perl API over https? 
>> For what it’s worth, the java API is just fine. And, I’m going to put a pull request on github to correct a problem with generating the service URL for the auto generated code.
>> Thanks,
>> Joe Carlson

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.intermine.org/pipermail/dev/attachments/20160217/045fad2c/attachment-0001.html>

More information about the dev mailing list