[InterMine Dev] Using perl client library with https

Josh Goodman jogoodma at indiana.edu
Wed Feb 17 20:12:03 GMT 2016


Hi Joe,

You can use type unions for this.  Given how this has been implemented
here, I would do the following.

Add a new type for NetHTTPS in Webservice/InterMine/Types.pm:

class_type NetHTTPS, { class => 'Net::HTTPS', };

Then modify ResultIterator to import the new type and use a type union on
the connection attribute.

use Webservice::InterMine::Types qw(Uri HTTPCode NetHTTP NetHTTPS RowFormat
JsonFormat RequestFormat RowParser);
...
has connection => (
    is        => 'ro',
    isa       => 'NetHTTP|NetHTTPS',
    writer    => 'set_connection',
    predicate => 'has_connection',
);

Josh

On Wed, Feb 17, 2016 at 2:10 PM Joe Carlson <jwcarlson at lbl.gov> wrote:

> For what it’s worth, I’ve verified that if I add NetHTTPS as a type in
> Webservice::InterMine::Types, change the ‘isa’ in connection to NetHTTPS
> and open the connection with Net::HTTPS, then I can use the perl API over
> https.
>
> But I don’t know enough about Moose to adapt the code to work with either
> Net::HTTP or Net::HTTPS. Line 520 in ResultIterator has a call
> $self->set_connection($connection) which checks that this is a specific
> class Net::HTTP. Is there a way to get Moose to accept alternative types in
> the type checking?
>
> Joe
>
>
> On Feb 17, 2016, at 10:15 AM, Joe Carlson <jwcarlson at lbl.gov> wrote:
>
> New problem.
>
> We have a mandate to use https exclusively for our web services. One of
> our users has seen that the perl client libraries do not work at our site.
> We have confirmed that this is a problem. There are some warning messages
> about deep recursion on subroutine
> "Webservice::InterMine::ResultIterator::set_headers” and the script never
> returns results.
>
> I discovered the trick of setting the DEBUG environment variable to get
> some verbosity in the output. The first few fetches seem to work fine.
> There are some calls to fetch the service version and model. That goes
> well.Here is the output:
>
> FETCHING https://phytozome.jgi.doe.gov/phytomine/service/version
> 1455730008.88403 at
> /opt/local/lib/perl5/site_perl/5.22/Webservice/InterMine/Service.pm line
> 830.
> FINISHED FETCHING https://phytozome.jgi.doe.gov/phytomine/service/version
> 1455730009.32112 at
> /opt/local/lib/perl5/site_perl/5.22/Webservice/InterMine/Service.pm line
> 839.
> FETCHING https://phytozome.jgi.doe.gov/phytomine/service/model?format=xml
> 1455730009.32364 at
> /opt/local/lib/perl5/site_perl/5.22/Webservice/InterMine/Service.pm line
> 830.
> FINISHED FETCHING
> https://phytozome.jgi.doe.gov/phytomine/service/model?format=xml
> 1455730011.81447 at
> /opt/local/lib/perl5/site_perl/5.22/Webservice/InterMine/Service.pm line
> 839.
> string=><model name="genomic" package="org.intermine.model.bio">
> <class name="DatabaseReference" is-interface="true">
>         <attribute name="identifier" type="java.lang.String"/>
>         <reference name="subject" referenced-type="Publication"
> reverse-reference="crossReferences"/>
>         <reference name="source" referenced-type="DataSource”/>
>
> ….
>
> This went well and it appears to use the https connection retrieve the
> contents.
>
>
> CONNECTING TO phytozome.jgi.doe.gov:443 at
> /opt/local/lib/perl5/site_perl/5.22/Webservice/InterMine/ResultIterator.pm
> line 493.
> UNENCODED-PARAM: op1==
> UNENCODED-PARAM: constraint1=Gene.primaryIdentifier
> UNENCODED-PARAM: format=json
> UNENCODED-PARAM: name=GO_terms_for_genes
> UNENCODED-PARAM: value1=Potri.014G066100
> UNENCODED-PARAM: code1=A
> SENDING REQUESTPOST /phytomine/service/template/results HTTP/1.1
> Connection: close
> Host: phytozome.jgi.doe.gov:443
> Content-Type: application/x-www-form-urlencoded
> User-Agent: Webservice::InterMine-1.0405/Perl client library
> Content-Length: 110
>
>
> op1=%3D&constraint1=Gene.primaryIdentifier&format=json&name=GO_terms_for_genes&value1=Potri.014G066100&code1=A
> SETTING HEADERS at
> /opt/local/lib/perl5/site_perl/5.22/Webservice/InterMine/ResultIterator.pm
> line 26.
> RETURNING LINE: <!DOCTYPE HTML PUBLIC "-//IETF//DTD HTML 2.0//EN"> at
> /opt/local/lib/perl5/site_perl/5.22/Webservice/InterMine/ResultIterator.pm
> line 468, <GEN3> line 1.
> RETURNING LINE: <html><head> at
> /opt/local/lib/perl5/site_perl/5.22/Webservice/InterMine/ResultIterator.pm
> line 468, <GEN3> line 2.
> RETURNING LINE: <title>400 Bad Request</title> at
> /opt/local/lib/perl5/site_perl/5.22/Webservice/InterMine/ResultIterator.pm
> line 468, <GEN3> line 3.
> RETURNING LINE: </head><body> at
> /opt/local/lib/perl5/site_perl/5.22/Webservice/InterMine/ResultIterator.pm
> line 468, <GEN3> line 4.
> RETURNING LINE: <h1>Bad Request</h1> at
> /opt/local/lib/perl5/site_perl/5.22/Webservice/InterMine/ResultIterator.pm
> line 468, <GEN3> line 5.
> RETURNING LINE: <p>Your browser sent a request that this server could not
> understand.<br /> at
> /opt/local/lib/perl5/site_perl/5.22/Webservice/InterMine/ResultIterator.pm
> line 468, <GEN3> line 6.
> RETURNING LINE: Reason: You're speaking plain HTTP to an SSL-enabled
> server port.<br /> at
> /opt/local/lib/perl5/site_perl/5.22/Webservice/InterMine/ResultIterator.pm
> line 468, <GEN3> line 7.
> HEADER Reason = You're speaking plain HTTP to an SSL-enabled server
> port.<br /> at
> /opt/local/lib/perl5/site_perl/5.22/Webservice/InterMine/ResultIterator.pm
> line 308, <GEN3> line 7.
> RETURNING LINE: Instead use the HTTPS scheme to access this URL,
> please.<br /> at
> /opt/local/lib/perl5/site_perl/5.22/Webservice/InterMine/ResultIterator.pm
> line 468, <GEN3> line 8.
> RETURNING LINE: <blockquote>Hint: <a href="https://gnweb01.nersc.gov/"><b>
> https://gnweb01.nersc.gov/</b></a></blockquote></p> at
> /opt/local/lib/perl5/site_perl/5.22/Webservice/InterMine/ResultIterator.pm
> line 468, <GEN3> line 9.
> HEADER <blockquote>Hint = <a href="https://gnweb01.nersc.gov/"><b>
> https://gnweb01.nersc.gov/</b></a></blockquote></p> at
> /opt/local/lib/perl5/site_perl/5.22/Webservice/InterMine/ResultIterator.pm
> line 308, <GEN3> line 9.
> RETURNING LINE: </body></html> at
> /opt/local/lib/perl5/site_perl/5.22/Webservice/InterMine/ResultIterator.pm
> line 468, <GEN3> line 10.
> RETURNING LINE:  at
> /opt/local/lib/perl5/site_perl/5.22/Webservice/InterMine/ResultIterator.pm
> line 468, <GEN3> line 10.
> FINISHED READING HEADERS at
> /opt/local/lib/perl5/site_perl/5.22/Webservice/InterMine/ResultIterator.pm
> line 312, <GEN3> line 10.
> RETURNING LINE:  at
> /opt/local/lib/perl5/site_perl/5.22/Webservice/InterMine/ResultIterator.pm
> line 468, <GEN3> line 10.
> FINISHED READING HEADERS at
> /opt/local/lib/perl5/site_perl/5.22/Webservice/InterMine/ResultIterator.pm
> line 312, <GEN3> line 10.
> RETURNING LINE:  at
> /opt/local/lib/perl5/site_perl/5.22/Webservice/InterMine/ResultIterator.pm
> line 468, <GEN3> line 10.
> FINISHED READING HEADERS at
> /opt/local/lib/perl5/site_perl/5.22/Webservice/InterMine/ResultIterator.pm
> line 312, <GEN3> line 10.
> RETURNING LINE:  at
> /opt/local/lib/perl5/site_perl/5.22/Webservice/InterMine/ResultIterator.pm
> line 468, <GEN3> line 10.
> FINISHED READING HEADERS at
> /opt/local/lib/perl5/site_perl/5.22/Webservice/InterMine/ResultIterator.pm
> line 312, <GEN3> line 10.
>
>
> The repeated ‘RETURNING LINE: ‘… is the start of the recursive descent
> into oblivion.
>
> The issue is that ResultIterator is attempting to open a Net::HTTP
> connection on port 443 in Webservice::InterMine::ResultIterator::connect
> and that fails. I’ve been seeing if I can replace this with Net::HTTPS but
> I’m not succeeding right away; it seems that I’m running into the Moose
> type checking. Being a Moose newbie, I need to see how to incorporate the
> netHTTPS type into the code.
>
> Is this something that has come up before? Did you have any suggestions of
> other code changes or workarounds to enable the perl API over https?
>
> For what it’s worth, the java API is just fine. And, I’m going to put a
> pull request on github to correct a problem with generating the service URL
> for the auto generated code.
>
> Thanks,
>
> Joe Carlson
>
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.intermine.org/pipermail/dev/attachments/20160217/e49a3e50/attachment.html>


More information about the dev mailing list