HTTPI::SSLError and Yodlee

[kamil-sourcebits]

Hi all!

I've encountered a strange problem. I'm using Savon to connect to Yodlee's API but when trying to initiate the client object I get:

HTTPI::SSLError: SSL_connect SYSCALL returned=5 errno=0 state=unknown state

I've already tried stuff like ssl_verify_mode: :none etc. But nothing seems to work. Most of similiar bugs I saw on the Web had sth. like state=SSLv3 at the end of the error but mine is different. Does anyone has a single clue what is going on here? Thx for any suggestions.

[rubiii]

no idea. you could try using a different httpi adapter to exclude possible adapter-specific problems.

[rubiii]

stackoverflow said you might want to try changing the ssl version.

[kamil-sourcebits]

From these listed adpaters HTTPI.adapter = :curb # or one of [:httpclient, :em_http, :net_http] I am only able to set the :net_http in other cases I get LoadError: cannot load such file -- curb when trying to run my code i rails console. first.

[rubiii]

please read the httpi documentation. you need to manage the adapter client gems yourself,
because otherwise httpi would require you to install all client gems.

[kamil-sourcebits]

Thx. Now I'm getting sth. much more readeable (with HTTPClient adapter):

HTTPI GET request to sdk11.yodlee.com (httpclient)
at depth 2 - 20: unable to get local issuer certificate
HTTPI::SSLError: SSL_connect returned=1 errno=0 state=SSLv3 read server certificate B: certificate verify failed

I tryied pointing explicitly by ssl_cert_file: and ssl_ca_cert_file: to /etc/ssl/certs/ca-certificates.crt but still the same

[kamil-sourcebits]

I've made some progress in finding what is going on here. I understand that Savon uses HTTPI and HTTPI uses now HTTPClient adapter (setted by me). So I gave a try this code from this issue

nahi/httpclient#79

and tryied;

client = HTTPClient.new
client.ssl_config.add_trust_ca("/etc/ssl/certs") !!! This seems to be vital
client.get('https://sdk11.yodlee.com/yodsoap/services/CobrandLoginService?wsdl2')

And it works I don't get the error above just proper response. So maybe You know how to get it done by somehow manage to do sth. similiar to this line client.ssl_config.add_trust_ca("/etc/ssl/certs") ? Becouse I want to use Savon, it really served me well.

EDIT:
However now I'm testing those simple pieces of code i rails console on stage server, but as soon I will overcome the error from above I will have to use the ssh tunnel and Savon's 'proxy:' param. So the code will run on my personal computer but request will go through the stage server. On those to machines the certs are ofcourse different so stil its a messy stuff :\

[rubiii]

great work. assuming you're using the latest version of savon and httpi,
here's how this should work under the hood:

1. savon accepts a ca cert file via the global :ssl_ca_cert_file option and passes it to httpi.
2. httpi uses its auth.ssl.ca_cert_file config value to add the trust ca to httpclient

[kamil-sourcebits]

But still there sth. wrong with passing the option I run this code:

Savon.client(wsdl: 'https://sdk11.yodlee.com/yodsoap/services/CobrandLoginService?wsdl2', ssl_ca_cert_file: "/etc/ssl/certs", ssl_verify_mode: :peer).operations

And the error still occurs.

[rubiii]

kinda hart to remote-debug this. are you using the correct ssl version?

[kamil-sourcebits]

Error shows always SSLv3:
HTTPI::SSLError: SSL_connect returned=1 errno=0 state=SSLv3 read server certificate B: certificate verify failed

Even if I specify for exmaple ssl_version: :SSLv2

[rubiii]

no idea what this response means, but it looks like the server uses SSLv3

[kamil-sourcebits]

Still your sugestion about how it should work failed :, It looks like this line https://github.com/savonrb/httpi/blob/f68f49ce912eebea1433b918c4d3ace6eb381645/lib/httpi/adapter/httpclient.rb#L56 is not fired. I'm debugging this now to be sure about that.

[kamil-sourcebits]

I've followed with debugger and find out that this line is not triggerred:
https://github.com/savonrb/httpi/blob/f68f49ce912eebea1433b918c4d3ace6eb381645/lib/httpi/adapter/httpclient.rb#L39

Becouse @request.auth.ssl? is nil so the code I would like to be started, doesn't gets fired up.

[rubiii]

can you come up with some simple code or maybe even a spec to reproduce this?

[kamil-sourcebits]

This is the only code You need:

require "savon"
require "httpclient"

HTTPI.adapter = :httpclient
Savon.client(wsdl: 'https://sdk11.yodlee.com/yodsoap/services/CobrandLoginService?wsdl2').operations

I've wrote the adapter assigment implicitly to point to the adapter I was debugging.

EDIT 1:
However like I said I run this code on stage server which IP is whitelisted.
Same code run on my personal computer gives TimeOut:
HTTPClient::ConnectTimeoutError: execution expired.

EDIT 2:
I've also checked this on my personal computer and I see that despite there is TimeOut error this code
@request.auth.ssl?
is still nil here https://github.com/savonrb/httpi/blob/f68f49ce912eebea1433b918c4d3ace6eb381645/lib/httpi/adapter/httpclient.rb#L39

I see that always at this point there is no object HTTPI::Auth::SSL it is being created by this method https://github.com/savonrb/httpi/blob/f68f49ce912eebea1433b918c4d3ace6eb381645/lib/httpi/auth/config.rb#L69

So even if I pass in Savon options this:
ssl_verify_mode: :none

I always get :peer becouse of this code:
https://github.com/savonrb/httpi/blob/f68f49ce912eebea1433b918c4d3ace6eb381645/lib/httpi/auth/ssl.rb#L51

[davidlesches]

We're having the same problem here using Savon v1 with Yodlee. Exact same error: HTTPI::SSLError: SSL_connect SYSCALL returned=5 errno=0 state=unknown state

If you would like to collaborate, feel free to give us (NYC Devshop) a call 212 380 8730

• Alec and David

[kamil-sourcebits]

It seems that it is sth. between Savon and HTTPI, becouse options are not passed right. The error you are getting guys is becouse you use default adapter net_http and the error is really unclear. Try installing gem 'httpclient'
and the HTTPI will use this adapter by default and you will get the same error and you will be at the same point that I'm now.

EDIT:
The more I'm looking into it the more I feel that is even the HTTPI issue itself.

[kamil-sourcebits]

I've modified this method
https://github.com/savonrb/httpi/blob/master/lib/httpi/adapter/httpclient.rb#L36-L40
as I think its adapter problem at this moment.
Change:

  def setup_client
    basic_setup
    setup_auth if @request.auth.http?
    setup_ssl_auth if @request.auth.ssl?
    setup_ssl_paths if @request.ssl?
  end

  def setup_ssl_paths
    unless @request.auth.ssl.verify_mode == :none
      @client.ssl_config.add_trust_ca("/etc/ssl/certs")
    end
  end

The error no longer appears but savon returns empty array for 'operations' so I can't make specific request.

[kamil-sourcebits]

I've tried it manually:

sc = Savon.client do
wsdl "https://sdk11.yodlee.com/yodsoap/services/CobrandLoginService?wsdl2"
endpoint "https://sdk11.yodlee.com/yodsoap/services/CobrandLoginService"
namespace "http://cobrandlogin.login.core.soap.yodlee.com"
end

Like was mentioned here #373
But still its empty, without wsdl parameter I get error saying I can't inspect document without wsdl

EDIT:
I'm using savon 2.0.2 maybe newer version handles this, but v1 has the same problem

[kamil-sourcebits]

I've finally managed to make it work. The tutorial I was following was a bit confusing in this matter.

Savon.client(wsdl: 'https://sdk11.yodlee.com/yodsoap/services/CobrandLoginService?wsdl', ssl_verify_mode: :none).operations

Maybe becouse I've updated to savon 2.1.0 from 2.0.2 and the verify_mode: :none is working like charm, though http://nycdevshop.com/blog/the-yodlee-api-and-rails-part-1-starting-the-conversation here it was :peer. But now I want to connect on my computer using Savon (not working in the stage server console all the time :P) through the stage server (becouse its IP is whitelisted). I'm trying to use this option proxy: "http://my_stage_server.com" but I'm getting;

HTTPClient::BadResponseError: connect to ssl proxy failed with status 401 Unauthorized

I tryied even this:

proxy: "http://name:password@my_stage_server.com"

But still the same please help with this final obstacle.

[kamil-sourcebits]

Local port forwarding did the trick ;)

[davidlesches]

@camol-sb I'm still not 100% what you changed to get this working - can you please elaborate? When I use :none it still doesn't work. How did you fix this?

(BTW I wrote that tutorial you are using - it worked fine for me with Yodlee dev, but when we switched to the Yodlee production credentials, we started having the issues)

[kamil-sourcebits]

That is odd. I was surprised that it suddenly worked. Try setting the ssl version to tlsv1 I had to do this when I use the ssh tunnel and for assurance try savon v2.1.0. For now I think those are the only differences between us.

[rubiii]

@camol-sb i tested the following code using the latest versions of savon and httpi:

client = Savon.client(
  wsdl: "https://sdk11.yodlee.com/yodsoap/services/CobrandLoginService?wsdl2",
  ssl_ca_cert_file: "/etc/ssl/certs",
  ssl_verify_mode: :peer
)

client.operations

when you follow the stack through httpi, you can see that it calls the add_trust_ca method inside the httpclient adapter. you're example didn't specify any ssl options, so that's why httpi doesn't have anything to configure.

am i missing something?

[kamil-sourcebits]

Does savon is able to work with wsdl2 or only with wsdl? Becouse this code:

  @client = Savon.client do |globals|
    globals.wsdl "https://sdk11.yodlee.com/yodsoap/services/TransactionSearchService_12_0?wsdl2"
    globals.open_timeout 120
    globals.read_timeout 120
    globals.ssl_verify_mode :none
    globals.ssl_version :TLSv1
    globals.endpoint "https://sdk11.yodlee.com/yodsoap/services/TransactionSearchService_12_0"
    globals.pretty_print_xml true
  end

@client.operations

gives empty array. I manged to go connect to CobrandLoginService and it is ok. But when I use https://sdk11.yodlee.com/yodsoap/services/TransactionSearchService_12_0?wsdl2 without "2" at the end I can't see the xml i get

Unable to generate WSDL 1.1 for this service

If you wish Axis2 to automatically generate the WSDL 1.1, then please +set useOriginalwsdl as false in your services.xml

[rubiii]

i don't know the correct address for the wsdl of your service. you should find it in the documentation for the yodlee service. the "axis2" error message indicates, that there is no wsdl for the service under that address.

[kamil-sourcebits]

@davidlesches maybe there is sth. wrong with your production credentials? Please mail me [email protected] so we will be able to talk more specifially about Yodlee stuff

[kamil-sourcebits]

@davidlesches could You please share with me some of your expirience with yodlee? Maybe the email way will be the best, becouse I'm having problems with getting transactions from Yodlee please mail me thx.

[jkoisch]

Late to the party, but we got Savon working very well with Yodlee, deployed on Heroku with a Proximo proxy IP. We are deploying all of the wsdl's locally, which I found to be a little slow, but it works. Thanks to @davidlesches for the great tutorial which served as my starting point.

[mdavidn]

The issue here seems to be that Wasabi::Resolver#load_from_remote, which gets called via Savon::Operation#ensure_exists!, does not use the SSL options given to Savon.new.