"Sorry, request timed out..." when using middleware.

[elorest]

I was really excited about using this gem instead of wicked_pdf or pdfkit, however I haven't been able to get the middleware to function correctly. I've added it to multiple rails projects including brand new ones but whenever I go to url/path.pdf it says "Preparing PDF..." and then after a few auto refreshes, "Sorry, request timed out..." even on very simple pages. Using the console command "Shrimp::Phantom.new(url).to_pdf!(target_path)" works fine and creates the pdf in just a second or two so it makes now sense that it's timing out. I've also tried increasing the timeout with no luck.

I looked through the closed issue regarding this but nothing in it solved my problem. #3

This seems like a really nice gem and I like that it's using phantomjs, I just really wish I could get it to work. Any help would be greatly appreciated. Thanks.

[TylerRick]

This seems to be the catch-all error that users/developers see if something goes wrong. I'm slowly figuring things out using trial and error and a debugger (byebug) and stepping through the source, but there's got to be a better way.

I'd like to add some debug logging that can be enabled to figure out what's going on behind the scenes and to figure out at what step things went wrong...

[bronson]

I'm finding the same thing. Shrimp works great from the command line but the middleware doesn't work at all, just times out. Or maybe it's conflicting with my app somehow?

[TylerRick]

It definitely seems to conflict with my Rails app in the development environment, at least, which is the only place I'm using it so far...

Specifically, it generates another request to my web server (I'm using thin in development) while the web server is still processing the first request... To get it to work for me, I had to start a second web server with rails s -p 3001 -P tmp/pids/server2.pid and hack the shrimp source to direct all of its requests to this other port. Is this the expected way to use Shrimp in development or am I doing something wrong?

[omnikron]

@isaacsloan @bronson are you all using thin as well? I wonder if that's the problem - I haven't been able to replicate this with webrick.

[elorest]

I tried it initially in webrick and then switched to puma with the same results.

[TylerRick]

Thanks for the tip about using puma. I switched from thin to puma in my Gemfile and now I only need a single development web server—yay! Thanks to puma supporting concurrent requests, it is able to process the 2nd HTTP request (that PhantomJS generates) even before the 1st request (that I generated from the web browser) has completed.

We should probably add this information to the Readme... Something like "Shrimp does not work well with single-threaded Ruby web servers like Webrick or Thin. You should use a Ruby web server that can handle concurrent requests, such as puma, instead." (I suppose multi-worker-process servers like Passenger should work as well...)

Or for those that are stuck using thin/webrick, maybe provide a config option that tells shrimp to direct all PhantomJS HTTP requests to a different port...

[omnikron]

@TylerRick sounds good, feel free to make a pull request
however @isaacsloan seems to say that he is experiencing the same problem with puma as well - can we get some rails and puma version information from both of you to see if there's maybe a difference in the puma config that might be causing @isaacsloan's to only manage one request at a time?

[Antiarchitect]

@isaacsloan @TylerRick @omnikron Hi folks, any success? I'm experiencing the same problem and puma doesn't help even with more than one worker.

[bhushangahire]

Hi I am also facing the same issue. I have ajax javascript request on page which are for Flot graph. As soon as I can see the ajax request getting executed I can see "Sorry, request timed out..." message. Is there any way to solve this?

[omnikron]

I'm still unable to replicate this - I wonder is this is maybe something to do with how many connections your database can handle? In rails, I have pool set to 25 in config/database.yml like this:

development:
  adapter: postgresql
  pool: 25

Does anybody who is getting these request timeouts have something different?

[bhushangahire]

I do have 5 for connection pool. I dont think this is something to do with connection pool. I also tried the Puma server with 5 workers. But still no luck. I must say this is the tool I am really in need. But somehow its not working.
I am still working on this. Does your application has Javascript requests?

[ghost]

I'm seeing this in development with passenger and 3 workers. Are there any suggestions?

ruby 2.1.1 + rails 3.2.17

[roa]

which version of phantomjs you are using?

[ghost]

phantomjs 1.9.7

OS X 10.9.2

[kemper]

I am also seeing this. I tried using webrick, puma with multiple threads, puma with a single thread and multiple processes, and thin. All web servers would time out. I'm using osx 10.9.2 and phantomjs 1.9.7.

When I use the middleware, I see the phantomjs process on my machine run for 90 seconds (using 'ps aux' to see running processes). Then it gives me the "Weird shit happened" message. I looked at the middleware source code, and didn't see anything obvious (though, I don't think forking a process in middleware is a good idea if you are running a threaded web server like puma).

In rails console, I can hit my server and generate a pdf of a page. It looks like I can also do it from a sidekiq job as well, so now I'm going down that route. However, I cannot do it from the middleware, and I cannot do it from inside the original web request either (inside the controller for that action).

I hope that helps. For now, I'll probably continue going down the path of generating the pdf from a background job as a workaround.

[breville]

I got it working as a middleware in a Sinatra app today, though it took some time adding my own debug logging to the code to figure out what was going wrong.

In my case, I had to provide a path to phantomjs like this:

Shrimp.configure do |config|
   config.phantomjs = Phantomjs.path
end

But all other options went via the middleware configuration:

use Shrimp::Middleware, :margin => '0.5cm', :format => 'Letter', :tmpdir => Dir.tmpdir

Setting tmpdir was crucial because it was defaulting to the root directory, which wasn't writable by the web process.

[bronson]

I never got the middleware to work reliably... To work around the problems, I extracted our application's background mode into a Shrimp-like gem: http://github.com/bronson/shrimple (it wasn't supposed to be a fork/rewrite, it just kinda happened...)

We now run 6+ Phantom processes in parallel all day without meaningful memory leaks.

Hoping others might find it useful.

[elorest]

I've tried puma, thin, unicorn, webrick, passenger and all the configuration suggestions in this thread with no luck. Has anyone gotten had this work for them lately?

[mikos]

I was experiencing the same issue and was able to fix it by setting the out_path parameter. Here's my middleware config line:

config.middleware.use Shrimp::Middleware, margin: '0.5cm',
                                          format: 'Letter',
                                          orientation: 'landscape',
                                          out_path: Rails.root.join('public')

[cesarizu]

I found that out_path is required. For me phantom was running just fine but it tried to copy the pdf to / and failed there. Adding config.middleware.use Shrimp::Middleware, :out_path => '/tmp' fixed that. Maybe it would be nice to have a default out_path = Dir.tmpdir