class Typhoeus::Hydra

Hydra manages making parallel HTTP requests. This is achieved by using libcurls multi interface: curl.haxx.se/libcurl/c/libcurl-multi.html The benefits are that you don't have to worry running the requests by yourself.

Hydra will also handle how many requests you can make in parallel. Things will get flakey if you try to make too many requests at the same time. The built in limit is 200. When more requests than that are queued up, hydra will save them for later and start the requests as others are finished. You can raise or lower the concurrency limit through the Hydra constructor.

Regarding the asynchronous behavior of the hydra, it is important to know that this is completely hidden from the developer and you are free to apply whatever technique you want to your code. That should not conflict with libcurls internal concurrency mechanism.

@example Use the hydra to do multiple requests.

hydra = Typhoeus::Hydra.new
requests = (0..9).map{ Typhoeus::Request.new("www.example.com") }
requests.each{ |request| hydra.queue(request) }
hydra.run

@note Callbacks are going to delay the request

execution.

Attributes

max_concurrency[RW]

@example Set max_concurrency.

Typhoeus::Hydra.new(max_concurrency: 20)
multi[R]

@api private

Public Class Methods

hydra() click to toggle source

Returns a memoized hydra instance.

@example Get a hydra.

Typhoeus::Hydra.hydra

@return [Typhoeus::Hydra] A new hydra.

# File lib/typhoeus/hydra.rb, line 66
def hydra
  Thread.current[:typhoeus_hydra] ||= new
end
new(options = {}) click to toggle source

Create a new hydra. All {rubydoc.info/github/typhoeus/ethon/Ethon/Multi#initialize-instance_method Ethon::Multi#initialize} options are also available.

@example Create a hydra.

Typhoeus::Hydra.new

@example Create a hydra with max_concurrency.

Typhoeus::Hydra.new(max_concurrency: 20)

@param [ Hash ] options The options hash.

@option options :max_concurrency [ Integer ] Number

of max concurrent connections to create. Default is
200.

@see rubydoc.info/github/typhoeus/ethon/Ethon/Multi#initialize-instance_method

Ethon::Multi#initialize
# File lib/typhoeus/hydra.rb, line 89
def initialize(options = {})
  @options = options
  @max_concurrency = @options.fetch(:max_concurrency, 200)
  @multi = Ethon::Multi.new(options.reject{|k,_| k==:max_concurrency})
end