Zyte API is replacing Smart Proxy Manager. See Migrating from Smart Proxy Manager to Zyte API.

Using Smart Proxy Manager with Ruby#


For the code below to work you must first install the Zyte CA certificate.


All the code in this documentation has been tested with Ruby 2.6.3p62, Curb 0.9.11, Typhoeus 1.4.0, and Mechanize 2.8.2.

Using Curb#

Making use of curb, a Ruby binding for libcurl:

require 'curb'

url = "https://toscrape.com"
proxy = "proxy.zyte.com:8011"
proxy_auth = "<CRAWLERA_APIKEY>:"

c = Curl::Easy.new(url) do |curl|
  curl.proxypwd = proxy_auth
  curl.proxy_url = proxy
  curl.verbose = true

puts c.body_str

Using Typhoeus#

Making use of typhoeus, another Ruby binding for libcurl:

require 'typhoeus'

url = "https://toscrape.com"
proxy_host = "proxy.zyte.com:8011"
proxy_auth = "<CRAWLERA_APIKEY>:"

request = Typhoeus::Request.new(
  method: :get,
  proxy: proxy_host,
  proxyuserpwd: proxy_auth,
  headers: {"X-Crawlera-Timeout" => "60000"}

print "Response Code: "
puts request.response.code
print "Response Time: "
puts request.response.total_time
print "Response Headers: "
puts request.response.headers
print "Response Body: "
puts request.response.body

Using Mechanize#

Making use of mechanize, a Ruby library for automated web interaction:

require 'rubygems'
require 'mechanize'

url = "https://toscrape.com"
proxy_host = "proxy.zyte.com"
proxy_api_key = "<CRAWLERA_APIKEY>"

agent = Mechanize.new
agent.set_proxy proxy_host, 8011, proxy_api_key, ''

res = agent.get(url)
puts res.body

To allow Mechanize to use the Zyte CA certificate, point the SSL_CERT_FILE environment variable to its path:

export SSL_CERT_FILE=/path/to/zyte-ca.crt