Fetching HTTPS pages with Zyte Smart Proxy Manager¶
Zyte Smart Proxy Manager (formerly Crawlera) performs HTTPS requests using CONNECT method, transparently tunneling packets over default HTTPS port 443 to and from the destination server in a secure session.
To fetch HTTPS web pages through Smart Proxy Manager you need to download and install the certificate file for Smart Proxy Manager Certificate Authority or disable SSL certificate verification in your HTTP client.
The Smart Proxy Manager Certificate Authority can be downloaded here: zyte-smartproxy-ca.crt.
Installing the certificate file on Windows 10¶
Win key + Rhotkey and input
mmcin Run to open the Microsoft Management Console window.
In the opened window select
Certificatesand press the
In the Certificates Snap-in window select
Computer account > Local Account, and press the
Finishbutton to close the window.
OKbutton in the Add or Remove Snap-in window.
Back in the Microsoft Management Console window, select
Certificatesunder Console Root and right-click
Trusted Root Certification Authorities.
From the context menu select
All Tasks > Importto open the Certificate Import Wizard window from which you can add Smart Proxy Manager’s certificate.
More details can be found here.
Installing the certificate file on Mac¶
Open Keychain Access window (
Launchpad > Other > Keychain Access).
Systemtab under Keychains, drag and drop the downloaded certificate file (or select File >
Import Items...and navigate to the file).
Enter the administrator password to modify the keychain.
Smart Proxy Manager CAcertificate entry, expand Trust, next to When using this certificate: select
Close the window and enter the administrator password again to update the settings.
Installing the certificate file on Linux/Ubuntu¶
Copy the downloaded certificate file using the following command:
sudo cp zyte-smartproxy-ca.crt /usr/local/share/ca-certificates/zyte-smartproxy-ca.crt
Update stored Certificate Authority files:
Installing the certificate file into the Mozilla Firefox browser¶
Open Preferences, visit Privacy & Security tab, scroll down to the Certificates section, click View Certificates… button to open Certificate Manager.
Under Authorities tab click Import… button, navigate to the certificate file.
In the opened window (You have been asked to trust a new Certificate Authority (CA)) check the first option Trust this CA to identify websites and click the OK button to finish importing the certificate.
Click the OK button to save settings and exit Certificate Manager.
Installing the certificate file into the Google Chrome browser¶
Click the triple-dot icon in the top right corner and choose
Scroll to the
Privacy and securitysection and click on
Scroll down to find and click
The next steps will depend on the operating system. In the case of Mac OS, the previous action will open the
Keychain Access, see Installing the certificate file on OS X section above. In the case of Windows, the Certificates application should appear, select
Trusted Root Certification Authoritiestab, click
Import...button, navigate to the certificate file, verify the import was successful and the installed certificate is displayed under Trusted Root Certification Authorities tab, close the Certificates window.
In case of requiring a certificate with PKCS#12 format, you can generate it with the following OpenSSL command:
openssl pkcs12 -export -nokeys -password pass: -in zyte-smartproxy-ca.crt -out zyte-smartproxy-ca.p12
The minimal example to check if the CA is working for you is to try:
curl -vx https://proxy.zyte.com:8014 -U <API key>: --cacert zyte-smartproxy-ca.crt https://httpbin.org/ip