Warning

Zyte API is replacing Smart Proxy Manager. It is no longer possible to sign up to Smart Proxy Manager. If you are an existing Smart Proxy Manager user, see Migrating from Smart Proxy Manager to Zyte API.

Using Smart Proxy Manager with Pyppeteer

Warning

For the code below to work you must first install the Zyte CA certificate.

Note

All the code in this documentation has been tested with Python 3.9.5 and Pyppeteer 0.2.6.

Installation

1. Setup the Zyte SmartProxy (formerly Crawlera) Headless Proxy as described in Using Headless Browsers with Zyte Smart Proxy Manager.

2. Install Pyppeteer using pip install pyppeteer==0.2.6.

3. Run the sample script mentioned below using python sample.py

Sample script

# sample.py
import asyncio
from pyppeteer import launch

async def main():
   browser = await launch(
      {
            'ignoreHTTPSErrors': True,
            'args': [
               '--proxy-server=localhost:3128',
               '--ignore-certificate-errors'
            ]
      }
   )
   page = await browser.newPage()
   await page.goto('https://toscrape.com', {'timeout': 180000})
   await page.screenshot({'path': 'screenshot.png'})
   await browser.close()

asyncio.get_event_loop().run_until_complete(main())