Using Smart Proxy Manager with Pyppeteer#

Warning

zyte-smartproxy-ca.crt should be installed in your OS for the below code to work. You can follow these instructions in order to install it.

Note

All the code in this documentation has been tested with Python 3.9.5 and Pyppeteer 0.2.6.

Installation#

  1. Setup the Zyte SmartProxy (formerly Crawlera) Headless Proxy as described in Using Headless Browsers with Zyte Smart Proxy Manager.

  2. Install Pyppeteer using pip install pyppeteer==0.2.6.

  3. Run the sample script mentioned below using python sample.py

Sample script#

# sample.py
import asyncio
from pyppeteer import launch

async def main():
   browser = await launch(
      {
            'ignoreHTTPSErrors': True,
            'args': [
               '--proxy-server=localhost:3128',
               '--ignore-certificate-errors'
            ]
      }
   )
   page = await browser.newPage()
   await page.goto('https://toscrape.com', {'timeout': 180000})
   await page.screenshot({'path': 'screenshot.png'})
   await browser.close()

asyncio.get_event_loop().run_until_complete(main())