Using Smart Proxy Manager with Pyppeteer

Note

All the code in this documentation has been tested with Python 3.9.5 and Pyppeteer 0.2.6.

Installation

  1. Setup the Zyte SmartProxy (formerly Crawlera) Headless Proxy as described in Using Headless Browsers with Zyte Smart Proxy Manager.

  2. Install Pyppeteer using pip install pyppeteer==0.2.6.

  3. Run the sample script mentioned below using python sample.py

Sample script

# sample.py
import asyncio
from pyppeteer import launch

async def main():
   browser = await launch(
      {
            'ignoreHTTPSErrors': True,
            'args': [
               '--proxy-server=localhost:3128',
               '--ignore-certificate-errors'
            ]
      }
   )
   page = await browser.newPage()
   await page.goto('https://toscrape.com', {'timeout': 180000})
   await page.screenshot({'path': 'screenshot.png'})
   await browser.close()

asyncio.get_event_loop().run_until_complete(main())