Using Smart Proxy Manager with Puppeteer#

Warning

zyte-smartproxy-ca.crt should be installed in your OS for the below code to work. You can follow these instructions in order to install it.

Note

Puppeteer requires Node.js, npm installed and an active Smart Proxy Manager account. Log in or signup here.

Note

All the code in this documentation has been tested with Node.js v14.15.1 and puppeteer 10.4.0

Option 1 - Plugin for Puppeteer-Extra#

Check out details here.

Option 2 - Zyte SmartProxy Puppeteer#

Installation#

Download and install Zyte SmartProxy Puppeteer:

$ npm install zyte-smartproxy-puppeteer

Sample script#

Zyte SmartProxy Puppeteer is a client library that provides Zyte Smart Proxy Manager related functionalities over Puppeteer.

In order to run the sample code present below save it in a file named sample.js:

const puppeteer = require('zyte-smartproxy-puppeteer');
(async () => {
    const browser = await puppeteer.launch({
        spm_apikey: '<Smart Proxy Manager API KEY>',
        ignoreHTTPSErrors: true,
        headless: false,
    });
    console.log('Before new page');
    const page = await browser.newPage();
    console.log('Opening page ...');
    try {
        await page.goto('https://toscrape.com/', {timeout: 180000});
    } catch(err) {
        console.log(err);
    }
    console.log('Taking a screenshot ...');
    await page.screenshot({path: 'screenshot.png'});
    await browser.close();
})();

Now run the sample code using Node:

node sample.js

To know more about what you can do with Zyte SmartProxy Puppeteer, check out the API Documentation.

Option 3 - Zyte SmartProxy Headless Proxy#

Installation#

Setup the Zyte SmartProxy (formerly Crawlera) Headless Proxy as described in Using Headless Browsers with Zyte Smart Proxy Manager.
Download and install Puppeteer:
```
$ npm install puppeteer
```

Sample script#

Here is a sample script you can use to test the integration of Puppeteer with Smart Proxy Manager, once you have started the Headless Proxy.

const puppeteer = require('puppeteer');

(async () => {
    const browser = await puppeteer.launch({
        ignoreHTTPSErrors: true,
        headless: false,
        args: [
            '--proxy-server=localhost:3128'
        ]
    });
    const page = await browser.newPage({ignoreHTTPSErrors: true});

    console.log('Opening page ...');
    try {
        await page.goto('https://toscrape.com/', {timeout: 180000});
    } catch(err) {
        console.log(err);
    }

    console.log('Taking a screenshot ...');
    await page.screenshot({path: 'screenshot.png'});
    await browser.close();
})();