Using Smart Proxy Manager with Puppeteer¶

Warning

zyte-smartproxy-ca.crt should be installed in your OS for the below code to work. You can follow these instructions in order to install it.

Warning

To use headless browsers properly, you then have to use Smart Proxy Manager in “Headless Mode”, which is different than the standard mode. You need to create a Headless user through the Smart Proxy Manager user dashboard for using this user in your headless browsers. More details in Using Headless Browsers with Zyte Smart Proxy Manager.

Note

Puppeteer requires Node.js, npm installed and an active Smart Proxy Manager account. Log in or signup here.

Note

All the code in this documentation has been tested with Node.js v14.15.1 and puppeteer 10.4.0

Option 1 - Zyte SmartProxy Puppeteer¶

Installation¶

Download and install Zyte SmartProxy Puppeteer:

$ npm install zyte-smartproxy-puppeteer

Sample script¶

Zyte SmartProxy Puppeteer is a client library that provides Zyte Smart Proxy Manager related functionalities over Puppeteer.

In order to run the sample code present below save it in a file named sample.js:

const puppeteer = require('zyte-smartproxy-puppeteer');
(async () => {
    const browser = await puppeteer.launch({
        spm_apikey: '<Smart Proxy Manager API KEY>',
        ignoreHTTPSErrors: true,
        headless: false,
    });
    console.log('Before new page');
    const page = await browser.newPage();
    console.log('Opening page ...');
    try {
        await page.goto('https://toscrape.com/', {timeout: 180000});
    } catch(err) {
        console.log(err);
    }
    console.log('Taking a screenshot ...');
    await page.screenshot({path: 'screenshot.png'});
    await browser.close();
})();

Now run the sample code using Node:

node sample.js

To know more about what you can do with Zyte SmartProxy Puppeteer, check out the API Documentation.

Option 2 - Zyte SmartProxy Headless Proxy¶

Installation¶

Setup the Zyte SmartProxy (formerly Crawlera) Headless Proxy as described in Using Headless Browsers with Zyte Smart Proxy Manager.
Download and install Puppeteer:
```
$ npm install puppeteer
```

Sample script¶

Here is a sample script you can use to test the integration of Puppeteer with Smart Proxy Manager, once you have started the Headless Proxy.

const puppeteer = require('puppeteer');

(async () => {
    const browser = await puppeteer.launch({
        ignoreHTTPSErrors: true,
        headless: false,
        args: [
            '--proxy-server=localhost:3128'
        ]
    });
    const page = await browser.newPage({ignoreHTTPSErrors: true});

    console.log('Opening page ...');
    try {
        await page.goto('https://toscrape.com/', {timeout: 180000});
    } catch(err) {
        console.log(err);
    }

    console.log('Taking a screenshot ...');
    await page.screenshot({path: 'screenshot.png'});
    await browser.close();
})();