Zyte API custom browser scripts#

Zyte API accounts on the Enterprise plan can write, debug, and deploy custom browser scripts to use as browser actions in browser-based data extraction requests.

Custom browser scripts can be written from Smart Browser DevTools IDE using our TypeScript scripting API.

src/ and dist/#

On the Explorer Explorer view of Zyte API Smart Browser DevTools IDE you get 2 folders:

  • src/ is the development folder, where you develop your custom browser scripts.

    Changes to this folder are saved on the Zyte API Smart Browser DevTools IDE cloud storage system.

    Multiple developers can work on this folder at the same time, and see the work of one another in real time.

  • dist/ is the deployment folder.

    It contains files built from the last version of src/ that has been deployed to Zyte API.

    You should never write anything in the dist/ folder, all its contents are removed during deployment.

Creating a new script#

To create a new custom browser script:

  1. Open Smart Browser DevTools IDE.

  2. Select Application Menu (top-left corner) → File → New File….

  3. On the Create New… dialog, select Smart Browser Interaction.

  4. On the Use Case dialog, select one of the following:

    • One of the special action interaction classes, to extend it in your new script.

    • Others, to create a script from scratch.

    • Utils, to create a file with utility code that you can reuse from browser scripts and other utility code files.

  5. On the Domain dialog, enter the domain of the website for which you are writing the script (e.g. toscrape.com).

A TypeScript file is created in the src/ folder based on your input data, and will open on a tab of Smart Browser DevTools IDE.

For example, if you select the Others use case and the toscrape.com domain, a src/Others/toscrape.com.ts file is created with the following content:

import { BaseInteraction, Page } from "smartbrowser-core-interactions/index.ts";

interface Args {
    // add your arguments here
    // arg1: string;
}

export default class Interaction extends BaseInteraction {
    domains = ["example.com"];
    async do(page: Page, args: Args): Promise<void> {
        // implement your logic here
    }
}

You can now use the scripting API to implement your custom script, and then debug and, eventually, deploy that script.

Debugging a script#

After you create an interaction class, you can run your interaction class on a webpage from Smart Browser DevTools IDE to see how it works:

  1. Select Run Smart Browser Interaction (top-right corner).

  2. On the URL dialog, enter a target URL (e.g. https://toscrape.com).

  3. On the Interaction Parameters(in JSON) dialog, enter a JSON object of arguments for your interaction class. You can leave it empty to pass no parameters.

On pressing Enter, the Smart Browser DevTools IDE view splits vertically, and on the right-hand view a tab loads a tool, Smart Browser DevTools, which runs your interaction against the specified URL, showing you the result in real time, and offering you debugging tools and data.

When your interaction finishes, an “execution finished” message pops up on the bottom-right corner.

Once your interaction class is working as expected, you can deploy it and use it as a browser action in your data extraction requests.

Deploying your changes#

You can debug scripts from Zyte API Smart Browser IDE, but to use them as browser actions in your data extraction requests you must first deploy your changes to Zyte API.

To deploy all your changes to Zyte API:

  1. Select Zyte Smart Browser Devtools on the Activity Bar (left-hand side).

  2. On the Zyte Smart Browser DevTools side view that opens, select Deploy.

The deployment process empties the dist/ folder, builds files from the src/ folder into the dist/ folder, and deploys the files in the dist/ folder to Zyte API.

Using a script with Zyte API#

Once you have deployed a script, you can invoke it as a browser action from your data extraction requests:

  1. Set the action field to "interaction"

  2. Set the id field to the file path of your script within the src/ directory, without the .ts file extension, and with path separators (/) replaced by hyphens (-).

For example, if your interaction file path is src/Others/toscrape.com.ts, use:

{
    "action": "interaction",
    "id": "Others-toscrape.com"
}

To pass arguments to your script, set the args field to an object. That object is passed to your script as the args parameter of BaseInteraction.do(). For example:

{
    "action": "interaction",
    "id": "Others-toscrape.com",
    "args": {
        "foo": "bar"
    }
}