Using the Zyte IDE#

The Zyte IDE, available on the Enterprise plan, lets you:

Write, debug, and deploy browser scripts, written using our TypeScript scripting API, to use as actions in browser requests.
Build Zyte API requests visually, and debug existing requests.

Zyte IDE requirements#

To use the Zyte IDE you need a modern browser.

You also need to enable third-party cookies on the zyte.group domain. If your browser blocks them, you will see an error like the following:

Error loading webview: Error: Could not register service workers: NotSupportedError: Failed to register a ServiceWorker for scope …

Browser script advantages#

Browser scripts are written using the scripting API, a TypeScript API to expose Zyte API actions.

The main advantage of browser scripts over action sequences is support for a non-linear flow: TypeScript allows using conditional statements, loops, and so on. For example, in a browser script you can check if an element is present in a webpage, and run different actions depending on that.

Browser scripts also allow accessing iframe elements through Page.getIframe().

`src/` and `dist/`#

On the Explorer view of the Zyte IDE you get 2 folders:

src/ is the development folder, where you develop your browser scripts.

Changes to this folder are saved on the Zyte IDE cloud storage system.

Multiple developers can work on this folder at the same time, and see the work of one another in real time.
dist/ is the deployment folder.

It contains files built from the last version of src/ that has been deployed to Zyte API.

You should never write anything in the dist/ folder, all its contents are removed during deployment.

Creating a new script#

To create a new browser script:

Open the Zyte IDE.
Select (top-left corner) → File → New File….
On the Create New… dialog, select Smart Browser Interaction.
On the Use Case dialog, select one of the following:
- One of the special action interaction classes, to extend it in your new script.
- Others, to create a script from scratch.
- Utils, to create a file with utility code that you can reuse from browser scripts and other utility code files.
On the Domain dialog, enter the domain of the website for which you are writing the script (e.g. toscrape.com).

A TypeScript file is created in the src/ folder based on your input data, and will open on a tab of the Zyte IDE.

For example, if you select the Others use case and the toscrape.com domain, a src/Others/toscrape.com.ts file is created with the following content:

import { BaseInteraction, Page } from "smartbrowser-core-interactions/index.ts";

interface Args {
    // add your arguments here
    // arg1: string;
}

export default class Interaction extends BaseInteraction {
    domains = ["example.com"];
    async do(page: Page, args: Args): Promise<void> {
        // implement your logic here
    }
}

You can now use the scripting API to implement your browser script, and then debug and, eventually, deploy that browser script. To get started, try out our our example browser scripts!

Debugging a script#

After you create an interaction class, you can run your interaction class on a webpage from the Zyte IDE to see how it works:

Select (top-right corner).
On the URL dialog, enter a target URL (e.g. https://toscrape.com).
On the Interaction Parameters(in JSON) dialog, enter a JSON object of arguments for your interaction class. You can leave it empty to pass no parameters.

On pressing Enter, the Zyte IDE view splits vertically, and on the right-hand view a tab loads a tool, Smart Browser DevTools, which runs your interaction against the specified URL, showing you the result in real time, and offering you debugging tools and data.

When your interaction finishes, an “execution finished” message pops up on the bottom-right corner.

Once your interaction class is working as expected, you can deploy it and use it as a browser action in your data extraction requests.

Deploying your changes#

You can debug scripts from the Zyte IDE, but to use them as browser actions in your data extraction requests you must first deploy your changes to Zyte API.

To deploy all your changes to Zyte API:

Select on the Activity Bar (left-hand side).
On the Zyte Smart Browser DevTools side view that opens, select Deploy.

The deployment process empties the dist/ folder, builds files from the src/ folder into the dist/ folder, and deploys the files in the dist/ folder to Zyte API.

Using a script with Zyte API#

Once you have deployed a script, you can get the interaction ID as follows:

Select on the Activity Bar (left-hand side).
On the Zyte Smart Browser DevTools side view that opens, select Interactions, right-click on your interaction, and click Copy Interaction ID.

The interaction ID will be copied to your clipboard.

You can then invoke your script as a browser action from your data extraction requests:

Set the action field to "interaction".
Set the id field to the interaction ID from your clipboard.

For example, if your interaction ID is Others-toscrape.com, use:

{
    "action": "interaction",
    "id": "Others-toscrape.com"
}

To pass arguments to your script, set the args field to an object. That object is passed to your script as the args parameter of BaseInteraction.do(). For example:

{
    "action": "interaction",
    "id": "Others-toscrape.com",
    "args": {
        "foo": "bar"
    }
}