Web Scraping Copilot user interface

Sidebar

The Web Scraping Copilot sidebar provides access to the main features of the extension, organized in the following views: Items, Page Objects, Spiders, Zyte API, Extension Status, and Feedback.

The Items view provides details and access to all your Scrapy items.

Specifically, it shows items:

• Defined in your items.py file.

• Declared as output items by your page objects.

• Provided by the installed version of zyte-common-items.

The Page Objects view provides details and access to all your web-poet page-objects.

It also makes it easy to use AI to add new page objects or to update some or all fields of existing page objects. See AI-assisted web scraping tutorial.

It also helps run, view, manage and debug tests for your page objects.

The Spiders view provides access to all your Scrapy spiders, and helps you to run them locally.

It also makes it easy to deploy them to Scrapy Cloud and monitor their jobs.

The Zyte API view helps you set up scrapy-zyte-api to use Zyte API.

The Extension Status view helps you set up the requirements of Web Scraping Copilot, and check if they are met.

The Feedback view provides a link to our issue tracker.

It may sometimes also include links to surveys or other feedback channels.