Zyte AI spiders tutorial

In this tutorial you will learn about AI spiders, ready-to-use Scrapy spiders that use Zyte API automatic extraction to automatically crawl and parse data from any website of a supported data type.

1. Run your first AI spider

Use Scrapy Cloud to create a project with Zyte AI-powered spiders in seconds, and run the e-commerce spider.

2. Choose an extraction source

Compare the output quality of HTTP and browser extraction sources to choose the best option for you.

3. Customize with page objects

Extend or override AI parsing with custom page object classes.

4. Customize through subclassing

Implement custom crawling logic by subclassing AI spiders.

5. Use virtual spiders and templates

Create virtual spiders out of spider templates, and create your own templates.

6. Implement post-processing logic

Implement website-independent post-processing logic through item pipelines, such as filling automatic fields or changing the output item schema.

See also

• AI-assisted web scraping tutorial, for an alternative approach to using AI for web scraping.

• Web scraping tutorial, featuring key features of Zyte API.

• Exporting scraped data, to learn how to export extracted data automatically to different data storage services.