# Automate parsing

Now that you are familiar with [browser automation](js.md#tutorial-js), it is
time to learn about *parsing automation*.

> [!TIP]
> This page covers AI-powered parsing on every request. See also the
> tutorials of [Coding Agent Add-Ons](../../../ai-code.md#ai-code) for an alternative approach where AI is used to
> generate parsing code instead. See also [Automatic extraction vs coding agents](../../../ai-code.md#zapi-extract-vs-ai-code).

Your [first spider](setup.md#first-spider) parsed 3 fields from book webpages of
[books.toscrape.com](http://books.toscrape.com/): `name`, `price`, `url`.

When targeting other websites, there are 2 challenges you are going to face:

- You will probably want more fields.

  For example, our [automatic extraction product schema](https://docs.zyte.com/zyte-api/usage/reference.html#operation/extract/response/200/product) has more than 25 fields. You need to write parsing
  logic for every combination of target field *and* target website.
  [Coding Agent Add-Ons](../../../ai-code.md#ai-code) can speed up this work significantly, but they cannot make
  it go away.
- Websites change, and when they do they can break your parsing code.

  You need to monitor your web scraping project for breaking website changes,
  and update your parsing code accordingly when they occur.

These issues are time-consuming and scale up with additional fields and
websites. To avoid them altogether, you can let Zyte API handle parsing for
you.

Create a file at
`web_scraping_tutorial/spiders/books_toscrape_com_extract.py` with the
following code:

```
from scrapy import Spider


class BooksToScrapeComExtractSpider(Spider):
    name = "books_toscrape_com_extract"
    custom_settings = {
        "CONCURRENT_REQUESTS_PER_DOMAIN": 8,
        "DOWNLOAD_DELAY": 0.01,
    }
    start_urls = [
        "http://books.toscrape.com/catalogue/category/books/mystery_3/index.html"
    ]

    def parse(self, response):
        next_page_links = response.css(".next a")
        yield from response.follow_all(next_page_links)
        book_links = response.css("article a")
        for request in response.follow_all(book_links, callback=self.parse_book):
            request.meta["zyte_api_automap"] = {"product": True}
            yield request

    def parse_book(self, response):
        yield response.raw_api_response["product"]
```

The code above is a modification of your [first spider](setup.md#first-spider)
that uses [automatic extraction](../../../zyte-api/usage/extract/index.md#zapi-extract), where:

- In requests for book URLs, at the end of the `parse` callback method, you
  include request metadata to have Zyte API give you structured data for an
  e-commerce product.
- The `parse_book` callback method yields the product data from the Zyte
  API response.

Now [run](setup.md#tutorial-run-spider) your new `books_toscrape_com_extract`
spider with `-O books.csv`.

Your code will now extract many more fields from each book, all without you
having to write a single line of parsing code.

> [!NOTE]
> [Zyte API automatic extraction](../../../zyte-api/usage/extract/index.md#zapi-extract) requires you to specify the kind of data you
> want to extract.
> 
> Your spider above uses [product](https://docs.zyte.com/zyte-api/usage/reference.html#operation/extract/request/product) to request the data of a
> single e-commerce product, but [automatic extraction supports many
> other types of data extraction](../../../zyte-api/usage/extract/index.md#zapi-extract-fields).
> 
> For example, if you need to extract a news article or a blog post, use the
> [article](https://docs.zyte.com/zyte-api/usage/reference.html#operation/extract/request/article) data extraction type instead.

This concludes our web scraping tutorial. The tutorial code is available [on
GitHub](https://github.com/zytedata/web-scraping-tutorial-project). To learn
more, check out our [web scraping guides](../../guides/index.md#guides), our documentation for
[Zyte API](../../../zyte-api/get-started.md#zyte-api) and [Scrapy Cloud](../../../scrapy-cloud/get-started.md#scrapy-cloud), and the
[Scrapy documentation](https://docs.scrapy.org/en/latest/index.html#topics-index). You can also visit our [Support
Center](https://support.zyte.com/support/home) or reach out to the wider [web
scraping](https://discord.gg/GjB8dHCCJS) and [Scrapy](https://scrapy.org/community/) communities.