Web scraping tutorial¶

In this tutorial you will build a production-ready web-scraping project from scratch:

1. Start a Scrapy project

Install Python and Scrapy, create a Scrapy project, and write your first spider.

2. Deploy and run on Scrapy Cloud

Deploy your project to Scrapy Cloud, run a job, and download the results.

3. Enable Zyte API to avoid bans

Install scrapy-zyte-api, and configure your project to use it in transparent mode.

4. Handle JavaScript content

Reproduce JavaScript code with HTTP requests, or execute it with browser automation.

5. Automate parsing

Use automatic extraction to get structured data without writing parsing code.

If you want to learn more, check out our guides!