Web scraping tutorial

In this tutorial you will build a production-ready web-scraping project from scratch:

1. Start a Scrapy project

Install Python and Scrapy, create a Scrapy project, and write your first spider.

2. Enable Zyte API to avoid bans

Install scrapy-zyte-api, and configure your project to use it in transparent mode.

3. Handle JavaScript content

Reproduce JavaScript code with HTTP requests, or execute it with browser automation.

4. Run your code on Scrapy Cloud

Deploy your project to Scrapy Cloud, run a job, and download the results.