Web scraping tutorial

Tip

Looking for a no-code approach? See Web scraping without code.

In this tutorial you will build a production-ready web-scraping project from scratch:

1. Start a Scrapy project

Install Python and Scrapy, create a Scrapy project, and write your first spider.

2. Deploy and run on Scrapy Cloud

Deploy your project to Scrapy Cloud, run a job, and download the results.

3. Enable Zyte API to avoid bans

Install scrapy-zyte-api, and configure your project to use it in transparent mode.

4. Handle JavaScript content

Reproduce JavaScript code with HTTP requests, or execute it with browser automation.

5. Automate parsing and crawling

Use automatic extraction to get structured data without writing parsing or crawling code.

If you want to learn more, check out our guides!