AI-assisted web scraping tutorial

With GitHub Copilot and Web Scraping Copilot you can write maintainable web scraping code using AI:

Warning

The GitHub Copilot Free plan is not recommended for AI-assisted web scraping. See Code generation requirements for details.

1. Set up a project

Prepare Visual Studio Code and a Scrapy project with all prerequisites.

2. Generate parsing code

Use AI to generate parsing code for a webpage.

3. Generate crawling code

Use AI to generate crawling code for a website.