Web Scraping Copilot requirements

See below the requirements of Web Scraping Copilot. If you can meet them, you can install Web Scraping Copilot.

Minimum requirements

Visual Studio Code 1.104+ is required.

Most functionality also requires a web scraping project that uses:

• Python 3.10+

• Scrapy 2.6.3+

Tip

Make sure you select the right Python executable in Visual Studio Code, and that a supported version of Scrapy is installed in the corresponding virtual environment or system path.

Code generation requirements

Code generation requires GitHub Copilot. While any GitHub Copilot plan is technically supported, the limited requests of the Free plan can run out quickly, so Pro or better is recommended. Alternatively, you can use a third-party model provider (e.g. OpenAI); for third-party models we recommend models with at least the level of quality of GPT-5 mini or o4-mini, and a context size of at least 128k tokens.

Code generation requires additional Python packages, but as long as you have a supported Scrapy project, the code generation workflow can help you install and set up the remaining requirements.

For reference, those additional Python packages are:

• itemadapter 0.12.2+

• packaging

• pytest 7.0.0+

• scrapy-poet 0.26.0+ (requires setup)

• web-poet 0.19.1+