Frequently Asked Questions about Scrapy Cloud

Can I use browser automation?

Of course!

We recommend using Zyte API browser automation, to enjoy automatic ban avoidance and a powerful API. If you already have browser automation code, see Migrating from browser automation to Zyte API.

Alternatively, you can use a third-party service or remote tool. If you are a paying customer, you can use Docker to run a browser automation tool like Playwright, Puppeteer, Selenium or Splash alongside your spider.

How can I configure an unlisted spider setting?

When adding a setting to your spider settings, select the first entry from the drop-down list of settings, Custom Name. You can then enter the name and value of your setting.

Tip

You can also use the Raw Settings tab to edit all of your spider settings as plain text.

Do I have to use Zyte API?

While Zyte API and Scrapy Cloud work great together, they are separate products that you can use independently.

Can I use third-party services in my spider code?

Yes, you can.