Warning

Zyte API is replacing Smart Proxy Manager. See Migrating from Smart Proxy Manager to Zyte API.

How to fix timeout errors?

Smart Proxy Manager automatically rotates IPs and retries requests internally when it receives a blocked page. It also adds artificial delays to ensure websites are crawl politely (this is often referred as “throttling”).

These mechanics often results in Smart Proxy Manager requests taking several seconds to be fulfilled, sometimes above a minute.

When you’re crawling a very popular domain in Smart Proxy Manager you may have to increase the timeout in your HTTP client to ensure these delays are accounted for. We recommend increasing to 5 minutes (300 seconds) to be safe.

How to do this depends on the particular technology or language you’re using to connect with Smart Proxy Manager.

If the HTTP client you use to connect to Smart Proxy Manager doesn’t wait long enough you will start receiving 499 errors. For more details on this error and other Smart Proxy Manager errors see Errors Reference.