# Zyte API proxy mode

To use Zyte API as a proxy, use the `api.zyte.com:8011` endpoint, with your
[Zyte API key](https://app.zyte.com/o/zyte-api/api-access) and [proxy
headers](#zapi-proxy-headers):

### curl

```bash
curl \
    --proxy api.zyte.com:8011 \
    --proxy-user YOUR_ZYTE_API_KEY: \
    --compressed \
    https://toscrape.com
```

### Scrapy

When using [scrapy-zyte-smartproxy](https://github.com/scrapy-plugins/scrapy-zyte-smartproxy), set the `ZYTE_SMARTPROXY_URL`
setting to `"http://api.zyte.com:8011"` and the
`ZYTE_SMARTPROXY_APIKEY` setting to [your Zyte API key](https://app.zyte.com/o/zyte-api/api-access) for Zyte API.

> [!NOTE]
> **Important**: Use your **Zyte API key** here, not a Scrapy Cloud API key. Make sure you get this from the Zyte API access page.

Then you can continue using Scrapy as usual and all requests will be
proxied through Zyte API automatically.

```python
from scrapy import Spider


class ToScrapeSpider(Spider):
    name = "toscrape_com"
    start_urls = ["https://toscrape.com"]

    def parse(self, response):
        print(response.text)
```

## Key differences

The proxy mode makes it easier to [migrate existing code](../migration/index.md#zapi-migrate)
that uses a proxy service. However, the proxy mode and the [HTTP API](reference.md#zapi-reference) have some key differences:

| Feature                 | HTTP API                                    | Proxy mode                                     |
|-------------------------|---------------------------------------------|------------------------------------------------|
| Parameter definition    | [Request body](reference.md#zapi-reference) | [Request headers](#zapi-proxy-headers)         |
| Browser HTML            | [Yes](browser.md#zapi-browser-html)         | [Yes](#zyte-browser-html) (new!)               |
| Screenshots             | [Yes](browser.md#zapi-screenshot)           | No                                             |
| Browser actions         | [Yes](browser.md#zapi-actions)              | No                                             |
| Network capture         | [Yes](browser.md#zapi-network-capture)      | No                                             |
| Disable JS on browser   | [Yes](browser.md#zapi-javascript)           | No                                             |
| Automatic extraction    | [Yes](extract/index.md#zapi-extract)        | No                                             |
| Server-managed sessions | [Yes](features.md#zapi-session-contexts)    | No                                             |
| Echo data               | [Yes](features.md#zapi-metadata)            | No                                             |
| Overhead                | [Some](#zapi-overhead)                      | [Minimum](#zapi-overhead)                      |
| Cookie definition       | [Multi-domain](#zapi-proxy-diff-cookies)    | [Target domain only](#zapi-proxy-diff-cookies) |

### Overhead

When using [HTTP requests](http.md#zapi-http), the HTTP API introduces some
overhead in responses due mainly to the base64-encoding of
[httpResponseBody](https://docs.zyte.com/zyte-api/usage/reference.html#operation/extract/response/200/httpResponseBody), increasing network traffic and latency, and
requiring base64-decoding on your end.

In contrast, with proxy mode the only overhead you get is some additional
[response headers](#zapi-proxy-response-headers).

> [!NOTE]
> ### See also
> 
> [Optimizing Zyte API usage](optimize.md#zapi-optimize)

### Cookie definition

With proxy mode, you can only set [cookies](features.md#zapi-cookies) for the domain
of the target URL, you cannot manually set cookies for additional domains that
may be reached through [redirection](http.md#zapi-redirection).

## Request headers

The following headers allow changing how a request is sent through Zyte API in
proxy mode.

### Zyte-Browser-Html

Sets [browserHtml](browser.md#zapi-browser-html).

This is not compatible with [Zyte-Disable-Follow-Redirect](#zyte-disable-follow-redirect).

### Example

> [!NOTE]
> Install and configure [code example requirements](https://docs.pytest.org/en/stable/example/index.html#examples) and
> the [Zyte CA certificate](../../misc/ca.md#ca) to run the example below.

### curl

```shell
curl \
    --proxy api.zyte.com:8011 \
    --proxy-user YOUR_ZYTE_API_KEY: \
    --compressed \
    -H "Zyte-Browser-Html: true" \
    https://toscrape.com
```

### Scrapy

```python
from scrapy import Request, Spider


class ToScrapeSpider(Spider):
    name = "toscrape_com"

    async def start(self):
        yield Request("https://toscrape.com", headers={"Zyte-Browser-Html": "true"})

    def parse(self, response):
        print(response.text)
```

Output (first 5 lines):

```html
<!DOCTYPE html><html lang="en"><head>
        <meta http-equiv="Content-Type" content="text/html; charset=UTF-8">
        <title>Scraping Sandbox</title>
        <link href="./css/bootstrap.min.css" rel="stylesheet">
        <link href="./css/main.css" rel="stylesheet">
```

### Zyte-Client

May be used to report to Zyte the software being used to access Zyte API.

It should be formatted with the syntax of the [User-Agent](https://developer.mozilla.org/en-US/docs/Web/HTTP/Headers/User-Agent) header, e.g.
`curl/1.2.3`.

### Zyte-Cookie-Management

Sets [cookieManagement](features.md#zapi-cookie-management).

### Zyte-Device

Sets [device emulation](http.md#zapi-device).

### Zyte-Disable-Follow-Redirect

When set to `true`, disables [redirect following](http.md#zapi-http-redirection), which is enabled by default.

### Zyte-Geolocation

Sets a [geolocation](features.md#zapi-geolocation).

### Zyte-IPType

Sets [ipType](features.md#zapi-ip-type).

### Zyte-JobId

Sets the ID of the [Scrapy Cloud](../../scrapy-cloud/get-started.md#scrapy-cloud) job that is sending the
request.

[scrapy-zyte-smartproxy](https://github.com/scrapy-plugins/scrapy-zyte-smartproxy) sets this header automatically when used from a Scrapy
Cloud job.

### Zyte-Override-Headers

Zyte API automatically sends some request headers for [ban avoidance](errors.md#zapi-bans).

Custom headers from your request will override most automatic headers, but not
these:

`Accept`
`Accept-Encoding`
`User-Agent`

To override any of these 3 headers, set `Zyte-Override-Headers` to a
comma-separated list of names of headers to override, e.g.
`Zyte-Override-Headers: Accept,Accept-Encoding`.

> [!WARNING]
> Overriding headers can break Zyte API ban avoidance.

### Zyte-Session-ID

Sets [session.id](https://docs.zyte.com/zyte-api/usage/reference.html#operation/extract/request/session.id) for a [client-managed session](features.md#zapi-session-id).

### Zyte-Tags

Sets the [tags](https://docs.zyte.com/zyte-api/usage/reference.html#operation/extract/request/tags) dictionary in the request.

The header is a JSON object, such as `Zyte-Tags: {"foo": "bar", "baz": null, "435":"true"}`.
Value MUST be valid ASCII under 512 bytes long.

### Invalid request headers

The following headers are not allowed, and any request with one or more of them
will result in an HTTP 400 response:

`Client-IP`
`Cluster-Client-IP`
`Forwarded-For`
`True-Client-IP`
`Via`
`X-Client-IP`
`X-Forwarded`
`X-Forwarded-For`
`X-Forwarded-Host`
`X-Host`
`X-Original-URL`
`X-Originating-IP`
`X-ProxyUser-IO`
`X-ProxyUser-IP`
`X-Remote-Addr`
`X-Remote-IP`

## Response headers

Responses include some headers injected by Zyte API.

Note that the response body of [unsuccessful responses](errors.md#zapi-unsuccessful-responses) is always the actual JSON response from the
[HTTP API](reference.md#zapi-reference) that provides error details.

### Zyte-Error-Title

A short summary of the problem type. Written in English and readable for
engineers, usually not suited for non-technical stakeholders, and not
localized.

It matches the `title` JSON field of the error response.

### Zyte-Error-Type

A URI reference that uniquely identifies the problem type, only in the context
of the provided API.

Opposed to the specification in RFC-7807, it is neither recommended to be
dereferencable and point to human-readable documentation nor globally unique
for the problem type.

It matches the `type` JSON field of the error response.

### Zyte-Request-ID

A unique identifier of the request.

When reporting an issue about the outcome of a request to us, please include
the value of this response header when possible.

## HTTPS proxy

> [!TIP]
> The [main endpoint](#zapi-proxy-http) works both for HTTP and
> HTTPS URLs, you do not need an HTTPS proxy interface to access HTTPS URLs.

You can use the `api.zyte.com:8014` endpoint for an HTTPS proxy interface,
provided your tech stack supports HTTPS proxies and you have [installed
our CA certificate](../../misc/ca.md#ca):

### curl

```bash
curl \
    --proxy https://api.zyte.com:8014 \
    --proxy-user YOUR_ZYTE_API_KEY: \
    --compressed \
    https://toscrape.com
```

## Use with browser automation tools

The proxy mode is not optimized for use in combination with browser automation
tools. Please, consider using [Zyte API’s browser automation features](browser.md#zapi-browser) instead. See [Migrating from browser automation to Zyte API](../migration/browser-automation/index.md#zapi-browser-automation).