# Zyte API HTTP requests
To send HTTP requests through Zyte API, without [browser rendering](browser.md#zapi-browser), set the [httpResponseBody](https://docs.zyte.com/zyte-api/usage/reference.html#operation/extract/request/httpResponseBody) request field to
`true`, and read the [Base64](https://en.wikipedia.org/wiki/Base64)-encoded response body from the
[httpResponseBody](https://docs.zyte.com/zyte-api/usage/reference.html#operation/extract/response/200/httpResponseBody) response field.
### Example
> [!NOTE]
> Install and configure [code example requirements](https://docs.pytest.org/en/stable/example/index.html#examples) and
> the [Zyte CA certificate](../../misc/ca.md#ca) to run the example below.
### CLI client
input.jsonl
```json
{"url": "https://toscrape.com", "httpResponseBody": true}
```
```shell
zyte-api input.jsonl \
| jq --raw-output .httpResponseBody \
| base64 --decode \
> output.html
```
### curl
input.json
```json
{
"url": "https://toscrape.com",
"httpResponseBody": true
}
```
```shell
curl \
--user YOUR_ZYTE_API_KEY: \
--header 'Content-Type: application/json' \
--data @input.json \
--compressed \
https://api.zyte.com/v1/extract \
| jq --raw-output .httpResponseBody \
| base64 --decode \
> output.html
```
### Proxy mode
With the [proxy mode](proxy-mode.md#zapi-proxy), you always get a response
body.
```shell
curl \
--proxy api.zyte.com:8011 \
--proxy-user YOUR_ZYTE_API_KEY: \
--compressed \
https://toscrape.com \
> output.html
```
### Python client
```python
import asyncio
from base64 import b64decode
from zyte_api import AsyncZyteAPI
async def main():
client = AsyncZyteAPI()
api_response = await client.get(
{
"url": "https://toscrape.com",
"httpResponseBody": True,
}
)
http_response_body = b64decode(api_response["httpResponseBody"]).decode()
print(http_response_body)
asyncio.run(main())
```
### Scrapy
In [transparent mode](https://scrapy-zyte-api.readthedocs.io/en/latest/usage/transparent.html#transparent), when you target a text
resource (e.g. HTML, JSON), regular Scrapy requests work out of the
box:
```python
from scrapy import Spider
class ToScrapeSpider(Spider):
name = "toscrape_com"
start_urls = ["https://toscrape.com"]
def parse(self, response):
http_response_text: str = response.text
```
While regular Scrapy requests also work for binary responses at the
moment, they may stop working in future versions of
[scrapy-zyte-api](https://scrapy-zyte-api.readthedocs.io/en/latest/index.html), so passing
[httpResponseBody](https://docs.zyte.com/zyte-api/usage/reference.html#operation/extract/request/httpResponseBody) is recommended when targeting binary
resources:
```python
from scrapy import Request, Spider
class ToScrapeSpider(Spider):
name = "toscrape_com"
async def start(self):
yield Request(
"https://toscrape.com",
meta={
"zyte_api_automap": {
"httpResponseBody": True,
},
},
)
def parse(self, response):
http_response_body: bytes = response.body
```
Output (first 5 lines):
```html
Scraping Sandbox
```
For HTTP requests, Zyte API also supports:
- HTTP request attributes for [method](#zapi-set-method),
[body](#zapi-set-body), and [headers](#zapi-body-request-headers).
- [Redirection](#zapi-http-redirection),
[device emulation](#zapi-device).
- [Geolocation](features.md#zapi-geolocation),
[IP type](features.md#zapi-ip-type),
[cookies](features.md#zapi-cookies),
[sessions](features.md#zapi-sessions),
[response headers](features.md#zapi-headers),
and [metadata](features.md#zapi-metadata).
> [!TIP]
> HTTP responses do not reflect HTML content rendered by a web browser
> that executes JavaScript code. To get [browser HTML](browser.md#zapi-browser-html), use a [browser request](browser.md#zapi-browser).
> See also [HTML and browser HTML](#zapi-raw-vs-browser).
## Request method
HTTP requests use the `GET` [HTTP method](https://developer.mozilla.org/en-US/docs/Web/HTTP/Methods) by default. Use the
[httpRequestMethod](https://docs.zyte.com/zyte-api/usage/reference.html#operation/extract/request/httpRequestMethod) field to set a different HTTP method.
> [!TIP]
> When using `POST`, `PUT` or similar, you probably want to also
> [set a request body](#zapi-set-body).
### Example
> [!NOTE]
> Install and configure [code example requirements](https://docs.pytest.org/en/stable/example/index.html#examples) and
> the [Zyte CA certificate](../../misc/ca.md#ca) to run the example below.
### CLI client
input.jsonl
```json
{"url": "https://httpbin.org/anything", "httpResponseBody": true, "httpRequestMethod": "POST"}
```
```shell
zyte-api input.jsonl \
| jq --raw-output .httpResponseBody \
| base64 --decode \
| jq .method
```
### curl
input.json
```json
{
"url": "https://httpbin.org/anything",
"httpResponseBody": true,
"httpRequestMethod": "POST"
}
```
```shell
curl \
--user YOUR_ZYTE_API_KEY: \
--header 'Content-Type: application/json' \
--data @input.json \
--compressed \
https://api.zyte.com/v1/extract \
| jq --raw-output .httpResponseBody \
| base64 --decode \
| jq .method
```
### Proxy mode
With the [proxy mode](proxy-mode.md#zapi-proxy), the request method
from your requests is used automatically.
```shell
curl \
--proxy api.zyte.com:8011 \
--proxy-user YOUR_ZYTE_API_KEY: \
--compressed \
-X POST \
https://httpbin.org/anything \
| jq .method
```
### Python client
```python
import asyncio
import json
from base64 import b64decode
from zyte_api import AsyncZyteAPI
async def main():
client = AsyncZyteAPI()
api_response = await client.get(
{
"url": "https://httpbin.org/anything",
"httpResponseBody": True,
"httpRequestMethod": "POST",
}
)
http_response_body: bytes = b64decode(api_response["httpResponseBody"])
method = json.loads(http_response_body)["method"]
print(method)
asyncio.run(main())
```
### Scrapy
```python
import json
from scrapy import Request, Spider
class HTTPBinOrgSpider(Spider):
name = "httpbin_org"
async def start(self):
yield Request(
"https://httpbin.org/anything",
method="POST",
)
def parse(self, response):
method = json.loads(response.text)["method"]
```
Output:
```json
"POST"
```
## Request body
To include a body in your request, use one of the following fields:
- [httpRequestText](https://docs.zyte.com/zyte-api/usage/reference.html#operation/extract/request/httpRequestText), for UTF-8-encoded text.
- [httpRequestBody](https://docs.zyte.com/zyte-api/usage/reference.html#operation/extract/request/httpRequestBody), for anything else. It supports binary data
as well, so the value must be [Base64](https://en.wikipedia.org/wiki/Base64)-encoded.
### `httpRequestText` example
> [!NOTE]
> Install and configure [code example requirements](https://docs.pytest.org/en/stable/example/index.html#examples) and
> the [Zyte CA certificate](../../misc/ca.md#ca) to run the example below.
### CLI client
input.jsonl
```json
{"url": "https://httpbin.org/anything", "httpResponseBody": true, "httpRequestMethod": "POST", "httpRequestText": "{\"foo\": \"bar\"}"}
```
```shell
zyte-api input.jsonl \
| jq --raw-output .httpResponseBody \
| base64 --decode \
| jq --raw-output .data
```
### curl
input.json
```json
{
"url": "https://httpbin.org/anything",
"httpResponseBody": true,
"httpRequestMethod": "POST",
"httpRequestText": "{\"foo\": \"bar\"}"
}
```
```shell
curl \
--user YOUR_ZYTE_API_KEY: \
--header 'Content-Type: application/json' \
--data @input.json \
--compressed \
https://api.zyte.com/v1/extract \
| jq --raw-output .httpResponseBody \
| base64 --decode \
| jq --raw-output .data
```
### Proxy mode
With the [proxy mode](proxy-mode.md#zapi-proxy), the request body from
your requests is used automatically, be it plain text or binary.
```shell
curl \
--proxy api.zyte.com:8011 \
--proxy-user YOUR_ZYTE_API_KEY: \
--compressed \
-X POST \
-H "Content-Type: application/json" \
--data '{"foo": "bar"}' \
https://httpbin.org/anything \
| jq .data
```
### Python client
```python
import asyncio
import json
from base64 import b64decode
from zyte_api import AsyncZyteAPI
async def main():
client = AsyncZyteAPI()
api_response = await client.get(
{
"url": "https://httpbin.org/anything",
"httpResponseBody": True,
"httpRequestMethod": "POST",
"httpRequestText": '{"foo": "bar"}',
}
)
http_response_body = b64decode(api_response["httpResponseBody"])
body = json.loads(http_response_body)["data"]
print(body)
asyncio.run(main())
```
### Scrapy
```python
import json
from scrapy import Request, Spider
class HTTPBinOrgSpider(Spider):
name = "httpbin_org"
async def start(self):
yield Request(
"https://httpbin.org/anything",
method="POST",
body='{"foo": "bar"}',
)
def parse(self, response):
body = json.loads(response.body)["data"]
print(body)
```
Output:
```json
{"foo": "bar"}
```
### `httpRequestBody` example
> [!NOTE]
> Install and configure [code example requirements](https://docs.pytest.org/en/stable/example/index.html#examples) and
> the [Zyte CA certificate](../../misc/ca.md#ca) to run the example below.
### CLI client
input.jsonl
```json
{"url": "https://httpbin.org/anything", "httpResponseBody": true, "httpRequestMethod": "POST", "httpRequestBody": "Zm9v"}
```
```shell
zyte-api input.jsonl \
| jq --raw-output .httpResponseBody \
| base64 --decode \
| jq --raw-output .data
```
### curl
input.json
```json
{
"url": "https://httpbin.org/anything",
"httpResponseBody": true,
"httpRequestMethod": "POST",
"httpRequestBody": "Zm9v"
}
```
```shell
curl \
--user YOUR_ZYTE_API_KEY: \
--header 'Content-Type: application/json' \
--data @input.json \
--compressed \
https://api.zyte.com/v1/extract \
| jq --raw-output .httpResponseBody \
| base64 --decode \
| jq --raw-output .data
```
### Proxy mode
With the [proxy mode](proxy-mode.md#zapi-proxy), the request body from
your requests is used automatically, be it plain text or binary.
```shell
curl \
--proxy api.zyte.com:8011 \
--proxy-user YOUR_ZYTE_API_KEY: \
--compressed \
-X POST \
-H "Content-Type: application/octet-stream" \
--data foo \
https://httpbin.org/anything \
| jq .data
```
### Python client
```python
import asyncio
import json
from base64 import b64decode
from zyte_api import AsyncZyteAPI
async def main():
client = AsyncZyteAPI()
api_response = await client.get(
{
"url": "https://httpbin.org/anything",
"httpResponseBody": True,
"httpRequestMethod": "POST",
"httpRequestBody": "Zm9v",
}
)
http_response_body: bytes = b64decode(api_response["httpResponseBody"])
body = json.loads(http_response_body)["data"]
print(body)
asyncio.run(main())
```
### Scrapy
```python
import json
from scrapy import Request, Spider
class HTTPBinOrgSpider(Spider):
name = "httpbin_org"
async def start(self):
yield Request(
"https://httpbin.org/anything",
method="POST",
body=b"foo",
)
def parse(self, response):
body = json.loads(response.body)["data"]
print(body)
```
Output:
```none
foo
```
## Request headers
In HTTP requests, use [customHttpRequestHeaders](https://docs.zyte.com/zyte-api/usage/reference.html#operation/extract/request/customHttpRequestHeaders) to set request
headers. You can set any header except `Cookie` (see
[Cookies](features.md#zapi-cookies)).
> [!TIP]
> You can also set headers like `Accept`, `Accept-Encoding`,
> `Accept-Language` or `User-Agent`, but it is usually best to let Zyte
> API set those headers; it will use values consistent with the network stack
> and other request parameters (e.g. [device](#zapi-device),
> [geolocation](features.md#zapi-geolocation)).
### Example
> [!NOTE]
> Install and configure [code example requirements](https://docs.pytest.org/en/stable/example/index.html#examples) and
> the [Zyte CA certificate](../../misc/ca.md#ca) to run the example below.
### CLI client
input.jsonl
```json
{"url": "https://httpbin.org/anything", "httpResponseBody": true, "customHttpRequestHeaders": [{"name": "Accept-Language", "value": "fa"}]}
```
```shell
zyte-api input.jsonl \
| jq --raw-output .httpResponseBody \
| base64 --decode \
| jq .headers
```
### curl
input.json
```json
{
"url": "https://httpbin.org/anything",
"httpResponseBody": true,
"customHttpRequestHeaders": [
{
"name": "Accept-Language",
"value": "fa"
}
]
}
```
```shell
curl \
--user YOUR_ZYTE_API_KEY: \
--header 'Content-Type: application/json' \
--data @input.json \
--compressed \
https://api.zyte.com/v1/extract \
| jq --raw-output .httpResponseBody \
| base64 --decode \
| jq .headers
```
### Proxy mode
With the [proxy mode](proxy-mode.md#zapi-proxy), the request headers
from your requests are used automatically.
```shell
curl \
--proxy api.zyte.com:8011 \
--proxy-user YOUR_ZYTE_API_KEY: \
--compressed \
-H "Accept-Language: fa" \
https://httpbin.org/anything \
| jq .headers
```
### Python client
```python
import asyncio
import json
from base64 import b64decode
from zyte_api import AsyncZyteAPI
async def main():
client = AsyncZyteAPI()
api_response = await client.get(
{
"url": "https://httpbin.org/anything",
"httpResponseBody": True,
"customHttpRequestHeaders": [
{
"name": "Accept-Language",
"value": "fa",
},
],
}
)
http_response_body: bytes = b64decode(api_response["httpResponseBody"])
headers = json.loads(http_response_body)["headers"]
print(json.dumps(headers, indent=2))
asyncio.run(main())
```
### Scrapy
```python
import json
from scrapy import Request, Spider
class HTTPBinOrgSpider(Spider):
name = "httpbin_org"
async def start(self):
yield Request(
"https://httpbin.org/anything",
headers={"Accept-Language": "fa"},
)
def parse(self, response):
headers = json.loads(response.text)["headers"]
```
Output (first 5 lines):
```json
{
"Accept": "text/html,application/xhtml+xml,application/xml;q=0.9,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.7",
"Accept-Encoding": "gzip, deflate, br",
"Accept-Language": "fa",
"Host": "httpbin.org",
```
## Redirection
HTTP requests follow [HTTP redirection](https://developer.mozilla.org/en-US/docs/Web/HTTP/Redirections) by default. Set
[followRedirect](https://docs.zyte.com/zyte-api/usage/reference.html#operation/extract/request/followRedirect) to `False` to change that.
> [!NOTE]
> Redirection [works differently in browser requests](browser.md#zapi-browser-redirection).
## Device emulation
In HTTP requests, use [device](https://docs.zyte.com/zyte-api/usage/reference.html#operation/extract/request/device) to set a type of device emulation,
either `desktop` (default) or `mobile`, to use for your request.
This option exists because some websites return different content depending on
the type of device used to access them.
> [!NOTE]
> In a request where you set [device](https://docs.zyte.com/zyte-api/usage/reference.html#operation/extract/request/device) to `mobile`, you
> cannot use [sessionContextParameters.actions](https://docs.zyte.com/zyte-api/usage/reference.html#operation/extract/request/sessionContextParameters.actions).
### Example
> [!NOTE]
> Install and configure [code example requirements](https://docs.pytest.org/en/stable/example/index.html#examples) and
> the [Zyte CA certificate](../../misc/ca.md#ca) to run the example below.
### CLI client
input.jsonl
```json
{"url": "https://httpbin.org/user-agent", "httpResponseBody": true, "device": "mobile"}
```
```shell
zyte-api input.jsonl \
| jq --raw-output .httpResponseBody \
| base64 --decode \
| jq --raw-output '.["user-agent"]'
```
### curl
input.json
```json
{
"url": "https://httpbin.org/user-agent",
"httpResponseBody": true,
"device": "mobile"
}
```
```shell
curl \
--user YOUR_ZYTE_API_KEY: \
--header 'Content-Type: application/json' \
--data @input.json \
--compressed \
https://api.zyte.com/v1/extract \
| jq --raw-output .httpResponseBody \
| base64 --decode \
| jq --raw-output '.["user-agent"]'
```
### Proxy mode
With the [proxy mode](proxy-mode.md#zapi-proxy), use the
[Zyte-Device](proxy-mode.md#zyte-device) header.
```shell
curl \
--proxy api.zyte.com:8011 \
--proxy-user YOUR_ZYTE_API_KEY: \
--compressed \
-H "Zyte-Device: mobile" \
https://httpbin.org/user-agent \
| jq --raw-output '.["user-agent"]'
```
### Python client
```python
import asyncio
import json
from base64 import b64decode
from zyte_api import AsyncZyteAPI
async def main():
client = AsyncZyteAPI()
api_response = await client.get(
{
"url": "https://httpbin.org/user-agent",
"httpResponseBody": True,
"device": "mobile",
}
)
http_response_body: bytes = b64decode(api_response["httpResponseBody"])
user_agent = json.loads(http_response_body)["user-agent"]
print(user_agent)
asyncio.run(main())
```
### Scrapy
```python
import json
from scrapy import Request, Spider
class HTTPBinOrgSpider(Spider):
name = "httpbin_org"
async def start(self):
yield Request(
"https://httpbin.org/user-agent",
meta={
"zyte_api_automap": {
"device": "mobile",
}
},
)
def parse(self, response):
user_agent = json.loads(response.text)["user-agent"]
print(user_agent)
```
Example output (may vary):
```none
Mozilla/5.0 (Linux; Android 10; K) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/117.0.0.0 Mobile Safari/537.36
```
## Submitting HTML forms
While it may be easier to submit HTML forms using a [browser request](browser.md#zapi-browser) with [actions](browser.md#zapi-actions), it is also possible to
reproduce form-submission requests with HTTP requests.
Reproducing an HTML form request usually requires:
- Setting the right value of [httpRequestMethod](https://docs.zyte.com/zyte-api/usage/reference.html#operation/extract/request/httpRequestMethod), often
`POST`.
- Setting the `Content-Type` header to
`application/x-www-form-urlencoded` through
[customHttpRequestHeaders](https://docs.zyte.com/zyte-api/usage/reference.html#operation/extract/request/customHttpRequestHeaders).
- Setting the right payload, i.e. key-value pairs set by the form.
For `GET` requests, that means setting those key-value pairs in the URL
query string.
For `POST` requests, that means encoding those key-value pairs as a query
string (without the starting `?`) and using that as
[httpRequestText](https://docs.zyte.com/zyte-api/usage/reference.html#operation/extract/request/httpRequestText) or [httpRequestBody](https://docs.zyte.com/zyte-api/usage/reference.html#operation/extract/request/httpRequestBody).
> [!TIP]
> Your key-value pairs may need to include hidden form fields, often
> used for [CSRF tokens](https://en.wikipedia.org/wiki/Cross-site_request_forgery) or to keep
> the state of stateful pages (e.g. ASP.NET’s `__VIEWSTATE` field).
### Example
> [!NOTE]
> Install and configure [code example requirements](https://docs.pytest.org/en/stable/example/index.html#examples) and
> the [Zyte CA certificate](../../misc/ca.md#ca) to run the example below.
In [https://quotes.toscrape.com/search.aspx](https://quotes.toscrape.com/search.aspx) you get an HTML form that could be
stripped down to:
```html
```
When you select an **Author** (e.g. Albert Einstein), a form request is sent,
and the **Tag** options fill up.
To reproduce that:
### Python
Install [form2request](https://form2request.readthedocs.io/en/latest/index.html), which makes it easier
to handle HTML forms in Python.
Then:
```python
from base64 import b64decode
from form2request import form2request
from parsel import Selector
import requests
api_response_1 = requests.post(
"https://api.zyte.com/v1/extract",
auth=("YOUR_ZYTE_API_KEY", ""),
json={
"url": "https://quotes.toscrape.com/search.aspx",
"httpResponseBody": True,
},
)
api_response_1_data = api_response_1.json()
http_response_body_1 = b64decode(api_response_1_data["httpResponseBody"])
selector_1 = Selector(body=http_response_body_1, base_url=api_response_1_data["url"])
form = selector_1.css("form")
request = form2request(form, {"author": "Albert Einstein"}, click=False)
api_response_2 = requests.post(
"https://api.zyte.com/v1/extract",
auth=("YOUR_ZYTE_API_KEY", ""),
json={
"url": request.url,
"httpRequestMethod": request.method,
"customHttpRequestHeaders": [
{"name": k, "value": v} for k, v in request.headers
],
"httpRequestText": request.body.decode(),
"httpResponseBody": True,
},
)
http_response_body_2 = b64decode(api_response_2.json()["httpResponseBody"])
selector_2 = Selector(body=http_response_body_2)
print(len(selector_2.css("select[name='tag'] option")))
```
### Python client
Install [form2request](https://form2request.readthedocs.io/en/latest/index.html), which makes it easier
to handle HTML forms in Python.
Then:
```python
import asyncio
from base64 import b64decode
from form2request import form2request
from parsel import Selector
from zyte_api import AsyncZyteAPI
async def main():
client = AsyncZyteAPI()
api_response_1 = await client.get(
{
"url": "https://quotes.toscrape.com/search.aspx",
"httpResponseBody": True,
}
)
http_response_body_1 = b64decode(api_response_1["httpResponseBody"])
selector_1 = Selector(body=http_response_body_1, base_url=api_response_1["url"])
form = selector_1.css("form")
request = form2request(form, {"author": "Albert Einstein"}, click=False)
api_response_2 = await client.get(
{
"url": request.url,
"httpRequestMethod": request.method,
"customHttpRequestHeaders": [
{"name": k, "value": v} for k, v in request.headers
],
"httpRequestText": request.body.decode(),
"httpResponseBody": True,
}
)
http_response_body_2 = b64decode(api_response_2["httpResponseBody"])
selector_2 = Selector(body=http_response_body_2)
print(len(selector_2.css("select[name='tag'] option")))
asyncio.run(main())
```
### Scrapy
Install [form2request](https://form2request.readthedocs.io/en/latest/index.html), which makes it easier
to handle HTML forms in Scrapy.
Then, use it and let [transparent mode](https://scrapy-zyte-api.readthedocs.io/en/latest/usage/transparent.html#transparent) take care of
the rest:
```python
from form2request import form2request
from scrapy import Spider
class QuotesToScrapeComSpider(Spider):
name = "quotes_toscrape_com"
start_urls = ["https://quotes.toscrape.com/search.aspx"]
def parse(self, response):
form = response.css("form")
request = form2request(form, {"author": "Albert Einstein"}, click=False)
yield request.to_scrapy(callback=self.parse_tags)
def parse_tags(self, response):
print(len(response.css("select[name='tag'] option")))
```
Output (number of **Tag** options):
```json
25
```
## Decoding HTML
HTML extracted as a [response body](#zapi-body) needs to be decoded.
HTML content can be encoded with one of many character encodings, and
you must determine the character encoding used so that you can decode that
HTML content accordingly.
The best way to determine the encoding of HTML content is to follow the
[encoding sniffing algorithm](https://html.spec.whatwg.org/#determining-the-character-encoding) defined in the HTML standard.
In addition to the HTML content, the HTML encoding sniffing algorithm takes
into account any character encoding provided in the optional `charset`
parameter of media types declared in the `Content-Type` response header, so
make sure you [get the response headers](features.md#zapi-response-headers) in
addition to the response body if you are following the HTML encoding sniffing
algorithm.
### Example
> [!NOTE]
> Install and configure [code example requirements](https://docs.pytest.org/en/stable/example/index.html#examples) and
> the [Zyte CA certificate](../../misc/ca.md#ca) to run the example below.
### curl
Use [file](https://www.darwinsys.com/file/) to find the media type of a [previously-downloaded
response](#zapi-get-body) based solely on its body (i.e. not
following the HTML encoding sniffing algorithm).
```shell
file --mime-encoding output.html
```
### Scrapy
In [transparent mode](https://scrapy-zyte-api.readthedocs.io/en/latest/usage/transparent.html#transparent), regular Scrapy requests
targeting HTML resources decode them by default. See
[Zyte API HTTP requests](#zapi-text).
## HTML and browser HTML
HTML found in [httpResponseBody](https://docs.zyte.com/zyte-api/usage/reference.html#operation/extract/response/200/httpResponseBody) is usually different from HTML
found in [browserHtml](https://docs.zyte.com/zyte-api/usage/reference.html#operation/extract/response/200/browserHtml) ([browser HTML](browser.md#zapi-browser-html)):
- [httpResponseBody](https://docs.zyte.com/zyte-api/usage/reference.html#operation/extract/response/200/httpResponseBody) does not reflect changes that a webpage
makes at run time using JavaScript, such as loading content from additional
URLs, or moving or reformatting content within the webpage.
- [browserHtml](https://docs.zyte.com/zyte-api/usage/reference.html#operation/extract/response/200/browserHtml) includes a normalization of the HTML from the
underlying HTTP response, which web browsers perform according to the HTML5
specification. So the content of HTML and browser HTML could be different
even when there is no JavaScript involved.
Parsing HTML from [httpResponseBody](https://docs.zyte.com/zyte-api/usage/reference.html#operation/extract/response/200/httpResponseBody) with libraries that do
not implement HTML5 parsing, such as [lxml.html](https://lxml.de/lxmlhtml.html) (used by [Scrapy](https://scrapy.org/)
by default), results in a different tree structure.
With an HTML5-compatible parser the resulting tree structure would be the
same, provided JavaScript does not cause any other difference.
Because of these differences, switching between these HTML inputs can break
your existing parsing code and require changes, such as updating XPath or CSS
selectors.