Product Extraction#
Product extraction supports pages which contain a single product. Many fields are extracted, such as product name, brand, price, availability, sku.
This supports use-cases such as price monitoring, product intelligence, product analytics and many others.
Related page types are Product List Extraction which supports pages with multiple products, Review Extraction which supports reviews on single product pages, Real Estate Extraction which supports pages with a single real estate item, and Vehicle Extraction which supports pages with a single vehicle item.
Request example#
If you request a product extraction, and the extraction succeeds,
then the product
field is available in the query result:
from autoextract.sync import request_raw
query = [{
'url': 'http://books.toscrape.com/catalogue/a-light-in-the-attic_1000/index.html',
'pageType': 'product'
}]
results = request_raw(query, api_key='[api key]')
print(results[0]['product'])
Available fields#
The following fields are available for product
:
name
: stringThe name of the product.
offers
: list of dictionariesProduct offers. Each offer may contain
price
,currency
,regularPrice
andavailability
string fields. All fields are optional butcurrency
is present only ifprice
is also present.price
field is a string with a valid number (a dot is used as decimal separator). It is the price a customer has to pay after discounts or special offers.currency
is the currency as given on the website, without extra normalization (for example, both “$” and “USD” are possible currencies). It is present only ifprice
is also present.regularPrice
is the price before any discount or special offer. It is present only when theprice
is different fromregularPrice
.availability
is the product availability, as a string. Allowed values:"InStock"
- includes limited availability, presale, preorder, and in-store only."OutOfStock"
- includes discontinued and sold out.
Example:
[ { "price": "42", "regularPrice": "45.00", "currency": "USD", "availability": "InStock" } ]
sku
: stringStock Keeping Unit identifier for the product assigned by the seller.
mpn
: stringManufacturer part number identifier for product. It is issued by the manufacturer and is same across different websites for a product.
gtin
: list of dictionaries withtype
andvalue
string fieldsStandardized GTIN product identifier which is unique for a product across different sellers. It includes the following
type
: isbn10, isbn13, issn, ean13, upc, ismn, gtin8, gtin14.gtin14 corresponds to former names EAN/UCC-14, SCC-14, DUN-14, UPC Case Code, UPC Shipping Container Code.
ean13 also includes the jan (japanese article number). Example:
[{"type": "isbn13", "value": "9781933624341"}]
brand
: stringBrand or manufacturer of the product.
breadcrumbs
: list of dictionaries withname
andlink
optional string fieldsA list of breadcrumbs (a specific navigation element) with optional name and URL. Example:
[ {"name": "Foo", "link": "http://example.com/foo"}, {"name": "Bar", "link": "http://example.com/foo/bar"}, {"name": "Baz"}, ]
mainImage
: stringA URL or data URL value of the main image of the product.
images
: list of stringsA list of URL or data URL values of all images of the product (may include the main image).
description
: stringDescription of the product.
descriptionHtml
: stringSimplified HTML of the description, including sub-headings, image captions and embedded content.
aggregateRating
: dictionaryAggregate information about the product rating and reviews.
ratingValue
is the average rating value, as a float.bestRating
is the best possible rating value, as a float.reviewCount
is the number of reviews or ratings for the product, as int.
Example - 4.5 out of 5, based on 12 reviews:
{ "ratingValue": 4.5, "bestRating": 5, "reviewCount": 12 }
All fields are optional but one of
reviewCount
orratingValue
must be present.color
: stringColor of the product.
size
: stringA standardized size of a product, specified through a simple textual string (for example “XL”, “32Wx34L”). A single product dimension (height, width) is not considered as the size.
style
: stringStyle of the product. It can be referred as pattern/finish on the product page. Example values: “Polka dots”, “Striped”, “Nickel finish with Translucent glass”, etc.
additionalProperty
: list of dictionaries withname
andvalue
fieldsA list of product properties or characteristics.
name
field contains the property name,value
field contains the property value.
Example:
[ {"name": "color", "value": "blue"}, {"name": "brand", "value": "McBrand"}, {"name": "best for", "value": "special events"}, ]
hasVariants
: list of ProductA list of product variants, using the same Product schema. Represents extra information available about the variants of a product. All variants are included into this array, including the variant shown on the page. If some field in this array is empty, it means that either the value is the same as in the top-level product, or that extraction API did not manage to extract it.
probability
: floatProbability that the requested page is a single product page.
canonicalUrl
: stringCanonical URL of the product, if available.
url
: stringURL a of page where this product was extracted.
All fields are optional, except for url
and probability
.
Fields without a valid value (null or empty array) are excluded from extraction results.
Response example#
Below is an example response with all product fields present:
[
{
"product": {
"name": "Product name",
"offers": [
{
"price": "42",
"regularPrice": "45.00",
"currency": "USD",
"availability": "InStock"
}
],
"sku": "product sku",
"mpn": "product mpn",
"gtin": [
{
"type": "ean13",
"value": "978-3-16-148410-0"
}
],
"brand": "product brand",
"breadcrumbs": [
{
"name": "Level 1",
"link": "http://example.com"
}
],
"mainImage": "http://example.com/image.png",
"images": [
"http://example.com/image.png"
],
"description": "product description",
"descriptionHtml": "<article>HTML description for Product ...",
"aggregateRating": {
"ratingValue": 4.5,
"bestRating": 5.0,
"reviewCount": 31
},
"color": "product color",
"size": "product size",
"style": "product style",
"additionalProperty": [
{
"name": "property 1",
"value": "value of property 1"
}
],
"hasVariants": [
{
"offers": [{"price": "42", "availability": "InStock"}],
"color": "red",
"sku": "pruqkj-r"
},
{
"offers": [{"price": "45", "availability": "OutOfStock"}],
"color": "black",
"sku": "pruqkj-b"
}
],
"probability": 0.95,
"canonicalUrl": "https://example.com/product/",
"url": "https://example.com/product"
},
"webPage": {
"inLanguages": [
{"code": "en"},
{"code": "es"}
]
},
"query": {
"id": "1564747029122-9e02a1868d70b7a2",
"domain": "example.com",
"userQuery": {
"pageType": "product",
"url": "https://example.com/product"
}
},
"algorithmVersion": "20.8.1"
}
]