The Automatic Extraction is a service for automatically extracting information from web pages.
You provide the page URLs that you are interested in, and what type of content you expect to find there: article, article list, comments, forum posts, job posting, product, product list, product reviews, real estate or vehicle.
The service will then fetch the content, and apply a number of techniques behind the scenes to extract as much information as possible. Finally, the extracted information is returned to you in structured form.
The following page types are supported:
Price intelligence & ecommerce¶
Media & discussion monitoring¶
1. Sign up (or login)¶
To get started, you need to sign up for an Automatic Extraction subscription. You can start your free 14-day trial here. You will need a credit card to subscribe but you won’t be charged if you cancel in the first 14 days.
2. Get API Key¶
Once subscribed to the free trial, you will receive an API key. If you haven’t received one, you can contact the Automatic Extraction support team directly at firstname.lastname@example.org.
3. Integrate Automatic Extraction¶
If you just want to extract data using CLI, use the scrapinghub-autoextract client library.
Otherwise, see our code examples:
See Automatic Extraction API for the detailed description of the Automatic Extraction HTTP API.