# Exporting to Google Drive with Scrapy

To configure a [Scrapy](https://scrapy.org) project or spider to export scraped data to
[Google Drive](https://www.google.com/drive/):

1. You need Python 3.8 or higher.

   If you are using [Scrapy Cloud](../../../../scrapy-cloud/get-started.md#scrapy-cloud), make sure you are
   using [stack](https://support.zyte.com/support/solutions/articles/22000200402-changing-the-deploy-environment-with-scrapy-cloud-stacks) `scrapy:1.7-py38` or higher. Using the latest stack
   (`scrapy:2.14-20260217`) is generally recommended.
2. Install [scrapy-feedexporter-google-drive](https://github.com/scrapy-plugins/scrapy-feedexporter-google-drive):
   ```bash
   pip install git+https://github.com/scrapy-plugins/scrapy-feedexporter-google-drive
   ```

   If you are using [Scrapy Cloud](../../../../scrapy-cloud/get-started.md#scrapy-cloud), remember to add the
   following line to your `requirements.txt` file:
   ```none
   scrapy-feedexporter-google-drive @ git+https://github.com/scrapy-plugins/scrapy-feedexporter-google-drive
   ```
3. In your `settings.py` file, define `FEED_STORAGES` as follows:
   settings.py
   ```python
   FEED_STORAGES = {
       "gdrive": "scrapy_gdrive_exporter.gdrive_exporter.GoogleDriveFeedStorage",
   }
   ```

   If the setting already exists in your `settings.py` file, modify the existing
   setting to add the key-value pair above, instead of re-defining the setting.
4. Add a [FEEDS](https://docs.scrapy.org/en/latest/topics/feed-exports.html#std-setting-FEEDS)
   setting to your project or spider, if not added yet.

   The value of `FEEDS` must be a JSON object (`{}`).

   If you have `FEEDS` already defined with key-value pairs, you can keep
   those if you want — `FEEDS` supports exporting data to multiple file
   storage service locations.

   To add `FEEDS` to a project, define it in your [Scrapy Cloud project
   settings](https://support.zyte.com/support/solutions/articles/22000200670-customizing-scrapy-settings-in-scrapy-cloud)
   or add it to your `settings.py` file:
   settings.py
   ```python
   FEEDS = {}
   ```

   To add `FEEDS` to a spider, define it in your Scrapy Cloud
   spider-specific settings (open a spider in Scrapy Cloud and select the
   **Settings** tab) or add it to your spider code with the [update_settings](https://docs.scrapy.org/en/latest/topics/spiders.html#scrapy.Spider.update_settings)
   method or the [custom_settings](https://docs.scrapy.org/en/latest/topics/spiders.html#scrapy.Spider.custom_settings) class variable:
   spiders/myspider.py
   ```python
   class MySpider:
       custom_settings = {
           "FEEDS": {},
       }
   ```
5. Add the following key-value pair to `FEEDS`:
   ```python
   {
       "gdrive://drive.google.com/<FOLDER ID>/<PATH>": {
           "format": "<FORMAT>"
       }
   }
   ```

   Where:
   - `<FOLDER ID>` is the ID of the target root folder, e.g.
     `1uWBpSBe3CvF8u21qTrzDqjZ6uexample`.
     > [!TIP]
     > When inside a folder, the URL ends with the folder ID, e.g:
     > `https://drive.google.com/drive/folders/1uWBpSBe3CvF8u21qTrzDqjZ6uexample`.
   - `<PATH>` is the path where you want to store the scraped data file, e.g.
     `scraped/data.csv`.

     The path can include [placeholders](https://docs.scrapy.org/en/latest/topics/feed-exports.html#storage-uri-parameters) that are replaced at run time, such
     as `%(time)`, which is replaced by the current timestamp.
     > [!NOTE]
     > [scrapy-feedexporter-google-drive](https://github.com/scrapy-plugins/scrapy-feedexporter-google-drive) does not support
     > overwriting or appending to files, it can only create new files
     > every time.
   - `<FORMAT>` is the desired [output file format](https://docs.scrapy.org/en/latest/topics/feed-exports.html#serialization-formats).

     Possible values include: `csv`, `json`, `jsonlines`, `xml`. You can
     also [implement support for more formats](https://docs.scrapy.org/en/latest/topics/exporters.html).
     > [!WARNING]
     > If you export in CSV format, and in your spider code you yield
     > items as Python dictionaries, only the fields present on the first yielded
     > item are exported for all items.
     > 
     > One solution is to [customize output fields](https://docs.scrapy.org/en/latest/topics/exporters.html#scrapy.exporters.BaseItemExporter.fields_to_export) through the `fields` [feed
     > option](https://docs.scrapy.org/en/latest/topics/feed-exports.html#feed-options) of [FEEDS](https://docs.scrapy.org/en/latest/topics/feed-exports.html#feeds) or
     > through the [FEED_EXPORT_FIELDS](https://docs.scrapy.org/en/latest/topics/feed-exports.html#feed-export-fields) Scrapy setting to explicitly indicate all
     > fields to export.
     > 
     > You can alternatively yield something other than a Python dictionary that
     > supports declaring all possible fields, such as an [Item object](https://docs.scrapy.org/en/latest/topics/items.html#item-objects) or an
     > [attrs object](https://docs.scrapy.org/en/latest/topics/items.html#attr-s-objects).
6. Define the `GDRIVE_SERVICE_ACCOUNT_CREDENTIALS_JSON` setting as a Python
   string containing your [service account credentials](https://developers.google.com/identity/protocols/oauth2/service-account) in JSON format:
   settings.py
   ```python
   GDRIVE_SERVICE_ACCOUNT_CREDENTIALS_JSON = '{ "type": "service_account", "project_id": "myproject", "private_key_id": "…", "private_key": "…", "client_email": "…@email.iam.gserviceaccount.com", "client_id": "…", "auth_uri": "…", "token_uri": "…", "auth_provider_x509_cert_url": "…", "client_x509_cert_url": "…" }'
   ```

   Make sure you give your service account write access on the target folder.
   You can do that by sharing the folder with the email of the service account
   (`client_email` in the JSON above).

Running your spider now, locally or on Scrapy Cloud, will export your scraped
data to the configured Google Drive location.