The purpose of the fs_service utility #

The utility is designed to work with data in Firestore.

The tool can:

• Import Firestore documents and collections and print them to a console or write to JSON files.
• Export documents and collections from a console or JSON files to Firestore.
• Delete Firestore documents and collections.
• Perform all these operations recursively, preserving the internal structure of Firestore and all data types that exist in Firestore.

Add tool to your project #

Add this line manually to your dev_dependencies:

dev_dependencies:
  fs_service: ^1.0.0

or alternatively run this command in the root folder of your app, where the pubspec.yaml file is located:

dart pub add dev:fs_service

Getting access to Firestore project #

⚠️ To access the data, a Firestore service account is used. What is needed for work:

• Create a service account for your Firestore project.
• Download JSON with credentials to access the account.
• Set the GOOGLE_APPLICATION_CREDENTIALS environment variable and specify the path to the credentials file in it.

For example:

export GOOGLE_APPLICATION_CREDENTIALS=secret/myProject-firebase-adminsdk-asda-23423hgh32.json

You can read more about this here: https://cloud.google.com/docs/authentication/application-default-credentials#GAC

General structure of documents and collections in the Firebase #

ℹ️ It’s important to understand that the structure has the following form:

• Documents are located at the root of the project.
• A document consists of fields with different types of data.
• Each document can have a nested collection, and not just one.
• Several documents can be nested in each collection.

⚠️ From this, the following important conclusions are drawn:

• If colX is the name of a collection and docX is the name of a document, then the relative path from the root of the project will look something like this: col1/doc1/col2/doc2, i.e., the path starts with a collection and alternately contains a document and a collection nested in the document.
• You can only add a document to a collection, and a collection only to a document.
• If you delete the collection col1, then you delete all documents and collections down this path.
• If you delete the document doc1, then you delete all collections and documents down this path.

The absolute path of the root of the project looks like this:

projects/{projectId}/databases/{databaseId}/documents

You have to pass the projectId value and optionally the databaseId value as the arguments to the application, see examples below.

get-doc Command #

Here is an example of a document lUAEoKptpXKT1CshnZKX which contains all types of fields provided by Firestore. There is also one nested collection with one document lUAEoKptpXKT1CshnZKX/col1/lUAEoKptpXKT1CshnZKX - a copy of the first one.

dart run fs_service get-doc test/lUAEoKptpXKT1CshnZKX --project=myProject

Since the output file is not specified in the options, the result will be printed to STDOUT as follows:

{
  "numberF": 1234.5432,
  "null1": null,
  "ref": "reference://projects/myProject/databases/(default)/documents/en/YLTunxHK6rgPTWHxjJYe",
  "timestamp": "datetime://2023-10-21T11:26:40.152Z",
  "translit": "",
  "geopoint": "location://34.3456/-23.432",
  "word": "four",
  "map": {
    "key1": {
      "key2": "value2"
    }
  },
  "transcript": "fɔːr",
  "boolean": true,
  "number": 12345,
  "array": [
    "array1",
    "array2"
  ],
  "id": "95il61U47MVonL027u3V",
  "$name": "lUAEoKptpXKT1CshnZKX",
  "$createTime": "2023-10-26T12:52:01.608133Z",
  "$updateTime": "2023-10-26T12:52:01.608133Z",
  "$collections": [
    {
      "$name": "col1",
      "$documents": [
        {
          "map": {
            "key1": {
              "key2": "value2"
            }
          },
          "number": 12345,
          "transcript": "fɔːr",
          "word": "four",
          "ref": "reference://projects/myProject/databases/(default)/documents/en/YLTunxHK6rgPTWHxjJYe",
          "numberF": 1234.5432,
          "id": "95il61U47MVonL027u3V",
          "translit": "",
          "geopoint": "location://34.3456/-23.432",
          "boolean": true,
          "array": [
            "array1",
            "array2"
          ],
          "null1": null,
          "timestamp": "datetime://2023-10-21T11:26:40.152Z",
          "$name": "lUAEoKptpXKT1CshnZKX",
          "$createTime": "2024-01-02T11:40:47.284577Z",
          "$updateTime": "2024-01-02T11:40:47.284577Z"
        }
      ]
    }
  ]
}

More sophisticated examples are in the doc-2.json and col-2.json files.

Time will always be converted to UTC to avoid confusion. The default geolocation is stored in the form of location://{LATITUDE}/{LONGITUDE}

By default the fields starting with $ are the meta-data fields. These fields are used to restore structure of the Firestore database.

In case the field names of your your database are clash with meta-data field names you can change meta-prefix. Run the following command to read more about it:

dart run fs_service help get-doc

If you are not happy with reference:// and other such prefixes you can also change them with the tool options.

add-doc Command #

If you use the get-doc command from the example above and use the add-doc command through the | operator, then the result of the get-doc command will be passed to the add-doc command.

dart run fs_service get-doc test/lUAEoKptpXKT1CshnZKX --project=myProject | \
  dart run fs_service add-doc test -c doc4 --project=ella500

In this case, the document test/lUAEoKptpXKT1CshnZKX will be copied to the same collection test but with the name doc4 (option -c). All nested collections and documents will also be copied.

⚠️ It's important to understand that the values of the '$createTime' and '$updateTime' fields will not be written to the Firestore document, because Firestore ignores these values and overwrites them automatically.

ℹ️ You can delete the meta-data field $name of the document or set it to null. In this case Firestore will assign random unique id to this document.

del-doc Command #

It is used to delete a document and all its nested collections and documents.

dart run fs_service del-doc test/lUAEoKptpXKT1CshnZKX --project=myProject

Be careful, as a result of the operation, data is irretrievably destroyed.

get-col, add-col and del-col Commands #

They work similarly to the get-doc, add-doc, and del-doc commands, but are designed to work with collections.

Detailed information #

You can read the detailed information about the tool and each of the command with the following commands

dart run fs_service --help

dart run fs_service help add-doc