Data Source - MongoDB
  • 03 Mar 2022
  • 1 Minute to read
  • Dark
    Light

Data Source - MongoDB

  • Dark
    Light

Overview

Help page for setting up a data transfer from MongoDB.

Supported Protocol

Data Loader: Embulk
Using the embulk-input-mongodb plugin.

Limitations

None.

Settings

STEP 1: General Settings

Field Name
Required
Default Value
Description
MongoDB Connection Configurations Yes - From the MongoDB Connection Configurations created in advance, select the one with the necessary permissions for this transfer.
If you have not previously created any Connection Configurations, click the Add Connection Configuration button to set up a new one.
Database Name Yes - Enter the MongoDB database name containing the data you wish to transfer.
Collection Name Yes - Enter the collection name containing the data you wish to transfer.
Query Yes - In this field, write a query to extract the data from MongoDB.
Column Settings Yes - Enter the columns (fields), that you wish to extract from each record.
Note: trocco does not support extraction of nested keys, and can only extract top level fields of the JSON.
Incremental Load Yes Disabled Select whether or not to enable Incremental Load. When enabled, only new data since the Last Record Loaded from the previous job will be transferred.
Incremental Load Column Name No - You can enter into this field only when Incremental Load is enabled.
For each transfer, the last value loaded from the column specified here will be saved as the Last Record Loaded.
From the next job, only values higher than the Last Record Loaded, will be transferred.
When using a unique and auto-increment column (e.g. id, created_at, etc.), enter it here.
Use comma separation to specify multiple columns. If unspecified, a primary key is used.
Last Record Loaded No - You can enter into this field only when Incremental Load is enabled.
Only data with a value higher than the value specified here will be transferred.
If unspecified, a full load will occur.
Do not change this value without reason (such as if you need to re-load data from a specific date range), as there is a chance of data duplication.

Was this article helpful?