Data Source - MongoDB
  • 06 Dec 2022
  • Dark
  • PDF

Data Source - MongoDB

  • Dark
  • PDF

Article Summary


Help page for setting up a data transfer from MongoDB.

Supported Protocol

Data Loader: Embulk
Using the embulk-input-mongodb plugin.




STEP 1: General Settings

Field Name
Default Value
MongoDB Connection ConfigurationsYes-From the MongoDB Connection Configurations created in advance, select the one with the necessary permissions for this transfer.
If you have not previously created any Connection Configurations, click the Add Connection Configuration button to set up a new one.
Database NameYes-Enter the MongoDB database name containing the data you wish to transfer.
Collection NameYes-Enter the collection name containing the data you wish to transfer.
QueryYes-In this field, write a query to extract the data from MongoDB.
Column SettingsYes-Enter the columns (fields), that you wish to extract from each record.
Note: trocco does not support extraction of nested keys, and can only extract top level fields of the JSON.
Incremental LoadYesDisabledSelect whether or not to enable Incremental Load. When enabled, only new data since the Last Record Loaded from the previous job will be transferred.
Incremental Load Column NameNo-You can enter into this field only when Incremental Load is enabled.
For each transfer, the last value loaded from the column specified here will be saved as the Last Record Loaded.
From the next job, only values higher than the Last Record Loaded, will be transferred.
When using a unique and auto-increment column (e.g. id, created_at, etc.), enter it here.
Use comma separation to specify multiple columns. If unspecified, a primary key is used.
Last Record LoadedNo-You can enter into this field only when Incremental Load is enabled.
Only data with a value higher than the value specified here will be transferred.
If unspecified, a full load will occur.
Do not change this value without reason (such as if you need to re-load data from a specific date range), as there is a chance of data duplication.

Was this article helpful?