Contents x
    Data Source - MongoDB
    • 06 Dec 2022
    • Print
    • Dark
      Light
    • PDF
    Contents

    Data Source - MongoDB

    • Updated on 06 Dec 2022
    • Print
    • Dark
      Light
    • PDF
    Article Summary

    Overview

    Help page for setting up a data transfer from MongoDB.

    Supported Protocol

    Data Loader: Embulk
    Using the embulk-input-mongodb plugin.

    Limitations

    None.

    Settings

    STEP 1: General Settings

    Field Name
    Required
    Default Value
    Description
    MongoDB Connection ConfigurationsYes-From the MongoDB Connection Configurations created in advance, select the one with the necessary permissions for this transfer.
    If you have not previously created any Connection Configurations, click the Add Connection Configuration button to set up a new one.
    Database NameYes-Enter the MongoDB database name containing the data you wish to transfer.
    Collection NameYes-Enter the collection name containing the data you wish to transfer.
    QueryYes-In this field, write a query to extract the data from MongoDB.
    Column SettingsYes-Enter the columns (fields), that you wish to extract from each record.
    Note: trocco does not support extraction of nested keys, and can only extract top level fields of the JSON.
    Incremental LoadYesDisabledSelect whether or not to enable Incremental Load. When enabled, only new data since the Last Record Loaded from the previous job will be transferred.
    Incremental Load Column NameNo-You can enter into this field only when Incremental Load is enabled.
    For each transfer, the last value loaded from the column specified here will be saved as the Last Record Loaded.
    From the next job, only values higher than the Last Record Loaded, will be transferred.
    When using a unique and auto-increment column (e.g. id, created_at, etc.), enter it here.
    Use comma separation to specify multiple columns. If unspecified, a primary key is used.
    Last Record LoadedNo-You can enter into this field only when Incremental Load is enabled.
    Only data with a value higher than the value specified here will be transferred.
    If unspecified, a full load will occur.
    Do not change this value without reason (such as if you need to re-load data from a specific date range), as there is a chance of data duplication.
    Was this article helpful?
    What's Next