Data Destination - Snowflake
  • 21 Apr 2022
  • 2 Minutes to read
  • Dark
    Light

Data Destination - Snowflake

  • Dark
    Light

Overview

Help page for setting up a data transfer to Snowflake.

Supported Protocol

Dataloader: Embulk
Using embulk-output-snowflake plugin.

Limitations

None.

Settings

Step 1: General Settings

Field Name
Required
Default Value
Description
Snowflake Connection Configuration Yes - From the Snowflake Connection Configuration created in advance, select the one with the required permissions for this transfer.
If you have not previously created any Connection Configurations, click the Add Connection Configuration button to set up a new one.
Warehouse Yes - Enter the destination warehouse name to transfer to.
Database Yes - Enter the database name at the destination to transfer to.
Schema Yes - Enter the schema name at the destination to transfer to.
Table Yes - Enter the table name at the destination to transfer to.
Transfer Mode Yes insert Select a transfer mode. For details on each transfer mode, please refer to the Transfer Mode section below.

Step 2: Advanced Settings

Field Name
Default Value
Details
Column Settings - Specify the columns to use when creating the temporary table.
You can select from the following data types:

  • boolean: BOOLEAN
  • string: VARCHAR(65535)
  • long: BIGINT
  • double: DOUBLE PRECISION
  • timestamp: TIMESTAMP_NTZ(9)

    See here for further information on the data types you can specify.
    If you wish to specify a data type not listed above, please contact your trocco customer success agent.
  • Maximum Retries 12 -
    Retry Wait Time (milliseconds) 1800000 -
    Default Time Zone UTC If there are columns with the timestamp data type, you can specify the corresponding time zone in this field.

    Transfer Modes

    Mode Details
    INSERT This transfer mode will append the table at the destination.
    First, the data is inserted into temporary tables that are created by trocco. Next, using the INSERT command, the data will be transferred from the temporary tables to the table at the destination.
    This ensures that if the transfer fails during execution, there will not be any incomplete data left in the table at the destination.
    If a table does not exist at the destination, it will automatically be created by trocco.
    INSERT_DIRECT This transfer mode will append the table at the destination.
    Data will be inserted directly into the table at the destination, without using a temporary table.
    If the transfer happens to fail during execution, there is a chance that the data at the destination table can end up incomplete.
    If a table does not exist at the destination, it will be automatically created by trocco.
    TRUNCATE_INSERT This mode will overwrite the data in the table at the destination.
    First, the data is inserted into temporary tables that are created by trocco. Next, the TRUNCATE command will be executed on the table at the destination, followed by the INSERT command.
    REPLACE This mode will recreate the table at the destination.
    First, the data is inserted into a temporary table created by trocco. Next, if all the data is transferred successfully, trocco will drop the existing table in the destination, then replace the name of the temporary table with the name of the previously existing table in the destination.

    Was this article helpful?