Contents x
    Data Destination - Snowflake
    • 06 Dec 2022
    • Print
    • Dark
      Light
    • PDF
    Contents

    Data Destination - Snowflake

    • Updated on 06 Dec 2022
    • Print
    • Dark
      Light
    • PDF
    Article Summary

    Overview

    Help page for setting up a data transfer to Snowflake.

    Supported Protocol

    Dataloader: Embulk
    Using embulk-output-snowflake plugin.

    Limitations

    None.

    Settings

    Step 1: General Settings

    Field Name
    Required
    Default Value
    Description
    Snowflake Connection ConfigurationYes-From the Snowflake Connection Configuration created in advance, select the one with the required permissions for this transfer.
    If you have not previously created any Connection Configurations, click the Add Connection Configuration button to set up a new one.
    WarehouseYes-Enter the destination warehouse name to transfer to.
    DatabaseYes-Enter the database name at the destination to transfer to.
    SchemaYes-Enter the schema name at the destination to transfer to.
    TableYes-Enter the table name at the destination to transfer to.
    Transfer ModeYesinsertSelect a transfer mode. For details on each transfer mode, please refer to the Transfer Mode section below.

    Step 2: Advanced Settings

    Field Name
    Default Value
    Details
    Column Settings-Specify the columns to use when creating the temporary table.
    You can select from the following data types:

  • boolean: BOOLEAN
  • string: VARCHAR(65535)
  • long: BIGINT
  • double: DOUBLE PRECISION
  • timestamp: TIMESTAMP_NTZ(9)

    See here for further information on the data types you can specify.
    If you wish to specify a data type not listed above, please contact your trocco customer success agent.
    • Maximum Retries12-
    Retry Wait Time (milliseconds)1800000-
    Default Time ZoneUTCIf there are columns with the timestamp data type, you can specify the corresponding time zone in this field.

    Transfer Modes

    ModeDetails
    INSERTThis transfer mode will append the table at the destination.
    First, the data is inserted into temporary tables that are created by trocco. Next, using the INSERT command, the data will be transferred from the temporary tables to the table at the destination.
    This ensures that if the transfer fails during execution, there will not be any incomplete data left in the table at the destination.
    If a table does not exist at the destination, it will automatically be created by trocco.
    INSERT_DIRECTThis transfer mode will append the table at the destination.
    Data will be inserted directly into the table at the destination, without using a temporary table.
    If the transfer happens to fail during execution, there is a chance that the data at the destination table can end up incomplete.
    If a table does not exist at the destination, it will be automatically created by trocco.
    TRUNCATE_INSERTThis mode will overwrite the data in the table at the destination.
    First, the data is inserted into temporary tables that are created by trocco. Next, the TRUNCATE command will be executed on the table at the destination, followed by the INSERT command.
    REPLACEThis mode will recreate the table at the destination.
    First, the data is inserted into a temporary table created by trocco. Next, if all the data is transferred successfully, trocco will drop the existing table in the destination, then replace the name of the temporary table with the name of the previously existing table in the destination.
    Was this article helpful?
    What's Next