Contents x
    dbt job settings
    • 17 Jul 2024
    • Print
    • PDF
    Contents

    dbt job settings

    • Updated on 17 Jul 2024
    • Print
    • PDF
    Article summary

    summary

    This article describes dbt job settings.

    List of setting items

    common setting

    item nameindispensabledefault valueContents
    Name✓-Enter the name of the dbt job setting.
    memo--You can enter notes on dbt job settings.
    dbt repository✓-Select the dbt Git repository linked to TROCCO.
    Execution command✓dbt buildSelect the command to execute.
    You can also set options and their values.
    See the corresponding dbt command for details.
    Number of Threads✓1Sets the number of models that dbt can process at one time.
    The maximum number of threads that can be set is 16.
    See UNDERSTANDING-THREADS for more information on the number of threads.
    target--You can specify the value of target in the profiles.yml generated after the dbt job is executed.
    You can also use the value of target as a flag for conditional branching in a query.
    Note that if no entry is made, the value trocco_default is passed to target.
    See Custom target names for details.
    Number of threads

    When the dbt command is executed, a directed acyclic graph (DAG) of the links between the models is first created.
    The number of threads specifies the maximum number of graph paths that dbt can process at one time.

    In general, increasing the number of threads reduces execution time, but on the other hand increases the load on the adapter.
    Therefore, the optimal number of threads depends on the data warehouse and its configuration.

    Setting items for each adapter

    Subsequent configuration items will vary depending on the adapter configured in the dbt Git repository.

    If the adapter is Google BigQuery

    item nameindispensableContents
    Google BigQuery connection information✓Select Google BigQuery connection information.
    Please refer to BigQuery's connection information for details.
    data-set✓Enter the name of the data set to which the data will be output.
    Location-You can enter the location of the data set for the data output destination.
    If you do not enter a location, US (multi-region) will be specified as the location for the data set if the specified data set does not exist in the output destination.

    If the adapter is Snowflake

    item nameindispensableContents
    Snowflake Connection Information✓Select Snowflake connection information.
    See Snowflake's connection information for details.
    Both "user password authentication" and "key pair authentication" can be used as authentication methods for connection information.
    warehouse✓Enter the name of the warehouse to be used for processing.
    database✓Enter the database name of the data output destination.
    schema✓Enter the name of the schema to which the data will be output.
    roll-Enter a role name that summarizes the privileges required to use Snowflake.
    For more information about roles, see the Snowflake Connection Information supplement.

    If the adapter is Amazon Redshift

    item nameindispensableContents
    Redshift connection information✓Select Amazon Redshift connection information.
    For more information, see Amazon Redshift Connection Information.
    database✓Enter the database name of the data output destination.
    schema✓Enter the name of the schema to which the data will be output.

    Corresponding dbt command

    The following are the dbt commands supported by TROCCO.
    For more information on the dbt command, see the dbt Command reference.

    functions associated with the dbt command

    TROCCO allows optional arguments and custom variables for the dbt command.

    command namesummary
    dbt buildBuild all resources in the project.
    dbt depsInstall the package.
    dbt runRun the model file.
    dbt seedTransfers CSV files to the adapter.
    dbt snapshotRun the snapshot job.
    dbt testRun the test.
    dbt run-operationExecute the macro.
    Please enter the macro name in the box below.
    image

    Various operations of dbt job setup

    dbt job settings list screen

    005-p.png

    dbt job setting details screen

    image

    Replicate dbt job settings

    dbt job settings cannot be duplicated.

    (1) Newly added

    Add a new dbt job setting. A new creation screen will appear.

    2) Editing

    Edit dbt job settings. The detail screen will be displayed.

    Execution

    Run the dbt job. The execution screen is displayed.

    Deleted

    Delete dbt job settings.

    5) Creator

    If you are using the Team feature, the creator of a dbt job setting has the same operating privileges as a user with the Resource Group Administrator role for that setting.
    For more information, see "Role Permissions" below.

    role permission

    If you use the team function

    dbt job settings are treated as resources in TROCCO.
    * See Resource Definition: Team Functions.
    * Permissions to operate on resources: see Resource Groups.

    Resource Group AdministratorResource Group EditorResource Group OperatorResource Group ViewerNo Role
    dbtジョブ設定の編集✓✓---
    Delete dbt job settings✓✓✓✓- - - - - - - -
    Run dbt job✓✓✓✓✓
    View dbt job settings✓✓✓✓✓
    View dbt job execution history✓✓✓✓✓
    Add new dbt job settings✓✓✓✓✓
    Was this article helpful?
    What's Next