Contents x
    dbt Job Configuration
    • 24 Jan 2023
    • Print
    • Dark
      Light
    • PDF
    Contents

    dbt Job Configuration

    • Updated on 24 Jan 2023
    • Print
    • Dark
      Light
    • PDF
    Article Summary

    Note

    This is a machine-translated version of the original Japanese article.
    Please understand that some of the information contained on this page may be inaccurate.

    summary

    This article describes dbt job settings.

    List of setting items

    Common settings

    Itemnamerequireddefault
    		valuecontent
    name✓-Enter a name for the dbt job configuration.
    memorandum--You can enter notes for dbt job settings.
    dbt repository✓-Select the dbt Git repository that is linked to trocco.
    Execution Command✓dbt buildSelect the Run command.
    You can also set options and their values.
    For details, refer to the corresponding dbt command.
    Number of threads✓1Sets the number of models that dbt can process at one time.
    The maximum number of threads that can be set is 16.
    For more information on the number of threads, see understanding-threads.
    Number of threads

    When the dbt command is executed, it first creates a directed cycle graph (DAG) of the links between the models.
    The number of threads specifies the maximum number of graph paths that dbt can process at one time.

    Note that increasing the number of threads generally shortens execution time, but increases the load on the adapter.
    Therefore, the optimal number of threads depends on the data warehouse and its configuration.

    Settings for each adapter

    The following settings differ depending on the adapter set in the dbt Git repository.

    If the adapter is Google BigQuery

    Item nameRequired
    		content
    Google BigQuery connection information✓Select the Google BigQuery connection information.
    For details, please refer to the connection information of BigQuery.
    Dataset✓Enter the dataset name to output data to.

    If the adapter is Snowflake

    Item nameRequired
    		content
    Snowflake connection information✓Select the connection information for Snowflake.
    See Snowflake Connection Information for more information.
    You can use either "User Password Authentication" or "Key Pair Authentication" as the authentication method for the connection information.
    Warehouse✓Enter the name of the warehouse that you want to use for processing.
    database✓Enter the name of the database to which you want to output data.
    schema✓Enter the schema name of the data destination.
    roll-Enter a role name that summarizes the permissions required to use Snowflake.
    For more information about roles, see Snowflake Connection Information Supplement.

    If the adapter is Amazon Redshift

    Item nameRequired
    		content
    Redshift connection information✓Choose your Amazon Redshift connection information.
    For more information, see Amazon Redshift connection information.
    database✓Enter the name of the database to which you want to output data.
    schema✓Enter the schema name of the data destination.

    Corresponding dbt command

    The following are the dbt commands supported by trocco:
    For more information about the dbt command, see dbt Command reference.

    Functions attached to the dbt command

    trocco provides optional arguments and custom variables for dbt commands.

    command namesoverview
    dbt buildBuild all resources in the project.
    dbt depsInstall the package.
    dbt runRun the model file.
    dbt seedTransfer the CSV file to the adapter.
    dbt snapshotRun the snapshot job.
    dbt testRun the test.
    dbt run-operationRun the macro.
    Please enter the macro name in the box below.
    image

    Various operations of dbt job settings

    DBT job settings list screen

    005-p.png

    DBT job setting details screen

    image

    Duplicate dbt job settings

    dbt job settings cannot be replicated.

    (1) New addition

    Add a new dbt job setting. The new screen appears.

    (2) Edit

    Edit the dbt job settings. The details screen appears.

    (3) Execution

    Run the dbt job. The execution screen appears.

    (4) Delete

    Delete the dbt job configuration.


    Once you delete the dbt job settings, you cannot undo them. Please be careful.

    (5) Author

    When using the team function, the creator of the dbt job setting has the same operation privileges as a user with the administrator role for the setting.
    For more information, see "Role permissions" below.

    Role permissions

    If you use the Teams feature

    DBT job settings are treated as resources in Trocco.


    		Operation\
    		RolesAdministratorEditorOperatorReaderNo
    		role
    Editing dbt job settings✓✓---
    Deleting dbt job settings✓✓---
    Running a dbt job✓✓✓--
    Viewing dbt job settings✓✓✓✓-
    Viewing dbt job execution history✓✓✓✓-
    Adding a new dbt job setting✓✓✓✓✓


    Was this article helpful?
    What's Next