summary
This article describes the dbt Job Setting.
List of setting items
common setting
| item name | indispensable | default value | Contents |
|---|---|---|---|
| Name | ✓ | - | Enter the name of the dbt Job Setting. |
| memo | - | - | You can enter notes on dbt Job Settings. |
| dbt repository | ✓ | - | Select the dbt Git Integration to TROCCO. |
| Execution command | ✓ | dbt build |
Select the command to execute. You can also set options and their values. See the corresponding dbt command for more information. |
| Number of Threads | ✓ | 1 |
Sets the number of models that dbt can process at one time. The maximum number of threads that can be set is 16. See UNDERSTANDING-THREADS for more information on the number of threads. |
| target | - | - | You can specify the value of target in the profiles.yml generated after the dbt job is run.You can also use the value of target as a flag for conditional branching in a query.Note that if no entry is made, the value trocco_default is passed to target.See Custom target names for details. |
When the dbt command is executed, a directed acyclic graph (DAG) of the links between the models is first created.
The number of threads specifies the maximum number of graph paths that dbt can process at one time.
In general, increasing the number of threads reduces execution time, but on the other hand increases the load on the adapter.
Therefore, the optimal number of threads depends on the data warehouse and its configuration.
Setting items for each adapter
Subsequent settings will vary depending on the adapter set up in the dbt Git repository.
When the adapter is Google BigQuery
| item name | indispensable | Contents |
|---|---|---|
| Google BigQuery Connection Configuration | ✓ | Select Connection Configuration for Google BigQuery. Please refer to BigQuery's Connection Configuration for more information. |
| data-set | ✓ | Enter the name of the data set to which the data will be output. |
| Location | - | You can enter the location of the data set for the data output destination. If no location is entered, US (multi-region) will be specified as the location for the data set if the specified data set does not exist in the output destination. |
If the adapter is Snowflake
| item name | indispensable | Contents |
|---|---|---|
| Snowflake Connection Configuration | ✓ | Select Connection Configuration for Snowflake. See Snowflake's Connection Configuration for more information. Both "User Password Authentication" and "Key Pair Authentication" can be used as the authentication method for Connection Configuration. |
| warehouse | ✓ | Enter the name of the warehouse to be used for processing. |
| database | ✓ | Enter the database name of the data output destination. |
| schema | ✓ | Enter the name of the schema to which the data will be output. |
| roll | - | Enter a role name that summarizes the permissions required to use Snowflake. For more information about roles, please see Snowflake's Connection Configuration supplement. |
If the adapter is Amazon Redshift
| item name | indispensable | Contents |
|---|---|---|
| Redshift Connection Configuration | ✓ | Select Connection Configuration for Amazon Redshift. For more information, see Amazon Redshift Connection Configuration. |
| database | ✓ | Enter the database name of the data output destination. |
| schema | ✓ | Enter the name of the schema to which the data will be output. |
Corresponding dbt command
The following are the dbt commands supported by TROCCO.
For more information on the dbt command, see the dbt Command reference.
TROCCO allows the use of Optional Arguments and Custom Variables for the dbt command.
| command name | summary |
|---|---|
dbt build |
Build all resources in the project. |
dbt deps |
Install the package. |
dbt run |
Run the model file. |
dbt seed |
Transfers CSV files to the adapter. |
dbt snapshot |
Run the snapshot job. |
dbt test |
Run the test. |
dbt run-operation |
Execute the macro. Please enter the macro name in the box below.  |
Various operations of dbt Job Setting
dbt Job Setting List screen
dbt Job Setting Details Screen
dbt Job Settings cannot be duplicated.
(1) Newly added
Add a new dbt Job Setting. A new creation screen will appear.
2) Editing
Edit dbt Job Settings. The detail screen will be displayed.
Execution
Run the dbt job. The execution screen is displayed.
Deleted
Delete dbt Job Setting.
5) Creator
If you are using the Team feature, the creator of a dbt Job Setting has the same rights to operate on that setting as a user with the Resource Group Administrator role.
For more information, see "Role Permissions" below.
role permission
The dbt Job Setting is treated as a resource in TROCCO.
- Resource Definition: See Team Functions.
- Permissions to operate on resources: see Resource Group.
| Operation backspace | Resource Group Manager | Resource Group Editor | Resource Group Operator | Resource Group Viewer | No roll |
|---|---|---|---|---|---|
| Edit dbt Job Settings | ✓ | ✓ | - | - | - |
| Delete dbt Job Setting | ✓ | ✓ | - | - | - |
| Running a dbt job | ✓ | ✓ | ✓ | - | - |
| Viewing dbt Job Settings | ✓ | ✓ | ✓ | ✓ | - |
| View dbt job execution history | ✓ | ✓ | ✓ | ✓ | - |
| Add new dbt Job Setting | ✓ | ✓ | ✓ | ✓ | ✓ |