- 24 Jan 2023
- Print
- DarkLight
- PDF
dbt Job Configuration
- Updated on 24 Jan 2023
- Print
- DarkLight
- PDF
This is a machine-translated version of the original Japanese article.
Please understand that some of the information contained on this page may be inaccurate.
summary
This article describes dbt job settings.
List of setting items
Common settings
Item | namerequireddefault | valuecontent | |
---|---|---|---|
name | ✓ | - | Enter a name for the dbt job configuration. |
memorandum | - | - | You can enter notes for dbt job settings. |
dbt repository | ✓ | - | Select the dbt Git repository that is linked to trocco. |
Execution Command | ✓ | dbt build | Select the Run command. You can also set options and their values. For details, refer to the corresponding dbt command. |
Number of threads | ✓ | 1 | Sets the number of models that dbt can process at one time. The maximum number of threads that can be set is 16. For more information on the number of threads, see understanding-threads. |
When the dbt command is executed, it first creates a directed cycle graph (DAG) of the links between the models.
The number of threads specifies the maximum number of graph paths that dbt can process at one time.
Note that increasing the number of threads generally shortens execution time, but increases the load on the adapter.
Therefore, the optimal number of threads depends on the data warehouse and its configuration.
Settings for each adapter
The following settings differ depending on the adapter set in the dbt Git repository.
If the adapter is Google BigQuery
Item nameRequired | content | |
---|---|---|
Google BigQuery connection information | ✓ | Select the Google BigQuery connection information. For details, please refer to the connection information of BigQuery. |
Dataset | ✓ | Enter the dataset name to output data to. |
If the adapter is Snowflake
Item nameRequired | content | |
---|---|---|
Snowflake connection information | ✓ | Select the connection information for Snowflake. See Snowflake Connection Information for more information. You can use either "User Password Authentication" or "Key Pair Authentication" as the authentication method for the connection information. |
Warehouse | ✓ | Enter the name of the warehouse that you want to use for processing. |
database | ✓ | Enter the name of the database to which you want to output data. |
schema | ✓ | Enter the schema name of the data destination. |
roll | - | Enter a role name that summarizes the permissions required to use Snowflake. For more information about roles, see Snowflake Connection Information Supplement. |
If the adapter is Amazon Redshift
Item nameRequired | content | |
---|---|---|
Redshift connection information | ✓ | Choose your Amazon Redshift connection information. For more information, see Amazon Redshift connection information. |
database | ✓ | Enter the name of the database to which you want to output data. |
schema | ✓ | Enter the schema name of the data destination. |
Corresponding dbt command
The following are the dbt commands supported by trocco:
For more information about the dbt command, see dbt Command reference.
trocco provides optional arguments and custom variables for dbt commands.
command namesoverview | |
---|---|
dbt build | Build all resources in the project. |
dbt deps | Install the package. |
dbt run | Run the model file. |
dbt seed | Transfer the CSV file to the adapter. |
dbt snapshot | Run the snapshot job. |
dbt test | Run the test. |
dbt run-operation | Run the macro. Please enter the macro name in the box below. ![]() |
Various operations of dbt job settings
DBT job settings list screen
DBT job setting details screen
dbt job settings cannot be replicated.
(1) New addition
Add a new dbt job setting. The new screen appears.
(2) Edit
Edit the dbt job settings. The details screen appears.
(3) Execution
Run the dbt job. The execution screen appears.
(4) Delete
Delete the dbt job configuration.
Once you delete the dbt job settings, you cannot undo them. Please be careful.
(5) Author
When using the team function, the creator of the dbt job setting has the same operation privileges as a user with the administrator role for the setting.
For more information, see "Role permissions" below.
Role permissions
DBT job settings are treated as resources in Trocco.
- See Defining Resources: About Team Features.
- Operation permissions for resources: See Resource groups.
Operation\ | RolesAdministratorEditorOperatorReaderNo | role | |||
---|---|---|---|---|---|
Editing dbt job settings | ✓ | ✓ | - | - | - |
Deleting dbt job settings | ✓ | ✓ | - | - | - |
Running a dbt job | ✓ | ✓ | ✓ | - | - |
Viewing dbt job settings | ✓ | ✓ | ✓ | ✓ | - |
Viewing dbt job execution history | ✓ | ✓ | ✓ | ✓ | - |
Adding a new dbt job setting | ✓ | ✓ | ✓ | ✓ | ✓ |