Datamart - BigQuery
- 07 Dec 2022
- 1 Minute to read
-
Print
-
DarkLight
Datamart - BigQuery
- Updated on 07 Dec 2022
- 1 Minute to read
-
Print
-
DarkLight
This is a machine-translated version of the original Japanese article.
Please understand that some of the information contained on this page may be inaccurate.
summary
This page is a help page for setting up datamart definitions using BigQuery on Google Cloud Platform.
Setting items
STEP1 Basic settings
| Item | namerequireddefault | valuecontent | |
|---|---|---|---|
| BigQuery connection information | Yes | - | Select the connection information for BigQuery. For details, please refer to the connection information of BigQuery. |
| Custom variables | No | - | Custom variables set here can be used for queries and table names. For more information, refer to About Custom Variables. |
Query settings
| Item | namerequireddefault | valuecontent | |
|---|---|---|---|
| Query Execution Mode | Yes | Data Transfer Mode | You can choose from the following two modes. ・Data transfer mode By simply specifying the SQL and the output destination table, you can easily change and append to the table. ・Free description mode You can freely execute arbitrary queries (DDL, DELETE, INSERT, etc.) against the connected DWH. |
| query | Yes | - | Enter the SQL to retrieve the transferred data from BigQuery. Custom variables can also be used to dynamically determine the setting value during trocco data transfer. Note that the query should not include the account name in the FROM clause and write the query in the format "FROM dataset_name.table_name". |
In addition, the following settings can only be specified when the query execution mode is data transfer mode.
| Item | namerequireddefault | valuecontent | |
|---|---|---|---|
| Destination dataset | Yes | - | Enter the dataset name to output data to. The dataset name must consist only of letters, numbers, and underscores. |
| Destination table | Yes | - | Enter the name of the table to which you want to output data. Table names can consist of Unicode-equivalent letters, numbers, underscores, hyphens, spaces, etc. For details on the characters that can be entered in table names, please refer to the official BigQuery documentation. |
| Write Mode | Yes | - | You can choose the following writing methods: ・P.S. ・Overwrite |
| Partitioning the destination table | No | - | You can choose from the following partitionings: ・ Divided by import time ・Split by field |
| Partitioning Fields | No | - | The column to use for partitioning. Specify a column of type DATE, TIMESTAMP, or DATETIME. |
| Partitioning Type | No | - | You can choose from the following partitioning types: ・Every 1 hour ・Every 1 day |
Required Permissions
The permissions required to use this service are as follows.
- bigquery.datasets.get
- bigquery.jobs.create
- bigquery.tables.create
- bigquery.tables.delete
- bigquery.tables.get
- bigquery.tables.getData
- bigquery.tables.list
- bigquery.tables.update
- bigquery.tables.updateData
Was this article helpful?