Datamart - BigQuery
  • 07 Dec 2022
  • Dark
  • PDF

Datamart - BigQuery

  • Dark
  • PDF

Article Summary


This is a machine-translated version of the original Japanese article.
Please understand that some of the information contained on this page may be inaccurate.


This page is a help page for setting up datamart definitions using BigQuery on Google Cloud Platform.

Setting items

STEP1 Basic settings

BigQuery connection informationYes-Select the connection information for BigQuery.
For details, please refer to the connection information of BigQuery.
Custom variablesNo-Custom variables set here can be used for queries and table names.
For more information, refer to About Custom Variables.

Query settings

Query Execution ModeYesData Transfer ModeYou can choose from the following two modes.
・Data transfer mode
By simply specifying the SQL and the output destination table, you can easily change and append to the table.
・Free description mode
You can freely execute arbitrary queries (DDL, DELETE, INSERT, etc.) against the connected DWH.
queryYes-Enter the SQL to retrieve the transferred data from BigQuery.
Custom variables can also be used to dynamically determine the setting value during trocco data transfer.
Note that the query should not include the account name in the FROM clause and write the query in the format "FROM dataset_name.table_name".

In addition, the following settings can only be specified when the query execution mode is data transfer mode.

Destination datasetYes-Enter the dataset name to output data to.
The dataset name must consist only of letters, numbers, and underscores.
Destination tableYes-Enter the name of the table to which you want to output data.
Table names can consist of Unicode-equivalent letters, numbers, underscores, hyphens, spaces, etc.
For details on the characters that can be entered in table names, please refer to the official BigQuery documentation.
Write ModeYes-You can choose the following writing methods:
Partitioning the destination tableNo-You can choose from the following partitionings:
・ Divided by import time
・Split by field
Partitioning FieldsNo-The column to use for partitioning.
Specify a column of type DATE, TIMESTAMP, or DATETIME.
Partitioning TypeNo-You can choose from the following partitioning types:
・Every 1 hour
・Every 1 day

Required Permissions

The permissions required to use this service are as follows.

  • bigquery.datasets.get
  • bigquery.tables.create
  • bigquery.tables.delete
  • bigquery.tables.get
  • bigquery.tables.getData
  • bigquery.tables.list
  • bigquery.tables.update
  • bigquery.tables.updateData

Was this article helpful?