Contents x
    Datamart - BigQuery
    • 07 Dec 2022
    • Print
    • Dark
      Light
    • PDF
    Contents

    Datamart - BigQuery

    • Updated on 07 Dec 2022
    • Print
    • Dark
      Light
    • PDF
    Article Summary

    Note

    This is a machine-translated version of the original Japanese article.
    Please understand that some of the information contained on this page may be inaccurate.

    summary

    This page is a help page for setting up datamart definitions using BigQuery on Google Cloud Platform.

    Setting items

    STEP1 Basic settings

    Itemnamerequireddefault
    		valuecontent
    BigQuery connection informationYes-Select the connection information for BigQuery.
    For details, please refer to the connection information of BigQuery.
    Custom variablesNo-Custom variables set here can be used for queries and table names.
    For more information, refer to About Custom Variables.

    Query settings

    Itemnamerequireddefault
    		valuecontent
    Query Execution ModeYesData Transfer ModeYou can choose from the following two modes.
    ・Data transfer mode
    By simply specifying the SQL and the output destination table, you can easily change and append to the table.
    ・Free description mode
    You can freely execute arbitrary queries (DDL, DELETE, INSERT, etc.) against the connected DWH.
    queryYes-Enter the SQL to retrieve the transferred data from BigQuery.
    Custom variables can also be used to dynamically determine the setting value during trocco data transfer.
    Note that the query should not include the account name in the FROM clause and write the query in the format "FROM dataset_name.table_name".

    In addition, the following settings can only be specified when the query execution mode is data transfer mode.

    Itemnamerequireddefault
    		valuecontent
    Destination datasetYes-Enter the dataset name to output data to.
    The dataset name must consist only of letters, numbers, and underscores.
    Destination tableYes-Enter the name of the table to which you want to output data.
    Table names can consist of Unicode-equivalent letters, numbers, underscores, hyphens, spaces, etc.
    For details on the characters that can be entered in table names, please refer to the official BigQuery documentation.
    Write ModeYes-You can choose the following writing methods:
    ・P.S.
    ・Overwrite
    Partitioning the destination tableNo-You can choose from the following partitionings:
    ・ Divided by import time
    ・Split by field
    Partitioning FieldsNo-The column to use for partitioning.
    Specify a column of type DATE, TIMESTAMP, or DATETIME.
    Partitioning TypeNo-You can choose from the following partitioning types:
    ・Every 1 hour
    ・Every 1 day

    Required Permissions

    The permissions required to use this service are as follows.

    • bigquery.datasets.get
    • bigquery.jobs.create
    • bigquery.tables.create
    • bigquery.tables.delete
    • bigquery.tables.get
    • bigquery.tables.getData
    • bigquery.tables.list
    • bigquery.tables.update
    • bigquery.tables.updateData
    Was this article helpful?
    What's Next