BigQuery Data Check
  • 07 Dec 2022
  • PDF

BigQuery Data Check

  • PDF

Article summary

Note

This is a machine-translated version of the original Japanese article.
Please understand that some of the information contained on this page may be inaccurate.

summary

This is a help page about BigQuery data check that can be set in a workflow definition.

What is BigQuery Data Check?

It is one of the tasks set in the workflow, and it is a function that matches the data on the BigQuery table with the error condition, and if the condition is met, the corresponding task is an error.

For example, it can be used to count a specific string in a column and consider it as a data duplication error if the result is 2 or more.

How to set it up

  1. Specify the connection information of BigQuery registered in advance.
    This connection information must have access to the table being data-checked.
    You can also create a new connection information for BigQuery from Add Connection Information.

datacheck.png

  1. Enter a query to perform a data check.
    The data to be matched is specified by writing a query in a Select statement that results in one row and one column.
    Queries can embed custom variables. Learn more about custom variables.
    If you click Run Preview, you can check the results of the written query on the spot.

query.png

  1. Specify the error condition.
  • The query result is judged to be an error under conditions equal to or greater than the specified number or less.
  • If the query result is NULL, you can choose to fail or succeed.

errorconditions.png


Was this article helpful?