Data Catalog Settings
- 24 Jan 2023
- 4 Minutes to read
-
Print
-
DarkLight
Data Catalog Settings
- Updated on 24 Jan 2023
- 4 Minutes to read
-
Print
-
DarkLight
This is a machine-translated version of the original Japanese article.
Please understand that some of the information contained on this page may be inaccurate.
summary
In Data Catalog Settings, you can configure various settings related to the entire Data Catalog.
Click Data Catalog Settings in the upper right corner of the screen.
The Data Catalog Settings screen appears.
supplement
For details on the various metadata that is subject to Data Catalog settings, refer to the table information.
Trocco basic metadata settings
Choose whether to display each item in the basic metadata as a field in the column information on the Table Info tab.
| Item Name | Default |
|---|---|
| Logical name | show |
| explanation | Do not display |
Templates for user-defined metadata
You can create a template that summarizes each item of user-defined metadata.
- If you want to create a template, click New.
- If you want to edit an existing template, click the icon in the red frame.
Precautions
- You can create only one user-defined metadata template for tables and columns.
- Once you add a field, you can't delete it later.
- The settings in Make input required cannot be changed later.
| Item | nameRequired | remarks |
|---|---|---|
| Template name | ✓ | Enter a template name. |
| Template Type | ✓ | Select either for the table or for the column. |
| Field Name | ✓ | Enter a field name. |
| hint | - | The contents entered in this field are displayed on the user-defined metadata input screen. |
| data type | ✓ | Select String, Text(Markdown), Integer, or Boolean. If you select Integer or Boolean, you cannot change to any other data type later. If you select String or Text(Markdown), you cannot change it to Integer or Boolean later. |
| Make input mandatory | - | If this item is enabled, you must enter a value for the field when editing user-defined metadata to save it. This item cannot be changed later. |
Metadata Import
You can use a CSV file to import basic metadata values.
Target of metadata import using CSV file
Metadata import using CSV files only supports basic metadata.
The following metadata import is not supported.
- User-defined metadata
- Metadata about the datastore
- Metadata about the data source
Character code of CSV file
UTF-8 is recommended.
In the case of character codes other than UTF-8, some characters may not be read.
File size limits for CSV uploads
The maximum file size of the CSV file you can upload is 100MB.
If metadata has already been entered
If metadata values have already been populated, the imported values overwrite the existing values.
CSV file format
The CSV file used for metadata import must be configured as follows.
- 1st line (header row): Metadata item name
- From the second line onwards: the value of the metadata you want to import
Depending on your datastore and the metadata to be imported, the contents in the header row of the CSV file will differ.
Please refer to the following according to the usage status of the data catalog.
If you are using BigQuery: Header row
For the first line of the CSV file, enter the following:
- Table basic metadata
project_id,dataset_id,table_id,logical_name,description
- Column base metadata
project_id,dataset_id,table_id,column_id,logical_name,description
If you are using BigQuery: from line 2 onwards
Please enter the second and subsequent lines of the CSV file based on the table below.
| column namesrequired | information | |
|---|---|---|
project_id | ✓ | Project name |
dataset_id | ✓ | Dataset name |
table_id | ✓ | Table name |
column_id | ✓ | Column name associated with the table (required only when importing column basic metadata) |
logical_name | - | The value you want to enter in the logical name field |
description | - | The value you want to enter in the description field |
If you're using Snowflake: Header row
For the first line of the CSV file, enter the following:
- Table basic metadata
database_name, schema_name, table_name, logical_name, description
- Column base metadata
database_name, schema_name, table_name, column_name, logical_name, description
If you are using Snowflake: Line 2 onwards
Please enter the second and subsequent lines of the CSV file based on the table below.
| column namesrequired | information | |
|---|---|---|
database_name | ✓ | Database name |
schema_name | ✓ | Schema name |
table_name | ✓ | Table name |
column_name | ✓ | Column name associated with the table (required only when importing column basic metadata) |
logical_name | - | The value you want to enter in the logical name field |
description | - | The value you want to enter in the description field |
Example of input of CSV file
The following is an example of a CSV file when importing table basic metadata when using BigQuery.
blank
- It is also possible to leave the values of the logical name and description fields blank.
- Either value can be blank, or both values can be blank.
- If you want a value to be blank, you do not need to type the value immediately after the comma in the field.
Upload Procedure
Select the import target and click Upload CSV.
Select a CSV file that follows the aforementioned format
Click Run Import
If the import is successful, the values in the basic metadata are overwritten as follows:
You can check the import results in the import history.
Datastore integration management
In Datastore Linkage Management, you can check the administrator of each datastore and reauthenticate your account.
Also, if you are using multiple datastores as your data catalog, you can set which datastores are displayed by default.
Reauthenticate your account
Click Reauthenticate.
For more information on the following steps, please refer to Account Verification.
Reauthenticate your Data Catalog administrator account
If your Data Catalog administrator reauthorizes your account, you may no longer see table data that was previously displayed in the Data Catalog. Please be careful.
This is because the data displayed on the data catalog is retrieved based on the data catalog administrator's access to the data store.
For details on Data Catalog administrators, refer to Specifications for Metadata Retrieval.
New datastore integration
If you want to use Data Catalog with a new data store, click Link.
For more information on the following steps, please refer to Account Verification.
Datastore default display settings
If you are using a data catalog with multiple data stores, the default data store displayed on the Data Catalog screen displays the default.
Note that only Data Catalog administrators can change the data store displayed by default.
If you want to change it, click Set as default.
If you want to display the data catalog screen of a data store that is not the default display
Click the link of the datastore name to display the data catalog screen of the corresponding datastore.
Was this article helpful?