Contents x
    Data Catalog Settings
    • 24 Jan 2023
    • 4 Minutes to read
    • Print
    • Dark
      Light
    Contents

    Data Catalog Settings

    • Updated on 24 Jan 2023
    • 4 Minutes to read
    • Print
    • Dark
      Light
    Article Summary

    Note

    This is a machine-translated version of the original Japanese article.
    Please understand that some of the information contained on this page may be inaccurate.

    summary

    In Data Catalog Settings, you can configure various settings related to the entire Data Catalog.
    Click Data Catalog Settings in the upper right corner of the screen.
    image

    The Data Catalog Settings screen appears.
    image

    supplement

    For details on the various metadata that is subject to Data Catalog settings, refer to the table information.

    Trocco basic metadata settings

    Choose whether to display each item in the basic metadata as a field in the column information on the Table Info tab.

    Item NameDefault
    Logical nameshow
    explanationDo not display

    Templates for user-defined metadata

    You can create a template that summarizes each item of user-defined metadata.

    • If you want to create a template, click New.
    • If you want to edit an existing template, click the icon in the red frame.
      image
    Precautions
    • You can create only one user-defined metadata template for tables and columns.
    • Once you add a field, you can't delete it later.
    • The settings in Make input required cannot be changed later.
    ItemnameRequiredremarks
    Template name✓Enter a template name.
    Template Type✓Select either for the table or for the column.
    Field Name✓Enter a field name.
    hint-The contents entered in this field are displayed on the user-defined metadata input screen.
    data type✓Select String, Text(Markdown), Integer, or Boolean.
    If you select Integer or Boolean, you cannot change to any other data type later. 
    If you select String or Text(Markdown), you cannot change it to Integer or Boolean later. 
    Make input mandatory-If this item is enabled, you must enter a value for the field when editing user-defined metadata to save it.
    This item cannot be changed later. 

    Metadata Import

    You can use a CSV file to import basic metadata values.
    image

    Target of metadata import using CSV file

    Metadata import using CSV files only supports basic metadata.
    The following metadata import is not supported.

    • User-defined metadata
    • Metadata about the datastore
    • Metadata about the data source
    Character code of CSV file

    UTF-8 is recommended.
    In the case of character codes other than UTF-8, some characters may not be read.

    File size limits for CSV uploads

    The maximum file size of the CSV file you can upload is 100MB.

    If metadata has already been entered

    If metadata values have already been populated, the imported values overwrite the existing values.

    CSV file format

    The CSV file used for metadata import must be configured as follows.

    • 1st line (header row): Metadata item name
    • From the second line onwards: the value of the metadata you want to import

    Depending on your datastore and the metadata to be imported, the contents in the header row of the CSV file will differ.
    Please refer to the following according to the usage status of the data catalog.

    If you are using BigQuery: Header row

    For the first line of the CSV file, enter the following:

    • Table basic metadata
      • project_id,dataset_id,table_id,logical_name,description
    • Column base metadata
      • project_id,dataset_id,table_id,column_id,logical_name,description

    If you are using BigQuery: from line 2 onwards

    Please enter the second and subsequent lines of the CSV file based on the table below.


    		column namesrequiredinformation
    project_id✓Project name
    dataset_id✓Dataset name
    table_id✓Table name
    column_id✓Column name associated with the table (required only when importing column basic metadata)
    logical_name-The value you want to enter in the logical name field
    description-The value you want to enter in the description field

    If you're using Snowflake: Header row

    For the first line of the CSV file, enter the following:

    • Table basic metadata
      • database_name, schema_name, table_name, logical_name, description
    • Column base metadata
      • database_name, schema_name, table_name, column_name, logical_name, description

    If you are using Snowflake: Line 2 onwards

    Please enter the second and subsequent lines of the CSV file based on the table below.


    		column namesrequiredinformation
    database_name✓Database name
    schema_name✓Schema name
    table_name✓Table name
    column_name✓Column name associated with the table (required only when importing column basic metadata)
    logical_name-The value you want to enter in the logical name field
    description-The value you want to enter in the description field

    Example of input of CSV file

    The following is an example of a CSV file when importing table basic metadata when using BigQuery.
    image

    blank
    • It is also possible to leave the values of the logical name and description fields blank.
    • Either value can be blank, or both values can be blank.
    • If you want a value to be blank, you do not need to type the value immediately after the comma in the field.
      image

    Upload Procedure

    1. Select the import target and click Upload CSV.
      image

    2. Select a CSV file that follows the aforementioned format

    3. Click Run Import
      image

    If the import is successful, the values in the basic metadata are overwritten as follows:
    image

    You can check the import results in the import history.
    image

    Datastore integration management

    In Datastore Linkage Management, you can check the administrator of each datastore and reauthenticate your account.
    Also, if you are using multiple datastores as your data catalog, you can set which datastores are displayed by default.

    image

    Reauthenticate your account

    Click Reauthenticate.
    For more information on the following steps, please refer to Account Verification.

    Reauthenticate your Data Catalog administrator account

    If your Data Catalog administrator reauthorizes your account, you may no longer see table data that was previously displayed in the Data Catalog. Please be careful.

    This is because the data displayed on the data catalog is retrieved based on the data catalog administrator's access to the data store.
    For details on Data Catalog administrators, refer to Specifications for Metadata Retrieval.

    New datastore integration

    If you want to use Data Catalog with a new data store, click Link.
    For more information on the following steps, please refer to Account Verification.

    Datastore default display settings

    If you are using a data catalog with multiple data stores, the default data store displayed on the Data Catalog screen displays the default.

    image

    Note that only Data Catalog administrators can change the data store displayed by default.
    If you want to change it, click Set as default.

    If you want to display the data catalog screen of a data store that is not the default display

    Click the link of the datastore name to display the data catalog screen of the corresponding datastore.
    image

    Was this article helpful?
    What's Next