Contents x
    Release Notes - August 2022
      • Print
      • PDF
      Contents

      Release Notes - August 2022

        • Print
        • PDF
        Article summary

        Hello! Release information for August 2022!

        Data Catalog

        Basic metadata is displayed on the column information screen

        The basic metadata items "logical name" and "description" can now be selected whether or not to be displayed in the column list.
        image

        The above can be set from "TROCCO Basic Metadata Settings" in "Data Catalog Settings".
        image

        Download query execution results in CSV

        Query Editor can now download query execution results in CSV format.
        image

        Query execution result download file size limit

        The maximum file size of the CSV file that can be downloaded is 10 MB.
        Lines that exceed the file size limit are truncated.

        Edit metadata in the Markdown editor

        The "Description" field of basic metadata and any field value of user-defined metadata can now be edited in the Markdown editor.
        image

        image

        User-defined metadata fields can be configured in the "Template for User-defined Metadata" under "Data Catalog Setting".
        Specify Text(Markdown) as the data type for the fields for which you want to use the Markdown editor.
        image

        Help Document Publication

        Help documentation on Data Catalog is now available.
        For more information, see About Data Catalog.

        dbt linkage

        dbt integration functionality has been released 🎉 .

        • We provide a runtime environment for the OSS version of dbt and GitHub integration.
        • The ETL to data modeling flow can be defined on the TROCCO Workflow function.
        • As of the end of August 2022, the following three dbt Git repository adapters are supported
          • Google BigQuery
          • Snowflake
          • Amazon Redshift
        • Please refer to the press release for details, including future release plans.

        Managed ETL

        Managed ETL is now available 🎉 .

        Managed ETL is an evolution of the existing batch registration feature.
        In addition to batch import of tables on the data source side, the new system automatically detects Created/Dropped Tables on the data source side and notifies Slack, and allows for easy batch execution of ETL Jobs on the workflow function.

        Supported Connector for Managed ETL

        As of the end of August 2022, only Data Source MySQL and Data Destination Google BigQuery are supported.

        Adding Managed ETL Configuration

        Created/Dropped Tables can be detected automatically.

        In STEP 3 "Notification of Created/Dropped Tables Detection Results and Batch Label Settings", you can configure the settings for notification of Created/Dropped Tables detection results.
        image

        Managed ETL Configuration List

        You can see the list of Managed ETL Configurations you added earlier.
        In addition to the detailed screen for each Managed ETL Configuration, you can also go to the detailed screen for individual ETL Configurations under each Managed ETL Configuration with a single click.
        image

        Managed ETL Configuration Details Screen

        For the relevant Managed ETL Configuration, you can manually add tables or delete individual ETL Configurations under your control.
        image

        A dialog box is also displayed when Created/Dropped Tables are detected.
        image

        Workflow

        Batch Execution of ETL Jobs under Managed ETL Configuration

        TROCCO Managed ETL Configuration" has been added as a task that can be added to the Flow Editor in Workflow Definitions.
        This allows ETL Jobs under each Managed ETL Configuration to be executed in one go.
        image

        image

        image

        ETL Configuration

        HubSpot added to Data Destination🎉.

        A merge can be performed on a contact object.
        image

        TROCCO added to Data Source 🎉.

        Data related to "User-Defined Metadata" and "Column Setting Reference List" held in TROCCO's Data Catalog can be transferred.
        image

        Data Source Marketo Target for Folders

        Data Source Marketo can now retrieve the folder's data.
        image

        Added Ability to Support Custom Variables 🎉.

        Custom Variables are now supported in the following functions

        • ETL Configuration Filter Settings in STEP2 "Data Preview and Advanced Settings" Filter Setting, Filter Value
          image

        • Keys and values in the "HTTP Header" of the Data Source HTTP (S)
          image

        UI・UX

        ETL Configuration List

        Added "Author" to filter field.

        ETL Configurations listed can now be narrowed down by "Author".
        image

        Filter item "name (partial match)" supports AND search.

        AND searches can now be performed by entering entries separated by spaces.

        • When searching for a single word
          image
        • When an AND search is performed on two words
          image

        Labels can now be added to selected ETL Configurations in batches.

        By clicking the buttons in the order described within the screen below, the Add Label Bulk screen will appear.
        image

        Select the Label of your choice and click Add.
        image

        Account/User Screen

        Displays the authentication method for each user

        You can check if 2-step verification and SAML verification are enabled respectively.
        image

        In addition, the following modifications and others have been added

        • When line_item is selected as the report type in Data Source Twitter Ads, campaign_id is now included in the transfer content.

        **The contents of this release are described above. **
        **Please feel free to contact our Customer Success Representative if you have a release you are interested in. **
        Happy Data Engineering!

        Was this article helpful?
        What's Next