Release Notes

Here are the trocco release notes.

2023-01-23

Data Catalog

• The Data Catalog feature for Snowflake is now available.
  • You can create a Data Catalog for data associated with your Snowflake account.
  • There are some features that are not supported in the Snowflake version. For details, refer to the list of functions supported by each data store.

Workflow Definition

• You can now duplicate a workflow definition.
  • You can duplicate it from the hamburger menu at the top right of the workflow definition details screen.
    

API Updates

Transfer toFacebook offline conversions

• The Facebook API used for the above transfer destination has been updated from v14 to v15.
• See the Meta for Developers documentation for the new version.

2023-01-16

Forwarding settings

Transferred from Google Ads

• The following resource types (report types) have been added as retrievable resource types.
  • ad_group_asset
  • customer_asset

DBT Integration

• The supported version has been updated from v1.2 to v1.3.

Data Catalog

• A link to the column reference list has been added to the table information screen.
  • Click the link to display a list of column references defined in the table.
  • For more information about column references, see Column references.

2023-01-10

API Updates

Transfer fromFacebook Ad Insights

• The Facebook API used for the above transfer source has been updated from v14 to v15.
• See the Meta for Developers documentation for the new version.

Connection Information

PostgreSQL

• You can now choose which version of the JDBC driver connects to the PostgreSQL server.
  • Select the driver version that matches your version of PostgreSQL.
    • For PostgreSQL 8.2 and above:JDBC Driver 42.5.1
    • For PostgreSQL 8.2 and below:9.4 1205 JDBC 41

2022-12-26

API Updates

Forwarded fromFacebook lead ads and forwardedfromFacebook Ads creatives

• The Facebook API used for the above transfer source has been updated from v14 to v15.
• See the Meta for Developers documentation for the new version.

Workflow Definition

• On the workflow job details screen, you can now easily display the execution log of each task.
  • Click a task on the flow to view the task execution log.

2022-12-19

Notice

Forwarding source LINE ads

Other UI/UX

Notification settings

• For notifications about the following jobs, notifications are now also sent when a job is skipped:
  • Transfer Jobs
  • Sync Jobs
  • Workflow Jobs

Managed Data Transfer

• You can now easily see which workflows are using your managed transfer settings.
  • On the managed transfer setting details screen, under Workflows where this setting is used, a list of workflows that incorporate the managed transfer setting is displayed.

2022-12-12

Connection Information

Amazon S3

• Added support for connecting to Amazon S3 using authentication by IAM roles.
  • You can connect to Amazon S3 without placing your AWS access key and AWS secret key on trocco.
  • In the Amazon S3 connection information, you can select IAM roles as the AWS authentication method.

Forwarding settings

Transfer fromAppsFlyer

• A new source AppsFlyer has been added.
  • For details on retrievable report types, fields, various restrictions, etc., please refer to Transfer Source - AppsFlyer.

Forward to HubSpot

• Added support for updating a contact's subscription.
  • You can select for thesubscription object type.
  • For details such as the schema of the transferred data, refer to Updating Subscriptions.

Other UI/UX

Team features

• On the resource group details screen, "dbt job settings" and "dbt repository" are now displayed.

2022-12-05

Forwarding settings

Forward to Box

• Destination Box has been newly added.
  • You can transfer files to Box by specifying the folder path.

Workflow Definition

• Child workflows embedded in a workflow can now be looped.
  • You can add a workflow as a task on the flow edit screen and set up loop execution by editing the task.

Other UI/UX

Forwarding settings list

• In the transfer settings list, you can now delete schedules from the selected transfer settings in bulk.

Data Catalog

• You can now change the width of the Data Catalog query editor.
  • You can change the width of the sidebar by dragging it left or right to move it left.

2022-11-28

Connection Information

MongoDB

• For MongoDB's replica set, you can now select the nodes (members) you want to read.
  • In "Import Settings", you can select from the following five items.
    • primary
    • primaryPreferred
    • secondary
    • secondaryPreferred
    • nearest
  • For more information about import settings, please refer to the help documentation.

Forwarding settings

Transfer from TikTok Ads

• The dimensions retrieved have been expanded. The dimensions supported this time are as follows.
  • Basic data metrics
    • reach
    • frequency
    • result
  • Video play metrics
    • video_watched_2s
    • video_watched_6s
    • average_video_play
  • Page Event Metrics
    • web_event_add_to_cart
    • on_web_order
    • initiate_checkout
    • add_billing
    • page_event_search

2022-11-21

API Updates

Twitter Ads

• The API version has been updated from v11 to v12.
• For more information about the new version, see the Twitter Developers documentation.

Facebook offline conversions

• The Facebook API used for forwarding Facebook offline conversions has been updated from v14 to v15.
• See the Meta for Developers documentation for the new version.

Forwarding settings

Forward to HubSpot

• The target object types for transfer destinations have been expanded. The object types added this time are as follows.
  • company
  • deal
  • product
  • ticket
  • line_item
  • quote
• An append (INSERT) has been added to the transfer mode.
• You can now specify a UPSERT key for UPSERTin transfer mode.
  • For more information about UPSERT keys, see Transfer to - HubSpot.

Forwarding/forwarding fromGoogle BigQuery

• Under Dataset Locations, you can now select the following three new locations.
  • Europe-West8 (Milan)
  • Europe-West9 (Paris)
  • Europe-Southwest1 (Madrid)

Destination Snowflake

• JSON columns are now populated as VARIANT by default.
  • Until now, json columns were imported as VARCHAR types.
  • If you want to import a json type column as before, make the following settings.
    • STEP2: In "Column Settings" in the output option, select as theVARCHAR data type.

Other UI/UX

Forwarding settings list

• In the forwarding settings list, you can now add schedules to the selected transfer settings in bulk.

Data Catalog

• You can now change the width of the sidebar in the Data Catalog.
  • You can change the width of the sidebar by dragging it left or right to move it left.

Team features

• In the connection information list on the resource group details screen, the service name of the connection information is now displayed in the form linked to each connection information.

2022-11-14

Datamarts

• Datamarts - Snowflake now supports output to databases with hyphens in their names.
  • When you select the data transfer mode in the query execution mode, you can now specify a database with a hyphen in the output database.

2022-11-07

API Updates

Yahoo! JAPAN Ads

The API version has been updated from v8 to v9.
For more information about the new version, please refer to the API Reference.

• Yahoo! JAPAN Ads Display Ads API Reference v9
• Yahoo! JAPAN Ads Search Ads API Reference v9

Forwarding settings

Transfer from Google Drive

• You can now also transfer files on shared drives.

Transfer toGoogle Drive

• It was also available for transfer to folders on shared drives.

DBT Integration

• Added as adbt run-operationRun Job Settings command.
  • You can call a macro by entering the macro name.

2022-10-24

Workflow

• You can now loop through jobs in workflows based on Amazon Redshift query results.
  • You can set the expansion value of a custom variable in a loop execution based on the results of an Amazon Redshift query.
  • By storing the values of the table you want to deploy in a table in Amazon Redshift, you can define a workflow in which the deployment value changes each time it is executed.

Data Catalog

• Basic metadata values can now be imported using a CSV file.
  • You can import from "Metadata Import" in Data Catalog settings.
  • For details, refer to Metadata Import.

2022-10-17

Connection Information

• Added support for connecting to Oracle Autonomous Database.
  • You can now upload a wallet file when you select Use tnsnames.ora file in "Connection Method" of the Oracle Database connection information.
  • By uploading the wallet file, you can connect to Oracle Autonomous Database.

Data Catalog

• Summary statistics now display the minimum and maximum values of type for date and time.
  • You can view summary statistics in column information and previews.

2022-10-11

Forwarding settings

• For split tables supported by the STEP2 and output options of the transfer destination BigQuery, it is now possible to divide the table in more granular time units.
  • Before the change, it only supported splitting by date.
  • With this change, you can now choose from the following four units for table partitioning.
    • Every 1 hour
    • Every 1 day
    • Every 1 month
    • Every 1 year

2022-10-03

Connection Information

• PostgreSQL connection information now supports connecting via AWS Systems Manager Session Manager.

Forwarding settings

• The File Storage connector now supports Microsoft Excel files (xlsx and xls) as input file formats.
  Supported connectors are as follows:
  • Forwarding from - Amazon S3
  • Source - Azure Blob Storage
  • Transfer from - Box
  • Transfer source - FTP/FTPS
  • Forwarding from - Google Cloud Storage
  • Forward from - Google Drive
  • Forwarding from - HTTP(S)
  • Forward from - SFTP
  • Source - Local Files

Workflow

• For automatic retries when a workflow fails, you can now specify the time interval before the next retry execution.
  • You can specify the time when the number of retries is set to 1 or more on the workflow setting screen.

Data Catalog

• For columns generated using the TroCCO data mart function, metadata about the data source of the generating column can now be automatically inherited.
  However, in order to automatically take over metadata, the following conditions are met.
  • In the data mart definition, the column value is not processed.
  • Data transfer mode is used as the query execution mode of the data mart definition.

2022-09-20

API Updates

Google Ads

• The API version has been updated to v11.
• For more information about the new version, see the Google Ads API release notes.

Managed transfer settings

• The edit screen for managed transfer settings has been expanded.
  • It is now possible to change the settings related to the transfer source and destination of registered transfer settings and common settings at once.

Other UI/UX

Forwarding settings list

• You can now narrow down the forwarding settings by whether or not you have notification settings.
• You can now narrow down the forwarding settings by whether or not you have a schedule.
• You can now remove labels attached to selected transfer settings in bulk.

2022-09-12

API Updates

Yahoo! JAPAN Ads

• The API version has been updated to v8.
  • Due to API updates, the old indicator has been deprecated.
  • From now on, if a column containing "(old)" is specified in the column name, the new column will be automatically retrieved.
• For details on the new version, please refer to the Yahoo! JAPAN Ads API v8 System Release Completion Announcement.

Criteo

• API version has been updated to v2022.07.
• For details on the new version, please refer to Version 2022.07 release notes.

Data Catalog

• In the Table Info tab, a button to open the query editor has been added.
  • Click the button to launch the query editor.
  • Displays a query in which the table that was open in the Table Info tab is specified in the FROM clause.
• You can now re-authenticate your Google account in the data catalog settings.

2022-09-05

Forwarding settings

• trocco has been added as a transfer source.
  • You can transfer data related to "user-defined metadata" and "column reference list" held in TroCCCO's data catalog.
• In the transfer destination Marketo, you can now specify a static list ID.

Managed transfer settings

• The Slack notification content for table increase / decrease detection now includes the name of the detected table.
  • Previously, the number of tables added and deleted was displayed.
  • In the future, in addition to the number of tables added and deleted, the name of the table will also be displayed.

DBT Integration

• Custom variables are now available in the option values of the Run Job Settings command.

Data Catalog

• You can now edit the "User Defined Metadata" value in the Markdown editor.
  • If you want to edit the value of a field in the Markdown editor, specify "Text(Markdown)" as the data type of "Template for user-defined metadata".
• In the preview under "Table Info", the logical name is now displayed directly below the column name.

Other UI/UX

• On the account user screen, you can now check the authentication method of each user.
  • You can check if two-factor authentication and SAML authentication are valid respectively.

2022-08-29

Notice

• HubSpot API keys will no longer be used as an authentication method to access the HubSpot API after November 30, 2022.
  • From now on, please use OAuth 2.0 as the authentication method.
  • For more information, please refer to the official documentation.

Forwarding settings

• In the filter settings of Transfer Settings STEP2 "Data Preview / Advanced Settings", custom variables can now be used for filter values.
• In the transfer source HTTP (S), custom variables can now be used for keys and values of HTTP headers.

Data Catalog

• When editing the Basic Metadata "Description" field, the Markdown editor is now available.
• In the query editor, you can now download the query execution results in CSV format.
  • You can download up to 10 MB of data, and rows that exceed the size limit are truncated.

Managed transfer settings

More information on managed transfer settings will be introduced in the August 2022 release notes.

Other UI/UX

• In the list of forwarding settings, "Author" has been added to the filter item.
  • You can list only the transfer settings created by the selected user.
• In the forwarding setting list, the narrowing item "Name (partial match)" now supports AND search.
  • You can search for AND by entering them separated by spaces.
• In the transfer settings list, you can now add labels to selected forwarding settings in bulk.

2022-08-22

DBT Integration

• Adapters for dbt Git repositories now include Snowflake and Redshift.

Data Catalog

• Added trocco basic metadata settings.
  • You can now select whether to display the basic metadata items "Logical Name" and "Description" specified by TROCCO in the column list.

2022-08-08

Forwarding settings

• On the source Marketo target, you can now select a folder.
• We've added HubSpot as a forwarding destination.
  • Corresponds to the Contact object. Corresponding objects will be added in the future.
• When you select line_item as the report type in the source Twitter Ads, the transfer content now includes campaign_id.

DBT Integration

• DBT linkage function has been released.
  • It provides an OSS dbt execution environment and GitHub linkage.
  • The flow of data modeling from ETL can be defined on the trocco workflow function.
  • For details on future release schedules, please refer to the press release.

2022-08-01

Forwarding settings

• In the source Marketo target, you can now select program members.
• In the STEP2 output option of the transfer destination BigQuery, you can now select the NUMERIC type as the data type of the column setting.

2022-07-25

Workflow

• When a workflow fails, automatic retry is now possible.
  • On the workflow settings screen, you can specify the number of times to perform automatic retryes.

2022-07-19

Forwarding settings

• Google Drive has been added as a destination.
• When you select BigQuery as the source / destination, the region that can be specified in the dataset location has been updated.

Datamarts

• When you select BigQuery in your datamart definition, we've updated the regions you can specify in your data processing location.

Data Catalog

• You can now set trocco original metadata.
  • Apart from user-defined metadata, there is a frame for setting default metadata on the TROCCO side.

Other UI/UX

• All screens of transfer settings, data mart definitions, workflow definitions, and various account settings are now available in English.

2022-07-11

Forwarding settings

• In the source Shopify, you can now specify a transaction object as the target.
  • You can now retrieve data stored in a transaction object.
• In the transfer source Marketo, when you specify a lead by static list as the target, you can now specify the list ID you want to get.

Workflow

• If a nested workflow fails in the middle, it will now be rerun from where the child workflow stopped when it is rerun.

Data Catalog

• In the user-defined metadata template, you can now select multiple fields to display in the column list.
  • The column information in the Data Catalog is displayed in the order set on the Edit Template for User-Defined Metadata screen.
• BigQuery tables created in trocco datamarts can now inherit primary table metadata.
  • Between tables that have inherited metadata, column-based lineage can now be displayed.

2022-07-04

Forwarding settings

• If you are using CDC at the transfer source, you can now retry up to 5 times at once.
  • If an error occurs during the binlog transfer phase, it will automatically retry up to 5 times.

Data Catalog

• The query editor now displays suggestions in the ON clause.
  • Condition suggestions are displayed based on dependencies registered in the Data Catalog.

2022-06-27

Workflow

• Trocco bulk registration has been added to workflow tasks.
  • You can execute the transfer settings linked to the batch registration.

2022-06-20

Forwarding settings

• In the source Shopify, you can now specify Metafield as the target.
  • You can now retrieve data stored in Metafield resources.

Data Catalog

• Tab information is now stored in URLs on the Data Catalog screen.
  • When you transition tabs and return to the original screen in the browser back, you can now return to the point before the transition.

2022-06-06

Forwarding settings

• In the source Google Ad Manager, you can now specify whether to show the top-level ad units or all units.
  • You can now also get reports on deep hierarchies at the child and grandchild ad level.

Datamarts

• In the datamart definition, a free description mode has been added in Redshift.
  • You can issue arbitrary queries such as DDL statements to Redshift's DWH.

2022-05-30

Datamarts

• In the datamart definition, Snowflake added a free-form mode.
  • You can issue arbitrary queries to Snowflake's DWH, such as DDL statements.

2022-05-23

Forwarding settings

• We've added enhanced conversions to destination Google Ads conversions.
  • Until now, trocco only supported offline conversions as the forwarding destination Google Ads, but now it also supports enhanced conversions.
  • Accordingly, "Forwarding Google Ads Offline Conversions" has been renamed "Forwarding Google Ads Conversions".
• In the transfer source local file, you can now specify the character code in STEP1.
• In the source Facebook Ads Insights, the report template feature is now available.
  • In Step 1 of the transfer settings edit screen of the transfer source Facebook Ads Insights, you can now retrieve the main fields for each report type in the template.

Datamarts

• When using free-description mode in a datamart definition in BigQuery, you can now specify the data processing location.
  • Previously, queries were only available in the US region, but you can specify the region of the target dataset.

Other UI/UX

• You can now display up to 50 definition information on the datamart definition list screen and workflow definition list screen.

2022-05-16

Forwarding settings

• The transfer source Shopify now supports custom apps.

  • You can now also choose a custom app for Shopify connection creation.

• Zoho CRM has been added as a forwarding destination.

• You can now display up to 50 transfer settings on the transfer setting list screen.

Data Catalog

• In the query dita, you can now view the scan volume of BigQuery.
  • At the same time, issuing an invalid query now returns an error point.

2022-05-02

Forwarding settings

• In the transfer source Amazon S3, the tar .gzformat can now be selected as the decompression setting.
  • When decompressing in tar.gz format, the relative path after decompression can now be set with a regular expression.

Data Catalog

• When you update user-defined metadata, the index is now updated at the same time.
  • User-defined metadata is now updated and the search hits at the same time.

2022-04-25

Workflow

• Icons are separated for each task type in the workflow.
  • It's now easier to intuitively understand what the tasks in the flow represent.

Data Catalog

• In the query editor, you can now use the project selected at the last startup.
  • You no longer need to select your primary project every time you start the query editor.
• You can now view and delete table column dependencies.
• PostgreSQL has been added to the metadata collection target.

2022-04-18

Forwarding settings

• Rtoaster insight+ Google account linkage has been added as a transfer source.
  • You can now specify Rtoaster insight+ Google account linkage to transfer data.
  • If you want to use Rtoaster insight+ Google Account integration as your source, you will need to add your Google account to Rtoaster insight+.
    To learn how to add it to Rtoaster insight+, please refer to this help page.

Data Catalog

• Registered user-defined metadata can now be searched from the search bar at the top of the screen.
  • If your user-defined metadata has a field of type string, you can search for values in that field.
    (Example: If you stored the value "Revenue" in the "Logical Name" metadata for a table, typing "Revenue" in the search bar will now appear in the search results.) ）
• When searching, you can now move to the "Search Results" screen to check detailed information and filter and sort.

2022-04-11

Forwarding settings

• Zoho CRM has been added as a forwarding source.
  • It is now possible to extract and transfer data using SQL-like queries (COQL) dedicated to ZohoCRM.
• Added wording to deprecate the trigger job feature.
  • Please move to a workflow function that can be operated with a GUI and can define more flexible dependencies.

Data Catalog

• In the query editor, you can now save queries and load saved queries.

2022-04-04

Connection Information

• SSH connection is now supported in Oracle Database connection information.
• In your HubSpot connection information, you can now select OAuth2.0 as the authentication method.

Forwarding settings

• In the source HubSpot, the fields "Pipeline", "Pipeline stage", and "Assignee" can now be retrieved.
  • You can now get a list of "deal" and "ticket" pipelines and pipeline stages.

security

• When previewing data in the transfer settings, etc., it is now possible to select whether to save (persist) the preview contents inside the Trocco service.
  • If you enable this function, the preview contents will be automatically deleted after a certain period of time after the setting is completed, so you can use it more securely.

2022-03-28

Forwarding settings

• In kintone, "update" and "upsert" have been added to the transfer mode.
  • Data linkage with kintone is now possible.
• In the transfer source RTB House, the items "subCampaignId" and "subCampaginName" can now be obtained.
• In the transfer source Salesforce, deleted and archived records can now be retrieved.
  • Records of (archived) Task and Event objects created more than one year ago can now also be retrieved.

Data Catalog

• Sharding tables under any dataset are now displayed together.
• The completion function of the query editor has been enhanced to make it easier to use.
  • Metadata about the entered table and how to use the entered function are now suggested.

2022-03-21

Datamarts

• Snowflake now makes it easier to understand error messages when connection information is misspecified.

Other UI/UX

• "FTP" listed as a data source has been changed to "FTP/FTPS", and "http" has been changed to "HTTP(S)".
  • There is no change in the function itself. The UI notation will be in a form that matches the function.

2022-03-14

Forwarding settings

• The source Facebook Ads creative now suggests the ad account ID.
• In the transfer destination Google Spreadsheets, "All items wash (transfer existing sheets)" has been added to the transfer mode.
  • Since the transferred contents are directly reflected in the target sheet, references in Google Data Studio, etc. are no longer broken.
• In the destination Snowflake, there are no stages left on Snowflake when the transfer job succeeds.

Data Catalog

• kintone has been added to the metadata collection target.