- Print
- PDF
List of Release Notes
- Print
- PDF
This page contains weekly releases.
2024-07-29
UI・UX
ETL Configuration list can now be filtered by Connection Configuration type.
In the last update, filtering by Connector was supported.
In response to the above, this update supports filtering by individual Connection Configuration.
This makes it easy to see which Connection Configuration is used for any given ETL Configuration.
2024-07-22
UI・UX
Changes to user management screens
The design of the user management screen has been changed.
This allows each user's permissions (operations allowed on TROCCO) to be checked at a glance.
When used in conjunction with COMETA, users can also be narrowed down.
2024-07-16
ETL Configuration
Data Source Databricks
New Data Source Databricks added.
For more information, see Data Destination - Databricks.
Notice
Increased memory size for ETL Job
The release within the week of 07/16/2024 will increase the memory size used for data transfer.
- Memory size before change: 2 GiB
- Modified memory size: 6 GiB
This change applies to ETL Configurations created after the above release.
Since the ETL Job will be up-specified, performance may be improved compared to the job before the change.
However, the following Connectors are exceptions to the current memory size of 15 GiB, and the 15 GiB will continue to apply after this change.
List of Connectors with exceptional memory size of 15 GiB
Elimination of Direct and Aggregate transfer functions
The following transfer functions have been eliminated
- Direct Transfer (selectable when Data Source Amazon S3 -> Data Destination SFTP combination is used)
- Aggregate Transfer (selectable when Data Source Google BigQuery -> Data Destination Amazon S3 combination)
ETL Configuration
When an OutOfMemoryError occurs, the execution log will clearly indicate this.
When an OutOfMemoryError occurs, it is now clearly indicated in the execution log.
If this message is displayed, please refer to the section on what to do when OutOfMemoryError occurs.
Input restrictions added for Data Source HTTP and HTTPS
ETL Configuration STEP2 > Input Option now has an upper and lower limit for the value that can be entered for each setting item.
For more information, see Data Source - HTTP/HTTPS.
2024-07-01
Notice
Google Analytics (Universal Analytics) Discontinued Support
In response to Google's discontinuation of Universal Analytics, the following Connectors will be discontinued on July 01, 2024.
- Data Source - Google Analytics
- Data Destination - Google Analytics Measurement Protocol
After 07/01/2024, it will no longer be possible to create new ETL Configuration and Connection Configuration. Also, running a Job from ETL Configuration will result in an error.
Please consider switching to Google Analytics 4 and using Data Source - Google Analytics 4 and Google Analytics 4 Connection Configuration in the future.
2024-06-24
ETL Configuration
Data Destination Databricks
New Data Destination Databricks added.
For more information, see Data Destination - Databricks.
2024-06-17
Notice
Limit job execution by the maximum number of simultaneous executions
TROCCO limits the number of jobs that can run simultaneously within an account.
Following the rate plan change in 04/2024, data mart jobs are now also subject to this limitation.
For more information on this limitation, please refer to Job Concurrency Limit.
dbt linkage
Compatible with dbt versions 1.7 and 1.8
dbt Core v1.7 and dbt Core v1.8 can now be specified.
The dbt version can be selected from the dbt Git repository.
2024-06-10
Notice
Restrictions on Job Execution in the Free Plan
If you are using the Free plan, you can no longer run jobs when the cumulative monthly processing time exceeds the processing time quota.
The accumulated processing time returns to 0 hours at midnight (UTC+9) on the first day of the following month. If any jobs were not executed, they should be rerun in the following month.
ETL Configuration
Record ID can be specified as update key in update/upsert of
Data Destination kintone.
Record IDs can now be specified as update keys.
If you wish to specify a record ID, enter $id
as the update key.
2024-06-03
API Update
Data Destination Google Ads Conversions
Regarding extended conversions, the version of the Google Ads API used for transfer has been updated from v14.1 to v16.
Please refer to the Google Ads API documentation for information on the new version.
2024-05-27
Data Mart Configuration
Datamart Snowflake
The write setting for the output destination table can now be selected between TRUNCATE INSERT and``REPLACE
for the all-wash mode.
- In the case of
TRUNCATE INSERT
, the schema of the existing table is not deleted. - In the case of
REPLACE
, the schema of the existing table is deleted.
For more information on the difference between the two, see Data Mart - Snowflake.
API Update
Data Source Google Ads / Data Destination Google Ads Conversions
The version of Google Ads API used in the transfer was updated from v14.1 to v16.
Regarding Data Destination Google Ads Conversions, only offline conversions have been updated.
Extended conversion updates will be available next week.
Please refer to the Google Ads API documentation for information on the new version.
Search Ads and Data Source Yahoo! Search Ads and Data Source Yahoo! Display Ads (managed)
The version of YahooAdsAPI used for transfer has been updated from v11 to v12.
Please refer to the following documents for each new version
- Yahoo! Search Ads API | Search Ads API v12 Release Notes
- YahooAdsAPI | Display Ads API v12 Release Notes
2024-05-20
Notice
API Update for Data Source Google Ads
On Wednesday, May 22, 2024, between 10:00 am and 4:00 pm, there will be an update to the Google Ads API used by Data Source Google Ads.
Some disruptive changes will occur with the API update.
Please see 2024/05/16 Destructive Changes to Data Source Google Ads for more information on the resources/fields that will be removed or changed and what to do about it.
2024-05-13
Notice
On May 9, 2024, we announced our brand renewal.
With the rebranding, the logotype of the
product was changed from TROCCO``to TROCCO
, as well as the color scheme of the logo.
The new logo image files and logo guidelines are available on our website.
Also, please refer to the press release regarding the brand renewal.
Connection Configuration
Allow SSH private key passphrase to be entered in Microsoft SQL Server Connection Configuration.
Added SSH private key passphrase to the configuration item.
This allows you to connect to Microsoft SQL Server with a private key with a passphrase.
2024-04-30
security
TROCCO API is now restricted by IP address when executed.
Execution of the TROCCO API is now subject to IP address restrictions.
This allows for more secure use of the TROCCO API.
If you are already using the TROCCO API and have set Account Security >AllowedIP Addresses for at least one IP address, you must add the IP address used to run the TROCCO API to the Allowed IP Addresses.
Data Mart Configuration
Data Mart Azure Synapse Analytics
New data mart Azure Synapse Analytics has been added.
For more information, see Data Mart - Azure Synapse Analytics.
ETL Configuration
Data Source Google Sheets columns can be extracted.
Previously, it was necessary to manually enter the name and data type of the column to be retrieved in ETL Configuration STEP 1.
In contrast, the ability to extract column information from a spreadsheet to be transferred has been added.
After entering the various setting items, click Extract Column Information to automatically set the column name and Data Setting.
With the addition of the above functionality, an entry field has been added to specify the starting column number for capturing.
For more information, see Data Source - Google Spreadsheets.
2024-04-22
ETL Configuration
Search Ads and Data Source Yahoo! Search Ads and Data Source Yahoo! Display Ads (managed)
Due to the discontinuation of YahooAdsAPI v11, the Base Account ID entry field has been added to the following Data Source Connector Configuration in order to update the version of Yahoo!
- Data Source Yahoo! Search Ads
- Data Source Yahoo! Display Ads (managed)
For details, see "MCC Multi-Tiered" in v12 Upgrade Information.
The transition to v12 is scheduled for mid-May 2024.
As soon as the migration to v12 is complete, ETL Job Settings with a Base Account ID not yet entered will be marked as an error.
Please edit your existing ETL Configuration prior to v12 migration.
2024-04-15
ETL Configuration
Data Destination kintone to be able to transfer to table
Data can now be transferred to tables (formerly subtables) in the kintone application.
For details on how to transfer, please refer to Updating Tables (formerly Subtables) in the Data Destination kintone application.
Data Source Google BigQuery allows users to select "Bucket Only" for temporary data export specification.
When transferring data from Data Source Google BigQuery, data is temporarily output to Google Cloud Storage.
Only buckets can now be specified as the output destination for temporary data in this case.
Note that the conventional format of entering a Google Storage URL will output temporary data to the same path, unless Custom Variables are used.
As a result, data on Google Cloud Storage could be overwritten.
On the other hand, if only buckets are specified, an internally unique path is created and temporary data is output to that path.
This avoids the aforementioned situation where data in Google Cloud Storage is overwritten and deleted.
2024-04-08
UI・UX
Allow organization name to be set in TROCCO account
You can now set an organization name for your TROCCO account.
The organization name makes it easier to identify which TROCCO account you are logging into if you are managing multiple TROCCO accounts, for example.
For more information, see About Organization Names.
Managed ETL
Add Amazon Redshift as a Data Destination
Amazon Redshift can now be selected as a Managed ETL Data Destination.
ETL Configuration, which retrieves Data Sources from a batch of Data Sources and transfers them to Amazon Redshift, can be created and managed centrally.
API Update
Data Source Google Ad Manager
The version of the Google Ad Manager API used during transfer has been updated from v202305 to v202311.
For more information on the new version, see Google Ad Manager API.
2024-04-01
Notice
Effective April 1, 2024, the rate plan will be revised.
For details, please refer to the fee plan.
ETL Configuration
Data Source Shopify supports retrieval of collections
.
Data Source Shopify targets can now select a collection
object.
See Data Source - Shopify for more information, including each item to be retrieved.
Additional types can be specified in Data Destination Amazon Redshift
The following items have been added to the Data Type in the STEP2 Output Option Column Setting for ETL Configuration Amazon Redshift.
TIME
DATE
Update Processing Configuration for NULL values transferred in Data Destination kintone added.
When the update data for an existing record in kintone contains a null
value, you can now select the update process for that record.
You can choose to update with NULL or****skip updating in the advanced settings of ETL Configuration STEP 1.
2024-03-25
ETL Configuration
Data Destination Azure Synapse Analytics
New Data Destination Azure Synapse Analytics.
For more information, see Data Destination - Azure Synapse Analytics.
2024-03-18
ETL Configuration and Managed ETL
Temporary stage can be deleted when ETL Job to Snowflake fails.
If an ETL Job to Snowflake fails, you can now choose to delete the temporary stage.
For more information, see Data Destination - Snowflake > STEP1 Advanced Configuration.
UI・UX
Maximum number of ETL Configuration Data Setting data previews changed to 20.
The maximum number of data items displayed in the Data Preview of ETL Configuration STEP2 and ETL Configuration Details has been changed to 20.
This change shortens the time it takes for the data preview to appear in the preview of schema data in E TL Configuration STEP 2.
API Update
Data Destination Facebook Custom Audience(Beta Feature)
The version of the Facebook API used for transfer has been updated from v17 to v18.
Please refer to the Meta for Developers documentation for the new version.
2024-03-11
Workflow
Allow the error log of an ETL Job to be viewed within the execution log of a Workflow Job.
When an ETL Job embedded in a workflow fails, the error log for the relevant ETL Job can now be viewed from the workflow execution log.
You can check the error log by clicking the corresponding task in the workflow execution log.
API Update
The Facebook API used for the following Connector has been updated from v17 to v18.
- Data Source Facebook Ad Insights
- Data Source Facebook Ad Creative
- Data Source Facebook Lead Ads
- Data Destination Facebook Conversions API
Please refer to the Meta for Developers documentation for the new version.
audit log
Removed "Restore Past Revisions" action in ETL Configuration from audit log capture.
Removed "Update ETL Configuration (restore past revisions of change history)" from actions eligible for audit logging.
For more information, please refer to the Change History of the Audit Log function.
2024-03-04
Notice
About TROCCO Web Activity Log Help Documentation
Until now, the help documentation for TROCCO Web Activity Log has been available on the Confluence space.
The help documentation has now been transferred to the TROCCO Help Center. Please refer to the TROCCO Web Activity Log in the future.
The help documentation on the Confluence side will be closed soon.
UI・UX
ETL Configuration list can now be filtered by Connection Configuration.
Connection Configuration has been added as a filter item in the ETL Configuration list.
You can filter by the ETL Configuration in which the specified Connection Configuration is used.
For more information, see List of ETL Configurations > Filter.
ETL Configuration
Allow regular expressions to be used to specify the path to the file to be retrieved by Data Source SFTP.
In ETL Configuration STEP1, the path to the file to be retrieved can now be specified with a regular expression.
For example, if you enter .csv$
in the path regular expression, only csv files under the directory specified by the path prefix will be retrieved.
Workflow
Manually Executed Workflow Jobs to show the user who executed them
Previously, when a Workflow Job was manually executed, it was not indicated which user executed it.
With this change, the email address of the user who executed the following cases is now displayed.
- When executed by clicking the Execute button on the Workflow definition details screen
- When executed by clicking Re-Execute from the stop position on the Workflow Job Details screen .
To display a link to the Workflow Job from which it was run
Previously, Workflow Jobs that were executed as tasks of another Workflow Job did not indicate by which Workflow Job they were executed.
With this change, a link to the Workflow Job from which it was run is now displayed.
2024-02-26
ETL Configuration
Microseconds and nanoseconds added to time units for UNIX Time conversion.
The number of UNIX Time units that can be handled in UNIX Time conversion in ETL Configuration STEP 2 has been expanded.
Microseconds and****nanoseconds can now be selected as the unit of UNIX Time conversion before and after conversion.
For more information, see UNIX Time conversion.
2024-02-19
Data Catalog
Snowflake version of Data Catalog supports "Automatic Metadata Inheritance" and "Column Lineage".
Until now, automatic metadata takeover andcolumn linage for Data Catalogs were supported only in the Google BigQuery version.
With this change, the same functionality is now available in the Snowflake version of the Data Catalog.
ETL Configuration
Changed how to set merge key for Data Destination PostgreSQL.
The method of setting the merge key when the transfer mode is set to UPSERT (MERGE) in
ETL Configuration STEP1 has been changed.
Previously, a merge key had to be set in ETL Configuration STEP 2.
With this change, when UPSERT (MERGE)
is selected as the transfer mode in ETL Configuration STEP 1, the Merge Key setting item will appear directly below.
API Update
Data Source Shopify
The version of the Shopify API used for transfers has been updated from 2023-01 to 2024-01.
Please refer to the documentation in the Shopify API reference docs for the new version.
2024-02-13
Managed ETL
Add Microsoft SQL Server as Data Source
Microsoft SQL Server can now be selected as the Data Source for Managed ETL.
Microsoft SQL Server tables can be imported in bulk, and the associated ETL Configuration can be created centrally.
See Managed ETL Configuration > Data Source Microsoft SQL Server for various entry fields.
ETL Configuration
Expanded columns of master data for ads that can be retrieved from Data Source LINE Ads.
Added small_delivery
to the column of data retrieved as master data for ads.
Master data for advertisements can be obtained when Master Data (Advertisements) is selected for the Download Typeand Advertisements is selected for the Master Data Type in STEP 1 of ETL Configuration.
Note that to incorporate the small_delivery
column in an existing ETL Configuration, you must edit the ETL Configuration and run the Automatic Data Setting.
Select "Execute Automatic Data Setting" on the screen that appears when moving from STEP1 to STEP2 of the Edit ETL Configuration screen, and save it.
2024-02-05
ETL Configuration
Expanded the types of dimensions that can be specified in Data Source Criteo
CampaignId and``Campaign
can now be selected in Dimension Name in ETL Configuration STEP 1.
Dimension name is an item that appears when statistics is
selected as the report type.
API Update
Search Ads and Data Source Yahoo! Search Ads and Data Source Yahoo! Display Ads (managed)
The version of YahooAdsAPI used for transfer has been updated from v10 to v11.
Please refer to the YahooAdsAPI | Developer Center documentation for information on the new version.
Due to an API update, the old indicator has been discontinued.
From now on, if a column containing "(old)" is specified in the column name, the new column will be automatically obtained.
2024-01-29
ETL Configuration
Data Destination Snowflake supports schema tracking.
Data Destination Snowflake now supports schema tracking.
Schema Tracking is a function that automatically corrects the schema of the Incremental Data Transfer table and resolves the schema difference between the Data Destination and the Connector's table.
From now on, it will no longer be necessary to manually modify the schema on the Snowflake side in the event of a difference in the above schema.
UI・UX
ETL Configuration list can be filtered by regular expression.
The ETL Configuration list can now be narrowed down by regular expression.
See Filtering ETL Configuration Names with Regular Expressions for more information on the notation of regular expressions that can be entered.
Time Zone Configuration values are now applied by default when creating Managed ETL Configuration.
The time zone value specified in the Time Zone Configuration is now entered by default in the time zone value selected in STEP 1 when creating the Managed ETL Configuration.
API Update
Data Destination Google Ads Conversions
The version of Google Ads API used during transfer has been updated from v13.1 to v14.1.
Both offline and****extended conversions have been updated.
Please refer to the Google Ads API documentation for information on the new version.
2024-01-22
Managed ETL
Enabled bulk selection and deselection of tables and filtering of table names
Previously, only pagination units (up to 100 tables/times) could be selected.
This change allows for batch selection and batch de-selection regardless of pagination.
In addition, it is now possible to filter by table name.
This change will be applied to the following screens.
- New creation STEP2
- List of unadded tables
- Check Created/Dropped Tables
ETL Configuration
Added "Change event" to the resource type of Data Source Google Ads.
Change event (change_event)
has been added to "Resource Type (Report Type)" in STEP 1 of ETL Configuration.
You can now get a report of changes that have occurred in your account.
For more information on change_event
, please refer to the Google Ads API documentation.
Data Source ValueCommerce to get reports for advertisers.
Previously, only affiliate sites were eligible to obtain reports.
With this change, advertiser reports can also be retrieved.
For more information, see Data Source - ValueCommerce.
UI・UX
Redesigned pop-up menu in the upper right corner of the screen
The design of the pop-up menu that appears when the user clicks on his/her own e-mail address area has been redesigned.
In addition to being able to check the organization ID and own privileges, users can now move to various settings related to accounts and users with a single click.
In addition, links to the following pages have been moved from the pop-up menu to the sidebar on the left side of the screen with this change.
- GitHub access token (under external collaboration )
- TROCCO API Key (under External Linkage )
- Audit log output
API Update
Data Source Google Ads / Data Destination Google Ads Conversions
The version of Google Ads API used during transfer has been updated from v13.1 to v14.1.
Please refer to the Google Ads API documentation for information on the new version.
2024-01-15
ETL Configuration
Data Source TROCCO Web Activity Log data acquisition period can be specified.
Data Retrieval Period can now be specified in ETL Configuration STEP1.
TROCCO Web Activity Log data can be retrieved for any time period by specifying a start and****end date.
For more information, see Data Source - TROCCO Web Activity Log.
Connection Configuration
HTTP/HTTPS Connection Configuration using Client Credentials can be created.
HTTP/HTTPS Connection Configuration, Grant Type can now be selected from Authorization Code or****Client Credentials.
Previously, the grant type was fixed and was an authorization code, but with this release, Client Credentials can now also be selected.
For details, please refer to the HTTP/HTTPS Connection Configuration.
Data Catalog
Changed specifications for importing partitioned tables
In Google BigQuery Data Catalog, the specification to retrieve partitioned tables as catalog data has been changed.
From now on, for partitioned tables, only the table with the latest date will be retrieved as catalog data.
Previously, all segments in a partitioned table were obtained as catalog data.
Because each segment was considered a separate table in the Data Catalog, there were multiple hits for essentially the same table when searching for tables, and manual metadata entry operations such as basic metadata and user-defined metadata were difficult.
From now on, only tables with the most recent dates will be retrieved, making tables more searchable and facilitating the operation of manual metadata entry.
2023-12-26
Workflow
Added "HTTP Request" to workflow task
Added an HTTP request to the workflow task that allows external API requests to be executed.
By incorporating tasks that communicate with external systems, you can build a more flexible and powerful workflow.
HTTP Request Tasks can be configured and added on the Flow Edit screen of Workflow definitions.
ETL Configuration
TSV file input/output setting delimiter can now be entered as ɑt
TSV file input and output settings can now be set to enter Јt
as the delimiter character.
Please use this when setting the delimiter character to a tab character in an ETL Configuration that uses a Connector that handles TSV files.
You can set the delimiter in the Input Option and****Output Option in ETL Configuration STEP 2.
Connection Configuration
Elasticsearch Connection Configuration allows you to choose whether or not to use SSL communication.
You can now choose whether or not to use SSL communication in Elasticsearch Connection Configuration.
If you need to communicate with the connecting Elasticsearch using the HTTPS method, select Enabled.
UI・UX
Revamped UI for Programming ETL
The UI for Data****Setting>Programming ETL in STEP 2 of ETL Configuration has been revamped.
Programming ETL allows for flexible conversion processes.
For more information, see Programming ETL.
2023-12-19
Data Mart Configuration
Data Mart Configuration change history to allow restoration of past configurations
The Change History tab on the Data Mart Configuration details screen now allows restoration of settings for previous changes.
If, for example, a data mart job run after a configuration change does not produce the expected results, you can immediately revert to the settings that were in place before the change.
From the Change History tab of the Data Mart Configuration Details screen, click on Advanced/Restore>Restore Revision
Saving the settings displayed will restore the settings on the specified change history.
ETL Configuration
Connector selection screen to show "Recently used Connectors".
When creating a new ETL Configuration, the Connector selection screen now displays the most recently used Connector at the top.
It is now easier to find frequently used connectors among the many available connectors.
Data Source Google Drive folder ID to be displayed in link text.
The folder ID of the Data Source Google Drive displayed on the ETL Configuration details screen is now displayed as link text.
Clicking on the folder will take you to the corresponding folder screen on Google Drive.
It is now easier to check what files are stored in the folder for data acquisition.
2023-12-13
Data Mart Configuration
Data Mart Configuration change history can now be viewed.
The Data Mart Configuration detail screen now lists the history of past changes.
You can also check the change differences from the previous change history.
Workflow
Workflow Job notifications can now include information about the results of the job execution.
In the Workflow Job Notification Setting, it is now possible to embed a variable that contains information on the results of the job execution in the notification content.
Embed information such as thenumberofoverall workflow tasks andthe number of failed workflow tasks, allowing for flexible customization of notification content.
ETL Configuration
UNIX Time conversion and date/time formats can be transferred to each other
UNIX Time conversion and date/time formats can now be set in the Data Setting tab of ETL Configuration STEP 2.
You can specify a column where date/time data is stored and transfer UNIX Time conversion and date/time format to each other.
For more information, see UNIX Time conversion.
API Update
Data Source Facebook Lead Ads
The version of the Facebook API used for transfer has been updated from v16 to v17.
See Meta for Developers for the new version.
2023-12-05
ETL Configuration
Data Destination Snowflake to specify batch size
Batch Size (MB)" can now be specified in the advanced settings of ETL Configuration STEP1.
If an error occurs during transfer due to insufficient memory, the batch size can be adjusted to eliminate the error.
For more information, see Data Destination - Snowflake.
Connection Configuration
Allow Key Pair Authentication Snowflake Connection Configuration to be used in ETL Configuration
Snowflake Connection Configuration created by Key Pair Authentication can now be used in ETL Configuration.
This allows Data Source/Data Destination Snowflake and Data Mart Snowflake to use the same Connection Configuration.
API Update
Workflow Task Tableau Data Extraction
The version of the Tableau REST API used for Tableau data extraction in the workflow task has been updated from 3 . 7 to 3.21.
See Tableau REST API Help for information on the new version.
Other
Korean can be selected in the language settings.
Korean can now be set as the language displayed on the TROCCO screen.
For background on Korean language support, please see this article.
ASCII.jp: primeNumber to Fully Expand "TROCCO" Overseas through Partnership with Korean SaaS Company
2023-11-28
ETL Configuration
Data Source Yahoo! Search Ads to get reports related to ad display options.
CAMPAIGN_ASSET" and "ADGROUP_ASSET (beta)" have been added to the types of reports retrieved by Data Source Yahoo! Search Ads.
Each can obtain the following report data, which will be available in March 2022.
- CAMPAIGN_ASSET: Ad Display Options Report (Campaign)
- ADGROUP_ASSET (beta): Ad Display Option Report (Ad Group)
Account User
Allow account privilege administrators to disable two-factor authentication for users in the account
Only the account privileged administrator can now disable two-factor authentication set by each user in the account.
For more information, see About Account Privilege Administrators.
2023-11-20
ETL Configuration
Data Source HTTP/HTTPS allows cursor-based paging requests
Cursor-based paging requests are now possible when retrieving data via Data Source HTTP/HTTPS.
When the cursor base is selected in the paging setting of ETL Configuration STEP 1, various setting items for the cursor base will be displayed.
For more information, see Data Source - HTTP/HTTPS.
Data Source Google Analytics 4 to allow specifying the number of rows to retrieve in a single request.
In the Advanced Settings of ETL Configuration STEP 1, you can now specify the number of rows to retrieve in a single request when retrieving data from Google Analytics 4.
If OutOfMemoryError
occurs during job execution, adjusting the value of this item may eliminate the error.
For more information, see Data Destination - Google Analytics 4.
Workflow
Workflow definitions can be duplicated from the Workflow Definition List screen.
Workflow definitions can now be duplicated in the Workflow Definition List screen.
You can duplicate any Workflow definition without going to the Workflow Definition Details screen.
TROCCO API
Allow the account privileged administrator to manage all API Keys in the account
Only the account privileged administrator can now view, edit, and delete all API Keys issued by users in the account.
Previously, only the user who issued the API Key could manage it, but now all API Keys in an account can be centrally managed.
For more information, see About Account Privilege Administrators.
The API Key list screen can be accessed from the Settings icon menu at the top of the screen.
Other
Email notification settings to apply line breaks in messages when they are sent.
In the various notification settings when the notification method is set to Email, the line breaks in the text entered in the message are now applied to the outgoing email.
*If the notification method is Slack, line breaks will be applied from before.
2023-11-14
ETL Configuration
Data Destination kintone
Automatic acquisition of field codes is now supported.
Previously, each field code had to be entered manually in STEP 1 of ETL Configuration.
From now on, it will be automatically retrieved in ETL Configuration STEP2 according to the specified app ID.
In addition, when the application to be transferred contains a table (formerly a sub-table), the user can now select whether to transfer the records by dividing them by row in the table (formerly a sub-table) or by combining them into a single record.
Please refer to Data Source - kintone for more information regarding the above.
Data Destination Google Cloud Storage
Naming conventions for multiple file output can now be specified.
When parallel transfer is selected as the transfer mode, the transferred data may be split into multiple files.
From now on, you can specify the naming of multiple files in the advanced settings of ETL Configuration STEP 1.
Data Source Google Analytics 4
The status of the "(other)" row when it is used can now be selected.
You can choose whether to set the
job to Succeeded
or Error
when the retrieved data contains "(other)" rows.
For more information on the " ( other ) " row, please refer to the official Google Analytics documentation at [GA4]About the "(other)" row.
Managed ETL
Data Destination Snowflake
UPSERT (MERGE)
has been added as a Data
Destination output mode.
If there is a record in the existing table based on the merge key, the record is updated; if there is no record, the record is appended.
For more information, see Data Destination - Snowflake.
2023-11-06
ETL Configuration
Renaming of former Twitter-based Connector
The following Connectors have been renamed
- Data Source Twitter Ads → X Ads (Formerly Twitter Ads)
- Data Destination Twitter Ads Web Conversions → Data Destination X Ads (Formerly Twitter Ads) Web Conversions
Other
Revised TROCCO Terms of Use
Revised TROCCO Terms of Use.
TROCCO Terms of Use - available in the latest version.
2023-10-31
Account User
Added the ability to delete users
Users registered to a TROCCO account can now be deleted from TROCCO.
In the future, users can be removed without contacting our Customer Success.
See Deleting Users for more information, including permissions required to delete users.
Connection Configuration
JDBC driver version can be specified in Snowflake Connection Configuration.
JDBC driver version can now be specified in Snowflake Connection Configuration.
When creating a new Connection Configuration, 3.14.2
is selected as the default value.
2023-10-24
Managed ETL
Add Snowflake to Data Destination
Snowflake can now be selected as a Data Destination for Managed ETL.
ETL Configuration, which retrieves Data Sources from the Data Source in bulk and transfers them to Snowflake, can be created and managed centrally.
See Managed ETL Configuration for more information.
UI・UX
Support for drag-and-drop column reordering of ETL Configuration
Columns can now be rearranged by drag-and-drop operation in Column Setting of STEP 2 of ETL Configuration.
The order of columns can be rearranged with intuitive operations.
ETL Configuration
Extends the normal system judgment when acquiring Data Source HTTP/HTTPS data.
In STEP 1 of ETL Configuration, the status code that determines a normal system when acquiring ETL Configuration data can now be specified.
For more information, see Data Source - HTTP/HTTPS.
TROCCO API
The TROCCO API can now retrieve a list of ETL Jobs.
You can specify any ETL Configuration and get a list of ETL Jobs using that ETL Configuration.
For more information, see About the TROCCO API.
API Update
Data Source Yahoo! Search Ads
The version of YahooAdsAPI used in the transfer has been updated from v9 to v10.
For more information about the new version, please visit the YahooAdsAPI | Developer Center.
2023-10-16
organization ID
Organization ID to be visible on the TROCCO screen
The organization ID is now displayed in the pop-up menu for logged-in users.
Organization ID is a required field when logging in.
In the unlikely event that you forget your organization ID, please ask a user who is already logged in to confirm your organization ID from the menu above and notify us.
Account User
Connection Configuration operation restrictions can now be applied on a per-user basis.
It is now possible to prohibit individual users from creating, editing, or deleting Connection Configuration.
By limiting the number of users who can create Connection Configurations, you can prevent connections to data sources to which they are not intended to connect, such as privately managed storage.
This reduces the risk of unintended data leakage.
For more information, see User Settings.
API Update
Data Source Google Ad Manager
The version of the Google Ad Manager API used during transfer has been updated from v202211 to v202305.
For more information on the new version, see Google Ad Manager API.
2023-10-10
Notice
On Monday, October 2, 2023, the method of logging into TROCCO was changed.
For more information, please refer to the Change of Login Method to TROCCO.
dbt linkage
Allow any dbt version to be used in a dbt job
The dbt version, previously fixed at dbt Core v1.3, can now be specified arbitrarily.
On the New/Edit screen of the dbt Git repository, you can choose from the following versions
- dbt Core v1.6
- dbt Core v1.5
- dbt Core v1.4
- dbt Core v1.3
UI・UX
List of Data Mart Jobs
Faster loading time
The data loading process has been sped up, and the time until the list of data mart jobs is displayed has been shortened.
Expanded filtering capabilities
The filtering format has been changed from the traditional text entry format to a format where the name of the Data Mart Configuration is selected.
This makes it possible to narrow the list of Data Mart Jobs by multiple Data Mart Configuration names.
API Update
Data Source Facebook Ad Creative and Data Destination Facebook Offline Conversions
The version of the Facebook API used for transfer has been updated from v16 to v17.
See Meta for Developers for the new version.
2023-10-02
security
Changed the period of time that login status is retained to 48 hours.
For enhanced security, the retention period of login status has been changed to 48 hours.
After 48 hours have elapsed since the last operation of TROCCO, the system enters a logout state.
The next time you access TROCCO, you will need to log in.
ETL Configuration
Data Destination Facebook Custom Audience(Beta Feature)
New Data Destination Facebook Custom Audience (Beta Feature).
See Data Destination - Facebook Custom Audience (Beta Feature) for more information on the various input fields and column mappings.
Added Update Processing Configuration for NULL values forwarded to Data Destination Salesforce.
You can now select the update process when the update data for an existing record in Salesforce contains a null
value.
You can choose to update with NULL or****skip updating in the advanced settings of ETL Configuration STEP 1.
API Update
Data Source Google Ads / Data Destination Google Ads Conversions
The version of Google Ads API used in the transfer was updated from v12 to v13.1.
Please refer to the Google Ads API documentation for information on the new version.
2023-09-25
On September 5, 2023, a new privilege , Account Privilege Manager, was added to TROCCO.
The Account Privilege Manager is the strongest of TROCCO's privileges and is therefore a special privilege that can only be granted to one user per account.
Due to the nature of this authorization, we will be taking a process to allow customers to choose which user will be the account privileged administrator for accounts that existed prior to September 5, 2023.
Please respond to the following email sent on September 14, 2023 notifying the following users about the transition.
Earliest user registered to a TROCCO account that has not been removed from the account
If you have any questions, such as if you do not know where to send an e-mail, please contact our Customer Success.
ETL Configuration
Data Destination File and Storage System Connector supports zip file compression
Some Data Destination file/storage system Connectors now support compression of files in zip format.
Zip
can be selected for file compression in ETL Configuration STEP 1 with the following Connector as the Data Destination.
- Data Destination Azure Blob Storage
- Data Destination FTP/FTPS
- Data Destination Google Cloud Storage
- Data Destination KARTE Datahub
- Data Destination S3
- Data Destination SFTP
dbt linkage
dbt Git repository settings can now specify subdirectories
Subdirectories in the Git Integration repository can now be specified as project directories for dbt integration.
Previously, the directory for integration was fixed to the root directory of the Git repository.
From now on, you can specify any directory in the Git Integration repository as the destination for dbt integration.
2023-09-20
Data Mart Configuration
Data Mart Configuration in Google BigQuery allows clustering settings.
Clustering configuration item was added to Data Mart Configuration in Google BigQuery.
Clustering settings can now be configured for tables newly created by executing a data mart job.
However, if a table already exists in the output destination, the settings of the existing table will be taken over instead of the contents of this setting.
ETL Configuration
Data Source Google BigQuery Job Waiting Timeout can be Job Setting.
In the Advanced Settings of ETL Configuration STEP1, you can specify the timeout period in seconds for waiting for a Job.
When there are many queries running in BigQuery, slot limits may cause jobs to wait until they are executed. If this waiting time reaches the timeout period, the relevant ETL Job will fail.
In such cases, increasing the "Job Waiting Timeout (sec)" will avoid ETL Job failures.
Data Destination Snowflake's NUMBER type output can specify precision and scale.
You can specify the precision and scale of the NUMBER type in the Output Option tab > Column Settings > Data Type in ETL Configuration STEP 2.
Use this function to convert data to be transferred to Snowflake to a NUMBER type of any precision and scale.
For more information on the precision and scale of the NUMBER type, please refer to the official Snowflake documentation - NUMBER.
### UI・UX
ETL Configuration list sidebar filter button fixed display
The Filter by this content button in the Filter by area of the ETL Configuration list is now fixed so that it always appears on the screen.
dbt linkage
dbt Job Setting in Google BigQuery supports selective location input.
Locations for dbt Job Settings can now be entered from a selection.
2023-09-12
Managed ETL
Schema Change Detection for ETL Configuration can now be set in bulk.
Schema Change Detection for Managed ETL Configuration can now be set in STEP 3 in a batch.
Receive notification of schema changes without having to configure them individually in each ETL Configuration.
ETL Configuration
Expanded items for Date/Time Columns to be added in Transfer Date Column Setting.
Transfer Date Column Setting in the Transfer Settings STEP 2 Advanced Settings > Data Setting tab now allows for flexible ETL Configuration.
When the
Transfer Date Column Data Typeis set to string
, the following items can be ETL Configuration.
- Format: A Specifies the format of the date/time expansion value.
- Time zone Select the time zone to be expanded in the format's time zone specifier from
Etc/UTC or``Asia/Tokyo
.
Extended the data types that can be transferred from Data Source Oracle Database
Data to be imported from Data Source Oracle Database can now be converted to string type and transferred.
Click on Set Details in ETL Configuration STEP 1, specify the target column name, and select string for
the Data Type.
For example, numerical values that had a large number of digits and were missing data during transfer can now be converted to string type and transferred, thereby avoiding missing data.
Data Mart Configuration
Custom Variable embedding support for Google BigQuery partition fields.
Custom Variable embedding is now available in the Partition field of Data Mart Configuration in Google BigQuery.
The value of the partition field can be dynamically specified at job execution.
The *Partition field is a setting item that can be entered when partitioning is selected by field.
Data Catalog
Google BigQuery Data Catalog adds service accounts as an authentication method.
Service Account (JSON Key)" can now be selected as the authentication method for Google BigQuery Data Catalog.
For details, please refer to the "For First-Time Users" page.
security
Allowed IP Addresses to be specified in CIDR format
Allowed IP Addresses can now be specified in CIDR format (a writing style in which the IP address and subnet mask are expressed simultaneously).
For example, if you enter 192.0.2.0/24
, access is allowed from``192.0.2.0 to``192.0.2.255
.
Click the AddAllowed IPAddress button on the Security screen to go to the Add Allowed IP Address screen.
API Update
Data Source Facebook Ad Insights
The version of the Facebook API used above has been updated to v17.
Please refer to the Meta for Developers documentation for the new version.
2023-09-04
ETL Configuration
Data Source Microsoft Advertising
Data Source Microsoft Advertising is newly added.
Data can now be retrieved and transferred from Microsoft Advertising reports such as keywords and campaigns.
See Data Source - Microsoft Advertising for details on the various input fields.
Time Zone Setting
Time zone can now be set.
Any time zone can be applied to date and time specifications, such as the date and time displayed on the screen or the date and time of a scheduled execution of ETL Configuration.
For more information, see About Time Zone Settings.
2023-08-28
Data Mart Configuration
Data Mart Configuration in Google BigQuery Expands Choice of Partition Types
In Data Mart Configuration in Google BigQuery, we have added the options of monthly andyearlypartition types that can be specified in the query settings.
When setting up partitions, there are four partition types to choose from
- Hourly
- per day
- Every 1 month
- Every 1 year
2023-08-21
ETL Configuration
Add "card" to Data Source Twitter Ads to get data.
Data Source Twitter Ads now includes card as a target for retrieval.
This allows us to retrieve and transfer information such as the card's website URL.
For more information, see Data Source - Twitter Ads.
Enable/Disable header can be selected when output file format is CSV/TSV.
In the Output Option of ETL Configuration STEP2 , where the File/Storage Connector is the Data Destination, you can now select whether to enable or disable the CSV/TSV header for output.
If you do not need a header line, select Disable.
For more information, please refer to the section on output file format settings.
Connection Configuration
HTTP/HTTPS Connection Configuration, Authorization URL parameter can be added.
HTTP/HTTPS Connection Configuration, you can now add a parameter for the authorization URL.
Click on Configure Connection Configuration Details to see the parameters of the authorization URL.
Some services require certain parameters to be passed when obtaining authorization codes as a condition for obtaining a token. Please use this item in such cases.
For details, please refer to the HTTP/HTTPS Connection Configuration.
2023-08-07
Notice
On Monday, October 2, 2023, between 10:00 a.m. and 4:00 p.m., the method of logging into TROCCO will change.
For more information, please refer to the Change of Login Method to TROCCO.
ETL Configuration
Add Data Destination HubSpot Engagement Object
Engagement objects can now be selected in the object type in ETL Configuration STEP 1.
Data Destination for interaction-related data.
For more information, see Data Destination - HubSpot.
Added Replace Empty Characters option to Data Destination Snowflake Configuration.
In the Advanced Settings of STEP 1 of ETL Configuration, it is now possible to specify whether or not empty characters in ETL Configuration data are to be replaced with NULL.
Uncheck the box if you want Snowflake to transfer empty characters in the data to be transferred.
UI・UX
Workflow definitions for the destination are displayed on the various detail pages.
Various detail pages, such as ETL Configuration and Data Mart Configuration, which are embedded as tasks in Workflow, now display the embedded Workflow definitions.
You can check the details on the various pages below.
- ETL Configuration
- Managed ETL Configuration
- Data Mart Configuration
- Workflow
- dbt Job Setting
Added links to various detail screens to list items in the Add Workflow Task modal
Added links to the respective detail screens in the modal for adding various tasks on the flow edit screen.
This allows the contents of tasks to be viewed immediately from the flow editing screen.
- Managed ETL Configuration
- Data Mart Configuration
- Workflow
- dbt Job Setting
Improved searchability of the Data Mart Sync Job List screen
The Data Mart Configuration list can now be filtered by definition name in the sidebar of the Data Mart Sync Job List screen.
You can now easily view a list of any Data Mart Job by filtering by Data Mart Configuration name.
API Update
The Facebook API used for the following Connector has been updated to v16.
- Data Source Facebook Ad Insights
- Data Destination Facebook Offline Conversions
- Data Source Facebook Lead Ads
Please refer to the Meta for Developers documentation for the new version.
2023-07-31
Workflow
Expanded data warehouse with data checking within workflow
Data checking tasks for the following data warehouses can now be executed in a workflow.
- Snowflake
- Amazon Redshift
The results of the query against the data warehouse are checked against the error condition, and if the condition is met, the corresponding task is set to error.
For more information, see Workflow Data Check.
Custom Variable Loop Execution Edit Form with Snowflake Warehouse Suggestions
In the input field for specifying a Snowflake warehouse as the target of Custom Variable loop execution, the warehouse associated with the Connection Configuration to be used is now displayed as a suggestion.
ETL Configuration
Data Source Adobe Marketo Engage added an item that allows Custom Variables to be embedded.
Custom Variables can be embedded in the following input fields.
- Filter type when custom object is selected as target
- Workspace when folder is selected as target
API Update
Facebook API updated to v16
The Facebook API used for the following Connector has been updated from v15 to v16.
- Data Source Facebook Ad Creative
- Data Destination Facebook Conversions API
Please refer to the Meta for Developers documentation for the new version.
UI・UX
Added ETL Configuration details link when selecting ETL Configuration for workflow.
A link to the ETL Configuration Details screen has been added to the Add TROCCO Transfer Job Task modal in the Workflow.
This allows the user to move to the ETL Configuration details screen when selecting an ETL Configuration, and to make a selection after reviewing the contents.
2023-07-24
ETL Configuration
OAuth 2.0 added to Data Source HTTP/HTTPS authorization method.
OAuth 2.0 is added as an authorization method.
This allows data to be retrieved from data sources that require OAuth authentication.
For more information, see Data Source - HTTP/HTTPS.
Added Data Source Zendesk Support to retrieve data.
Added ticket_comments to be retrieved by Data Source Zendesk Support.
It is now possible to retrieve and forward comment data related to tickets, including the text of the comment and the ID of the sender of the comment.
For more information, see Data Source - Zendesk Support.
Data Source Adobe Marketo Engage added an item that allows Custom Variables to be embedded.
Custom Variables can now be embedded in the following input fields.
- Program ID filter when selecting a program member as target
- List ID filter when selecting leads by static list as target
XML (beta version) support for input file format
XML (beta version)" has been added in the ETL Configuration of the following Connector.
- Data Source Google Drive
- Data Source HTTP/HTTPS
For more information, see About input file format settings.
TROCCO API
Added parameter to specify time zone in job execution request
Added time_zone as a request parameter for ETL Job and Workflow Job execution.
This allows you to specify the time zone for the date and time specified in context_time.
For more information, see About the TROCCO API.
2023-07-18
API Update
Data Source Criteo
API version has been updated to v2023.04.
Please refer to Version 2023.04 release notes for more information about the new version.
ETL Configuration
Expanded columns transferred from Data Source Facebook Ad Creative.
Three columns have been added to the data transferred from Data Source Facebook Ad Creative.
The following column names are displayed in the preview of ETL Configuration STEP2 and ETL Configuration Details.
ad_creative_object_story_spec_video_data_video_id
ad_creative_object_story_spec_child_attachments_image_hash
ad_creative_asset_feed_spec_image_hash
Data Destination HubSpot custom object support
Custom objects can now be selected for the object type in ETL Configuration STEP 1.
This can be used to transfer data that cannot be categorized by HubSpot's standard objects.
Custom objects are displayed as "xxxxxx (custom object)".
Connection Configuration
Roles can now be specified in Snowflake Connection Configuration.
Roles can now be specified in Snowflake Connection Configuration.
Roles that grant the necessary permissions to access Snowflake from TROCCO can be tied to Connection Configuration.
If not entered, Snowflake's default settings are used.
Additional hosts selectable in Braze Connection Configuration
The rest.fra-02.braze.eu has been added to the host choices in Connection Configuration.
UI・UX
Fixed header size of various data mart screens changes in response to scrolling
Scrolling down in the New, Detail, and Edit pages of the Data Mart will reduce the height of the fixed header.
The display area for the contents of Data Mart Configuration has been widened, increasing the amount of information displayed on the screen.
Before Scroll | After scrolling |
---|---|
2023-07-10
ETL Configuration
Data Source Yahoo! Search Ads now supports CampaignExportService.
CampaignExportService has been added to the services for data acquisition that can be selected in ETL Configuration STEP1.
See Data Source - Yahoo! Search Ads for the fields retrieved by the CampaignExportService.
2023-07-03
ETL Configuration
Data Source TROCCO enables ETL Job execution history for Data Mart and Workflow
Data Mart and****Workflow have been added to the list of data to be transferred.
ETL Job transfers historical data about previously executed data mart and workflow jobs.
See Data Source - TROCCO for details.
Data Destination Amazon Redshift allows batch size to be specified.
The batch size can now be specified in the advanced settings of ETL Configuration STEP 1.
If an error due to insufficient memory occurs during transfer, the batch size can be adjusted to eliminate the error.
For more information, see Data Destination - Amazon Redshift.
Managed ETL
Add PostgreSQL as Data Source
PostgreSQL can now be selected as the Data Source for Managed ETL.
You can import PostgreSQL tables in a batch and create the associated ETL Configuration in one place.
See Managed ETL Configuration for various entry fields.
UI・UX
Expanded edit area on workflow edit screen
The add task sidebar on the workflow editing screen can now be opened and closed.
By closing the sidebar, a larger display area can be used for workflow editing.
Sidebar (open) | Sidebar (closed) |
---|---|
Improved visibility of data lineage on data mart detail pages
Data lineage information on the data mart detail page is now displayed in a hierarchical structure.
Compared to before the change, where the hierarchical structure was not represented, it is now easier to see the relationship between data sets and tables.
2023-06-27
ETL Configuration
XML (beta version)" added as input file format
XML files can now be selected as the input file format for ETL Configuration for some Data Source file and storage-based Connectors.
For more information, see About input file format settings.
Data Source Zendesk Support
Ticket_metrics
has been added to the data acquisition target.
Various indicator data about the ticket, such as the date and time it was resolved and the time of the first reply, can now be retrieved and forwarded.
For more information, see Data Source - Zendesk Support.
Connection Configuration
JDBC Driver Selection for MySQL
The JDBC driver can now be selected in the MySQL Connection Configuration.
For details, please refer to the Data Source - RDBMS Version Mapping Table.
2023-06-19
ETL Configuration
Data Destination Snowflake
UPSERT (MERGE)
has been added as a transfer mode.
If there is a record in the existing table based on the merge key, the record is updated; if there is no record, the record is appended.
For more information, see Data Destination - Snowflake.
Data Source TikTok Ads
Support for loading of advertiser IDs has been added.
By clicking on "Load Advertiser ID, " the advertiser ID associated with the Connection Configuration you selected earlier will be suggested.
This allows you to create ETL Configurations without having to manually enter the advertiser ID.
Connection Configuration
Google Analytics 4
Google account (OAuth) has been added as an authentication method.
This allows Connection Configuration to be created without creating a JSON Key in the service account.
UI・UX
Custom Variable Definition Form
The design has been substantially redesigned.
2023-06-12
ETL Configuration
Data Source Google Analytics 4
Data Source Google Analytics 4 is newly added.
Please refer to Data Source - Google Analytics 4 for more information on the various input fields.
2023-05-29
ETL Configuration
Data Source TROCCO supports ETL Job execution history transfer
The history of ETL Job execution can now be transferred from Data Source TROCCO.
Historical data of ETL Jobs executed up to one year in the past can be transferred.
See Data Source - TROCCO for details.
Data Destination Google Sheets to allow sorting of data.
Data Order can now be set from the Advanced Settings in ETL Configuration STEP 1.
Records can be sorted by sort key name and sort order.
See Data Destination - Google Sheets for more information.
Data Source Box can now select the decompression format for data transfers.
The decompression format can now be selected in ETL Configuration STEP1.
If the data to be transferred from Box is compressed, it will be decompressed and transferred in the selected format.
The decompression method can be selected from the following four options
bzip2
gzip
tar.gz
zip
Custom Variable embedding support
ETL Configuration and Data Mart have been expanded to include more input fields in which Custom Variables can be embedded.
Custom Variables, which can be dynamically populated at runtime, are now available in more Connector services.
ETL Configuration
- Data Source Google BigQuery
- Temporary Table Creation Destination Dataset
- Data Destination Amazon S3
- Bucket."
- Data Destination Google Cloud Storage
- Bucket."
- Data Destination Google BigQuery
- Dataset."
- Data Destination FTP/FTPS
- Path prefix."
- Data Destination PostgreSQL
- Database," "schema," "table."
- Data Destination SFTP
- Path prefix."
datamart
- Snowflake
- Warehouse," "output destination database," and "output destination schema."
2023-05-22
ETL Configuration
Expanded Decompression Format Options for Data Source Google Cloud Storage
bzip2
" and " gzip"
have been added to the "Extract Format" options in STEP 1 of ETL Configuration.
When transferring compressed data from Google Cloud Storage, you can choose from the following four decompression formats
bzip2
gzip
tar.gz
zip
Data Destination FTP/FTPS now has selectable transfer mode.
The "Transfer Mode" can now be selected in ETL Configuration STEP 1.
You can choose from the following two options
Parallel transfer."
Apply parallel processing to perform the transfer.
Compared to "output file number suppression transfer," the transfer time is reduced.
Due to parallel processing, a file retrieved from Data Source may be split into multiple files and sent to Data Destination.
Output File Count Suppression Transfer
Performs transfers without applying parallel processing.
Unlike "Parallel Transfer," files retrieved from Data Source are sent to Data Destination without being split.
Custom Variable embedding support
ETL Configuration and Data Mart have been expanded to include more input fields in which Custom Variables can be embedded.
Custom Variables, which can be dynamically populated at runtime, are now available in more Connector services.
ETL Configuration
- Data Source Amazon Redshift
- Database, Schema.
- Data Source Amazon S3
- Bucket."
- Data Source Azure Blob Storage
- Path prefix."
- Data Source Box
- Folder ID.
- Data Source Google Sheets
- URL of the spreadsheet."
- Data Source Google BigQuery
- Temporary Table Creation Destination Dataset
- Data Source Google Cloud Storage
- Bucket."
- Data Source Google Drive
- Folder ID.
- Data Source PostgreSQL
- Database, Schema.
- Data Source Snowflake
- Warehouse," "database," and "schema."
- Data Destination PostgreSQL
- Database," "schema," "table."
- Data Destination Google BigQuery
- Dataset."
- Data Destination Google Drive
- Folder ID.
datamart
- Google BigQuery
- Destination data set.
2023-05-15
ETL Configuration
Added "Upper Case Conversion" and "Upper Snake Case Conversion" as batch conversion formats for column names
In ETL Configuration STEP 2 Column Setting, "Upper Case Conversion" and "Upper Snake Case Conversion" have been added to the conversion format for Batch Column Name Conversion.
Custom Variable Support for Data Destination Google Ads Conversions
Custom Variables are now available for the following items in the ETL Configuration of Data Destination Google Ads Conversions.
- Customer ID
- Conversion Action ID
UI・UX
Fixed header size in ETL Configuration details changes according to scrolling
Scrolling down in the ETL Configuration details screen will reduce the height of the fixed header.
The display area for the contents of ETL Configuration has been widened, increasing the amount of information displayed on the screen.
Before Scroll | After scrolling |
---|---|
Data Catalog
Full text display of table logical names in ER diagrams
Full-text display was previously available only when the table name was long, but now full-text display is available on hover even when the logical name is long.
2023-05-08
Custom Variable support for DWH services
**Custom Variable Loop Execution in Workflow allows for flexible loop processing when Custom Variables are set in the DWH service. **
The items that are now supported for Custom Variable input are as follows
ETL Configuration
- Data Destination Snowflake
- warehouse
- database
- schema
- table
- Data Destination Redshift
- database
- schema
- table
- Amazon S3 bucket
- Amazon S3 key prefix
- Data Source BigQuery
- Data export destination Google Cloud Storage URI
datamart
- Redshift
- database
- schema
- table
ETL Configuration
Page Size" can be specified in Data Source Zendesk Support
Data Destination Zendesk Support's "Page Size" can now specify the number of items that can be retrieved in a single request.
Improved error handling in Data Destination Salesforce
Data Destination Salesforce forwarding can now set the forwarding status to Error when a record fails to send.
dbt linkage
Target" and "Location" can now be specified in dbt Job Settings
Target" and "Location" can now be specified in dbt Job Settings.
The "Location" will only appear in the dbt Job Setting with the BigQuery selected dbt Git repository as the adapter.
2023-04-24
UI・UX
Improved usability of sidebar
The hierarchical structure and order of items in the sidebar displayed on the left side of the screen has been changed.
Functions with multiple pages can now be collapsed in item-by-item chunks, and items on the currently displayed page will be collapsed and expanded.
2023-04-17
ETL Configuration
Data Destination Yahoo!JAPAN Ads Display Ads Conversion Measurement API (Beta Feature)
Data Destination Yahoo! JAPAN Ads Display Ads Conversion Measurement API (Beta Feature) has been newly added.
For details on the various input fields and column mappings, please refer to Data Destination - Yahoo!
Data Destination Snowflake
In ETL Configuration STEP1, a list of items that can be entered in the following resources can now be read based on the specified Connection Configuration.
- Warehouse Name
- database
- schema
UI・UX
Expanded workflow task information
Workflow tasks now display the creator. Also, a message is now displayed if you do not have permission to view the file.
2023-04-10
dbt linkage
Official Release
The dbt integration feature, which was offered as an optional feature in the beta version, is now available as an official version.
This allows you to use this function without having to contact your contact person.
For more information, see About dbt Linkage.
Location settings for BigQuery datasets to connect to
On the Create/Edit dbt Job Setting screen, you can now enter a location when you select a dbt Git repository with BigQuery selected as the adapter.
A destination data set is created at the location entered.
ETL Configuration
Data Destination Twitter Ads Web Conversions
Data Destination Twitter Ads Web Conversion is a new addition.
See Data Destination - Twitter Ads Web Conversions for more information on the various input fields and column mappings.
Data Source Snowflake
In ETL Configuration STEP1, a list of items that can be entered in the following resources can now be read based on the specified Connection Configuration.
- Warehouse Name
- database
- schema
UI・UX
Improved visibility of workflow loop execution forms
The layout of the workflow loop execution form has been revised to facilitate deletion and other operations.
2023-04-03
ETL Configuration
Data Destination LINE Conversion API
A new Data Destination LINE Conversion API has been added.
For more information on the various input fields and column mappings, please refer to Data Destination - LINE Conversion API.
Workflow
Custom Variable loop execution supports Snowflake queries
Custom Variable loop execution now supports Snowflake queries.
Custom Variable expansion values in a loop run can be set based on the results of a Snowflake query.
API Update
Data Source Shopify
- The Shopify API used for the above Connector has been updated to 2023-01.
- Please refer to the documentation in the Shopify developer documentation for the new version.
2023-03-27
UI・UX
Expanded dashboard on TROCCO home screen
- Workflow definitions are now displayed on the dashboard of TROCCO's home screen.
2023-03-20
ETL Configuration
Data Source Google Search Console
- The data to be acquired can now be filtered by dimension filters.
- Clicking on the Set Details in E TL Configuration STEP 1 will bring up the Dimension Filters.
- Currently, only PAGE items are supported.
Data Catalog
Logic for obtaining metadata when using team functions
- Metadata about the data source (e.g., metadata about MySQL) is obtained by connecting to the data source using Connection Configuration on TROCCO, which is available to Data Catalog managers.
- When the Team function is used in conjunction with the Team function, only Connection Configuration for which the Data Catalog administrator has the Operator role or higher privileges will be used.
For more information on this matter, please refer to the specifications for obtaining catalog data and metadata.
UI・UX
Visualization of Connector usage
- Connector usage is now displayed on the TROCCO home screen.
Improved visibility of Connector list
- Each Connector is now more clearly displayed on the Service Selection screen when creating ETL Configuration and on the list of supported services on the Home screen.
API Update
Data Destination Facebook Conversions API
- The Facebook API used for the above Connector has been updated from v14 to v15.
- Please refer to the Meta for Developers documentation for the new version.
2023-03-06
Notice
The name of each Connector of TROCCO has been changed as follows.
Before change | After change |
---|---|
Aurora MySQL | Amazon Aurora MySQL |
Cisco AMP | Cisco Secure Endpoint |
CloudWatch Logs | Amazon CloudWatch Logs |
DynamoDB | Amazon DynamoDB |
GitHub(GraphQL) | GitHub GraphQL API |
Marketo | Adobe Marketo Engage |
Pardot | Salesforce Marketing Cloud Account Engagement |
SQL Server | Microsoft SQL Server |
Tableau CRM | Tableau CRM Analytics |
2023-02-27
ETL Configuration
Data Destination Google Analytics 4 Measurement Protocol
Data Destination Google Analytics 4 Measurement Protocol is newly added.
For more information on the various input fields and column mappings, see Data Destination - Google Analytics 4 Measurement Protocol.
2023-02-20
UI・UX
tutorial dialog
- A tutorial dialog is now available for first-time TROCCO users.
- By following the three steps in the dialog, you will understand the basic functions of TROCCO.
- This dialog will be hidden after the three steps of "Creating Connection Configuration," "Creating ETL Configuration," and "Executing ETL Job" are completed.
Markdown notation support for memos in each setting
- Markdown notation is now supported for the memo fields in the following settings.
- ETL Configuration
- Data Mart Configuration
- Workflow
ETL Configuration
Microsoft Excel file
- Microsoft Excel files can now be transferred faster.
- In the File/Storage Connector, when a Microsoft Excel file is selected as the input file format, you can now choose how to retrieve the values.
- By selecting the use of cache as****the method for retrieving values, they are transferred faster than before.
If you select "Use Cache" in the column configuration, ETL Configuration will use the values at the time the Microsoft Excel Files are saved locally for the transfer.
Therefore, functions that change the result of calculation each time, such as date/time functions (e.g., TODAY function) and random number generation functions (e.g., RAND function), are not recalculated when the transfer is executed. Please note
Select Recalculate on Transfer if the formulas in the cells need to be recalculated when the transfer is executed.
Data Catalog
Metadata Import
- For CSV files used for metadata import, template files can now be downloaded from the screen.
- You can download a CSV file with pre-loaded header rows for your Data Catalog.
- See Metadata Import for more information.
Workflow
Flow screen
Multiple tasks can now be selected and moved together.
Error indication during loop execution in query results
An error message is now displayed when a query written in a loop in****a Google BigQuery query result or a loop in an Amazon Redshift query result fails to execute.
Managed ETL
Label Setting
- Labels can now be installed and removed in batches, even after Managed ETL Configuration has been created.
- Batch installation and removal can be performed from both the Advanced and Edit screens of Managed ETL Configuration.
Connection Configuration
Google BigQuery
The list of projects that can be selected can now be read for the project ID to be specified when a Google account (OAuth) is selected as the authentication method.
API Update
Data Source Google Ads
The version of Google Ads API has been updated from v11 to v12.
Please refer to the Google Ads API documentation for information on the new version.
2023-02-13
Managed ETL
Data Source Salesforce added
- Data Source has been added to Salesforce.
- ETL Configuration can be created for all objects connected to Connection Configuration at once.
2023-02-06
Notice
Data Source LINE Ads
As of 00:00, 02/01/2023, Data Source Using Scraping - LINE Ads (Discontinued) is no longer offered.
Thereafter, executing an ETL Job using the Data Source - LINE Ads (to be discontinued) Job Setting will result in an error.
If you wish to create a new ETL Configuration in the future, please use the LINE Ads API-based ETL Configuration - Data Source - LINE Ads.
Workflow
Google BigQuery data check
- Job IDs are now displayed in the execution log.
2023-01-23
Data Catalog
- The Data Catalog feature of the Snowflake version is now available.
- Data Catalogs can be created for the data associated with a Snowflake account.
- Some features are not supported by the Snowflake version. For details, please refer to the list of functions supported by each data store.
Workflow
- Workflow definitions can now be duplicated.
- You can duplicate a Workflow definition from the hamburger menu in the upper right corner of the Workflow definition details screen.
- You can duplicate a Workflow definition from the hamburger menu in the upper right corner of the Workflow definition details screen.
API Update
Data Destination Facebook Offline Conversions
- The Facebook API used for the above Data Destination has been updated from v14 to v15.
- Please refer to the Meta for Developers documentation for the new version.
2023-01-16
ETL Configuration
Data Source Google Ads
- The following resource types (report types) have been added
ad_group_asset
customer_asset
dbt linkage
- The supported version has been updated from v1.2 to v1.3.
Data Catalog
- A link to the column reference list has been added to the table information screen.
- Clicking on the link will display a list of Column Setting references defined in the table in question.
- For more information on column references, see Column References.
2023-01-10
API Update
Data Source Facebook Ad Insights
- The Facebook API used for the above Data Source has been updated from v14 to v15.
- Please refer to the Meta for Developers documentation for the new version.
Connection Configuration
PostgreSQL
- You can now select the version of the JDBC driver that connects to the PostgreSQL server.
- Please select the driver version according to the version of PostgreSQL you are using.
- PostgreSQL 8.2 or higher:
JDBC Driver 42.5.1
- Less than PostgreSQL 8.2:
9.4 1205 JDBC 41
- PostgreSQL 8.2 or higher:
- Please select the driver version according to the version of PostgreSQL you are using.