About Managed ETL
- Print
- PDF
About Managed ETL
- Print
- PDF
Article summary
Did you find this summary helpful?
Thank you for your feedback
Help page on Managed ETL.
This page provides an overview of Managed ETL.
See Managed ETL Configuration for specific configuration procedures and detailed configuration items.
Plan Restrictions
Managed ETL is available only for accounts subscribed to the Free or Essential plans and above.
summary
Managed ETL allows you to import Data Sources (e.g., RDBMS tables, Salesforce objects, etc.) from the Data Source in bulk and create a centralized ETL Configuration that is tied to the source.
You can also manage all the ETL Configurations you have created at once.
Managed ETL eliminates the need to manually create ETL Configurations one by one, thus saving time.
Managed ETL Configuration and ETL Configuration
When a Managed ETL Configuration is created, one ETL Configuration is created for each individual Data Source (e.g., RDBMS table, Salesforce object, etc.) from which it is transferred.
These individual ETL Configurations are created in conjunction with the Managed ETL Configuration.
In addition, settings made when creating or editing a Managed ETL Configuration apply to each individual ETL Configuration associated with it.
Format of individual ETL Configuration
The names of the individual ETL Configurations will be in the following format
<Managed ETL Configuration Name>_<Data Source Name>.
Created/Dropped Tables Detection
Managed ETL can detect Created/Dropped Tables in the Data Source (e.g., RDBMS tables, Salesforce objects, etc.).
Any increase or decrease in the number of data sources detected can also be reflected in the existing Managed ETL Configuration.
- When a new Data Source addition is detected at the Data Source
- You can create a new ETL Configuration that is tied to your Managed ETL Configuration.
- When a Data Source deletion is detected at the Data Source
- You can delete forwarding configurations that were already tied to Managed ETL Configuration.
Job execution through workflow
Workflow definitions can be used to run Jobs on a Managed ETL Configuration basis.
At this time, all ETL Job Settings associated with the Managed ETL Configuration will be executed.
Incorporate the TROCCO Managed ETL Configuration into your flow in the Flow Editor.
Notification function for Created/Dropped Tables detection
When Data Source detects Created/Dropped Tables in Data Source (e.g., RDBMS tables, Salesforce objects, etc.), it can also notify Slack of the detection results.
Created/Dropped Tables detection is notified when a workflow incorporating Managed ETL Configuration is executed with this feature enabled.
For details, see "Notification Settings for Table Created/Dropped Tables Detection Results" in Managed ETL Configuration.
Supported Connectors
Data Source
- Microsoft SQL Server
- MySQL
- PostgreSQL
- Salesforce
Data Destination
- Amazon Redshift
- Databricks
- Google BigQuery
- Snowflake
precautions
Deleting Managed ETL Configuration
Deleting Managed ETL Configuration will delete the **associated ETL Configuration. **
Note that the Data Settings on TROCCO will be deleted, but the Data Source (e.g., RDBMS tables, Salesforce objects, etc.) in the ETL Configuration will not be deleted.
Change in forwarding configuration tied to Managed ETL Configuration
Forwarding Configurations tied to Managed ETL Configuration can be edited separately.
However, if you edit Managed ETL Configuration after editing the associated ETL Configuration, any changes you make to the associated ETL Configuration will be overwritten.
If you are using the Team function
Managed ETL Configuration is treated as a resource in TROCCO.
- Resource Definition: See Team Functions.
- Permissions to operate on resources: see Resource Group.
Resource Group to which the individual ETL Configuration belongs
- When creating or editing Managed ETL Configuration
- If you add to a Resource Group or change your affiliation to another Resource Group, any associated ETL Configuration will belong to the same Resource Group as the Managed ETL Configuration.
- When editing individual ETL Configurations
- If you change the Resource Group to which you belong, only the Resource Group for that ETL Configuration will be changed.
- Managed ETL Configuration and other associated ETL Configurations will not be affected.
Was this article helpful?