Data Destination - S3
- 06 Dec 2022
- Print
- DarkLight
- PDF
Data Destination - S3
- Updated on 06 Dec 2022
- Print
- DarkLight
- PDF
Article Summary
Share feedback
Thanks for sharing your feedback!
Overview
Help page for setting up a data transfer to AWS S3.
Supported Protocol
Dataloader: Embulk
- Using the embulk-output-s3 plugn
- Using the embulk-output-s3 plugn
Single File Output (limited to when the source is BigQuery)
Settings
Step 1: General Settings
| S3 Connection Configuration | Yes | - | From the S3 Connection Configuration created in advance, select the one with the required permissions for this transfer. If you have not previously created any Connection Configurations, click the Add Connection Configuration button to set up a new one. |
| Region | Yes | ap-northeast-1 | Enter the region that you specified when creating the bucket. For further details regarding regions, please refer to the following AWS documentation. |
| Bucket | Yes | - | Specify the name of bucket at the destination. The bucket specified must exist before creating this ETL configuration. |
| Path Prefix | Yes | - | Specify the path prefix at the destination for the data that you wish to transfer. Starting from the path prefix you specify, trocco will output multiple files into the bucket at the destination. |
| Access Control List | Yes | - | The access control list (ACL) manages the access permissions for your objects and buckets. Choose an ACL predefined by Amazon S3. For further details regarding ACL, please refer to the following AWS documentation. |
| Output File Format | Yes | CSV/TSV | Select the output file format. For further details, please refer to trocco's Output File Format support documentation. |
| File Compression | No | No Compression | If you wish to compress your files, you can select between the gzip or bzip2 compression methods. |
Step 2: Advanced Settings
| Naming Convention | .%03d.%02d | In this field you can set the rules for file naming, when outputting multiple files |
Was this article helpful?