Data Source - Google Cloud Storage
- 24 Jan 2023
- Print
- DarkLight
- PDF
Data Source - Google Cloud Storage
- Updated on 24 Jan 2023
- Print
- DarkLight
- PDF
Article Summary
Share feedback
Thanks for sharing your feedback!
This is a machine-translated version of the original Japanese article.
Please understand that some of the information contained on this page may be inaccurate.
Overview
This is a help page for setting up data transfer from Google Cloud Storage on the Google Cloud Platform.
Supported Protocols
- Data Transfer (Embulk)
Using embulk-input-gcs
Limitations
Settings
STEP 1: General Settings
Field Name | Required | Default Value | Description |
---|---|---|---|
Google Cloud Storage connection information | Yes | - | From the connection information registered in advance, select the one that has the necessary permissions for this transfer setting. Please refer to the separate page for how to set the connection information. |
bucket | Yes | - | Specify the name of the bucket from which you want to transfer data. |
Path prefix | Yes | - | Specify the path prefix to which the data is transferred. trocco transfers multiple files starting with the path prefix of the source bucket. |
Input file format | Yes | CSV/TSV | Specify the format of the input file. For details, refer to About setting the input file format. |
Number of transfers | Yes | Transfer All | Here you can choose to transfer all records or transfer. Learn more about differential transfers. |
STEP2: Advanced settings
Field Name | Required | Default Value | Description |
---|---|---|---|
Whether to continue the transfer if the file does not exist in the specified path | Yes | Continue the transfer | You can choose whether to continue the transfer if the file is not found in the path specified in the basic settings. If you select Continue transfer, the transfer job will be completed as if 0 records have been transferred. If you select Abort transfer and Make error, aborts the transfer job as an error. |
Required Permissions
The permissions required to use this service are as follows.
- storage.buckets.get
- storage.buckets.list
- storage.objects.get
- storage.objects.list
Was this article helpful?