Contents x
    Data Source - Amazon S3
      • Print
      • PDF
      Contents

      Data Source - Amazon S3

        • Print
        • PDF
        Article summary

        summary

        Help Page for ETL Configuration of Data Settings from Amazon Web Services S3.

        constraints

        Setting items

        STEP1 Basic settings

        item nameindispensabledefault valueContents
        S3 Connection ConfigurationYes-Select the previously registered Connection Configuration that has the necessary permissions for this ETL Configuration.
        Please refer to another page for Connection Configuration.
        regionYesap-northeast-1Enter the Region of the Data Source S3 bucket.
        Please refer to the official AWS page for an explanation of regions.
        bucketYes-Enter the name of the Data Source S3 bucket.
        path prefixNo-Specify the data to be transferred by path prefix.
        If you specify up to the directory name, all files under the directory (matching the prefix) are transferred.
        On the other hand, if you specify up to the file name, only that file will be transferred.
        Custom Variables can also be used to dynamically determine the value of a setting during TROCCO's data ETL Configuration.
        path regular expressionNo-By entering a regular expression, you can specify which files under the path prefix are to be transferred.
        If not entered, all files matching the path prefix will be included.
        decompression formatYesUncompressed files or bzip2/gzipIf the Data Source data is compressed, it can be decompressed and transferred.
        The following decompression formats are supported Select according to the compression format of the data to be transferred.
      • Uncompressed files or bzip2/gzip
      • zip
      • tar.gz

        • If you change the decompression format later, be sure to perform Automatic Data Setting.
        Relative path after decompression (regular expression)No-This can be entered when zip or tar.gz is selected as the decompression format.
        If you include a path separator, escape it using a backslash ( ⌘).
        Input file formatYesCSV/TSVSpecify the format of the input file.
        For more information, see About input file format settings.
        CSV/TSV input settingsYesCSV/TSV files with header columnsSelect this option when CSV/TSV is selected as the input file format.
        If you select a CSV/TSV file that does not contain a header column, a temporary header name generated on the TROCCO side will be assigned to the column name.
        Transfer MethodYesFull Data TransferSelect either Full Data Transfer or Incremental Data Transfer.
        For more information on Incremental Data Transfer, see Incremental Data Transfer Function.

        STEP2 Detailed settings

        item nameindispensabledefault valueContents
        Whether to continue the transfer if the file does not exist in the specified path.YesContinue transferYou can choose whether or not to continue the transfer if the file is not found in the path specified in the Basic ETL Configuration.
        If you choose to continue the transfer, the ETL Job will be completed assuming that 0 items were transferred.
        If you choose to abort the transfer and make an error, the transfer job is aborted as an error.
        Was this article helpful?
        What's Next