Contents x
    Configuring Output Files
    • 07 Dec 2022
    • Print
    • Dark
      Light
    • PDF
    Contents

    Configuring Output Files

    • Updated on 07 Dec 2022
    • Print
    • Dark
      Light
    • PDF
    Article Summary

    Note

    This is a machine-translated version of the original Japanese article.
    Please understand that some of the information contained on this page may be inaccurate.

    summary

    This is the help page of "Output file format" that can be specified at the transfer destination in the transfer settings.

    What is the output file format?

    If the destination is file storage such as S3 or SFTP, you need to specify the format of the file to be transferred. Currently, you can choose from two types: CSV and JSON Lines. You can also make detailed settings for each format when importing files.

    About advanced settings

    .CSV

    Here is the site I used as a reference.

    Item namedefault valuedescription
    Delimiter,You can set the data delimiter for CSV.
    Newline characterCRLFYou can choose the rule for line breaks: CR, CRLF, or LF.
    Newline character in fieldsLFYou can choose between CR, CRLF, and LF rules for line breaks in the field.
    Character codeUFT-8You can set the character encoding.
    How to assign quotesMINIMALRules of quotes
    ・ALL (quote all fields)
    ・MINIMAL (quotes only when necessary, only when there is a delimiter etc. in the field)
    ・NONE (do not do quotes)
    You can choose from:
    Escape characters\You can set escape characters.
    Characters to convert to NULL-You can set the character to be treated as NULL.
    Default time zoneUTCYou can set the time zone for time output.
    You can also override this setting with column settings.
    Column settingsformat:%Y-%m-%d %H:%M:%S.%6N %zFor timestamp columns, you can specify the name of the column and set the time zone and format for that column.

    JSON Lines

    I'm using embulk-formatter-jsonl.

    Item namedefault valuedescription
    Character codeUTF-8Character code
    ・UTF-8
    ・UTF-16LE
    ・UTF-32BE
    ・UTF-32LE
    ・UTF-32BE
    You can choose from:
    Newline characterLFYou can choose between CRLF, LF, CR, NUL, and NO for newline character rules.
    Learn more about each rule here.
    Date format-You can specify how the date is formatted.
    Please see here for details.
    Time zone-You can set the time zone.
    To use this option, you must also set the date format.
    Was this article helpful?
    What's Next