Contents x
    Data Source - Azure Blob Storage
    • 07 Dec 2022
    • Print
    • Dark
      Light
    • PDF
    Contents

    Data Source - Azure Blob Storage

    • Updated on 07 Dec 2022
    • Print
    • Dark
      Light
    • PDF
    Article Summary

    Overview

    Help page for setting up a data transfer from Azure Blob Storage.

    Supported Protocol

    Data Loader: Embulk
    Using the embulk-input-azure_blob_storage plugin.

    Limitations

    None.

    Settings

    STEP 1: General Settings

    Field Name
    Required
    Default Value
    Description
    Azure Blob Storage Connection SettingsYes-From the Azure Blob Storage Connection Settings registered in advance, select the one with the necessary permissions for this transfer.
    If you have not previously registered any connection settings, click the Add Connection Settings button to create a new one.
    ContainerYes-Enter the container name storing the data that you wish to transfer.
    Path PrefixYes-Specify the path prefix of the data you wish to transfer.
    If you specify up to the directory name, all files under the directory (matching the prefix) will be transferred.
    If you specify up to a file name, only that file will be transferred.
    In this field, you can use custom variables to embed a value that will be dynamically set when the job executes.
    Input File FormatYesCSVSelect the input file format from the following options:
  • CSV
  • JSON Lines
  • JSONPath
  • LTSV
    • Transfer MethodYesFull LoadChoose between the Full Load or Incremental Load transfer methods.
    If Incremental Load is selected, only new data since the previous job will be transferred.

    STEP 2: Advanced Settings

    Field Name
    Default Value
    Description
    Path Regex-In this field, you can specify a regular expression to determine whether or not to transfer certain files under the path prefix.
    If unspecified, all files under the prefix will be transferred.
    Maximum Number of Files to Load-Here you can set the maximum number of files to load in one transfer (by default there is no limit).
    Was this article helpful?
    What's Next