Data Source - Azure Blob Storage
  • 07 Dec 2022
  • PDF

Data Source - Azure Blob Storage

  • PDF

Article summary

Overview

Help page for setting up a data transfer from Azure Blob Storage.

Supported Protocol

Data Loader: Embulk
Using the embulk-input-azure_blob_storage plugin.

Limitations

None.

Settings

STEP 1: General Settings

Field Name
Required
Default Value
Description
Azure Blob Storage Connection SettingsYes-From the Azure Blob Storage Connection Settings registered in advance, select the one with the necessary permissions for this transfer.
If you have not previously registered any connection settings, click the Add Connection Settings button to create a new one.
ContainerYes-Enter the container name storing the data that you wish to transfer.
Path PrefixYes-Specify the path prefix of the data you wish to transfer.
If you specify up to the directory name, all files under the directory (matching the prefix) will be transferred.
If you specify up to a file name, only that file will be transferred.
In this field, you can use custom variables to embed a value that will be dynamically set when the job executes.
Input File FormatYesCSVSelect the input file format from the following options:
  • CSV
  • JSON Lines
  • JSONPath
  • LTSV
  • Transfer MethodYesFull LoadChoose between the Full Load or Incremental Load transfer methods.
    If Incremental Load is selected, only new data since the previous job will be transferred.

    STEP 2: Advanced Settings

    Field Name
    Default Value
    Description
    Path Regex-In this field, you can specify a regular expression to determine whether or not to transfer certain files under the path prefix.
    If unspecified, all files under the prefix will be transferred.
    Maximum Number of Files to Load-Here you can set the maximum number of files to load in one transfer (by default there is no limit).

    Was this article helpful?