Contents x
    Data Source - MySQL
    • 11 Apr 2022
    • Print
    • Dark
      Light
    • PDF
    Contents

    Data Source - MySQL

    • Updated on 11 Apr 2022
    • Print
    • Dark
      Light
    • PDF
    Article Summary

    Overview

    Help page for setting up a data transfer from MySQL.

    Supported Protocol

    Data Loader: Embulk
    Using the embulk-input-mysql plugin.

    Limitations

    None.

    Settings

    Step 1: General Settings

    Field Name
    Required
    Default Value
    Description
    MySQL Connection SettingsYes-From the MySQL Connection Configurations created in advance, select the one with the necessary permissions for this transfer.
    If you have not previously created any Connection Configurations, click the Add Connection Configuration button to set up a new one.
    DatabaseYes-Enter the MySQL database name containing the data you wish to transfer.
    By pressing the Load Database List button, you can update your MySQL database list, used to auto-complete the field.
    Transfer MethodYesQueryChoose between either the Query or Incremental Load transfer methods.
    QueryYes-You must complete this field when the transfer method is set to Query.
    In this field, write an SQL query to extract the data you wish to transfer to from MySQL.
    Here you can use custom variables to embed a value that will be dynamically set when the job executes.
    TableYes-You must complete this field when the transfer method is set to Incremental Load.
    Enter the table name that contains the data you wish to transfer.
    Incremental Load Column NameNo-You can enter into this field when the transfer method is set to Incremental Load.
    For each transfer, the last value loaded from the column specified here will be saved as the Last Record Loaded.
    From the next job, only values higher than the Last Record Loaded, will be transferred.
    When using a unique and auto-increment column (e.g. id, created_at, etc.), enter it here.
    Use comma separation to specify multiple columns.
    If unspecified, a primary key is used.
    Last Record LoadedNo-You can enter into this field when the transfer method is set to Incremental Load.
    Only data with a value higher than the value specified here will be transferred.
    If unspecified, a full load will occur.
    Do not change this value without reason (such as if you need to re-load data from a specific date range), as there is a chance of data duplication.
    Default Time ZoneNo-Enter the time zone set server side in MySQL.
    For example, if the time zone is set as Japan use Asia/Tokyo in this field.

    Advanced Settings

    Field Name
    Default Value
    Description
    Cursor Fetch Count10000In this field you can specify the number of rows the cursor will fetch at one time
    Connection Timeout (seconds)300In this field you can set the amount of time (in seconds), until connection timeout
    Socket Timeout (seconds)1800In this field, you can set the amount of time (in seconds), until socket timeout
    Was this article helpful?
    What's Next