Data Source - MySQL
  • 11 Apr 2022
  • 2 Minutes to read
  • Dark
    Light

Data Source - MySQL

  • Dark
    Light

Overview

Help page for setting up a data transfer from MySQL.

Supported Protocol

Data Loader: Embulk
Using the embulk-input-mysql plugin.

Limitations

None.

Settings

Step 1: General Settings

Field Name
Required
Default Value
Description
MySQL Connection Settings Yes - From the MySQL Connection Configurations created in advance, select the one with the necessary permissions for this transfer.
If you have not previously created any Connection Configurations, click the Add Connection Configuration button to set up a new one.
Database Yes - Enter the MySQL database name containing the data you wish to transfer.
By pressing the Load Database List button, you can update your MySQL database list, used to auto-complete the field.
Transfer Method Yes Query Choose between either the Query or Incremental Load transfer methods.
Query Yes - You must complete this field when the transfer method is set to Query.
In this field, write an SQL query to extract the data you wish to transfer to from MySQL.
Here you can use custom variables to embed a value that will be dynamically set when the job executes.
Table Yes - You must complete this field when the transfer method is set to Incremental Load.
Enter the table name that contains the data you wish to transfer.
Incremental Load Column Name No - You can enter into this field when the transfer method is set to Incremental Load.
For each transfer, the last value loaded from the column specified here will be saved as the Last Record Loaded.
From the next job, only values higher than the Last Record Loaded, will be transferred.
When using a unique and auto-increment column (e.g. id, created_at, etc.), enter it here.
Use comma separation to specify multiple columns.
If unspecified, a primary key is used.
Last Record Loaded No - You can enter into this field when the transfer method is set to Incremental Load.
Only data with a value higher than the value specified here will be transferred.
If unspecified, a full load will occur.
Do not change this value without reason (such as if you need to re-load data from a specific date range), as there is a chance of data duplication.
Default Time Zone No - Enter the time zone set server side in MySQL.
For example, if the time zone is set as Japan use Asia/Tokyo in this field.

Advanced Settings

Field Name
Default Value
Description
Cursor Fetch Count 10000 In this field you can specify the number of rows the cursor will fetch at one time
Connection Timeout (seconds) 300 In this field you can set the amount of time (in seconds), until connection timeout
Socket Timeout (seconds) 1800 In this field, you can set the amount of time (in seconds), until socket timeout

Was this article helpful?