Data Source - MySQL
  • 11 Apr 2022
  • PDF

Data Source - MySQL

  • PDF

Article summary

Overview

Help page for setting up a data transfer from MySQL.

Supported Protocol

Data Loader: Embulk
Using the embulk-input-mysql plugin.

Limitations

None.

Settings

Step 1: General Settings

Field Name
Required
Default Value
Description
MySQL Connection SettingsYes-From the MySQL Connection Configurations created in advance, select the one with the necessary permissions for this transfer.
If you have not previously created any Connection Configurations, click the Add Connection Configuration button to set up a new one.
DatabaseYes-Enter the MySQL database name containing the data you wish to transfer.
By pressing the Load Database List button, you can update your MySQL database list, used to auto-complete the field.
Transfer MethodYesQueryChoose between either the Query or Incremental Load transfer methods.
QueryYes-You must complete this field when the transfer method is set to Query.
In this field, write an SQL query to extract the data you wish to transfer to from MySQL.
Here you can use custom variables to embed a value that will be dynamically set when the job executes.
TableYes-You must complete this field when the transfer method is set to Incremental Load.
Enter the table name that contains the data you wish to transfer.
Incremental Load Column NameNo-You can enter into this field when the transfer method is set to Incremental Load.
For each transfer, the last value loaded from the column specified here will be saved as the Last Record Loaded.
From the next job, only values higher than the Last Record Loaded, will be transferred.
When using a unique and auto-increment column (e.g. id, created_at, etc.), enter it here.
Use comma separation to specify multiple columns.
If unspecified, a primary key is used.
Last Record LoadedNo-You can enter into this field when the transfer method is set to Incremental Load.
Only data with a value higher than the value specified here will be transferred.
If unspecified, a full load will occur.
Do not change this value without reason (such as if you need to re-load data from a specific date range), as there is a chance of data duplication.
Default Time ZoneNo-Enter the time zone set server side in MySQL.
For example, if the time zone is set as Japan use Asia/Tokyo in this field.

Advanced Settings

Field Name
Default Value
Description
Cursor Fetch Count10000In this field you can specify the number of rows the cursor will fetch at one time
Connection Timeout (seconds)300In this field you can set the amount of time (in seconds), until connection timeout
Socket Timeout (seconds)1800In this field, you can set the amount of time (in seconds), until socket timeout

Was this article helpful?