- 11 Apr 2022
- 2 Minutes to read
Data Source - MySQL
- Updated on 11 Apr 2022
- 2 Minutes to read
Help page for setting up a data transfer from MySQL.
Data Loader: Embulk
Using the embulk-input-mysql plugin.
Step 1: General Settings
|MySQL Connection Settings||Yes||-||From the MySQL Connection Configurations created in advance, select the one with the necessary permissions for this transfer.
If you have not previously created any Connection Configurations, click the Add Connection Configuration button to set up a new one.
|Database||Yes||-||Enter the MySQL database name containing the data you wish to transfer.
By pressing the Load Database List button, you can update your MySQL database list, used to auto-complete the field.
|Transfer Method||Yes||Query||Choose between either the Query or Incremental Load transfer methods.|
|Query||Yes||-||You must complete this field when the transfer method is set to Query.
In this field, write an SQL query to extract the data you wish to transfer to from MySQL.
Here you can use custom variables to embed a value that will be dynamically set when the job executes.
|Table||Yes||-||You must complete this field when the transfer method is set to Incremental Load.
Enter the table name that contains the data you wish to transfer.
|Incremental Load Column Name||No||-||You can enter into this field when the transfer method is set to Incremental Load.
For each transfer, the last value loaded from the column specified here will be saved as the Last Record Loaded.
From the next job, only values higher than the Last Record Loaded, will be transferred.
When using a unique and auto-increment column (e.g. id, created_at, etc.), enter it here.
Use comma separation to specify multiple columns.
If unspecified, a primary key is used.
|Last Record Loaded||No||-||You can enter into this field when the transfer method is set to Incremental Load.
Only data with a value higher than the value specified here will be transferred.
If unspecified, a full load will occur.
Do not change this value without reason (such as if you need to re-load data from a specific date range), as there is a chance of data duplication.
|Default Time Zone||No||-||Enter the time zone set server side in MySQL.
For example, if the time zone is set as Japan use Asia/Tokyo in this field.
|Cursor Fetch Count||10000||In this field you can specify the number of rows the cursor will fetch at one time|
|Connection Timeout (seconds)||300||In this field you can set the amount of time (in seconds), until connection timeout|
|Socket Timeout (seconds)||1800||In this field, you can set the amount of time (in seconds), until socket timeout|