Data Source - Amazon DynamoDB

summary

Help page for ETL Configuration for transferring items from Amazon DynamoDB.

Setting items

STEP1 Basic settings

item name	indispensable	default value	Contents
Amazon DynamoDB Connection Configuration	Yes	-	Select the previously registered Connection Configuration that has the necessary permissions for this ETL Configuration. Please refer to another page for Connection Configuration.
region	Yes	-	Please specify the AWS region.
table	Yes	-	Specify the target table from which to retrieve items.
secondary index	No	-	Specify global/local secondary index. See official documentation for details.
transfer column	Yes	-	Specify the columns to be retrieved. At least one column must be set up. The setting items differ depending on the type you select. - Column Name(required) The name of the column to be retrieved. - Type (required) The type of embulk to convert the retrieved value. - Amazon DynamoDB data type of the value The Data Type set for the Amazon DynamoDB value. Values of any type other than the specified data type are treated as null. - format Type: string Can be set when timestamp is selected. The format of the value to be retrieved. (Default: "0"). %Y-%m-%d %H:%M:%S.%N %z） - time zone Type: string Can be set when timestamp is selected. The time zone to be used if the value to be retrieved does not include a time zone. (Default: "0"). Asia/Tokyo） supplementary information - About Amazon DynamoDB types For example, in the following image, s_col is set to the string A (data type: S (string)) and an array of strings of {"A"} (data type:. SS (array of string)) in some items. If S (string) is specified as the data type, s_col of p_key:a, s_key:2 is treated as NULL.
operation	yes	-	Operations on Amazon DynamoDB. Either of the following can be specified - Full Data Transfer (scan) - Extraction by query (QUERY)
query condition expression	No	-	Query Conditional Expression. - Partition key (required) Specify the partition key set for the table. - Partition key value (required) Specify the value of the partition key to be squeezed. - Amazon DynamoDB type of the partition key (required) Specify the Amazon DynamoDB type of the partition key. - sort key Specify the sort key set for the table. - Sort Key Operators Specify the sort key operator. - Sort key value 1 Specify the value of the sort key to be narrowed. - Sort key value 2 Specify the value of the sort key to be narrowed. It can be specified when BETWEEN is selected as the sort key operator, resulting in the following query conditional expression. {sort key} between {sort key value 1} and {sort key value 2} - Amazon DynamoDB type of sort key Specify the Amazon DynamoDB type of the sort key.
Number of segments	No	-	Operation: Can be specified when Full Data Transfer (scan) is selected. Performs a parallel scan of a table, or secondary index, divided into a specified number of segments. If a large number of items are retrieved, a StackOverflow error may occur. StackOverflow errors can be avoided by increasing the number of segments and scanning in parallel. For more information, please refer to the official AWS documentation1 AWS official documentation2.
Number of items to be retrieved in one operation	No	-	OutOfMemory errors may occur when transferring large numbers of items. OutOfMemory errors can be avoided by adjusting the batch size. By default, there is no limit on the number of items, and 1 MB worth of items will be retrieved. Please refer to the official AWS documentation for details.
Strong consistent loading	Yes	invalid	Enable/disable strong consistency reads. If a global secondary index is specified for the secondary index, enabling strong consistent reads will result in an error during transfer. See official documentation for details.