This destination writes data to AWS DynamoDB.
The Airbyte DynamoDB destination allows you to sync data to AWS DynamoDB. Each stream is written to its own table under the DynamoDB.
- For Airbyte Open Source users using the Postgres source connector, upgrade your Airbyte platform to version
v0.40.0-alphaor newer and upgrade your DynamoDB connector to version
Each stream will be output into its own DynamoDB table. Each table will a collections of
json objects containing 4 fields:
_airbyte_ab_id: a uuid assigned by Airbyte to each event that is processed.
_airbyte_emitted_at: a timestamp representing when the event was pulled from the data source.
_airbyte_data: a json blob representing with the extracted data.
sync_time: a timestamp representing when the sync up task be triggered.
|Full Refresh Sync||✅||Warning: this mode deletes all previously synced data in the configured DynamoDB table.|
|Incremental - Append Sync||✅|
|Incremental - Deduped History||❌||As this connector does not support dbt, we don't support this sync mode on this destination.|
|Namespaces||✅||Namespace will be used as part of the table name.|
This connector by default uses 10 capacity units for both Read and Write in DynamoDB tables. Please provision more capacity units in the DynamoDB console when there are performance constraints.
- Allow connections from Airbyte server to your AWS DynamoDB tables (if they exist in separate VPCs).
- The credentials for AWS DynamoDB (for the COPY strategy).
- Fill up DynamoDB info
- DynamoDB Endpoint
- Leave empty if using AWS DynamoDB, fill in endpoint URL if using customized endpoint.
- DynamoDB Table Name
- The name prefix of the DynamoDB table to store the extracted data. The table name is \\.
- DynamoDB Region
- The region of the DynamoDB.
- Access Key Id
- See this on how to generate an access key.
- We recommend creating an Airbyte-specific user. This user will require read and write permissions to the DynamoDB table.
- Secret Access Key
- Corresponding key to the above key id.
- DynamoDB Endpoint
- Make sure your DynamoDB tables are accessible from the machine running Airbyte.
- This depends on your networking setup.
- You can check AWS DynamoDB documentation with a tutorial on how to properly configure your DynamoDB's access here.
- The easiest way to verify if Airbyte is able to connect to your DynamoDB tables is via the check connection tool in the UI.
|0.1.7||2022-11-03||#18672||Added strict-encrypt cloud runner|
|0.1.6||2022-11-01||#18672||Enforce to use ssl connection|
|0.1.5||2022-08-05||#15350||Added per-stream handling|
|0.1.4||2022-06-16||#13852||Updated stacktrace format for any trace message errors|
|0.1.3||2022-05-17||12820||Improved 'check' operation performance|
|0.1.1||2022-12-05||#9314||Rename dynamo_db_table_name to dynamo_db_table_name_prefix.|