What are the ingestion types and use-cases on Amorphic?

info

• Total time taken for this task: 5 Minutes.
• Pre-requisites: None

Ingestion patterns on Amorphic:

Source data	Ingestion patterns	Migration Types	Targets
1. RDBMS tables (JDBC)	Normal data load Bulk data load Bulk data load v2 Create connection and create tasks to automatically migrate data to datasets	Full load CDC Full load and CDC	S3 S3Athena Redshift
2. External files	Create a connection for the source server and use it in a destination dataset. Run the csclone ingestion command (provided in dataset)on the source machine to copy files automatically from server to dataset	Append Reload	S3 S3Athena Redshift
3. S3 files	Create a connection for source bucket and use it in a destination dataset	Append Reload	S3
4. External APIs	Create a connection with API endpoint and use it in a destination dataset	Append Reload	S3 S3Athena Redshift
5. Real-time streams	Create a stream and then create a consumer for s3 or s3athena	Append	S3 S3Athena

Ingestion use-cases and their relevant patterns on Amorphic:

Usecase	Pattern to be used	Supported DBs	Automation on Amorphic
1. Migrate one table from RDBMS to Amorphic	JDBC connection with normal data load	Amazon Redshift Amazon Relational Database Service Amazon Aurora Microsoft SQL Server MySQL Oracle PostgreSQL Publicly accessible databases Amazon Aurora MariaDB Microsoft SQL Server MySQL Oracle PostgreSQL IBM Db2 (LUW)	Data Ingestion scheduling
2. Migrate multiple tables from RDBMS to Amorphic. Automatically create target datasets	JDBC connection with Bulk data load	Oracle MySQL PostgreSQL Microsoft SQL Server IBM Db2 (LUW) Aurora MySQL	JDBC Full Load or JDBC CDC scheduling
3. Migrate multiple tables from RDBMS to Amorphic. Use existing dataset(s) for target. Multiple tables to be ingested into a single or multiple datasets	JDBC connection with Bulk data load V2	Oracle MySQL PostgreSQL Microsoft SQL Server IBM Db2 (LUW) Aurora MySQL	JDBC Full Load or JDBC CDC scheduling
4. Migrate data from external server to Amorphic	Ext-fs connection with a source server name Use this connection for a Amorphic dataset	S3 S3Athena Redshift	csclone command automation on server
5. Migrate data from one S3 bucket to Amorphic	S3 connection with a source bucket name Use this connection for a Amorphic destination dataset	S3	Data Ingestion scheduling
6. Migrate data from external APIs to Amorphic	API connection with a source API Endpoint Use this connection for a Amorphic dataset	S3 S3Athena Redshift	Data Ingestion scheduling
7. Migrate data from a Stream to Amorphic	Create a Stream Create a consumer with target dataset Publish data to stream	S3 S3Athena	Automation needed for Data producer. Consumer is automatic.

Does it cover your use-case?

Choose the ingestion type depending on the type of source data.