Data Migration
Data Warehouse Migration from Teradata to AWS Redshift
An American company that franchises and operates a chain of restaurants needed to migrate its on-premise Data Warehouse (Teradata Database) to AWS Cloud Infrastructure (Redshift Database)
Client Challenges and Requirements
- Large volume of data. Approx. size of data to be migrated was 8TB
- Data field mapping issues
- Time duration for the entire Migration
- Large size of a few fact tables
- Data Validation
Bitwise Solution
Bitwise recommended Amazon SCT for converting schema and loading chunks of data for large fact tables by applying
conditions for source data. Resolved data issues faced earlier while migrating with R programming using SCT. Time duration
was an important constraint for the entire migration. Recommended parallel tasks in SCT for processing different chunks of data
resulting in faster migration of data.
- Datatype
- Data length
- Stratification (Sum of amount fields)
Implementation Included:
- Systematic migration of the entire Data Warehouse including all dimensions / fact
- Data Validation for all tables
- Parallel tasks in SCT for speeding up the migration process
Bitwise recommended Amazon SCT for converting schema and loading chunks of data for large fact tables by applying
conditions for source data. Resolved data issues faced earlier while migrating with R programming using SCT. Time duration
was an important constraint for the entire migration. Recommended parallel tasks in SCT for processing different chunks of data
resulting in faster migration of data.
- Datatype
- Data length
- Stratification (Sum of amount fields)
Implementation Included:
- Systematic migration of the entire Data Warehouse including all dimensions / fact
- Data Validation for all tables
- Parallel tasks in SCT for speeding up the migration process
Tools & Technologies We Used
Amazon SCT
Teradata
Aginity for Redshift
Amazon S3
Key Results
Reduced maintenance and operating costs for the Data Warehouse
Faster execution of queries in comparison with the previous setup
The deployment time was reduced from 24 hours to less than an hour
An American company that franchises and operates a chain of restaurants needed to migrate their on-premise Data Warehouse (Teradata Database) to AWS Cloud Infrastructure (Redshift Database)
