A Direct Banking and Payment Services company needed to migrate their on-premise data warehouse residing on Teradata to cloud-based data warehouse Snowflake. The process involved migrating tables, DDL including historical and incremental data with support for cut over to on premise.
Client Challenges and Requirements
- Unavailability of any tool for migrating Teradata DDL to Snowflake DDL with constraintsÂ
- Time consuming manual process for migrating large number of tables and high volume of dataÂ
- Manual validation of migrated data resulting in inaccuracy and inefficiency of the process
- Challenges to migrate incremental data due to lack of CDC feature in TeradataÂ
- Migration of sensitive data with SecurityÂ
- No provision to handle pipeline failures and restartability
Bitwise Solution
Bitwise leveraged its data migration framework consisting of the following features:
Schema Mapping and Auto conversion: Mapped the schema of Teradata with Snowflake and auto convert the DDL
Data Validation: Developed a process/service to validate the historical as well as CDC data with provision of sample and full data after migration.
High Performance, Availability and Scalability: Implemented containerization to execute jobs for extraction, load and Validation
Auto Scheduling: Automatic scheduling of migration process for data extraction, load and validation
Tokenization: Implemented tokenization process to handle sensitive data
Monitoring, Logging and Reporting: Ability to monitor the data migration process to check the progress/status with user friendly dashboards and reports.
Tools & Technologies We Used
Teradata
Snowflake
Bitwise Cloud Data
Migration Framework
Using Java, Apache Spark
Containerization using
Docker and Kubernetes
Key Results
Fast processing of data migration and validation
100% accuracy of migrated data with sample and full validation
High scalability and availability with reduced manual intervention
Share This Case Study
Download Case Study
Bitwise provides comprehensive solutions for all your cloud migration projects
Cloud Migration Services
Our ready-to-go Cloud Migration Roadmap helps guide you through your initiative for moving on-premise data to the cloud
