Data Migration

Data Migration from Teradata to Snowflake

case-study-feature-img

A Direct Banking and Payment Services company needed to migrate their on-premise data warehouse residing on Teradata to cloud-based data warehouse Snowflake. The process involved migrating tables, DDL including historical and incremental data with support for cut over to on premise.

Client Challenges and Requirements

  • Unavailability of any tool for migrating Teradata DDL to Snowflake DDL with constraints
  • Time consuming manual process for migrating large number of tables and high volume of data
  • Manual validation of migrated data resulting in inaccuracy and inefficiency of the process
  • Challenges to migrate incremental data due to lack of CDC feature in Teradata
  • Migration of sensitive data with Security
  • No provision to handle pipeline failures and restartability

Bitwise Solution

Bitwise leveraged its data migration framework consisting of the following features:

solution-icon

Schema Mapping and Auto conversion: Mapped the schema of Teradata with Snowflake and auto convert the DDL

solution-icon

Data Validation: Developed a process/service to validate the historical as well as CDC data with provision of sample and full data after migration.

solution-icon

High Performance, Availability and Scalability: Implemented containerization to execute jobs for extraction, load and Validation

solution-icon

Auto Scheduling: Automatic scheduling of migration process for data extraction, load and validation

solution-icon

Tokenization: Implemented tokenization process to handle sensitive data

solution-icon

Monitoring, Logging and Reporting: Ability to monitor the data migration process to check the progress/status with user friendly dashboards and reports.

Tools & Technologies We Used

Teradata
Snowflake
Bitwise Cloud Data Migration Framework Using Java, Apache Spark
Containerization using Docker and Kubernetes

Key Results

skill-icon

Fast processing of data migration and validation

skill-icon

100% accuracy of migrated data with sample and full validation

skill-icon

High scalability and availability with reduced manual intervention

Download Case Study

Ready to start a conversation?