We are looking for 2 Senior Consultants with solid hands-on experience in Teradata-to-Databricks migration and ETL modernization. The ideal candidate should have experience working on large-scale data warehouse migration projects and be comfortable owning migration activities from assessment and design through development, validation and production deployment.
Key Responsibilities
Lead and execute Teradata-to-Databricks migration projects.
Analyze existing Teradata databases, tables, views, SQL logic, stored procedures and ETL workflows.
Design and implement migration solutions on Databricks Lakehouse.
Develop scalable data pipelines using PySpark and Databricks.
Build and optimize data processing using Delta Lake.
Implement data governance, security and access control using Unity Catalog.
Modernize legacy ETL processes and migrate them to Databricks-based pipelines.
Perform source-to-target mapping, data transformation and reconciliation.
Conduct migration validation, data quality checks and performance testing.
Optimize Databricks workloads for performance, scalability and cost.
Support migration cutover, production deployment and post-migration stabilization.
Collaborate with architects,
business stakeholders and engineering teams to resolve migration challenges.
Provide technical guidance to junior team members and contribute to migration best practices.
Mandatory Skills
Strong hands-on experience in Teradata-to-Databricks migration.
Strong Databricks experience.
Strong PySpark / Spark SQL skills.
Solid knowledge of Delta Lake.
Hands-on experience with Unity Catalog.
Experience in ETL/ELT modernization.
Robust SQL and data warehousing concepts.
Experience with data migration, reconciliation and validation.
Good understanding of Lakehouse architecture.
Preferred Experience
Experience with Teradata SQL, stored procedures and legacy ETL workloads.
Experience with Bronze/Silver/Gold or Medallion architecture.
Experience with Databricks Workflows / Jobs / DLT.
Experience with CI/CD and Git-based deployment.
Experience with Azure/AWS/GCP cloud data platforms.
Experience in performance tuning and optimization of Spark/Databricks workloads.
Ideal Candidate
Candidates who have personally executed Teradata-to-Databricks migration projects will be strongly preferred over candidates who only have generic Databricks development experience.