02 Aug
|
Tata Consultancy Services
|
Telangana
02 Aug
Tata Consultancy Services
Telangana
Role: Databricks DB Migration Engineer
Experience: 6 to 8 years
Location: PAN India
Key Responsibilities
Assess existing source database settings, schemas, data volumes, dependencies, jobs, stored procedures, ETL
pipelines, and reporting workloads for migration readiness.
Design migration approach for moving data and workloads from RDBMS, data warehouse, Hadoop, or cloud databases to
Databricks Lakehouse using batch and incremental migration patterns.
Develop scalable ingestion and transformation pipelines using PySpark, Spark SQL, Databricks Workflows, Delta Live
Tables, Auto Loader, and notebooks/jobs as applicable.
Convert and optimize SQL scripts, stored procedures, ETL logic, and business transformations into Spark SQL/PySpark-
based implementations.
Implement Delta Lake best practices including partitioning, OPTIMIZE, Z-ORDER, schema evolution, time travel, ACID
transactions, and data retention strategies.
Perform data profiling, cleansing, mapping, validation, reconciliation, and row/hash/count-level comparison between
source and target systems.
Tune Spark jobs and Databricks clusters for performance, cost, concurrency, workload isolation, and operational
efficiency.
Support migration dry runs, defect triage, production cutover, rollback planning, post-migration validation, hypercare,
and operational handover.
Collaborate with architects, DBAs, data engineers, cloud teams, security teams, QA teams, and business stakeholders to
ensure successful migration delivery.
Prepare technical documentation including migration design, mapping documents, runbooks, reconciliation reports,
deployment plans, and operational support guides.
📌 Databricks Db Migration Engineer Telangana
🏢 Tata Consultancy Services
📍 Telangana