Job Summary

A highly skilled 8 to 10 years of experience on SSIS. SQL Server, Azure Databricks, Pyspark, Hands-On Data Tech Lead to drive the modernization and scaling of our enterprise data platform to provide technical leadership while remaining deeply technical and actively writing code. Also lead the migration, integration, and development of data pipelines across our legacy SQLServer/SSIS environments and our modern cloud infrastructure built on Azure Databricks and PySpark and act as the principal architect and developer for end-to-end ELT/ETL pipelines, ensuring optimal performance, data governance, and scalability.

• Lead the migration of legacy SQL Server Integration Services (SSIS) packages to up-to-date, cloud-native Azure Databricks / PySpark Lakehouse environments.

• Architect end-to-end Modern Data Warehouse (MDW) and Medallion architecture systems (Bronze, Silver, Gold layers) within Azure.

• Mentor and guide a team of data engineers, conducting stringent code reviews, enforcing design patterns, and establishing development standards.





• Collaborate with stakeholders including Data Architects, Business Analysts, and DevOps teams to translate complex business requirements into technical blueprints.

• Develop and optimize scalable, modular, high-performance batch and real-time ETL/ELT pipelines using PySpark on Azure Databricks cluster environments.

• Maintain and refactor existing on-premise SSIS packages and complex T-SQL code (stored procedures, views, performance tuning) to ensure business continuity during transition phases.

• Implement Delta Lake features, managing transactions, schema evolution, and time-travel querying functionalities.

• Orchestrate workflows seamlessly by integrating Azure Data Factory (ADF) with on-premise databases, cloud storage (ADLS Gen2), and Databricks activities.

• Optimize Spark configurations and cluster utilization to drastically reduce cloud compute overhead and job execution windows.

• Enforce data quality fr

📌 Data Engineer Advisor (Hyderabad)
🏢 NTT DATA Services
📍 Hyderabad