We are looking for a skilled Data Engineer with expertise in Databricks, PySpark, Python, PL/SQL, and Azure Data Factory (ADF) to design, develop, and optimize scalable data pipelines and analytics solutions. The ideal candidate will have experience working with large-scale data processing, ETL/ELT frameworks, cloud-based data platforms, and data warehousing solutions.

Key Responsibilities
Design, develop, and maintain data pipelines using Databricks, PySpark, and Azure Data Factory (ADF).
Build scalable ETL/ELT solutions for processing structured and unstructured data.
Develop and optimize data transformation workflows using Python and PySpark.
Write, maintain, and tune complex PL/SQL queries, stored procedures, and database objects.
Integrate data from multiple sources into enterprise data warehouses and data lakes.
Monitor and troubleshoot data pipeline performance, ensuring reliability and data quality.
Collaborate with business stakeholders, analysts, and data scientists to understand data requirements.
Implement data governance, security,



and best practices across data platforms.
Support cloud-based data migration and modernization initiatives.
Participate in code reviews, testing, and production deployments.

Required Skills
410 years of experience in Data Engineering or related roles.
Robust hands-on experience with Databricks and PySpark.
Proficiency in Python programming for data processing and automation.
Robust experience in PL/SQL development and performance tuning.
Experience with Azure Data Factory (ADF) for data integration and orchestration.
Knowledge of data warehousing concepts and ETL/ELT processes.
Experience working with SQL databases and large datasets.
Understanding of Azure cloud services and data platform architecture.
Robust analytical and problem-solving skills.

Preferred Skills
Experience with Azure Data Lake Storage (ADLS).
Exposure to Delta Lake, Azure Synapse Analytics, or Snowflake.
Exper

📌 Databricks Lead Chennai
🏢 Tata Consultancy Services
📍 Chennai