We are looking for a skilled Data Engineer with expertise in Databricks, PySpark, Python, PL/SQL, and Azure Data Factory (ADF) to design, develop, and optimize scalable data pipelines and analytics solutions. The ideal candidate will have experience working with large-scale data processing, ETL/ELT frameworks, cloud-based data platforms, and data warehousing solutions.

Key Responsibilities

- Design, develop, and maintain data pipelines using Databricks, PySpark, and Azure Data Factory (ADF).
- Build scalable ETL/ELT solutions for processing structured and unstructured data.
- Develop and optimize data transformation workflows using Python and PySpark.
- Write, maintain, and tune complex PL/SQL queries, stored procedures, and database objects.
- Integrate data from multiple sources into enterprise data warehouses and data lakes.
- Monitor and troubleshoot data pipeline performance, ensuring reliability and data quality.
- Collaborate with business stakeholders, analysts, and data scientists to understand data requirements.
- Implement data governance, security, and best practices across data platforms.




- Support cloud-based data migration and modernization initiatives.
- Participate in code reviews, testing, and production deployments.

Required Skills

- 410 years of experience in Data Engineering or related roles.
- Strong hands-on experience with Databricks and PySpark.
- Proficiency in Python programming for data processing and automation.
- Strong experience in PL/SQL development and performance tuning.
- Experience with Azure Data Factory (ADF) for data integration and orchestration.
- Knowledge of data warehousing concepts and ETL/ELT processes.
- Experience working with SQL databases and large datasets.
- Understanding of Azure cloud services and data platform architecture.
- Solid analytical and problem-solving skills.

Preferred Skills

- Experience with Azure Data Lake Storage (ADLS).
- Exposure to Delta Lake, Azure Synapse Analytics, or Snowflake.
- Experience with CI/CD and DevOps practices.
- Knowledge of Data Governance and Data Quality frameworks.
- Azure certifications are a plus.

📌 Databricks Lead (Noida)
🏢 Tata Consultancy Services
📍 Noida