Key Responsibilities:

- Analyze existing BO reports and BODS ETL jobs to design equivalent modern data pipelines in Azure Databricks.
- Build robust, scalable, and reusable ETL/ELT pipelines using Databricks (PySpark/Scala) and Delta Lake.
- Collaborate with data architects and business analysts to translate business requirements into technical implementations.
- Optimize data ingestion, transformation, and storage processes across structured and semi-structured data.
- Integrate data from on-prem systems and legacy tools into Azure Data Lake, Synapse, and Databricks environments.
- Participate in data validation, testing, and reconciliation between source and target systems during migration.
- Implement best practices for data pipeline development, CI/CD, monitoring, version control, and documentation.




- Work with DevOps engineers to ensure automated deployment and operational support of data pipelines.
- Ensure data quality, security, and compliance standards are upheld throughout the migration process.

Qualifications:

- 8+ years of experience in data engineering, with at least 4 years working with Azure Databricks.
- Robust experience with SAP BusinessObjects and BODS, including understanding of ETL logic and reporting structures.
- Proficiency in PySpark, SQL, and Python, with hands-on experience building scalable pipelines in Databricks.
- Experience with Azure Data Lake, Azure Synapse, and Delta Lake.
- Familiarity with version control systems (Git), CI/CD tools (Azure DevOps, GitHub Actions), and job orchestration tools (Databricks Workflows, Airflow).

📌 Azure Data Engineer - S (Chennai)
🏢 Tata Consultancy Services
📍 Chennai