Key Responsibilities

- Design, develop, and maintain scalable data pipelines using Databricks, Python, and PySpark.

- Build and optimize ETL/ELT workflows for processing large-scale structured and unstructured data.

- Develop and maintain data lakes, data warehouses, and Lakehouse architectures.

- Implement batch and real-time data processing solutions to support business requirements.

- Collaborate with Data Architects, Data Scientists, Analysts, and Business stakeholders to deliver data-driven solutions.

- Optimize data ingestion, transformation, storage, and processing performance.

- Ensure data quality, integrity, security, and governance standards across data platforms.

- Develop reusable frameworks, utilities, and automation solutions for data engineering processes.

- Monitor, troubleshoot, and resolve production data pipeline issues.

- Participate in data platform modernization and cloud transformation initiatives.

- Document data flows, technical designs, and operational processes.





- Mentor junior engineers and contribute to engineering best practices.

Required Skills & Qualifications

- Bachelor's degree in Computer Science, Information Technology, Engineering, Mathematics, or a related field.

- 5+ years of experience in Data Engineering and Data Warehousing.

- Robust hands-on experience with Databricks platform.

- Expertise in Python and PySpark development.

- Strong proficiency in SQL and data modeling concepts.

- Experience building scalable ETL/ELT pipelines.

- Knowledge of distributed data processing and big data technologies.

- Experience working with large datasets in cloud environments.

- Understanding of Data Lake, Data Warehouse, and Lakehouse architectures.

- Strong analytical, debugging, and problem-solving skills.

- Experience with Agile development methodologies.

📌 Senior Data Engineer - S (Chennai)
🏢 Tata Consultancy Services
📍 Chennai