Role & responsibilities

- Design, develop, and maintain large-scale data pipelines and workflows.
- Build batch and real-time data ingestion solutions.
- Develop scalable ETL/ELT processes using Big Data technologies.
- Process and analyze large datasets using Spark and Hadoop ecosystem tools.
- Design and maintain Data Lakes and Data Warehouse solutions.
- Implement data quality checks, monitoring, and governance processes.
- Optimize data processing jobs for performance and scalability.
- Collaborate with Data Scientists, Analysts, Architects, and Business teams.
- Troubleshoot data pipeline failures and production issues.
- Participate in code reviews and follow best development practices.
- Document technical designs, workflows, and processes.

Preferred candidate profile

4-10 years of experience in Data Engineering/Big Data development.

- Solid hands-on experience with Apache Spark and SQL.
- Experience in Hadoop ecosystem and distributed computing.
- Experience building and maintaining ETL/ELT pipelines.
- Hands-on experience with at least one Cloud platform (AWS/Azure/GCP).
- Experience with Kafka or other streaming platforms.
- Strong knowledge of database concepts and data modeling.

📌 Big Data Engineer (Telangana)
🏢 Tata Consultancy Services
📍 Telangana