Role: Data Engineer Python & Kafka

Experience: 6-10 Years

Locations: Chennai, Bengaluru, Hyderabad, Pune, Kochi, Kolkata, NCR/Delhi

Virtual Interview: 01-OCT-26

Job Summary

We are looking for an experienced Data Engineer with strong expertise in Python and Apache Kafka to design, develop, and maintain scalable data pipelines and real-time data processing solutions. The ideal candidate should have experience working with large-scale data platforms, distributed systems, and cloud-based data engineering environments.

Key Responsibilities

- Design, develop, and optimize batch and real-time data pipelines using Python.
- Build and maintain Kafka-based streaming solutions for high-volume data processing.
- Develop scalable ETL/ELT frameworks and data ingestion pipelines.
- Integrate data from multiple sources and ensure data quality, reliability, and consistency.
- Work closely with Data Scientists, Analysts, and Application Development teams to support data requirements.
- Implement monitoring, logging, and performance tuning for data pipelines.




- Participate in architecture discussions and recommend best practices for data engineering solutions.
- Ensure compliance with data governance and security standards.

Required Skills

- Strong experience in Python programming.
- Hands-on experience with Apache Kafka, Kafka Streams, Kafka Connect, and message-based architectures.
- Experience in building ETL/ELT pipelines.
- Strong understanding of SQL and database concepts.
- Experience with distributed data processing frameworks such as Spark is preferred.
- Knowledge of REST APIs and data integration.
- Experience with cloud platforms such as GCP, AWS, or Azure.
- Familiarity with CI/CD, Git, and DevOps practices.
- Robust problem-solving and analytical skills.

Preferred Skills

- Experience with PySpark.
- Knowledge of Airflow or other workflow orchestration tools.
- Exposure to Data Lake/Data Warehouse architectures.
- Understanding of containerization technologies such as Docker and Kubernetes.

📌 Data Engineer -Python & Kafka (Bengaluru)
🏢 Tata Consultancy Services
📍 Bengaluru