AWS Senior Data Engineer (Kafka & PySpark) Experience: 10-15 Years Location: Chennai, Bengaluru, Hyderabad We are seeking an experienced AWS Data Engineer with robust expertise in Kafka, PySpark, Spark SQL, Python, and AWS to build and optimize large-scale batch and real-time data platforms. Key Skills PySpark, Spark SQL, Python, SQL Apache Kafka (Kafka Connect, Schema Registry, Streaming) AWS: S3, Glue, EMR, Lambda, IAM, Redshift, Step Functions, CloudWatch ETL/ELT & Real-Time Data Pipelines Spark Structured Streaming Data Modeling, Data Lakes & Lakehouse Architecture CI/CD, Git, Jenkins/GitHub Actions Data Quality, Governance & Security Responsibilities Design and develop scalable batch and streaming data pipelines on AWS. Build high-performance PySpark and Spark SQL applications.



Implement Kafka-based event-driven and real-time streaming solutions. Integrate and optimize AWS data services for reliability and cost efficiency. Implement data quality, governance, monitoring, and security best practices. Build CI/CD pipelines and support production deployments. Troubleshoot performance bottlenecks and production issues. Good to Have Amazon MSK, Kinesis Databricks, Delta Lake, Unity Catalog Airflow / MWAA Snowflake, DBT Terraform Primary Skills: AWS Data Engineering, Kafka, PySpark, Spark SQL, Python, Streaming Data Engineering.

📌 AWS Senior Data Engineer (Chennai)
🏢 Tata Consultancy Services
📍 Chennai