AWS Senior Data Engineer (Kafka & PySpark)
Experience: 10-15 Years
Location: Chennai, Bengaluru, Hyderabad

We are seeking an experienced AWS Data Engineer with strong expertise in Kafka, PySpark, Spark SQL, Python, and AWS to build and optimize large-scale batch and real-time data platforms.

Key Skills
• PySpark, Spark SQL, Python, SQL
• Apache Kafka (Kafka Connect, Schema Registry, Streaming)
• AWS: S3, Glue, EMR, Lambda, IAM, Redshift, Step Functions, CloudWatch
• ETL/ELT & Real-Time Data Pipelines
• Spark Structured Streaming
• Data Modeling, Data Lakes & Lakehouse Architecture
• CI/CD, Git, Jenkins/GitHub Actions
• Data Quality, Governance & Security

Responsibilities
• Design and develop scalable batch and streaming data pipelines on AWS.
• Build high-performance PySpark and Spark SQL applications.




• Implement Kafka-based event-driven and real-time streaming solutions.
• Integrate and optimize AWS data services for reliability and cost efficiency.
• Implement data quality, governance, monitoring, and security best practices.
• Build CI/CD pipelines and support production deployments.
• Troubleshoot performance bottlenecks and production issues.

Positive to Have
• Amazon MSK, Kinesis
• Databricks, Delta Lake, Unity Catalog
• Airflow / MWAA
• Snowflake, DBT
• Terraform

Primary Skills: AWS Data Engineering, Kafka, PySpark, Spark SQL, Python, Streaming Data Engineering.

📌 AWS Senior Data Engineer (Chennai)
🏢 Tata Consultancy Services
📍 Chennai