Position: AWS Data Engineer Lead

Experience: 8–10 Years

Location: Chennai, Bangalore

Key Skills: AWS Data Engineering, Databricks, PySpark, SQL

About the Role:

We are seeking an experienced AWS Data Engineer Lead to drive the design and delivery of robust, scalable data engineering solutions on AWS. You'll lead a team building high-performance data pipelines, lakehouse architectures, and analytics platforms, while championing engineering excellence and cloud-native best practices.

Key Responsibilities:

- Lead architecture and development of scalable data pipelines using AWS Glue, EMR, Lambda, and Step Functions
- Design and manage cloud-native data lakes using S3, Lake Formation, and Redshift
- Define engineering standards, review architecture decisions, and drive best practices across the team
- Build real-time and batch data processing solutions using Kinesis / Kafka, Spark, and Glue Streaming
- Partner with data architects and analytics teams to design efficient data models and pipelines
- Implement strong data governance, cataloging, and security controls (IAM, Lake Formation, KMS)
- Optimize infrastructure for performance, scalability, and cost using AWS best practices
- Set up monitoring, alerting, and observability (CloudWatch, CloudTrail)
- Mentor data engineers, lead code/design reviews, and manage sprint deliverables




- Evaluate and adopt new AWS data services to keep the platform modern and efficient

Required Skills:

- Strong hands-on experience with AWS Glue, EMR, Redshift, S3, Lambda, Athena
- Proficiency in PySpark, SQL, and Python for large-scale ETL/ELT development
- Solid understanding of data lake / lakehouse design (Delta/Iceberg/Hudi a plus)
- Experience with data modeling and building scalable data warehouses
- Hands-on experience with CI/CD (CodePipeline/Jenkins) and Infrastructure-as-Code (Terraform/CloudFormation)
- Strong knowledge of AWS security and governance (IAM, Lake Formation, KMS)
- Proven experience leading engineering teams in an Agile delivery workplace
- Excellent problem-solving, communication, and stakeholder management skills

Good to Have:

- Experience with Kafka/Kinesis for real-time streaming pipelines
- Exposure to QuickSight or other BI/visualization tools
- Familiarity with Apache Iceberg / Hudi for open table formats
- AWS certifications: AWS Certified Data Analytics – Specialty, AWS Certified Solutions Architect

Why Join Us?

- Lead impactful, large-scale data initiatives on a modern AWS stack
- Collaborative, growth-oriented culture with strong technical mentorship
- Attractive compensation, flexible working arrangements, and career advancement opportunities

📌 AWS Data Engineer Lead (Chennai)
🏢 Softview Infotech
📍 Chennai