Role: Data Engineer,

Exp:6 to 8 Years,

Location: Bengaluru,

Type: Full Time,

Mandatory Skills: Data Engineer, Team Lead, Python, Py-spark, IDMC, ETL/ELT, SQL,

AWS Services, S3, Lambda, Athena, Glue, EMR, Terraform, Git, CI/CD, DevOps, Data lake, Data warehouse,

:

We are seeking a Data Engineer with 5-8 years of experience and strong expertise in Python, AWS, and modern data engineering practices, along with solid experience in Git-based version control and Terraform for infrastructure autornation, The ideal candidate will focus on building scalable data pipelines, cloud-native solutions, and automated infrastructure, with a practical understanding of data quality and governance practices.

Key Responsibilities

1. Data Engineering & Pipeline Development

Design, build, and maintain scalable ETL/ELT pipelines using Python and AWS. Develop batch and near real-time data processing pipelines

Optimize data workflows for performance, scalability, and cost-efficiency .

Integrate and process data frorn diverse sources (databases, APls, files).

1. Python & Data Processing

- Develop production-grade solutions using Python (PySpark, Glue)
- Write clean, modular, and testable code. Implement logging, monitoring, and error handling mechanisms.

1. AWs Cloud & Data Platform Engineering
2. Hands-on experience with AWS services: S3, Glue, Lambda, Redshift, Athena, EMR .

Build and manage data lake: / warehouse architectures. Develop serverless and distributed data processing solutions.

Ensure rellability, scalability, and cost optimization In AWS

1. Infrastructure as Code (Terraform)
2. Design and deploy cloud infrastructure using Terraform. Manage reusable modules and enforce Infrastructure standards.
3. Automate provisioning of data platforms and environments.
4. Work closely with DeVops practices to ensure repeatable deployments.





5.Version Control & DevOps (Git) . Use Git (GitHub/GitLab/Bitbucket) for version control and collaboration.

Follow branching strategies, code reviews, and pull request workflows.

Contribute to C1/CD pipelines for automated testing and deployment.

1. Data Modelling & Architecture .

Design efficient data models for analytics and reporting . Support implementation of data lakes, warehouses, and Lakehouse architectures.

Ensure alignment with enterprise architecture standards.

1. Data Management (light Focus .

Apply basic data quality, validation, and governance practices. Ensure data is reliable, secure, and compliant with policies. Maintain metadata and documentation for datasets.

8.Collaboration & Delivery Work with cross-functional teams (analysts, data scientists, business users). . Translate business requirements into technical solutions. . Contribute to Agle/scurm deliverymodels . Support and mentor unlor team members as needed.

Required Skills & Experience

- Support and mentor junior team members as needed

Required Skills & Experience

- 5-8years experience in Data Engineering.
- Strong proficiency in Python.

Hands-on experience with AWS data services.

- Strong experience in Git version control and cooperative workflows,
- Strong Exposure in IDMC Services.

Proven experience using Terraform for infrastructure automation. Strong SOL skills and database knowledge.

Experience with ETL/ELT and orchestration tools (e.g., Airflow).

Preferred Skills

- Experience with PySpark / Apache Spark.
- Knowledge of CI/CD tools
- Familiarity with containerization (Docker, Kubernetes)

Soft Skills

- Strong problem-solving and analytical skills
- Good communication and stakeholder engagement ability.
- Ability to work independently with ownership.
- Focus on code quality, scalability, and reliability

📌 Data Engineer (Bengaluru)
🏢 Shell Infotech
📍 Bengaluru