Job Summary

Location: Pune, Bengaluru, NCR, Kolkata, Hyderabad

Exp: 6+yrs

We are looking for a talented Data Engineer with solid experience in AWS, Python, PySpark, and Databricks to build and maintain scalable data processing solutions. The ideal candidate will have expertise in developing data transformation pipelines, managing large datasets, working with SQL/NoSQL databases, and collaborating with cross-functional engineering teams.

The successful candidate will be responsible for:
Designing, building, and maintaining data transformation pipelines using AWS services (Lambda, Step Functions, Glue, S3, etc.) and Python.
Developing and optimizing data processing workflows using PySpark and Databricks.
Implementing and maintaining complex data calculation and transformation engines.
Managing and processing large-scale datasets efficiently.
Working with PostgreSQL and MongoDB databases.
Developing scalable and high-performance data solutions.
Collaborating closely with Software Engineers and other stakeholders.
Participating in code reviews, technical discussions, and software planning activities.
Contributing to automation, testing, CI/CD, and deployment processes.

Key Responsibilities




Build and support end-to-end ETL/ELT data pipelines.
Design data models and transformation frameworks for analytics and reporting.
Ensure data quality, reliability, and performance optimization.
Troubleshoot and resolve data pipeline and database issues.
Follow software engineering best practices and coding standards.
Create and maintain technical documentation.
Support cloud-based data architecture initiatives.

Mandatory Skills
Python Programming
AWS Services (Lambda, Step Functions, S3, Glue)
PySpark
Databricks
SQL
PostgreSQL
MongoDB
ETL/ELT Development
Large-scale Data Processing
Data Modeling and Transformation

Preferred Skills
AWS Glue
CI/CD and DevOps concepts
Data Quality & Monitoring Frameworks
Git Version Control

Education Requirements
Bachelor's degree in Computer Science, Information Technology, Engineering, or equivalent experience.
Educational qualification is versatile for candidates with robust relevant experience.

Certifications
AWS Certification (Preferred/Optional)
AWS Certified Cloud Practitioner
AWS Certified Developer Associate
AWS Certified Data Engineer Associate

📌 Aws Data Engineer West Bengal (India)
🏢 Tata Consultancy Services
📍 India