Job Summary

Location: Pune, Bengaluru, NCR, Kolkata, Hyderabad

Exp: 6+yrs

We are looking for a talented Data Engineer with strong experience in AWS, Python, PySpark, and Databricks to build and maintain scalable data processing solutions. The ideal candidate will have expertise in developing data transformation pipelines, managing large datasets, working with SQL/NoSQL databases, and collaborating with cross-functional engineering teams.

The successful candidate will be responsible for:

- Designing, building, and maintaining data transformation pipelines using AWS services (Lambda, Step Functions, Glue, S3, etc.) and Python.
- Developing and optimizing data processing workflows using PySpark and Databricks.
- Implementing and maintaining complex data calculation and transformation engines.
- Managing and processing large-scale datasets efficiently.
- Working with PostgreSQL and MongoDB databases.
- Developing scalable and high-performance data solutions.
- Collaborating closely with Software Engineers and other stakeholders.
- Participating in code reviews, technical discussions, and software planning activities.
- Contributing to automation, testing, CI/CD, and deployment processes.

Key Responsibilities





- Build and support end-to-end ETL/ELT data pipelines.
- Design data models and transformation frameworks for analytics and reporting.
- Ensure data quality, reliability, and performance optimization.
- Troubleshoot and resolve data pipeline and database issues.
- Follow software engineering best practices and coding standards.
- Create and maintain technical documentation.
- Support cloud-based data architecture initiatives.

Mandatory Skills

- Python Programming
- AWS Services (Lambda, Step Functions, S3, Glue)
- PySpark
- Databricks
- SQL
- PostgreSQL
- MongoDB
- ETL/ELT Development
- Large-scale Data Processing
- Data Modeling and Transformation

Preferred Skills

- AWS Glue
- CI/CD and DevOps concepts
- Data Quality & Monitoring Frameworks
- Git Version Control

Education Requirements

- Bachelor's degree in Computer Science, Information Technology, Engineering, or equivalent experience.
- Educational qualification is flexible for candidates with robust relevant experience.

Certifications

- AWS Certification (Preferred/Optional)
- AWS Certified Cloud Practitioner
- AWS Certified Developer Associate
- AWS Certified Data Engineer Associate

📌 Aws Data Engineer (West Bengal)
🏢 Tata Consultancy Services
📍 West Bengal