Job Description

Skill: Python, Pyspark, AWS
Exp: 6 to 12 years

Location: Pune

Job Summary

We are seeking a skilled Python + PySpark + AWS Developer to design, develop, and maintain scalable data engineering solutions. The role involves building high-performance ETL pipelines, processing large datasets, and integrating with AWS cloud services to support enterprise analytics and reporting needs.

Key Responsibilities

- Design, develop, and optimize ETL pipelines using PySpark and Python.
- Work with large-scale datasets to perform data cleansing, transformation, and aggregation.
- Integrate with AWS services such as S3, Glue, EMR, Lambda, Redshift, Athena, Kinesis.
- Implement data lake and data warehouse solutions on AWS.
- Ensure data quality, consistency, and security across all processing stages.
- Collaborate with data scientists, analysts, and business stakeholders to deliver data solutions.
- Monitor, troubleshoot, and optimize data workflows for performance and cost efficiency.
- Follow best practices for code versioning, CI/CD, and cloud resource management.
- Write unit tests and maintain documentation for all developed solutions.

Required Skills & Qualifications

- Strong programming skills in Python (data manipulation,



APIs, automation).
- Hands-on experience with PySpark for distributed data processing.
- Proficiency in AWS cloud services (S3, Glue, EMR, Lambda, Redshift, Athena, IAM).
- Solid understanding of SQL and relational database concepts.
- Experience with data lake and data warehouse architectures.
- Knowledge of ETL/ELT design patterns and best practices.
- Familiarity with Git and CI/CD tools (e.g., Jenkins, Git Hub Actions).
- Strong problem-solving and debugging skills.
- Bachelor's degree in Computer Science, Information Technology, or related field.

Preferred Skills

- Experience with streaming data (Kafka, Kinesis).
- Knowledge of Docker and containerized deployments.
- Familiarity with Terraform or Cloud Formation for infrastructure as code.
- Exposure to machine learning pipelines in a big data workplace.
- Understanding of data governance and compliance standards.

Work Environment

- Location: Onsite / Remote / Hybrid (as per company policy)
- Team: Works closely with data engineers, analysts, and cloud architects.
- Tools: AWS Cloud, Spark, Python, SQL, Git, CI/CD pipelines.

📌 Python, Pyspark, AWS (Chennai)
🏢 Cognizant Consulting
📍 Chennai