Job Description

Skill: Python, Pyspark, AWS
Exp: 6 to 12 years

Location: Pune

Job Summary

We are seeking a skilled Python + PySpark + AWS Developer to design, develop, and maintain scalable data engineering solutions. The role involves building high-performance ETL pipelines, processing large datasets, and integrating with AWS cloud services to support enterprise analytics and reporting needs.

Key Responsibilities
Design, develop, and optimize ETL pipelines using PySpark and Python.
Work with large-scale datasets to perform data cleansing, transformation, and aggregation.
Integrate with AWS services such as S3, Glue, EMR, Lambda, Redshift, Athena, Kinesis.
Implement data lake and data warehouse solutions on AWS.
Ensure data quality, consistency, and security across all processing stages.
Collaborate with data scientists, analysts, and business stakeholders to deliver data solutions.
Monitor, troubleshoot, and optimize data workflows for performance and cost efficiency.
Follow best practices for code versioning, CI/CD, and cloud resource management.
Write unit tests and maintain documentation for all developed solutions.

Required Skills & Qualifications




Solid programming skills in Python (data manipulation, APIs, automation).
Hands-on experience with PySpark for distributed data processing.
Proficiency in AWS cloud services (S3, Glue, EMR, Lambda, Redshift, Athena, IAM).
Solid understanding of SQL and relational database concepts.
Experience with data lake and data warehouse architectures.
Knowledge of ETL/ELT design patterns and best practices.
Familiarity with Git and CI/CD tools (e.g., Jenkins, Git Hub Actions).
Strong problem-solving and debugging skills.
Bachelor's degree in Computer Science, Information Technology, or related field.

Preferred Skills
Experience with streaming data (Kafka, Kinesis).
Knowledge of Docker and containerized deployments.
Familiarity with Terraform or Cloud Formation for infrastructure as code.
Exposure to machine learning pipelines in a big data workplace.
Understanding of data governance and compliance standards.

Work Workplace
Location: Onsite / Remote / Hybrid (as per company policy)
Team: Works closely with data engineers, analysts, and cloud architects.
Tools: AWS Cloud, Spark, Python, SQL, Git, CI/CD pipelines.

📌 Python, Pyspark, Aws Chennai (India)
🏢 Cognizant Consulting
📍 India