Job Description
Senior Data Engineer - Pyspark & AWS
nExperience: 5-7 Years
nLocation: Koregoan Park, Pune
nWork Mode: Work from Office (5 days working)
n

nJob Summary:
nWe are seeking an experienced and leadership-minded Senior Data Engineer - Pyspark & AWS to drive the reliability, scaling, and operation of our enterprise data infrastructure. In this role, you will balance core development with production support and operational excellence, ensuring high availability through proactive monitoring and debugging of cloud data streams.
nAs a senior member of the data team, you will take full ownership of the data ecosystem, troubleshoot complex live incidents, and actively mentor and upskill junior engineers.
n

nKey Responsibilities:
nData Engineering & Pipeline Design:
nOrchestration: Program complex workflow dependencies and multi-stage DAGs within Apache Airflow.
nETL Architecture: Construct serverless data preparation workflows using AWS Glue and PySpark.
nCompute Processing: Deploy targeted, cost-optimized scripts via AWS Lambda for event-driven backend tasks.




nStorage Management: Structure secure Amazon S3 data lakes with clear landing, raw, and gold curation tiers.
n

nLeadership & Mentorship:
nTechnical Ownership: Accept end-to-end accountability for platform health, security compliance, and architecture updates.
nTeam Mentorship: Guide junior developers through systematic code reviews, architectural advice, and pair programming sessions.
nProcess Standards: Establish baseline engineering documentation, testing frameworks, and on-call rotation protocols.
n

nTechnical Skills:
nPrimary Skills (Required)
nAWS Production Setting: Background supporting enterprise-scale cloud infrastructures under tight data freshness targets.
nObservability & Triage: Hands-on triage using AWS CloudWatch, CloudTrail, X-Ray, or third-party log collectors.
nServerless Ingestion: Advanced setup of AWS Glue (Crawlers, Catalogs, Jobs) and event-driven AWS

📌 Senior Data Engineer (Pune)
🏢 rbm Software
📍 Pune