About the Role

We are looking for a skilled and motivated Senior Data Engineer to design, build, and maintain robust, scalable data pipelines on cloud. You will work across multi-source data environments, collaborate with software engineers and data platform teams, and deliver high-quality, maintainable data products. We welcome robust data engineers from either an AWS or an Azure background mention your primary cloud when you apply.

Must-Have Skills (Core All Candidates)

- Strong hands-on Python for data transformation, automation, and scripting.
- PySpark for large-scale data processing and distributed computing.
- Proven experience designing, building, and maintaining data pipelines / ETL workflows.
- Experience managing large datasets and NoSQL databases (CosmosDB, MongoDB, etc.).
- Solid SQL and understanding of data modeling and data warehousing concepts.
- Exposure to automation, testing, CI/CD, and deployment processes.
- Active participation in code reviews, design discussions, and sprint planning.

Cloud-Specific Skills (based on your track)

1. AWS Track

- AWS Cloud services with Terraform (Infrastructure as Code).
- AWS Glue, Lambda, and CloudWatch for ETL, serverless processing, and monitoring.
- Data Lake architecture standardizing and managing raw and curated data across multiple sources.
- (Nice to have) IoT/edge data ingestion — Greengrass, MQTT, OPC-UA; Windows Server + PowerShell; Grafana dashboards; Salesforce and Power BI integration.

2. Azure Track





- Azure Cloud platform with Databricks for data engineering.
- Unity Catalog for data governance and cataloging.
- Building and optimizing PySpark + Databricks transformation pipelines.
- (Nice to have) Basic Data Science knowledge.

Key Responsibilities

- Design, build, and maintain scalable data transformation pipelines using Python, PySpark, and cloud-native services.
- Efficiently manage large datasets and NoSQL databases (CosmosDB).
- Standardize and govern raw and curated data across a multi-source data lake / lakehouse.
- Implement and maintain complex data calculations and business logic.
- Build infrastructure and automate deployments (Terraform / IaC) and monitor system health.
- Collaborate with software engineers and data platform teams on access, API provisioning, and data delivery.
- Contribute to automation, testing, deployment, and code quality best practices.

Soft Skills & Mindset

- Strong analytical, problem-solving, and debugging skills.
- Proactive, detail-oriented, and focused on code quality and documentation.
- Clear communication in English; comfortable working both independently and in cross-functional teams.
- Continuous-learning mindset; stays current with industry trends.

Qualifications

- Bachelor's / Master's degree in Computer Science, Engineering, or a related field.
- 7–13 years of relevant data engineering experience with strong cloud exposure (AWS or Azure).

📌 Senior Data Engineer (AWS / Azure) (Maharashtra)
🏢 Ancilar
📍 Maharashtra