? Hiring: Lead Data Engineer – Databricks / PySpark

? Location: Hinjewadi, Pune

? Experience: 8–14 years

⏳ Notice Period: Immediate to Serving

? Work Model: To be discussed

We are hiring a Lead Data Engineer for a leading AI and technology services organization with a 15,000+ strong workforce across the Asia-Pacific region. The organization delivers AI, digital transformation, cloud, infrastructure, cybersecurity, and technology solutions to enterprise clients across multiple industries.

? Key Responsibilities

- Design, build, and operate scalable data pipelines using Databricks
- Develop end-to-end data workflows from ingestion through transformation to consumption
- Build and optimize PySpark pipelines and Spark jobs
- Modernize legacy ETL processes and refactor code to PySpark
- Work with Delta Lake , Databricks Jobs, Workflows, and notebooks
- Implement data quality, monitoring, validation, and error-handling frameworks
- Troubleshoot production data pipeline issues
- Follow best practices around Git, testing, CI/CD, and code reviews
- Collaborate with architects, analysts, business stakeholders, Infrastructure, Applications, and Cyber teams
- Mentor junior data engineers and provide technical guidance

? Must-Have Skills ✅ 8–14 years of experience in Data Engineering or related roles

✅ 4–8 years of hands-on Databricks experience





✅ Strong PySpark, Python, and SQL skills

✅ Experience building production-scale data pipelines

✅ Strong understanding of ETL/ELT and data engineering principles

✅ Experience with Delta Lake and lakehouse/data modeling concepts

✅ Minimum 2–3 years of team handling/management experience

✅ Strong stakeholder management and mentoring experience

✅ Experience modernizing/refactoring legacy data pipelines

✅ Good understanding of cloud platforms – Azure, AWS, or GCP

✅ Mandatory: Databricks Certified Data Engineer Associate or Professional certification

✅ Job stability – minimum 2 years with an organization preferred

✅ Immediate to serving notice period

? Positive to Have

- Databricks Associate Developer for Apache Spark certification
- Azure/AWS/GCP data engineering certifications
- Structured Streaming
- DevOps and CI/CD
- Data quality and testing frameworks
- Data governance and security

? Interview Process 2 Technical Rounds + 1 Client Round + HR

If you are an experienced Data Engineer / Lead Data Engineer with strong Databricks + PySpark + Python + SQL expertise and team leadership experience, I'd be happy to connect.

? Interested candidates can DM me or share their updated resume at [email protected]

📌 Lead Data Engineer (Pune)
🏢 Arigo HR Solutions
📍 Pune