Immediate/Early Joiners preferred

We are seeking an experienced Senior Data Engineer with strong hands-on expertise in designing, developing, and optimising scalable data ingestion and processing pipelines using Databricks, PySpark, and AWS services.

Key Responsibilities

- Design and implement scalable batch and streaming data pipelines using Databricks and PySpark.
- Develop cloud-native data ingestion and ETL solutions using AWS streaming and data processing services.
- Implement Medallion Architecture (Bronze, Silver, and Gold layers) and modern Lakehouse solutions.
- Leverage Unity Catalogue for data governance, security, lineage, and metadata management.
- Integrate and support MLflow for machine learning lifecycle management and analytics use cases.
- Optimise
- pipeline performance, reliability, scalability, data quality, and cost efficiency.
- Collaborate with data science, analytics, and business teams to deliver reliable data solutions.
- Participate in technical design, code reviews, CI/CD, and engineering best practices.





Required Skills & Expertise

- Databricks and PySpark
- AWS Streaming and ETL Services
- Unity Catalog and MLflow
- Medallion Architecture and Lakehouse Architecture
- SQL, Data Modelling, and Data Lake concepts
- Robust experience in enterprise-scale data ingestion and processing pipelines

Good to Have

- Python scripting and automation
- Pandas for data transformation and analysis
- Workflow orchestration using Airflow or Databricks Workflows
- Exposure to DataOps/MLOps practices and CI/CD implementation

Ideal Candidate A hands-on Data Engineer with 6–8 years of relevant experience, strong problem-solving skills, and the ability to build secure, governed, scalable, and cost-efficient data platforms. ? Interested candidates can share their updated CV at [email protected]

#Hiring #SeniorDataEngineer #DataEngineering #Databricks #AWS #PySpark #ETL #DataPipelines #LakehouseArchitecture

📌 Senior Data Engineer – Databricks & AWS (Pune)
🏢 RefRelay
📍 Pune