Job Summary:

We are seeking a highly motivated and experienced Azure Databricks Engineer with strong expertise in Azure Databricks and cloud-based data platforms. In this role, you will design, develop, and optimize scalable data pipelines, support the implementation of enterprise-grade data and analytics solutions, and leverage Databricks best practices to deliver reliable, high-quality data products. You will work closely with data engineers, architects, business stakeholders, and analytics teams to enable data-driven decision-making across the organization.

Key Responsibilities:

- Design, develop, and optimize data pipelines using Azure Databricks (Apache Spark).
- Build and support large-scale, cloud-native data processing solutions on Azure.
- Collaborate with data scientists, analysts, and business stakeholders to ensure data availability, reliability, and quality.
- Implement and support Databricks Lakehouse architecture, Delta Lake, Unity Catalog, and MLflow solutions.
- Follow and promote best practices for CI/CD, version control, testing, and deployment in a Databricks environment.
- Work closely with project teams to deliver solutions aligned with business requirements and timelines.

Required Qualifications:

- Bachelor's or Master's degree in Computer Science, Data Engineering, or a related field.
- 7+ years of experience in data engineering.




- Hands-on expertise with Azure Databricks, including Spark SQL, Delta Lake, job orchestration, and performance tuning.
- Proficient in Python, SQL, and Apache Spark; knowledge of Scala is a plus.
- Experience working with Azure cloud services, including Databricks deployments.
- Strong understanding of data modeling, ETL/ELT, data warehousing, and up-to-date data lake architectures.
- Advanced SQL skills for data analysis. Expertise on problem-solving skills with a focus on finding innovative solutions to complex data challenges. Ability to proactively identify and address potential issues and bottlenecks.
- Fluent in multiple coding languages, such as Python or Java and able to learn new ones quickly
- Technical skills in metadata capturing and management using Collibra to document, organize, and ensure the quality of metadata assets
- Strong Knowledge of cloud-based data, compute, and storage services, including AWS S3, EC2, RDS, EBS, Lambda, Airflow (MWAA), Containerization Services (ECS, EKS) and so on.

Preferred Qualifications:

- Databricks certification (e.g., Databricks Certified Data Engineer Professional).
- Experience with Unity Catalog, MLflow, and building data products on the Lakehouse.
- Experience with Azure services such as Azure Data Factory (ADF) and Azure Data Lake Storage (ADLS).

📌 Data Engineer (Ahmedabad)
🏢 First Meridian Business Services
📍 Ahmedabad