The Senior / Principal Data Engineer will design, develop, and maintain scalable data platform and analytics solutions across data lakes and operational databases. This role requires hands-on expertise in Azure Databricks, Azure SQL, Python/PySpark, Notebooks, with a strong understanding of data modeling, ETL/ELT best practices, and CI/CD automation in Azure DevOps. The ideal candidate will have a proven record of building robust, efficient, and secure data pipelines that enable analytics, reporting, and AI/ML solutions, preferably in life sciences, clinical research, or healthcare domains.

Data Architecture & Engineering

- Design and implement end-to-end data pipelines using Azure Databricks, Azure Data Factory, and ADLS Gen2.

- Build scalable and performant data models for data lakes (Medallion architecture), data warehouses, and operational systems.

- Develop ELT/ETL frameworks for ingestion from APIs, relational sources, flat files, and third-party systems (e.g., Dynamics 365, Veeva, EDC).

- Optimize data transformations, partitioning, and delta lake performance for analytics workloads.

Data Integration & Automation

- Leverage Python and PySpark for data ingestion, cleansing, enrichment, and advanced transformations.

- Implement CI/CD pipelines for data workflows using Azure DevOps and Git, including automated testing, deployment, and monitoring.

- Develop and integrate RESTful APIs for cross-system data exchange and automation.

Analytics Enablement

- Collaborate with the BI team to ensure clean, high-quality, and accessible data for the Power BI platform.

- Support semantic modeling, metric layer design, and data governance best practices.

- Enable advanced analytics by provisioning data for ML/AI initiatives and predictive insights.





Cross Functional Collaboration

- Collaborate with product/system owners, analysts, and business stakeholders to translate analytical requirements into technical data solutions.

- Drive best practices in Agile development, version control, and DevOps workflows.

Education Requirements and Qualifications

Qualifications

- Bachelor’s degree in Computer Science, Information Systems, Engineering, or related field (Master’s preferred).

- Minimum 5–8 years of relevant experience building and maintaining data solutions (data lakes, data warehouses, operational databases).

- Expert-level proficiency in Azure Databricks, PySpark, SQL, and Azure DevOps.

- Proven experience with Azure Data Factory, ADLS Gen2, and Azure SQL Server.

- Working knowledge of CI/CD automation, version control (Git), and infrastructure as code (ARM or Terraform).

- Experience with Power BI or similar analytics platforms (Tableau, Looker) required; experience with Snowflake, Redshift, or Synapse Analytics is a plus.

- Strong analytical, debugging, and performance-tuning skills.

- Experience in life sciences or healthcare industries is a strong plus.

Skills

Core expertise: Expert-level in Databricks, PySpark, SQL, and Azure DevOps

Data engineering: Data modeling, Delta Lake optimization, ETL/ELT design, distributed processing.

Integration & Automation: Azure Data Factory, REST APIs, CI/CD pipelines, Git branching strategies.

Analytics & BI: Power BI (Tableau), semantic layer design, DAX/SQL tuning.

Cloud & DevOps: Azure ecosystem (ADF, ADLS, Azure SQL, Synapse), Infrastructure as Code

Data Governance & Quality: Metadata management, data validation frameworks, logging and monitoring.

Soft skills: Positive communication, mentoring, Agile teamwork, analytical thinking, collaboration.

📌 Data Developer (Bengaluru)
🏢 Important Company of the Sector
📍 Bengaluru