Senior Data Engineer

Strictly Immediate Joiners only(0 to 15 days NP)

Exp : Minimum 5 Years

Skills : AWS , Databricks

:

We are seeking a highly skilled Data Engineer with expertise in Databricks and AWS Cloud to design, build, and optimize enterprise-scale ETL pipelines and reporting solutions. This role is ideal for someone who thrives at the intersection of data engineering and business intelligence, with a strong focus on transforming complex datasets into inputs for actionable insights. You will play a critical role in enabling spend analytics, patient access, and commercial reporting initiatives.

Key Responsibilities Databricks ETL Development

- Design, develop, and maintain scalable ETL pipelines in Databricks using PySpark and SQL.
- Implement robust data ingestion, transformation, and validation processes to ensure high-quality datasets for analytics.
- Optimize workflows for performance, scalability, and reliability across large healthcare datasets.

Data Product Build

- Transform AWS assets (on EC2 and Redshift) to Databricks by recreating the running ETL jobs and orchestrating via Airflow or Databricks Workflows.
- Hands-on working experience of catalog management using Unity Catalog.
- Conversant with AI capabilities within AWS and Databricks.

Requirements

- 7 years of experience in Databricks data engineering roles.
- Strong hands-on proficiency in Databricks (PySpark, SQL) and AWS Cloud.
- Proven track record in ETL pipeline development and understanding of BI dashboards.
- Experience with US healthcare datasets.
- Robust SQL skills for data extraction, aggregation, and reporting.

- Excellent problem-solving abilities, with the capacity to work independently and in cross-functional teams.
- Detail-oriented with a commitment to delivering high-quality, reliable data solutions.

Good to Have

Domain Data Knowledge

- Knowledge of IQVIA claims (standard, eLaaD, Remit, Rejection,



NBRx, TRx, IGG4 Claims) and MMIT datasets to derive actionable insights.

End-to-End Production Deployment Orchestration

Candidates with exposure to deploying and orchestrating data pipelines in production environments will be preferred. While not a core requirement, the following experience is a strong differentiator:

- Experience deploying ETL or ML pipelines end-to-end in a production environment, including environment promotion across dev, staging, and production.
- Familiarity with CI/CD tooling (GitHub Actions, Azure DevOps, or Jenkins) for automating pipeline deployment and release management.
- Exposure to Databricks Asset Bundles (DABs) or equivalent frameworks for version- controlled, repeatable job deployments.
- Working knowledge of Apache Airflow for DAG authoring, scheduling, dependency management, and monitoring of production workflows.
- Awareness of infrastructure-as-code practices (Terraform or AWS CloudFormation) for managing cloud resources supporting data pipelines.
- Basic understanding of containerization concepts (Docker) for packaging and deploying data pipeline components.
- Experience with pipeline monitoring and alerting tracking job health, SLA adherence, failure notifications, and data freshness in production.
- Familiarity with secrets and configuration management across environments (AWS Secrets Manager, Databricks Secrets, or equivalent).

If Interested, Request you to share me your updated resume to [email protected]

Thanks & Regards,

Geetha Shree

Lead Recruiter

Talent Acquisition Team | Quantzig AI Solutions Pvt. Ltd.

www.quantzig.com

91 (phone hidden)

Role: Data Engineer, Industry Type: Analytics / KPO / Research, Department: Engineering - Software & QA, Employment Type: Full Time, Permanent Role Category: Software Development Education UG: Any Graduate Key Skills Skills highlighted with ‘‘ are preferred keyskills AWSData Bricks PysparkSQL

📌 Senior Data Engineer - Bangalore - Immediate Joiners only (Bengaluru)
🏢 Infiniti Research
📍 Bengaluru