Senior Data Engineer (Bengaluru)

Senior Data Engineer (Bengaluru)

15 Nov
|
Infiniti Research
|
Bengaluru

15 Nov

Infiniti Research

Bengaluru

## Job highlights
- 7+ years experience in Databricks data engineering with strong PySpark, SQL, and AWS Cloud skills
- Design, develop, and optimize scalable ETL pipelines in Databricks; transform AWS assets to Databricks; manage data catalog and support healthcare analytics

Job description
Senior Data Engineer Strictly Immediate Joiners only(0 to 15 days NP) Exp : Minimum 5+ Years

Skills:
AWS , Databricks

Job Description
We are seeking a highly skilled Data Engineer with expertise in Databricks and AWS Cloud to design, build, and optimize enterprise-scale ETL pipelines and reporting solutions. This role is ideal for someone who thrives at the intersection of data engineering and business intelligence, with a strong focus on transforming complex datasets into inputs for actionable insights. You will play a critical role in enabling spend analytics, patient access, and commercial reporting initiatives. Key Responsibilities Databricks ETL Development
- Design, develop, and maintain scalable ETL pipelines in Databricks using PySpark and SQL.
- Implement robust data ingestion, transformation, and validation processes to ensure high-quality datasets for analytics.
- Optimize workflows for performance, scalability, and reliability across large healthcare datasets. Data Product Build
- Transform AWS assets (on EC2 and Redshift) to Databricks by recreating the running ETL jobs and orchestrating via Airflow or Databricks Workflows.
- Hands-on working experience of catalog management using Unity Catalog.
- Conversant with AI capabilities within AWS and Databricks. Requirements
- 7+ years of experience in Databricks data engineering roles.
- Strong hands-on proficiency in Databricks (PySpark, SQL) and AWS Cloud.
- Proven track record in ETL pipeline development and understanding of BI dashboards.
- Experience with US healthcare datasets.
- Solid SQL skills for data extraction, aggregation, and reporting.
- Excellent problem-solving abilities,



with the capacity to work independently and in cross-functional teams.
- Detail-oriented with a commitment to delivering high-quality, reliable data solutions. Good to Have Domain Data Knowledge
- Knowledge of IQVIA claims (standard, eLaaD, Remit, Rejection, NBRx, TRx, IGG4 Claims) and MMIT datasets to derive actionable insights. End-to-End Production Deployment Orchestration *Candidates
* *with
* *exposure
* *to
* *deploying
* *and
* *orchestrating
* *data
* *pipelines
* *in
* *production environments
* *will
* *be
* *preferred.
* *While
* *not
* *a
* *core
* *requirement,
* *the
* *following
* *experience
* *is
* *a strong
* *differentiator:
*
- Experience deploying ETL or ML pipelines end-to-end in a production environment, including environment promotion across dev, staging, and production.
- Familiarity with CI/CD tooling (GitHub Actions, Azure DevOps, or Jenkins) for automating pipeline deployment and release management.
- Exposure to Databricks Asset Bundles (DABs) or equivalent frameworks for version
- controlled, repeatable job deployments.
- Working knowledge of Apache Airflow for DAG authoring, scheduling, dependency management, and monitoring of production workflows.
- Awareness of infrastructure-as-code practices (Terraform or AWS CloudFormation) for managing cloud resources supporting data pipelines.
- Basic understanding of containerization concepts (Docker) for packaging and deploying data pipeline components.
- Experience with pipeline monitoring and alerting tracking job health, SLA adherence, failure notifications, and data freshness in production.
- Familiarity with secrets and configuration management across environments (AWS Secrets Manager, Databricks Secrets, or equivalent). If Interested, Request you to share me your updated resume to [email protected] Thanks & Regards, Geetha Shree Lead Recruiter Talent Acquisition Team | Quantzig AI Solutions Pvt. Ltd. www.quantzig.com +91 (phone hidden)

📌 Senior Data Engineer (Bengaluru)
🏢 Infiniti Research
📍 Bengaluru

Reply to this offer

Impress this employer describing Your skills and abilities, fill out the form below and leave Your personal touch in the presentation letter.

Subscribe to this job alert:

Get the latest job offers by email for: senior data engineer (bengaluru) / bengaluru

Subscribe to this job alert:

Get the latest job offers by email for: senior data engineer (bengaluru) / bengaluru