Company Name: VARITE India Private Limited

About The Client

A global IT services and consulting company, multinational information technology (IT), headquartered in Tokyo, Japan. The Client offers a wide array of IT services, including application development, infrastructure management, and business process outsourcing. Their consulting services span business and technology, while their digital solutions focus on transformation and user experience design.

It excels in data and intelligence services, emphasizing analytics, AI, and machine learning. Additionally, their cybersecurity, cloud, and application services round out a comprehensive portfolio designed to meet the diverse needs of businesses worldwide. About The Job:

- Experience: 8+ Years
- You will combine clinical data expertise with strong data engineering and technical skills to generate well documented pipelines from source to curated data sets in common data models like CDISC SDTM.
- You will collaborate closely with clinical SMEs, data scientists, infrastructure, and other skilled data engineers.
- We are looking to expand this functionality to include Real World Data (from a broad range of regristries).
- You will help extend our medallion Databricks pipelines (CDISC SDTM) to incorporate Real World Data (RWD) from registries and other sources, working with clinical experts and AI teams to combine rule based and automated mapping approaches (including OMOP interoperability).

Essential Job Functions:

- Design, build and maintain production ETL pipelines in Databricks/Delta Lake to ingest RWD (registries, claims, EHR extracts) and transform into standard models.
- Implement harmonisation workflows to map incoming RWD to OMOP and to the internal CDISC SDTM canonical model; handle vocabulary mapping, units normalization and provenance.
- Extend the medallion architecture (bronze/silver/gold)



patterns with robust validation, lineage, partitioning and performance tuning.
- Develop configurable, inputdriven transformation frameworks so clinical experts can drive mapping rules via config files and catalogs.
- Integrate AI/automation components (e.g., model-assisted mapping, NLP for free text) with humanintheloop review and confidence scoring.
- Establish testing, CI/CD, monitoring and alerting for ETL jobs and automations; ensure reproducibility, versioning and governance.
- Collaborate with clinical data scientists, data stewards and stakeholders to define requirements, data contracts and success metrics.

Qualifications:

Required skills and qualifications

- Proven experience designing and implementing ETL pipelines in Databricks / Spark and Delta Lake.
- Strong knowledge of OMOP CDM and experience mapping datasets to OMOP; familiarity with CDISC SDTM is a plus.
- Expertise in data moClienting, partitioning, performance tuning, and best practices for large clinical/RWD datasets.
- Experience with vocabulary services and terminology mapping (OHDSI/Athena, UMLS, or similar).
- Experience integrating AI/NLP components into data pipelines (entity extraction, mapping suggestions) is desirable.
- Familiarity with testing frameworks for data (Great Expectations, Deequ), CI/CD, infrastructure as code, and orchestration tools (Databricks Jobs, Airflow).




- Positive communication skills and experience working with domain experts to capture requirements.

Preferred
- Prior experience in pharma or clinical research environments.
- Knowledge of data governance, privacy regulations and secure handling of patient data.
- Experience with Unity Catalog, Databricks Delta Sharing, and cloud infrastructure (Azure/AWS).

How to Apply: Interested candidates are encouraged to respond/submit their updated resumes, and for additional job opportunities, please visit Jobs In India – VARITE.

Unlock Rewards: Refer Candidates and Earn.

If you're not available or interested in this opportunity, please pass this along to anyone in your network who might be a good fit and interested in our open positions. VARITE offers a Candidate Referral program, where you'll receive a one-time referral bonus based on the following scale if the preferred candidate completes a three-month assignment with VARITE.

Experience Level Bonus Referral:

0-2 years INR 5,000 2-6 years INR 7,500 6+ years INR 10,000

About VARITE: VARITE is a global staffing and IT consulting company providing technical consulting and team augmentation services to Fortune 500 Companies in USA, UK, CANADA and INDIA. VARITE is currently a primary and direct vendor to the leading corporations in the verticals of Networking, Cloud Infrastructure, Hardware and Software, Digital Marketing and Media Solutions, Clinical Diagnostics, Utilities, Gaming and Entertainment, and Financial Services.

Equal Opportunity Employer:

VARITE is an equal opportunity employer. We celebrate diversity and are committed to creating an inclusive environment for all employees. We do not discriminate based on race, color, religion, sex, sexual orientation, gender identity or expression, national origin, age, marital status, veteran status, or disability status.

📌 Data Engineer (Bengaluru)
🏢 VARITE
📍 Bengaluru