17 Aug
|
Innovaccer Analytics
|
Noida
17 Aug
Innovaccer Analytics
Noida
Engineering at Innovaccer
With every line of code, we accelerate our customers' success, turning complex challenges into cutting-edge solutions. Collaboratively, we transform each data point we gather into valuable insights for our customers. Join us and be part of a team that's turning dreams of better healthcare into reality, one line of code at a time. Together, we're shaping the future and making a meaningful impact on the world.
About the Role
As a Senior Software Engineer on the Lakehouse team, you will build and operate the data pipelines at the heart of Innovaccer's on-premise platform: Spark ingestion jobs landing raw healthcare data into Apache Iceberg, and Trino SQL transforms building the layered tables thatpower analytics and applications. You will work hands-on across the full pipeline surface, from file validation and quarantine at ingestion to query performance and table health in serving.
A Day in the Life
● Build Spark ingestion jobs that land high-volume raw files into Iceberg tables with schema handling, bad-record quarantine, and idempotent batch replay.
● Develop and operate Trino SQL transform pipelines across data layers: validation and
typing, business-rule transforms, MERGE-based deduplication, and aggregate builds.
● Port existing warehouse SQL workloads to Trino and Spark SQL dialects, and validate results against source outputs.
● Automate Iceberg table maintenance: compaction, snapshot expiry, and orphan-file cleanup as scheduled workflows.
● Tune query and pipeline performance: partitioning strategy, file sizing, statistics, and resource-group behavior.
● Instrument pipelines with data-quality checks, reconciliation reports, and alerting, and participate in per-dataset validation during rollout phases.
● B.E., B.Tech., M.Sc. degree in Computer Science or a related technical field.
● 5+ years of data engineering experience building production pipelines at scale.
● Strong SQL skills and hands-on experience with Apache Spark for batch proce
📌 4467- Software Development Engineer-III (Noida)
🏢 Innovaccer Analytics
📍 Noida