08 Oct
|
NTT DATA Services
|
Bengaluru
08 Oct
NTT DATA Services
Bengaluru
Req ID: 392715
NTT DATA strives to hire exceptional, cutting-edge and passionate individuals who want to grow with us. If you want to be part of an inclusive, adaptable, and forward-thinking organization, apply now.
We are currently seeking a Databricks Engg - Systems Integration Specialist to join our team in Bangalore, Karnātaka (IN-KA), India (IN).
Key Responsibilities
• Design, build, and maintain production data pipelines on Azure Databricks using PySpark, Delta Lake, and a bronze/silver/gold medallion architecture - owning solutions end-to-end from ingestion through curated, consumption-ready datasets.
• Ingest data from diverse sources (Oracle, Netezza, Mongo DB, SQL Server, Synapse, Azure Data Lake, APIs, file feeds) using ADF and Databricks Workflows, handling incremental loads, CDC, schema drift, and late-arriving data.
• Build reusable, config-driven frameworks and libraries for ingestion, transformation, and data quality rather than one-off scripts, enabling the wider team to deliver faster.
• Embed testing and data quality into every pipeline - unit and integration tests, source-to-target reconciliation, and automated DQ gates (null, uniqueness, freshness, row-count thresholds) with clear failure handling and alerting.
• Own CI/CD and environment promotion through Azure DevOps/Git, Databricks Asset Bundles, and infrastructure-as-code, delivering repeatable, auditable deployments across dev, UAT, and production.
• Tune performance and manage cost - diagnose skew, shuffle, and small-file issues, optimize Delta tables and cluster configurations, and continuously reduce runtime and compute spend.
• Provide production support and reliability engineering - monitor SLAs, perform root-cause analysis on failures, and drive design changes and operational improvements that prevent recurrence.
• Contribute to data modelling, data analysis, and solution architecture on demand - profiling sources, producing logical/physical and dimensional models, and transla
📌 Databricks Engg (Bengaluru)
🏢 NTT DATA Services
📍 Bengaluru