19 Nov
|
Asper.ai
|
Bengaluru
19 Nov
Asper.ai
Bengaluru
We are looking for a technical anchor and day-to-day engineering leader of Asper Data Platform: the person who owns how the platform is built, how it ships, and how fast a new customer goes from raw files to trusted Gold data. This is a hands-on leadership role, roughly 60% technical (architecture, design and code reviews, deep debugging on Databricks) and 40% people and delivery leadership. You will lead a team of data and platform engineers, work closely with the forward-deployed and customer success teams who onboard customers, and partner with product, GenAI and ontology teams to turn the ADP roadmap into a reliable, multi-tenant product. We are looking for someone who has built and run a data platform as a product, not as a one-off project, and who still enjoys getting into a Spark plan or a Delta merge conflict when it matters.
What you'll do
Platform architecture and technical ownership
• Own the technical design of ADP end to end: ingestion and connectors, medallion layers, the CPG data model, multi-tenancy, governance and serving.
• Set and enforce platform standards: Bronze→Silver mapping rules, Silver/Gold modelling conventions, schema evolution and backward compatibility, naming and data contracts.
• Make build-vs-buy and Databricks-native-vs-external decisions (e.g. Lakebase vs external Postgres, SQL Warehouse vs other query engines) with clear cost, latency and operability trade-offs.
• Stay hands-on: review designs and PRs, write proofs of concept, and lead deep dives on Spark performance, Delta concurrency, cluster sizing and cost optimization. Delivery and engineering excellence
• Run sprint planning, roadmap execution and release management; deliver predictably against quarterly OKRs.
• Own platform reliability: SLOs, monitoring, data quality checks, incident response and root-cause reviews.
• Drive CI/CD for data (Databricks Asset Bundles, automated testing of pipelines and mappings, setting promotion) and AI-assisted development practices.
• Track and improve engineering North Star metrics such as time-to-value for new customers and platform availability.
Productization and customer onboarding
• Turn repeated customer-specific work into reusable, configuration-driven platform capabilities.
• Partner with forward-deployed engineering and customer success to cut onboarding time for new tenants and data sources; build the tooling and documentation that let them self-serve.
• Lead technical design for complex enterprise deployments: cross-tenant and cross-cloud data sharing, private networking, customer-owned Databricks workspaces. People leadership
• Lead, hire and grow a team of 10–15 engineers across levels; run 1:1s, goal setting and performance reviews.
• Build a culture of ownership, code quality and explicit written design.
• Coach senior engineers into tech leads; develop junior engineers through structured mentoring. Cross-functional partnership
• Work with Product, GenAI/ontology, RGM application and Solutions teams to align the ADP roadmap with product needs.
• Represent ADP engineering with Databricks and Azure partner teams, and in technical conversations with customer data and IT teams.
• Manage external vendors or partners delivering platform migrations where needed. Experience (must have)
• 12–15 years in software or data engineering, with at least 4 years managing engineering teams of 8+ people.
• Product-company background is mandatory: you have built and shipped a data platform, data product or SaaS platform used by many customers or internal teams. Careers spent only in IT services or consulting delivery will not be considered.
• A track record of taking a platform from early stage to production scale, including multi-tenant or multi-customer operation.
Data engineering and platform skills (hands-on)
• Deep, current hands-on experience with Apache Spark (PySpark and Spark SQL) and strong Python and SQL.
• Strong data modelling: dimensional/star schemas, medallion architecture, slowly changing dimensions, schema evolution.
• Batch and incremental pipeline design, orchestration, data quality frameworks and observability.
• Cloud data platforms on Azure (preferred) or AWS/GCP: object storage, networking basics, IAM, Key Vault, private endpoints.
• Engineering fundamentals: Git workflows, CI/CD, automated testing, infrastructure as code (Terraform or similar). Databricks (required)
• Production experience designing and running workloads on Databricks: Delta Lake, Unity Catalog, Workflows/Jobs, Lakeflow Declarative Pipelines (DLT), Databricks SQL.
• Working knowledge of Delta internals and tuning: merges, concurrency conflicts, liquid clustering, Z-ordering, optimize/vacuum, Photon.
• Experience with Databricks governance and sharing: catalogs and permissions, Delta Sharing, workspace and environment design.
• Comfort managing Databricks cost (cluster policies, serverless, DBU monitoring). Nice to have
• Ontology and knowledge graphs: experience with ontology design, semantic layers or knowledge graphs (Neo4j, FalkorDB or RDF/OWL), Cypher, and using an ontology to standardize or map heterogeneous data.
• GenAI on data platforms: RAG, Vector Search, model serving, LLM-assisted schema mapping or agentic data pipelines.
• CPG / retail domain: familiarity with POS, syndicated data (Nielsen, Circana), shipments, trade promotion or RGM analytics.
• Databricks depth: Databricks certifications (Data Engineer Professional, Solutions Architect), Lakebase, Databricks Apps, or the Built on Databricks / Marketplace programs.
• Connectors and integration: Airbyte or similar ingestion frameworks, event-driven integration (Azure Service Bus, Kafka).
• Startup or scale-up
experience:
building a platform team in a fast-growing product company.
📌 Senior Development Manager (Bengaluru)
🏢 Asper.ai
📍 Bengaluru