18 Nov
|
HackerRank Careers
|
Bengaluru
18 Nov
HackerRank Careers
Bengaluru
HackerRank helps companies like NVIDIA, Amazon, and Microsoft hire and upskill the next generation of developers based on skills, not pedigree. Software has entered an era where humans and AI build side by side. As this shift accelerates, the definition of strong technical talent is changing. We are seeking a Lead Data Engineer with deep expertise in distributed systems, real-time data processing, and modern data platforms to join our data team. This is a senior technical leadership role where you’ll help shape and stabilize our data architecture, drive business-critical initiatives, and mentor a high-performing team. You’ll be a senior individual contributor at the heart of the data organisation — owning complex platform decisions, collaborating cross-functionally with AI, product, and go-to-market teams, and shipping data-driven features that directly drive revenue. This is a greenfield opportunity to shape the next phase of data at HackerRank.
Architect and implement streaming-first data pipelines using Apache Kafka and real-time processing frameworks.
Design and operate distributed data systems using Apache Spark and Flink to support high-volume data processing.
Design and implement distributed query and federation capabilities using Trino, enabling unified access across data lakes, warehouses, and heterogeneous data sources.
Build scalable data-serving and analytical layers using StarRocks and cloud data warehouses such as BigQuery, Redshift, and Snowflake.
Design and maintain robust data models, data marts, and data cubes that power business intelligence, reporting, and self-serve analytics.
Build the next-gen AI-optimised data layer: clean,
structured datasets that power natural language querying and AI add-on features for HackerRank for Work customers.
Implement agile development best practices including Scrum, sprint planning, and documentation using JIRA, Linear, Notion and Confluence.
Collaborate with cross-functional teams including engineering, product, and data science to align technical solutions with business goals.
8+ years of hands-on experience in data engineering, distributed systems, streaming, and cloud-native architectures.
- Deep expertise in Kafka, Spark, Flink, and building scalable data pipelines using modern LakeHouse architectures (Delta Lake, Hudi, Iceberg, Parquet).
- Experience with distributed query engines and metadata management using Trino, Hive Metastore, and analytical serving platforms like StarRocks, ClickHouse, Druid, or Pinot.
- Solid understanding of data governance and security, including Apache Ranger, access controls, auditing, data quality, and lineage.
- Proficiency in Airflow, cloud platforms (AWS/GCP/Azure), observability (Prometheus, Grafana, Loki), and analytical data modeling.
- Hands-on experience with AI Coding Agents (Claude Code, Codex, Cursor), agentic workflows, and applying emerging AI capabilities to improve engineering productivity and operational efficiency.
- Passionate about engineering excellence, technical mentorship,
architectural ownership, and building scalable data products aligned with business outcomes
Proven ability to deliver both data ingestion (inbound) and data serving (outbound) capabilities as reusable platform products.
Good exposure in integrating Redash, Metabase and Looker BI with the data stores.
Prior experience building RAG (Retrieval-Augmented Generation) pipelines using enterprise data sources.
Experience enabling Natural Language Querying (NLQ) and conversational analytics over data warehouses and Lakehouse platforms.
Exposure to MLOps platforms such as SageMaker, Kubeflow, or MLflow, and how they interface with the data stack.
You believe in building data platforms that empower other engineering teams through self-service data ingestion, transformation, discovery, and consumption.
You enjoy translating ambiguous business problems into scalable data products, reusable platform capabilities, measurable engineering outcomes and take pride in engineering excellence, reliability, and long-term impact.
You're genuinely curious about AI and want to work at the intersection of Data Engineering, GenAI, RAG, Agentic AI, and intelligent analytics.
You don't wait for a PM to hand you a roadmap — you proactively identify opportunities, connect data capabilities to business outcomes, and drive initiatives from conception to delivery.
We provide equal opportunity to everyone for employment based on individual performance and qualification. We never discriminate based on race, religion, national origin, gender identity or expression, sexual orientation, age, marital, veteran, or disability status.
📌 Lead AWS Data Engineer (Bengaluru)
🏢 HackerRank Careers
📍 Bengaluru