16 Nov
|
Sigma Allied Services
|
Bengaluru
16 Nov
Sigma Allied Services
Bengaluru
Senior AI & Data Engineer - GraphTech Expert
Role Overview
We are seeking a Senior AI & Data Engineer with deep graph database expertise to architect and optimize large-scale knowledge graph systems powering enterprise GenAI applications. This hands-on technical role focuses on graph technologies, knowledge graph construction, semantic infrastructure, and AI-driven knowledge systems. You will own end-to-end design and optimization of graph-centric initiatives: entity/relationship extraction, knowledge graph enrichment, graph-backed RAG systems, and production graph analytics at scale.
Key Responsibilities
Design & architect scalable knowledge graph systems: Entity extraction, relationship inference, hierarchical graph construction, and enrichment pipelines handling billions of nodes/relationships, graph-backed RAG systems with entity linking.
Optimize graph database performance: Neo4j query optimization, index strategies (b-tree, HNSW, full-text), traversal efficiency, bulk operations, and large-scale analytics.
Implement distributed graph processing: High-volume ingest pipelines with idempotence, state management, consistency guarantees, and production monitoring.
Evaluate and integrate emerging technologies: graph databases, vector stores, semantic search, LLM APIs, and AI-driven data enrichment.
Own performance optimization across the stack: query planning, indexing strategies, caching, batch processing, and compute resource allocation.
Collaborate with cross-functional teams (data science, backend, platform) to translate requirements into robust, production-grade systems.
Technical Skills (Required)
Graph Databases & Technologies (Core Expertise):
o Neo4j: Cypher query language, index strategies (b-tree, full-text, HNSW), query optimization and execution planning.
o Graph data modeling: entity-relationship design, schema optimization, constraint management.
o Graph algorithms: shortest path, centrality (betweenness, closeness, PageRank), community detection, k-NN similarity, link prediction.
o Graph traversal optimization: query performance tuning, index selection, caching strategies.
o Knowledge graphs: entity linking, relationship extraction,
hierarchical organization, semantic enrichment.
o Graph analytics at scale: aggregation, pattern matching, temporal graphs, multi-hop reasoning.
Data Engineering:
o End-to-end pipeline design: ingestion, transformation, enrichment, indexing, serving.
o ETL/ELT workflows, distributed query optimization, data lineage and quality.
o Experience with structured and unstructured data at scale.
Databases & Storage:
o Relational databases (SQL optimization, query planning, schema design).
o NoSQL systems (document stores, key-value, time-series).
o Vector databases and semantic search (indexing, k-NN, HNSW, approximate nearest neighbor).
Cloud Platforms:
o Azure services: Compute (Functions, VMs, Kubernetes), Storage (Blob, Table, CosmosDB), Azure Durable Functions, Messaging & Queues (Service Bus, Event Hub, Event Grid systems at scale), Analytics (Synapse, Data Factory, Cognitive Search).
o Serverless patterns: Functions, Logic Apps, event-driven architectures.
o Infrastructure-as-Code (Bicep, Terraform, ARM templates).
o Service-to-service integration, security (Key Vault, Managed Identity, RBAC).
Distributed Systems:
o Asynchronous messaging patterns, message queues, event streaming.
o State management, eventual consistency, idempotence, retry strategies.
o Fault tolerance, graceful degradation, health monitoring.
o Load balancing, backpressure, throttling.
Backend & Systems:
o Python, Java, or equivalent production-grade language.
o API design, microservices architecture.
o Async/concurrent programming, dependency injection, testing.
o Observability: logging, metrics, tracing (Application Insights, ELK, Prometheus).
Generative AI Systems:
o LLM APIs (OpenAI, Azure OpenAI, Claude), prompt engineering, chain-of-thought patterns.
o RAG: chunking, retrieval optimization, context ranking; embedding models and vector operations.
o Entity/relationship extraction, semantic understanding, LLM application architecture.
o ML model serving, inference optimization, token management, cost optimization.
Technical Skills (Nice to Have)
Graph Expertise: Neo4j Certifications (Associate, Qualified); PyTorch Geometric, DGL, GraphSAGE; graph embeddings (Node2Vec, GNN); RDF/semantic web; temporal/property graphs
Azure Certifications: AZ-900, AZ-104, AZ-305, DP-200, DP-900.
Advanced Graph Systems: Distributed graph processing (Spark GraphX, Gremlin), graph partitioning, scaling 100M+ nodes, multi-tenancy, sharding
System Optimization: Query execution internals, cost modeling, workload characterization, backend optimization expertise
Container & Orchestration: Docker, Kubernetes, Helm.
Open-source: Contributions to graph technology projects (Neo4j, Gremlin, etc.)
Preferred Qualifications
5-8+ years in data engineering, backend engineering, or AI/ML infrastructure roles in production environments.
3+ years designing and building enterprise-scale data systems, pipelines, or platform services.
2+ years building knowledge graphs, graph systems, or graph-backed AI/search applications (Neo4j)
2+ years deploying generative AI systems (RAG, LLM APIs, prompt engineering)
Masters degree in Computer Science, Engineering, Mathematics, or related field; Ph.D. preferred.
Production experience with graph databases at scale (millions to billions of nodes)
Hands-on cloud experience (Azure, AWS, or GCP)
Expertise in vector databases or semantic search at scale.
Background in backend systems optimization (database engines, query optimization, indexing strategies).
Track record of shipping complex systems: migrations, rollouts, debugging production incidents.
Disclaimer: This job posting has been aggregated from external source. Role details, content, and availability are subject to change. Applicants are advised to confirm the latest information directly on the company website before applying.
📌 Senior AI & Data Engineer (Bengaluru)
🏢 Sigma Allied Services
📍 Bengaluru