19 Nov
|
Kyndryl
|
Hyderabad
Job Summary
As an AI Data Engineer at Kyndryl, you ll be the architect behind the high-performance data infrastructure that powers our autonomous systems. We arent just moving tables; we re building the real-time pipelines that allow agentic AI to reason over the world s most complex enterprise environments.
What you will do
- Architect for RAG: You ll design and scale the pipelines for Retrieval-Augmented Generation (RAG), transforming massive volumes of unstructured IT logs and documentation into optimized Vector Embeddings.
- Scale vector infrastructure: You will be responsible for the health and performance of our vector databases (e.g., Pinecone, Milvus, or Weaviate), ensuring sub-second retrieval speeds for agentic reasoning loops.
- Engineer semantic layers: Move beyond simple ETL to build knowledge graphs and semantic layers that provide agents with the necessary context to navigate complex infrastructure puzzles.
- Automate data excellence: Using a keen eye for detail, you ll build automated data guardrails to detect noise, bias, or PII (Personally Identifiable Information) before it reaches the model, ensuring our AI remains safe and impactful.
- Solve meaningful challenges: Serve as the bridge between raw, messy data sources and deep technical AI work, identifying and resolving quality issues at the source.
- Progress to production: With a well-defined methodology and software engineering prowess, you will build, deploy, and maintain the CI/CD pipelines for our data infrastructure, ensuring that our context window remains fresh and reliable.
Who You Are
You re good at what you do and possess the required experience to prove it. However, equally as important - you have a growth mindset; keen to drive your own personal and professional development.
You are customer-focused - someone who prioritizes customer success in their work. And finally, you re open and borderless - naturally inclusive in how you work with others.
Required Skills Experience
- 7+ years of experience in data engineering/data warehousing, 4+ years of hands-on experience with Databricks/Spark and Snowflake.
- Advanced hands-on experience with: Databricks Python SQL Spark/PySpark Delta Lake Snowflake
- Strong experience with Databricks Workflows/Jobs, notebooks, Unity Catalog, Git, and CI/CD.
- Experience building metadata-driven ELT frameworks, reusable pipeline templates, data-quality frameworks, reconciliation utilities, and automated testing.
- Strong knowledge of performance tuning, partitioning, incremental processing, schema evolution, and large-volume data processing.
- Knowledge of dimensional and normalized data modeling.
- Experience working with healthcare data, including patient identity, effective dating, PHI handling, lineage, and audit requirements.
- Robust troubleshooting, debugging, and root-cause-analysis skills.
Preferred Skills
- Experience building GenAI/RAG applications on Databricks, Agentic AI / AI Agents, Generative AI and LLMs, Retrieval-Augmented Generation (RAG), Prompt Engineering, Embeddings Vector Search
- Experience with MLflow and AI/ML lifecycle practices.
- Exposure to Azure OpenAI or equivalent enterprise LLM platforms.
- Knowledge of agent frameworks/orchestration approaches.
- Experience moving AI/GenAI solutions from POC to production, including CI/CD, monitoring, security, and governance.
- Healthcare domain experience and familiarity with PHI-sensitive datasets
Disclaimer: This job posting has been aggregated from external source. Role details, content, and availability are subject to change. Applicants are advised to confirm the latest information directly on the company website before applying.
📌 Lead Data Engineer (Hyderabad)
🏢 Kyndryl
📍 Hyderabad