Greater Noida, Uttar PradeshBengaluru, Karnataka

Job Summary
- Build and maintain data pipelines that ingest infrastructure metrics, logs, and health check outputs
- Design ETL processes to feed the RAG pipeline with structured runbook and operational data
- Implement vector embedding pipelines for knowledge base ingestion into pgvector/ChromaDB
- Create data quality checks and validation layers before AI reasoning layer ingestion
- Integrate data from ServiceNow, DB Run scripts, and observability platforms
- Optimize data throughput and storage for real-time weather map updates

Key Responsibilities
- Build and maintain data pipelines that ingest infrastructure metrics, logs, and health check outputs
- Design ETL processes to feed the RAG pipeline with structured runbook and operational data
- Implement vector embedding pipelines for knowledge base ingestion into pgvector/ChromaDB
- Create data quality checks and validation layers before AI reasoning layer ingestion




- Integrate data from ServiceNow, DB Run scripts, and observability platforms
- Optimize data throughput and storage for real-time weather map updates

Skill Requirements

Skills Required:
- Python (intermediate-advanced), SQL, Bash scripting
- ETL/ELT frameworks: Apache Airflow, Pandas, Polars
- Databases: PostgreSQL, MongoDB, Redis, SQLite
- Vector databases: pgvector, ChromaDB, FAISS
- Cloud data services: GCP BigQuery, Cloud Storage
- Data modeling, schema design, and data quality
- JSON/YAML parsing and data transformation

Other Requirements

NA
body.unify div.unify-button-container.unify-apply-now: focus, #body.unify div.unify-button-container.unify-apply-#body.unify div.unify-button-container.unify-apply-now: focus, #body.unify div.unify-button-container.unify-apply

📌 Track Lead (Greater Noida)
🏢 HCLTech
📍 Greater Noida