Job Title: Senior Data Engineer - GenAI & Unstructured Data Pipelines
Experience: 6-8 Years
Open Positions: 2
Location: Offshore
Employment Type: Full-Time

Job Summary

Key Responsibilities
- Own the end-to-end ML lifecycle, including data ingestion, feature engineering, model training, evaluation, deployment, monitoring, retraining, and rollback.
- Design, build, and maintain production-grade ML pipelines using Azure native services with CI/CD automation.
- Utilize Azure Machine Learning and MLflow for experiment tracking, model registry, and environment promotion (Dev/Test/Prod).
- Design and implement Generative AI solutions using Azure OpenAI, embeddings, vector search, and Retrieval-Augmented Generation (RAG) pipelines.
- Develop Agentic AI workflows with multi-step reasoning, tool integration, guardrails, observability, reliability, and cost optimization.
- Build scalable batch and streaming data pipelines using Azure Databricks.
- Design and optimize pipelines for processing unstructured and multi-modal data, including text, documents, logs, images, and PDFs.




- Develop robust RAG pipelines involving document chunking, embedding generation, vector indexing, and semantic retrieval.
- Design and manage vector databases and search solutions such as Azure AI Search, Pinecone, or similar platforms.
- Build batch and real-time data ingestion pipelines using Apache Spark and Apache Kafka.
- Collaborate with data scientists, ML engineers, and software development teams to deliver AI-powered solutions.
- Ensure scalability, performance, security, and reliability of enterprise data platforms.

Required Skills:
- 6-8 years of experience in Data Engineering.
- Robust programming skills in Python, PySpark, and SQL.
- Hands-on experience with:
- Apache Spark
- Apache Airflow
- Apache Kafka
- Experience processing unstructured data, including JSON, logs, documents, and PDFs.
- Strong knowledge of Azure cloud services (preferred).
- Experience building GenAI/LLM pipelines, i

📌 Senior Data Engineer (Noida)
🏢 WebSenor InfoTech
📍 Noida