Senior AI Platform Engineer

We are looking for an experienced Full-Stack, DevOps & Data Pipeline Engineer to develop and operate enterprise-scale AI and data platforms. The role combines backend development, data engineering, AI integration, cloud infrastructure, platform operations, testing, and CI/CD engineering.

You will work on systems that process large volumes of enterprise documents, orchestrate complex data pipelines, integrate AI services, and deliver reliable, observable, and production-grade business solutions.

Your Responsibilities

- Design, develop, test, and operate scalable backend services, APIs, and data-processing solutions.
- Build and maintain large-scale document ingestion, transformation, enrichment, classification, and indexing pipelines.
- Develop AI-enabled services integrating LLMs, retrieval systems, vector search, and enterprise data sources.
- Design and operate event-driven and message-driven architectures.
- Implement automated unit, integration, contract, end-to-end, performance, and regression testing.
- Establish quality gates and integrate automated testing into CI/CD and release workflows.
- Implement and maintain CI/CD pipelines, release automation, and deployment workflows.
- Design and operate containerized workloads running on Kubernetes.
- Manage databases, search platforms, vector stores, and application infrastructure.
- Implement end-to-end observability through metrics, logs, distributed tracing, dashboards, and alerting.
- Ensure traceability across source documents, pipeline transformations, AI processing steps, model interactions, and generated outputs.
- Establish data quality controls for completeness, accuracy, consistency, validity, timeliness, and duplication.
- Implement automated data validation, data-quality monitoring, reconciliation, anomaly detection, and issue-handling processes.
- Ensure reliability, security, scalability, performance, and cost efficiency of production systems.
- Support incident analysis, root-cause analysis, troubleshooting, and operational excellence initiatives.
- Collaborate with software developers, AI engineers, data engineers, and business stakeholders to deliver enterprise-grade solutions.

Core Requirements

Backend Engineering

- Strong professional experience in Python software development.
- Experience designing and implementing REST APIs, asynchronous services, and distributed systems.
- Experience building scalable enterprise applications with a focus on maintainability, reliability, and performance.
- Strong understanding of software architecture, clean code, code reviews, release management, and software quality.
- Experience with automated unit, integration, contract, end-to-end, and regression testing.




- Familiarity with test automation frameworks, mocking, test data management, code coverage, and CI quality gates.
- Experience testing distributed systems, asynchronous workflows, APIs, and failure scenarios.

Data Engineering & AI

- Experience building, testing, and operating data ingestion, transformation, and processing pipelines.
- Knowledge of document processing, OCR, content extraction, metadata enrichment, and search technologies.
- Practical experience with Generative AI, including LLM integration, retrieval-augmented generation, embeddings, and vector databases.
- Experience with LangChain, LangGraph, OpenAI APIs, or comparable AI orchestration frameworks.
- Understanding of agentic AI architectures, tool calling, workflow orchestration, structured outputs, and multi-step agent workflows.
- Experience integrating AI solutions into production business processes.
- Experience validating AI and retrieval workflows, including input validation, retrieval quality, output consistency, and failure handling.
- Understanding of AI evaluation methods, such as groundedness, relevance, completeness, citation quality, and hallucination detection.

Data Quality Assurance

- Experience defining and implementing data quality rules and validation frameworks.
- Ability to monitor data completeness, accuracy, consistency, validity, timeliness, uniqueness, and schema compliance.
- Experience implementing automated data profiling, reconciliation, anomaly detection, and quality reporting.
- Understanding of data contracts, schema evolution, lineage, metadata management, and quality gates.
- Ability to trace data-quality issues from downstream outputs back to source systems and transformation steps.
- Experience handling invalid, incomplete, duplicated, delayed, or corrupted data within production pipelines.

DevOps & Cloud

- Strong experience with Docker and Kubernetes.
- Experience operating cloud-native applications, preferably on Microsoft Azure.
- Hands-on experience with CI/CD pipelines, automated deployments, Git-based workflows, and release automation.
- Experience with Helm, secrets management, container registries, workload identities, and infrastructure automation.
- Experience implementing build, test, security, data-quality, and deployment gates within CI/CD pipelines.




- Ability to operate and troubleshoot business-critical production environments.

Monitoring, Observability & Traceability

- Strong understanding of monitoring, logging, distributed tracing, alerting, and operational support processes.
- Experience defining service-level indicators, service-level objectives, health checks, and actionable alerts.
- Ability to correlate logs, metrics, traces, events, and deployment information across distributed systems.
- Experience implementing end-to-end traceability across APIs, message flows, data pipelines, retrieval processes, and AI workflows.
- Understanding of audit trails, correlation IDs, data lineage, prompt and model versioning, and reproducibility.
- Ability to design dashboards and alerts that support proactive issue detection and efficient root-cause analysis.
- Familiarity with observability platforms such as Azure Monitor, Application Insights, OpenTelemetry, Prometheus, Grafana, or comparable technologies.

Databases & Integration Technologies

- Experience with PostgreSQL and MongoDB.
- Experience working with Kafka, Redis, RabbitMQ, or other messaging and event-streaming platforms.
- Experience integrating enterprise applications, APIs, search platforms, and external services.
- Knowledge of workflow orchestration platforms such as Dagster, Airflow, or comparable technologies.
- Understanding of distributed processing, fault handling, idempotency, retry mechanisms, dead-letter handling, and large-scale data operations.
- Experience monitoring pipeline status, processing quality, data freshness, throughput, and failure rates.

Preferred Qualifications

- Experience operating and testing production-grade AI platforms at scale.
- Experience with Azure AI Services, Azure AI Search, Azure Storage, Azure Key Vault, Azure Identity, Azure Monitor, and Application Insights.
- Experience managing Kubernetes-based workloads with demanding performance and reliability requirements.
- Familiarity with semantic release workflows, automated versioning, and enterprise software delivery processes.
- Experience with data-quality and observability frameworks such as Excellent Expectations, Soda, OpenTelemetry, Prometheus, Grafana, or comparable technologies.
- Experience implementing data lineage, model and prompt versioning, audit trails, and reproducible AI processing.
- Familiarity with evaluation and monitoring approaches for LLM, RAG, and agentic AI solutions.
- Experience in business-critical environments where operational stability, traceability, scalability, security, and cost optimization are essential.
- Familiarity with enterprise knowledge management, search, and document-centric AI systems.

📌 Senior AI Platform Engineer (Telangana)
🏢 Basf
📍 Telangana