22 Nov
|
AppXcess Technologies
|
Coimbatore
22 Nov
AppXcess Technologies
Coimbatore
Job Description
AI Engineer – On-Premises AI, LLM & InfrastructureExperience: 1–5 Years Employment Type: Full-Time Work Schedule: US Pacific Time (PT) Business Hours Job OverviewWe are looking for a hands-on AI Engineer to develop, deploy, optimize, and maintain AI applications on on-premises infrastructure. The candidate should understand Large Language Models (LLMs), Retrieval-Augmented Generation (RAG), intent parsing, tool/function calling, Model Context Protocol (MCP), AI orchestration, Python backend development, and GPU-accelerated computing. The role involves working with locally hosted models and NVIDIA GPU infrastructure, including RTX 6000 Blackwell and L40S, along with Docker, Microsoft Hyper-V, Windows Server/Linux, networking, security, and performance optimization. The engineer will contribute across the AI application lifecycle, from experimentation and integration to deployment and production support. Work Schedule – US Pacific Time
• Must be available to work full time in alignment with US Pacific Time (PT) business hours.
• The corresponding IST schedule may involve late-evening or night shifts.
• Participate in meetings, technical discussions, and coordination with US-based clients and teams during PT business hours.
• Accommodate seasonal changes between Pacific Standard Time (PST) and Pacific Daylight Time (PDT).
Key Responsibilities
1. LLM Deployment & On-Premises AI
• Deploy and manage locally hosted models such as Qwen, Mistral, Kimi, and other suitable open-weight LLMs.
• Use inference frameworks such as Ollama and vLLM to serve models.
• Deploy workloads on NVIDIA RTX 6000 Blackwell, L40S, and compatible GPU systems.
• Optimize inference, GPU utilization, VRAM usage, latency, and throughput.
• Evaluate model quality, accuracy, tool-calling capabilities, and hardware requirements.
• Apply quantization, batching, and streaming inference where appropriate.
• Troubleshoot CUDA, drivers, memory allocation, and model-serving issues.
• Maintain local deployments with attention to privacy, data governance, and reduced reliance on external AI services. 2. RAG, Intent Parsing & AI Orchestration
• Build RAG pipelines for documents, databases, and enterprise data sources.
• Implement document parsing, text extraction, OCR, chunking, metadata extraction, embeddings, vector search, and reranking.
• Develop intent detection, classification, and routing to select appropriate workflows.
• Implement LLM tool/function calling for APIs, databases, and connected applications.
• Build multi-step workflows covering retrieval, tool selection, execution, and response generation.
• Develop MCP servers/clients and connect models to authorized tools and data sources.
• Maintain tool registries, schemas, execution handlers, and permissions.
• Implement guardrails, validation, fallbacks, error handling, and controlled tool execution.
• Improve response accuracy, relevance, reliability, and source attribution. 3. Backend Development & Integration
• Develop Python backend services and REST APIs using frameworks such as FastAPI.
• Integrate AI applications with enterprise systems, databases, document repositories, and third-party APIs.
• Build reusable connectors for authorized data retrieval and application actions.
• Implement asynchronous processing, background jobs, session management, and streaming responses.
• Create modular services for inference, orchestration, retrieval, and integrations.
• Write automated tests, technical documentation, and deployment scripts.
• Collaborate with frontend and infrastructure teams to deliver end-to-end applications. 4. Real-Time Voice AI & Conversational Systems
• Develop voice-enabled conversational applications integrating speech-to-text, LLMs, text-to-speech, and optional avatar animation.
• Work with technologies such as Faster-Whisper/Whisper, Kokoro TTS, MuseTalk, and NVIDIA Audio2Face.
• Build low-latency audio and streaming pipelines.
• Implement conversation context, session and interruption handling, and error recovery.
• Optimize audio processing, inference latency, GPU usage, and concurrent sessions.
• Integrate voice components with backend APIs and application interfaces. 5. Hyper-V & On-Premises Infrastructure
• Provision and manage virtual machines using Microsoft Hyper-V.
• Maintain Windows Server and Linux environments for AI workloads.
• Configure virtual CPU, memory, storage, networking, and resource allocation.
• Enable and troubleshoot GPU access through supported GPU-P or passthrough configurations, subject to hardware and OS compatibility.
• Understand GPU virtualization requirements for NVIDIA RTX 6000 Blackwell and L40S.
• Deploy inference services across virtual machines and physical servers.
• Configure virtual switches, network segmentation, firewalls, DNS, and secure communication.
• Troubleshoot virtualization, GPU, driver, storage, and networking issues.
• Maintain infrastructure documentation, backups, and operational runbooks. 6. Containerization & Deployment
• Containerize models, backend services, RAG pipelines, MCP services, and voice components using Docker.
• Use Docker Compose or similar tools to manage multi-service applications.
• Configure GPU access for supported NVIDIA containers.
• Manage environment variables, secrets, persistent storage, dependencies, and deployment settings.
• Deploy across Hyper-V virtual machines and on-premises GPU servers.
• Implement health checks, logging, monitoring, service recovery, and deployment automation.
• Maintain reproducible and reliable deployment environments. 7. Performance, Scalability & Reliability
• Benchmark latency, time to first token, tokens per second, throughput, and concurrent-user capacity.
• Identify CPU, GPU, VRAM, storage, and network bottlenecks.
• Optimize resources for multiple models, applications, and concurrent users.
• Apply caching, asynchronous processing, queues, batching, and streaming where appropriate.
• Implement timeouts, retries, fallbacks, and graceful error handling.
• Monitor GPU utilization, memory, application health, and inference performance.
• Improve system performance, scalability, and reliability as workloads grow. 8. Security & Maintainability
• Implement authentication, authorization, role-based access, and secure API integrations.
• Configure TLS, reverse proxies, firewalls, and secure service communication.
• Restrict access to tools, documents, databases, and connected applications.
• Apply input/output validation, audit logging,
and secrets management.
• Ensure RAG retrieval and tool execution respect user permissions and tenant boundaries.
• Maintain modular, testable code and documented infrastructure.
• Troubleshoot production issues and improve security and stability. Example Use Cases1. Real-Time AI CompanionBuild a live, one-to-one conversational AI companion using locally hosted LLMs, Faster-Whisper for speech recognition, Kokoro TTS for speech generation, and MuseTalk or NVIDIA Audio2Face for avatar animation. Focus on natural conversation, context management, low latency, GPU optimization, and concurrent sessions. 2. Multi-Source Enterprise ChatbotBuild a chatbot that retrieves information from authorized data sources and interacts with connected applications. The solution may use Qwen 14B or Mistral, document processing, RAG, embeddings, vector databases, APIs, MCP, and a tool registry. Workflows may include intent routing, retrieval, tool calling, orchestration, guardrails, and response generation. Required Technical Skills
• Python programming and backend/API development.
• Understanding or experience with LLMs such as Qwen, Mistral, or Kimi.
• Familiarity with Ollama, vLLM, or similar inference frameworks.
• Knowledge of RAG, embeddings, vector databases, and semantic search.
• Understanding of intent parsing, tool calling, MCP, and AI orchestration.
• Docker and containerized application deployment.
• Practical knowledge of Microsoft Hyper-V and virtual machines.
• Understanding of NVIDIA GPUs, GPU memory, inference performance, and model optimization.
• Networking, security, troubleshooting, and REST API integration. Preferred Skills
• Experience with NVIDIA RTX 6000 Blackwell, L40S, or comparable GPUs.
• CUDA, NVIDIA drivers, and GPU monitoring.
• GPU virtualization or passthrough in Hyper-V, where supported.
• Windows Server, Linux, and virtual networking.
• Faster-Whisper, Kokoro, MuseTalk, NVIDIA Audio2Face, or related voice AI tools.
• MCP SDKs, LangGraph, or similar orchestration frameworks.
• OCR, reranking, hybrid retrieval, and document-processing pipelines.
• Reverse proxies, TLS, monitoring, logging, and CI/CD.
• Production deployment and maintenance of on-premises AI applications. Candidate Profile
• Strong analytical, troubleshooting, and problem-solving skills.
• Hands-on approach to AI development, deployment, and infrastructure.
• Ability to work across LLMs, backend applications, and infrastructure.
• Understanding of trade-offs between model quality, latency, GPU requirements, and scalability.
• Ability to build secure, modular, and maintainable applications.
• Good communication and collaboration skills.
• Willingness to learn emerging LLM, agentic AI, voice AI, and GPU technologies.
• Willingness to work in alignment with US Pacific Time business hours and coordinate with US-based stakeholders. Overall Role ExpectationsThe selected candidate will contribute to developing and maintaining on-premises AI solutions across model deployment, orchestration, backend integration, voice AI, containerization, and infrastructure. The role requires an understanding of how AI models operate in enterprise environments, connect with authorized tools and data, and use available hardware efficiently. The engineer will support applications from experimentation and development through deployment, monitoring, troubleshooting, and continuous improvement.
📌 AI Engineer – On-Premises AI Infra (Coimbatore)
🏢 AppXcess Technologies
📍 Coimbatore