AI Infrastructure Engineer for RAG Systems

We are seeking an AI Infrastructure Engineer to deploy, optimize, and maintain production-grade AI systems with a strong focus on Retrieval-Augmented Generation (RAG). The ideal candidate will have experience in cloud infrastructure, containerization, and CI/CD pipelines. You will work closely with our team to ensure the reliability and scalability of our AI systems. You will own the full stack: from GPU inference serving and vector database integration to production API endpoints. The ideal candidate has already shipped LLM infrastructure in production and understands the real-world challenges of GPU memory limits, multi-user concurrency, retrieval accuracy, and system reliability. Responsibilities Deploy and optimize RAG pipelines end-to-end — from document ingestion and chunking to embedding, vector retrieval, and LLM response generation Configure and run production inference servers (vLLM, llama.cpp, Ollama, TGI, TensorRT-LLM, or SGLang) for open-source LLMs Integrate deployed LLMs with existing RAG backends and vector databases (Pinecone, Qdrant, Chroma, Weaviate, or similar) Optimize GPU inference for VRAM usage, token latency, concurrency throughput, and inference speed Expose production-ready, OpenAI-compatible API endpoints for internal or external consumption Handle CUDA driver setup, GPU scaling decisions, and quantization strategies (GGUF, AWQ, GPTQ, EXL2) Build and maintain embedding pipelines, chunking strategies, and re-ranking workflows Set up monitoring, logging, alerting, and automatic restart/recovery for AI services Containerize AI workloads using Docker and deploy via CI/CD pipelines Advise on hardware upgrades, multi-GPU setups, and cost/performance tradeoffs Document deployment architecture and provide clear handover notes Skills: Strong, proven experience deploying RAG pipelines in production environments Hands-on experience with open-source LLMs: Gemma, Llama, Mistral, Qwen, or similar Proficiency with inference frameworks: vLLM, llama.cpp, Ollama, TGI, or TensorRT-LLM Experience with vector databases and embedding models (semantic search, context window optimization, metadata filtering) CUDA GPU optimization and Linux server environment experience Model quantization experience: GGUF, AWQ, GPTQ, or EXL2 Python proficiency — backend APIs using FastAPI, Flask, or equivalent Docker and containerized deployment experience Cloud infrastructure experience: AWS, GCP, or Azure (compute, storage, networking) Strong debugging skills across GPU, inference, and API layers