Hybrid NYC-based MLOps/AI Engineer for Full-stack Development

Summary We are deploying a 100% offline, private AI stack using LibreChat, vLLM, and RAG. The project requires setting up a local infrastructure for AI model deployment, ensuring security and privacy. The ideal candidate will have experience in MLOps and DevOps, with a focus on local AI deployments. Responsibilities include managing the deployment process, optimizing performance, and ensuring the system's reliability and security. Title: MLOps/DevOps Engineer Needed to Deploy Private, Local AI Stack (LibreChat + vLLM + RAG) Project Description: We are deploying a 100% offline, privacy-first AI portal on local hardware (Threadripper server with Dual RTX 6000 Blackwell 96GB GPUs). Must be able to work at least in a hybrid environment, preferably in-person at our NYC office. We need an expert to containerize and automate this infrastructure. The stack consists of: LibreChat frontend, MongoDB, MeiliSearch, local HuggingFace RAG, and local SLMs (Llama 3/Mistral) served via vLLM. Key Responsibilities: Configure Ubuntu server host, including NVIDIA drivers and CUDA environment. Optimize multi-GPU serving utilizing vLLM for high-throughput local inference. Build out a localized, containerized RAG pipeline with HuggingFace. Write Docker Compose and Ansible configuration scripts for automated deployment.