I am seeking an experienced ML engineer to provide insights on the design of a model I am planning to build. Your expertise in model design and architecture will be invaluable in helping me make informed decisions.

I want to build a private multi-model RAG-based Opportunity Intelligence Agent. It should support document ingestion, opportunity-specific workspaces, vector search, source citations, multi-model routing across OpenAI, Claude, Perplexity, and possibly DeepSeek, and generate strategic recommendations from both uploaded files and live web research. This is intended to become a reusable base agent capable of knowledge retrieval, web research, multi-model orchestration, document analysis, citation generation, and agent clonding and configuration. It will be used for analyzing & strategy development for project opportunities, responding to RFPs, and proposal assistance, as well as other applications.

I am looking for an experienced ASR engineer to build a production-ready speech-to-text system for a low-resource language. I already have approximately 3,000 prepared audio segments, totaling about 10 hours of audio, with clean and consistent transcripts ready for immediate use. Data preparation and segmentation are already handled. The initial 10 hours of audio will serve as the first milestone. After that, the engineer will be expected to continue training and improving the model with additional data until the system reaches a target WER of 10% or below. Your responsibility will focus on: Fine-tuning a Whisper-based model for high transcription accuracy Optimizing word error rate (WER) over time Providing inline/embedded start timestamps per phrase Building an efficient inference pipeline for both real-time and batch transcription Structuring evaluation and improvement workflows Preparing the system for deployment and integration into a web platform Providing clear documentation and guidance so I can independently continue training and improving the model over time without ongoing engineer involvement The goal is to reach strong accuracy at launch, with a clear process for continued improvement as more data becomes available. Please describe your experience with Whisper fine-tuning or similar ASR model training in your proposal.

We're building an internal AI system that runs entirely on our own hardware (no cloud inference) against our own company data. We have a working proof-of-concept and want to get the architecture right. We need an experienced consultant to review what we've built, pressure-test our decisions, and tell us where we're wrong. This is an advisory/validation role first — we have someone doing the hands-on work; what we want is a senior second opinion to make sure we're building this the right way. What we're running today: Inference: RTX 5090 (32GB, Blackwell), Ubuntu 24.04, running llama-server (llama.cpp + CUDA) serving Gemma 4 31B-it (Q4_K_M GGUF) at a 262,144 context window. Also hosts our MCP retrieval server, PostgreSQL, and Qdrant. Embeddings: separate machine with an RTX 3060 running vLLM serving Qwen3-Embedding-4B. RAG: hybrid retrieval — Postgres full-text search + Qdrant semantic search with RRF fusion, exposed through a custom MCP server with tool-calling. Data: ingesting our own internal operational data into Postgres + Qdrant. Planned stack: LiteLLM for model routing, n8n for automation, Open WebUI for the interface, Langfuse for observability, Vault or Infisical for secrets, Keycloak/Azure AD for SSO. What we need help with: Validating our two-machine split (inference vs. embeddings) and whether our VRAM/context budget holds up under real load — specifically whether a 256K context window is real and performant on a single 32GB card or just nominal. Model selection and routing strategy: which open-weight models for which tasks, and how to structure LiteLLM routes. RAG quality: chunking, embedding dimensionality, hybrid search tuning, reranking — making retrieval actually accurate on messy real-world data. Sanity-checking our overall architecture and telling us our blind spots. You should have done: Stood up local LLM inference in production — llama.cpp/llama-server and vLLM, not just Ollama on a laptop. You understand GGUF quantization (Q4_K_M, IQ-series), KV cache, KV-cache quantization, and how context length maps to actual VRAM consumption. Real fluency in GPU sizing math — given a model, a quant, and a context window, you can tell us whether it fits on a given card and what throughput to expect. Bonus if you've worked with Blackwell / sm_120a. Built production RAG — vector DBs (Qdrant, pgvector), hybrid search, RRF fusion, embedding model selection, reranking, evaluation. Worked with agentic/tool-calling systems and ideally MCP servers. Know the open-weight model landscape (Gemma, Qwen, Llama, Mistral, Phi, Nemotron, Hermes) and their licenses well enough to advise. Production ops: systemd, Docker, model gateways (LiteLLM or similar), observability (Langfuse), secrets management, SSO.