Applied AI / Multi-Agent Systems Engineer Needed with Agent Architecture and production experience

Title: Applied AI / Multi-Agent Systems Engineer Needed — Agent Architecture, Evals, Observability, Production Deployment I am looking for an experienced Applied AI / Multi-Agent Systems Engineer to help build a production-style demo platform for reliable AI agents. This is not a simple chatbot project. I want to build and learn a complete multi-agent architecture that demonstrates production-grade concepts: agent orchestration, tool use, context optimization, evaluation frameworks, observability, deterministic replay, and deployment. Project Goal: Build a working multi-agent customer support / billing assistant demo. The system should handle realistic support scenarios such as billing questions, duplicate charges, refund policy checks, escalation to human review, and safe tool execution. The project should demonstrate the following: 1. Agent Architecture * Multi-agent design using LangGraph, OpenAI Agents SDK, CrewAI, AutoGen, or similar framework * Clear agent roles such as Router Agent, Billing Agent, Policy/Grounding Agent, Tool Executor Agent, Response Agent, and Eval/Safety Agent * Support for sequential flow, routing/handoff, and supervisor-style orchestration * Conversation state and run state management 2. Model Orchestration * Ability to configure model/provider * Prompt/version management * Agent handoff logic * Retry, timeout, and fallback behavior * Clear explanation of why the selected framework was chosen 3. Context Optimization * RAG or knowledge-base integration * Token budget management * Context selection strategy * Conversation memory strategy * Ability to show what context was included or excluded for each run 4. Tool Use and Grounding * Mock business APIs such as get_customer_profile, get_invoice_history, check_refund_policy, create_refund_ticket, and escalate_to_human * Safe tool gateway/wrapper * Tool-call validation * Approval/human-in-the-loop for risky actions * Grounded responses based only on approved tool/KB data 5. Evaluation Framework * Offline eval dataset with test cases * Production-style evals from saved traces * Metrics for task success, grounding, tool correctness, policy compliance, escalation correctness, latency, and cost * Regression testing between prompt/model versions * Eval report output 6. Production Observability * Trace every agent step, prompt, model call, tool call, latency, cost, and error * UI or dashboard to inspect runs * Ability to replay a failed conversation * Failure categories and top failing intents * Basic alerts or metrics endpoint 7. Deployment * Docker Compose setup required * Clear local setup instructions * Optional Kubernetes manifests are a plus * Environment variable based provider configuration * README and architecture diagram Expected Deliverables: * Working codebase * Multi-agent demo application * Backend API * Simple frontend/dashboard * Mock tools/APIs * Eval runner * Trace/replay view * Docker Compose deployment * Architecture document * README with setup and demo steps * 1–2 walkthrough sessions explaining the design and trade-offs Ideal Candidate: * Strong experience building LLM agents in production or production-like environments * Experience with LangGraph, LangChain, OpenAI Agents SDK, CrewAI, AutoGen, or similar * Strong backend engineering skills * Understands evals, observability, prompt/model versioning, tool calling, and RAG * Can explain architecture trade-offs clearly * Has shipped or built real agentic workflows, not only toy chatbot demos Please include in your proposal: 1. Links to any multi-agent or LLM app projects you have built 2. Which framework you recommend and why 3. How you would design the agent architecture 4. How you would implement evals and observability 5. How you would deploy it 6. Estimated milestones and timeline 7. Any risks or trade-offs you see