You will get a calibrated RAG audit with a 20-page report and a 50-question eval set
Project details
Most "RAG audits" return adjectives. This one returns numbers. You give me your codebase and a representative slice of your corpus, and in five days you get back a 20-page report with measured faithfulness, citation accuracy, and answer relevance, scored by a calibrated LLM-as-judge. No vibes, no "looks decent."
The eval set I deliver is yours to keep. Re-run it after every change. Six months from now, when someone tweaks the chunker, you'll know within 10 minutes whether the system got better or worse.
What you'll see in the report: the top 5 fixes ranked by measured impact, with the test cases that prove each one. I do not pad with generic recommendations.
I built the open-source eval-kit Python library this same workflow runs on. Six years writing production AI for Société Générale, Allianz, and Decathlon. Currently shipping a Claude support agent at Decathlon with the same eval discipline.
You give me access on day 1. I deliver on day 5. If the audit doesn't surface at least three concrete, measured fixes, the engagement is on me.
The eval set I deliver is yours to keep. Re-run it after every change. Six months from now, when someone tweaks the chunker, you'll know within 10 minutes whether the system got better or worse.
What you'll see in the report: the top 5 fixes ranked by measured impact, with the test cases that prove each one. I do not pad with generic recommendations.
I built the open-source eval-kit Python library this same workflow runs on. Six years writing production AI for Société Générale, Allianz, and Decathlon. Currently shipping a Claude support agent at Decathlon with the same eval discipline.
You give me access on day 1. I deliver on day 5. If the audit doesn't surface at least three concrete, measured fixes, the engagement is on me.
AI Algorithms
Large Language Model, Transformer ModelAI Applications
AI Chatbot, AI Content Creation, AI Mobile App Development, AI Text-to-Image, AI Text-to-Speech, AI-Enhanced Medical Imaging, AI-Generated Code, AI-Generated Music, Conversational AI, Machine Translation, Natural Language Generation, Natural Language UnderstandingAI Development Language
PythonAI Tools
Azure OpenAI, Copy.ai, GitHub Copilot, Hugging Face, Jasper AI, NVIDIA AI Platform, PyTorch, Replit, TensorFlow, Word2vecAI Models
BERT, BLOOM, ChatGPT, DALL-E, Dolly, GPT-3, GPT-4, GPT-Neo, Jurassic-2, LaMDA, LLaMA, OpenAI CodexWhat's included
| Service Tiers |
Starter
$850
|
Standard
$1,500
|
Advanced
$3,000
|
|---|---|---|---|
| Delivery Time | 3 days | 5 days | 10 days |
Number of Revisions | 1 | 2 | 3 |
AI Model Integration | - | - | |
Batch Normalization | - | - | - |
Database Integration | - | ||
Detailed Code Comments | - | ||
Image Upscaling | - | - | - |
MLOps | - | - | |
Model Deployment | - | - | |
Model Documentation | |||
Model Monitoring | - | - | |
Model Testing & Optimization | |||
Model Tuning | - | - | |
Natural Language Processing | |||
NLP Tokenization | - | - | - |
Pre-Training | - | - | - |
Prompt Engineering | |||
Setup File | - | - | |
Source Code | - |
Optional add-ons
You can add these on the next page.
Re-run eval set 30 days later
(+ 2 Days)
+$300
1-hour live debugging call
(+ 1 Day)
+$200Frequently asked questions
About Amir
Senior AI/LLM Engineer | RAG, AI Agents, LangChain, LLM Evals
Paris, France - 12:32 pm local time
Currently at Decathlon building a Claude support agent with RAG over the product catalogue: three Sonnet calls (classify, draft, escalate) through Anthropic's tool-use API, per-call cost accounting, a 50-row eval set running in CI. Before that, three years at Allianz taking the insurer's first LLM applications from prototype to production: a document Q&A system for claim adjusters, a multi-step research agent on LangGraph, and a vendor-neutral wrapper around Anthropic and OpenAI. Three years before that at Société Générale on classical ML and NLP: a transformer-based document classifier for compliance, OCR for scanned regulatory filings, the team's first embeddings-based search.
What I publish on GitHub:
- A production RAG reference that refuses to answer when retrieval does not support the claim. 50-question eval gate blocks merges on more than 3 points of regression.
- A customer-support agent: three Sonnet calls, calibrated confidence, real cost accounting, eval gate in CI.
- A 7-node LangGraph DAG with a parallel fan-out and a critic retry loop. Streams per-node telemetry over SSE.
- An open-source Python library for LLM evals: calibrated LLM-as-judge, synthetic adversarial data, regression diff that fails the PR.
Stack: Python, FastAPI, Anthropic Claude, OpenAI, LangGraph, Voyage embeddings, pgvector, Postgres, Modal, Langfuse, Next.js, Tailwind. Bar: mypy strict, ruff, pytest with a coverage gate, golden-set evals in CI, structlog, per-call cost tracking. Every repo runs make ci clean from a fresh clone.
Best fit for:
- Building a RAG system that has to cite, not invent.
- Taking a multi-step agent prototype to production.
- Setting up the eval and quality engineering layer your team has been meaning to build.
EU timezone, 30+ hours per week, async-friendly.
Steps for completing your project
After purchasing the project, send requirements so Amir can start the project.
Delivery time starts when Amir receives requirements from you.
Amir works on your project following the steps below.
Revisions may occur after the delivery date.
Day 1: access and corpus review
I receive your repo, corpus sample, and example queries. I set up your stack locally or in an isolated environment and confirm I can reproduce a baseline retrieval pass.
Day 1-2: tailored 50-question eval set
I write a 50-question eval set against your specific corpus and the production queries you shared, covering positive cases, hard negatives, and edge cases.