FastAPI & LangChain RAG Pipeline Optimization

Diagnosed and resolved multiple critical issues in a live production RAG (Retrieval-Augmented Generation) pipeline built on Python and FastAPI. The system was returning irrelevant document chunks due to a misconfigured chunking strategy fixed by tuning chunk size and overlap parameters to match the document structure, which immediately improved retrieval precision. Identified and corrected an embedding pipeline bug where documents were being re-embedded on every query instead of at ingestion time, causing significant latency spikes in production. Refactored the pipeline to embed at upload andstore vectors in the database, reducing average response time by over 60%. Resolved LLM hallucination issues caused by insufficient context being passed in the system prompt — rewrote the prompt template to enforce strict grounding instructions and added a relevance score threshold so the LLM only responds when retrieved chunks exceed a confidence cutoff. Fixed a broken FastAPI endpoint that was silently swallowing retrieval errors and returning empty responses to the frontend — added proper exception handling, structured error responses, and request logging for observability. Stack: Python · FastAPI · LangChain · OpenAI API · Vector DB (ChromaDB /Pinecone) · Prompt Engineering · REST API