GenAI + Data Platform Architecture — Multi-Cloud RAG Blueprint

Looking for a principal-level engineer who lives at the intersection of GenAI and data platforms — not a prompt tinkerer, and not someone who only moves CSVs around. We’re building an applied AI product that has to stay grounded in real operational data. Think: messy multi-source events, lab/partner feeds, product telemetry, and user context — then RAG + agent workflows that don’t hallucinate policy, pricing, or clinical-ish language. The full build is a multi-quarter effort. This engagement is not that. I need the architecture done right before we hire the build team. What I need in ~10 hours A concrete, decision-grade architecture for a GenAI + data platform that we can hand to engineers and start implementing without re-litigating fundamentals every sprint. Scope (architecture / design only — no full production build) 1) Current-state teardown (1–2 hrs) I’ll share what we have today (sources, warehouses, APIs, any existing LLM experiments). I want a blunt read: what’s salvageable, what’s tech debt, where GenAI will break if we bolt it on. 2) Target architecture (majority of the hours) Design the end-to-end system across: • Ingestion + pipelines (batch + near-real-time where it matters) • Lakehouse / warehouse patterns (Iceberg or equivalent on object storage, plus analytics store) • Feature / context layer for retrieval (embeddings, chunking strategy, metadata filters, freshness SLAs) • GenAI serving path: RAG + tool-using agents, with evals and guardrails • Multi-cloud reality check — we are not religious about one vendor Tech direction I’m leaning toward (open to pushback if you’ve lived this): • AWS: Bedrock (Claude / Titan embeddings), S3, Glue/EMR or managed Spark, OpenSearch Serverless or Aurora pgvector, EventBridge/MSK as needed • GCP: Vertex AI (Gemini), BigQuery, Vertex Vector Search / AlloyDB, Pub/Sub, Cloud Run • Microsoft: Azure OpenAI, Fabric / Synapse or ADLS Gen2, Azure AI Search, Functions / Container Apps I want a primary path + a migration/hybrid story — not three half-built platforms. Prefer someone who’s actually shipped Bedrock and/or Vertex in production, not just blog-post diagrams. 3) Deliverables I will pay for • Architecture doc (decisions + rejected alternatives — why we didn’t do LangChain-everywhere, why not pure Lambda, etc.) • One clear diagram (C4 or equivalent: context + containers) • Data contracts sketch for the retrieval corpus (what lands in the index, PII handling, retention) • GenAI control plane: prompt/versioning, eval harness outline, grounding + safety guardrails • Rough cost model (steady-state monthly) and a 30/60/90 implementation sequence for a follow-on team • 45–60 min working session mid-engagement to pressure-test assumptions Out of scope for this 10-hour pass • Full pipeline implementation • Fine-tuning / training jobs • Production IaC of the whole estate • UI work Who this is for You’ve owned platforms that process serious volume, you’ve put LLMs behind real product surfaces (RAG / agents / MCP-style tool protocols), and you’ve made multi-cloud IaC choices under cost and reliability pressure. Bonus if you’ve dealt with regulated data (HIPAA/SOC2) and know where GenAI becomes a compliance problem. How we’ll work Async + one live architecture review. I’ll be decisive. If something is underspecified, call it out — don’t paper over it with “TBD microservice.” To apply Don’t send a generic AI cover letter. Tell me: 1) One GenAI system you designed (Bedrock / Vertex / Azure OpenAI — which, and why) 2) One data-platform decision you’d reverse if you could 3) How you’d spend the first 3 of these 10 hours If you’ve only done ChatGPT wrappers or only classic ETL with no retrieval layer, this isn’t the right fit — no hard feelings.