Run a Batch LLM Content-Generation Pipeline (Python + Claude API) — ~14,000 Short Stories
Worldwide
Run a Batch LLM Content-Generation Pipeline (Python + Claude API) — ~14,000 Short Stories Overview I run a collectible toy brand. Each toy links (via a QR code) to a unique ~270-word children's story about that character. I need roughly 14,000 of these stories generated at a consistent quality and reading level, and delivered as a single spreadsheet. A working Python script already exists (I'll provide it). It calls the Anthropic Claude API, generates each story from a "canon" reference file plus ~78 hand-written example stories, runs automated quality checks, assigns metadata, and writes the output. It has been tested in a free "dry-run" mode. Your job is to finalize it for a full production run, run it reliably and cost-effectively, and deliver clean, QA'd output. This is a finish-and-run job, not a build-from-scratch job. I want to own the finished script and be able to run future batches myself for only the API cost — no per-batch dev fees and no monthly hosting. What I'll provide The existing generator script (Python, ~330 lines, documented) A Canon Reference workbook (character voice guides, location/companion rules, winner logic, reading-level target) ~78 example stories (the style/quality bar and the anti-repetition seed set) An Anthropic API key (billed to my account) Scope of work Finalize the script. One file-I/O section in main() is stubbed and needs completing (straightforward). Review the rest. Harden it for scale. Add/improve concurrency or async for throughput, robust rate-limit and error handling, resumability, and running cost logging. Basic versions of these exist — make them production-grade. Costed test run. Generate ~50 stories against the real API. Report exact cost-per-story and a full-run cost estimate. Full sweep. On my approval, run the complete ~14,000-story generation. Deliver the completed workbook plus a short QA summary (pass rates, any skipped slots, final cost). Quality requirements (the script's gate enforces these — verify they hold at scale) ~240–300 words per story; reading level a 7-year-old understands (≈ grade 4, measured) No banned filler phrases; a specific character-numbering convention (provided) Anti-repetition: no reused openings or sentence structures across the whole library Correct per-story metadata: season rules, companions, winner flags, collection Output must match the exact 21-column spreadsheet schema I provide Deliverables & ownership (important) Full source code handover — I own the finished script and all code outright (work-for-hire). No dependencies on your private accounts or servers. A one-page, plain-English "How to run it" guide written so that a non-developer can generate future batches themselves — e.g., "to make 500 more, open this, type this command." No hosting, no subscription required. The script must run locally (on a normal laptop) with only an API key — no ongoing hosting or monthly costs. Re-running for a new batch should be a single command; the tool should resume/skip already-written stories so refills are cheap and never redo work. Required skills Strong Python Hands-on experience with LLM APIs (Anthropic Claude and/or OpenAI), especially large batch / async generation jobs Cost and rate-limit management for high-volume API runs (this job is ~15,000–20,000 calls) openpyxl / Excel data handling Careful, detail-oriented QA Nice to have Prompt-engineering experience Prior content-generation or data-pipeline work at scale Process & milestones This is structured in two milestones so we both stay low-risk. Milestone 1 is small and proves the approach before the full run. Milestone 1 — Finalize + costed test Complete the stubbed section of the script and get it production-ready (concurrency, retries, resumability, cost logging). Run a ~50-story test against the real API. Deliver: the 50 test stories, the exact cost-per-story, a full-run cost estimate, and confirmation the quality gate holds. Milestone 2 — Full run + handover (only after I approve Milestone 1) Run the complete ~14,000-story generation. Deliver: the finished workbook, a short QA summary (pass rates, any skipped slots, final cost), full source code, and the one-page "how to run it" guide. Terms API costs are separate (billed to my Anthropic account via my key). Please quote timeline and fee for each milestone separately in your proposal. Work-for-hire: I own all delivered code. To apply, please answer these three questions Have you run batch LLM generation jobs of 10,000+ API calls? Briefly describe one (what, scale, how you managed it). How would you handle cost, rate limits, and failures/retries for ~15,000–20,000 calls, and how would you keep the run resumable? Rough estimate: your fee and timeline to finalize the script, run the 50-story test, and complete the full run. (Applicants who don't answer these three questions will not be considered.)
- Less than 30 hrs/weekHourly
- < 1 monthDuration
- ExpertExperience Level
- Remote Job
- One-time projectProject Type
Skills and Expertise
Activity on this job
- Proposals:50+
- Last viewed by client:2 days ago
- Hires:1
- Interviewing:9
- Invites sent:11
- Unanswered invites:1
About the client
- United StatesWaukesha6:12 AM
- $24K total spent157 hires, 26 active
- 1,707 hours
Explore similar jobs on Upwork
How it works
Create your free profileHighlight your skills and experience, show your portfolio, and set your ideal pay rate.
Work the way you wantApply for jobs, create easy-to-by projects, or access exclusive opportunities that come to you.
Get paid securelyFrom contract to payment, we help you work safely and get paid securely.
About Upwork
- 4.9/5(Average rating of clients by professionals)
- G2 2021#1 freelance platform
- 49,000+Signed contract every week
- $2.3BFreelancers earned on Upwork in 2020
Find the best freelance jobs
Growing your career is as easy as creating a free profile and finding work like this that fits your skills.
Trusted by