Run a Batch LLM Content-Generation Pipeline (Python + Claude API) — ~14,000 Short Stories

Posted 2 days ago

Worldwide

Summary

Run a Batch LLM Content-Generation Pipeline (Python + Claude API) — ~14,000 Short Stories Overview I run a collectible toy brand. Each toy links (via a QR code) to a unique ~270-word children's story about that character. I need roughly 14,000 of these stories generated at a consistent quality and reading level, and delivered as a single spreadsheet. A working Python script already exists (I'll provide it). It calls the Anthropic Claude API, generates each story from a "canon" reference file plus ~78 hand-written example stories, runs automated quality checks, assigns metadata, and writes the output. It has been tested in a free "dry-run" mode. Your job is to finalize it for a full production run, run it reliably and cost-effectively, and deliver clean, QA'd output. This is a finish-and-run job, not a build-from-scratch job. I want to own the finished script and be able to run future batches myself for only the API cost — no per-batch dev fees and no monthly hosting. What I'll provide The existing generator script (Python, ~330 lines, documented) A Canon Reference workbook (character voice guides, location/companion rules, winner logic, reading-level target) ~78 example stories (the style/quality bar and the anti-repetition seed set) An Anthropic API key (billed to my account) Scope of work Finalize the script. One file-I/O section in main() is stubbed and needs completing (straightforward). Review the rest. Harden it for scale. Add/improve concurrency or async for throughput, robust rate-limit and error handling, resumability, and running cost logging. Basic versions of these exist — make them production-grade. Costed test run. Generate ~50 stories against the real API. Report exact cost-per-story and a full-run cost estimate. Full sweep. On my approval, run the complete ~14,000-story generation. Deliver the completed workbook plus a short QA summary (pass rates, any skipped slots, final cost). Quality requirements (the script's gate enforces these — verify they hold at scale) ~240–300 words per story; reading level a 7-year-old understands (≈ grade 4, measured) No banned filler phrases; a specific character-numbering convention (provided) Anti-repetition: no reused openings or sentence structures across the whole library Correct per-story metadata: season rules, companions, winner flags, collection Output must match the exact 21-column spreadsheet schema I provide Deliverables & ownership (important) Full source code handover — I own the finished script and all code outright (work-for-hire). No dependencies on your private accounts or servers. A one-page, plain-English "How to run it" guide written so that a non-developer can generate future batches themselves — e.g., "to make 500 more, open this, type this command." No hosting, no subscription required. The script must run locally (on a normal laptop) with only an API key — no ongoing hosting or monthly costs. Re-running for a new batch should be a single command; the tool should resume/skip already-written stories so refills are cheap and never redo work. Required skills Strong Python Hands-on experience with LLM APIs (Anthropic Claude and/or OpenAI), especially large batch / async generation jobs Cost and rate-limit management for high-volume API runs (this job is ~15,000–20,000 calls) openpyxl / Excel data handling Careful, detail-oriented QA Nice to have Prompt-engineering experience Prior content-generation or data-pipeline work at scale Process & milestones This is structured in two milestones so we both stay low-risk. Milestone 1 is small and proves the approach before the full run. Milestone 1 — Finalize + costed test Complete the stubbed section of the script and get it production-ready (concurrency, retries, resumability, cost logging). Run a ~50-story test against the real API. Deliver: the 50 test stories, the exact cost-per-story, a full-run cost estimate, and confirmation the quality gate holds. Milestone 2 — Full run + handover (only after I approve Milestone 1) Run the complete ~14,000-story generation. Deliver: the finished workbook, a short QA summary (pass rates, any skipped slots, final cost), full source code, and the one-page "how to run it" guide. Terms API costs are separate (billed to my Anthropic account via my key). Please quote timeline and fee for each milestone separately in your proposal. Work-for-hire: I own all delivered code. To apply, please answer these three questions Have you run batch LLM generation jobs of 10,000+ API calls? Briefly describe one (what, scale, how you managed it). How would you handle cost, rate limits, and failures/retries for ~15,000–20,000 calls, and how would you keep the run resumable? Rough estimate: your fee and timeline to finalize the script, run the 50-story test, and complete the full run. (Applicants who don't answer these three questions will not be considered.)

  • Less than 30 hrs/week
    Hourly
  • < 1 month
    Duration
  • Expert
    Experience Level
  • Remote Job
  • One-time project
    Project Type
Skills and Expertise
Mandatory skills
Python
Machine Learning
Activity on this job
  • Proposals:50+
  • Last viewed by client:2 days ago
  • Hires:
    1
  • Interviewing:
    9
  • Invites sent:
    11
  • Unanswered invites:
    1
About the client
Member since Sep 14, 2017
  • United States
    Waukesha6:12 AM
  • $24K total spent
    157 hires, 26 active
  • 1,707 hours

Explore similar jobs on Upwork

Local Lead GenerationHourly‐ Posted 3 weeks ago
Web Scraping
Data Scraping
Data Extraction
Lead Generation
Data Entry
Data Mining
Data Collection
Data Processing
Web Scraping Framework
Web Crawler Framework
Web Scraping Software
Web Scraping Plugin
Web API
Search Tool
Search Engine
Microsoft Word
Data Entry
Administrative Support
Microsoft Excel

How it works

  • Post a job icon
    Create your free profile
    Highlight your skills and experience, show your portfolio, and set your ideal pay rate.
  • Talent comes to you icon
    Work the way you want
    Apply for jobs, create easy-to-by projects, or access exclusive opportunities that come to you.
  • Payment simplified icon
    Get paid securely
    From contract to payment, we help you work safely and get paid securely.
Want to get started? Create a profile

About Upwork

  • Rating is 4.9 out of 5.
    4.9/5
    (Average rating of clients by professionals)
  • G2 2021
    #1 freelance platform
  • 49,000+
    Signed contract every week
  • $2.3B
    Freelancers earned on Upwork in 2020

Find the best freelance jobs

Growing your career is as easy as creating a free profile and finding work like this that fits your skills.

Trusted by

  • Microsoft Logo
  • Airbnb Logo
  • Bissell Logo
  • GoDaddy Logo