Prompt/QA Engineers for AI Study
Worldwide
I'm running a formal validation study of AI test-coverage methods and need two experienced prompt/QA engineers for a short, well-defined engagement. You'd be one of two independent specialists; you won't interact with each other. The work (4 tasks, ~5–8 hours total, spread over ~2 weeks): 1. Failure-scenario enumeration (~4 × 1 hr, async). For each of four real AI prompts (a support router, a triage assigner, and similar), you get the prompt and one hour: list the scenarios where you'd expect it to fail. AI tools are allowed and expected — use whatever assistants you normally would, with one restriction on which model family (disclosed at kickoff), and a short log of what tools you used. Deliverable: a failure-scenario list per prompt + one 15-minute debrief call. 2. Expert labeling session (~1 hr, live on a video call). You'll tag a set of test cases against provided definitions, sharing your screen, no AI assistance for this one task — the whole point is your unaided judgment. Paid at the same rate; scheduled at your convenience. 3. Failure clustering (~1 hr, async). Given a set of failing AI outputs (source hidden), group them into distinct failure modes. AI assistance fine here. 4. On-call tiebreaker pool (~0–1 hr). Occasionally adjudicate items where crowd raters split. Same no-AI rule as task 2; done in the labeling app, low volume. You're a fit if you: have shipped or QA'd production LLM features (prompt iteration, eval design, red-teaming, or test authoring); can articulate *why* a prompt fails, not just that it does; are comfortable being timed and logged (it's a study — process is data); and can commit to availability windows across two specific weeks (dates at kickoff). Logistics: $45–75/hr depending on experience; ~5 hrs guaranteed, up to ~8 if the tiebreaker pool activates. Async except the labeling session and one debrief call. NDA-free — everything you see is external/public material, and your deliverables are anonymized in any write-up. I'll ask two short screening questions before hiring (below). To apply, answer the two questions (a paragraph each, no essays) below.
- Less than 30 hrs/weekHourly
- < 1 monthDuration
- ExpertExperience Level
$40.00
-
$75.00
Hourly- Remote Job
- One-time projectProject Type
Skills and Expertise
Activity on this job
- Proposals:15 to 20
- Last viewed by client:3 days ago
- Interviewing:0
- Invites sent:0
- Unanswered invites:0
About the client
- United States1:55 AM
Explore similar jobs on Upwork
How it works
Create your free profileHighlight your skills and experience, show your portfolio, and set your ideal pay rate.
Work the way you wantApply for jobs, create easy-to-by projects, or access exclusive opportunities that come to you.
Get paid securelyFrom contract to payment, we help you work safely and get paid securely.
About Upwork
- 4.9/5(Average rating of clients by professionals)
- G2 2021#1 freelance platform
- 49,000+Signed contract every week
- $2.3BFreelancers earned on Upwork in 2020
Find the best freelance jobs
Growing your career is as easy as creating a free profile and finding work like this that fits your skills.
Trusted by