Prompt/QA Engineers for AI Study

Posted 3 days ago

Worldwide

Summary

I'm running a formal validation study of AI test-coverage methods and need two experienced prompt/QA engineers for a short, well-defined engagement. You'd be one of two independent specialists; you won't interact with each other. The work (4 tasks, ~5–8 hours total, spread over ~2 weeks): 1. Failure-scenario enumeration (~4 × 1 hr, async). For each of four real AI prompts (a support router, a triage assigner, and similar), you get the prompt and one hour: list the scenarios where you'd expect it to fail. AI tools are allowed and expected — use whatever assistants you normally would, with one restriction on which model family (disclosed at kickoff), and a short log of what tools you used. Deliverable: a failure-scenario list per prompt + one 15-minute debrief call. 2. Expert labeling session (~1 hr, live on a video call). You'll tag a set of test cases against provided definitions, sharing your screen, no AI assistance for this one task — the whole point is your unaided judgment. Paid at the same rate; scheduled at your convenience. 3. Failure clustering (~1 hr, async). Given a set of failing AI outputs (source hidden), group them into distinct failure modes. AI assistance fine here. 4. On-call tiebreaker pool (~0–1 hr). Occasionally adjudicate items where crowd raters split. Same no-AI rule as task 2; done in the labeling app, low volume. You're a fit if you: have shipped or QA'd production LLM features (prompt iteration, eval design, red-teaming, or test authoring); can articulate *why* a prompt fails, not just that it does; are comfortable being timed and logged (it's a study — process is data); and can commit to availability windows across two specific weeks (dates at kickoff). Logistics: $45–75/hr depending on experience; ~5 hrs guaranteed, up to ~8 if the tiebreaker pool activates. Async except the labeling session and one debrief call. NDA-free — everything you see is external/public material, and your deliverables are anonymized in any write-up. I'll ask two short screening questions before hiring (below). To apply, answer the two questions (a paragraph each, no essays) below.

  • Less than 30 hrs/week
    Hourly
  • < 1 month
    Duration
  • Expert
    Experience Level
  • $40.00

    -

    $75.00

    Hourly
  • Remote Job
  • One-time project
    Project Type
Skills and Expertise
Mandatory skills
LLM Prompt Engineering
Activity on this job
  • Proposals:15 to 20
  • Last viewed by client:3 days ago
  • Interviewing:
    0
  • Invites sent:
    0
  • Unanswered invites:
    0
About the client
Member since Jul 15, 2026
  • United States
    1:55 AM

Explore similar jobs on Upwork

iOS
Camera
Android Smartphone
Artificial Intelligence
AI Engineer (RAG)Fixed-price‐ Posted 5 days ago
Retrieval Augmented Generation
Artificial Intelligence
Machine Learning
Python

How it works

  • Post a job icon
    Create your free profile
    Highlight your skills and experience, show your portfolio, and set your ideal pay rate.
  • Talent comes to you icon
    Work the way you want
    Apply for jobs, create easy-to-by projects, or access exclusive opportunities that come to you.
  • Payment simplified icon
    Get paid securely
    From contract to payment, we help you work safely and get paid securely.
Want to get started? Create a profile

About Upwork

  • Rating is 4.9 out of 5.
    4.9/5
    (Average rating of clients by professionals)
  • G2 2021
    #1 freelance platform
  • 49,000+
    Signed contract every week
  • $2.3B
    Freelancers earned on Upwork in 2020

Find the best freelance jobs

Growing your career is as easy as creating a free profile and finding work like this that fits your skills.

Trusted by

  • Microsoft Logo
  • Airbnb Logo
  • Bissell Logo
  • GoDaddy Logo