Data Annotation & Labeling Operations Lead

Posted 3 weeks ago

Worldwide

Summary

Job description: We're 8x. We've built one of the fastest-growing human networks in the world (60K+ people in 5 months) plus an AI layer that orchestrates their work. We're opening a data labeling line and need someone to stand it up. This is a build role, not a labeling role. You'll turn our human supply and a set of annotation projects into a reliable, high-quality delivery engine. What you'll do: Do market research and commercialisation. Design annotation workflows: guidelines, edge cases, gold sets, QA/review. Set and enforce quality benchmarks (accuracy, agreement, throughput). Recruit, onboard, and manage a labeling team; build the feedback loop. Pick and configure tooling (Label Studio, CVAT, Labelbox, or similar). Own a first paid pilot end to end: scope, timeline, per-unit cost, quality. You're a fit if you: Have run or closely supported annotation work (text, image, audio, or RLHF) - depth matters more than years. Know QA methods (gold sets, consensus, audit sampling) and at least one annotation tool. Can manage a distributed team and hit accuracy and throughput together. Write clearly enough to make guidelines a non-expert can follow. Bonus: RLHF / LLM preference data, model eval, or a background at a labeling vendor (Scale, iMerit, Sama, Surge, Appen). Engagement: Part-time contract (10-20 hrs/week) for the pilot, path to a larger role if it works. Remote, async-friendly, some European-hours overlap preferred. Include your rate in your proposal. Apply with: A labeling project you ran or supported - task, team size, accuracy/throughput. How you measure and enforce annotation quality. Rough per-unit cost to label 10,000 short text items, and how you got there.

  • Less than 30 hrs/week
    Hourly
  • 1-3 months
    Duration
  • Intermediate
    Experience Level
  • $8.00

    -

    $20.00

    Hourly
  • Remote Job
  • Ongoing project
    Project Type
Skills and Expertise
Mandatory skills
Data Annotation
CVAT
Accuracy Verification
Activity on this job
  • Proposals:20 to 50
  • Last viewed by client:yesterday
  • Interviewing:
    4
  • Invites sent:
    0
  • Unanswered invites:
    0
About the client
Member since Feb 3, 2026
  • United States
    San Francisco6:07 AM
  • $23K total spent
    38 hires, 18 active
  • 1,379 hours
  • Tech & IT
    Small company (2-9 people)

Explore similar jobs on Upwork

German
German to English Translation
English to German Translation
AI Agent Setup Consultancy for Telecom
New
Hourly‐ Posted 20 hours ago
AI Agent Development
AI Bot
AI Builder
AI App Development
Lead Generation
Sales
Email Communication
Administrative Support
Price & Quote Negotiation
Python

How it works

  • Post a job icon
    Create your free profile
    Highlight your skills and experience, show your portfolio, and set your ideal pay rate.
  • Talent comes to you icon
    Work the way you want
    Apply for jobs, create easy-to-by projects, or access exclusive opportunities that come to you.
  • Payment simplified icon
    Get paid securely
    From contract to payment, we help you work safely and get paid securely.
Want to get started? Create a profile

About Upwork

  • Rating is 4.9 out of 5.
    4.9/5
    (Average rating of clients by professionals)
  • G2 2021
    #1 freelance platform
  • 49,000+
    Signed contract every week
  • $2.3B
    Freelancers earned on Upwork in 2020

Find the best freelance jobs

Growing your career is as easy as creating a free profile and finding work like this that fits your skills.

Trusted by

  • Microsoft Logo
  • Airbnb Logo
  • Bissell Logo
  • GoDaddy Logo