English Proficiency Scorer

We're hiring experienced language-assessment raters to score English-speaking recordings from learners. You'll listen to short learner responses and rate them using a structured 0 to 4 rubric (provided, with benchmark examples). The work is detail-oriented and rubric-driven. Your scoring will be measured on three dimensions throughout the engagement: 1. Validity: your scores accurately reflect a learner's true speaking proficiency. 2. Consistency: you apply the rubric the same way across clips, over time, and in line with other raters. 3. Task suitability: you correctly flag prompts or recordings that are unfair, unclear, or unusable. ENGAGEMENT DETAILS: TWO PHASES This role runs in two phases, and everyone starts with Phase 1. Phase 1: Calibration (paid, and required first) Before any main labeling, all raters complete the same set of ~100 calibration clips (about 8 hours). These have already been scored by our expert rater, so this round lets us confirm everyone is applying the rubric consistently, and lets you get comfortable with the task and ask questions. Calibration is how we decide who continues to Phase 2. If your scores meet our consistency and validity bar, you move forward. If they don't line up closely enough for this project, we may not continue together past calibration, and you'll be paid in full for all calibration work you completed regardless of the outcome. We want this transparent up front so there are no surprises: passing calibration is a condition of continuing. Phase 2: Main labeling (for raters who continue) After calibration, you'll work through assigned batches of new clips (about 250 more, roughly 18 hours), scoring every clip the same way using the same rubric. All scoring is double-blind, and you may be asked to join short tie-break discussions on clips where raters disagree. TIME COMMITMENT: About 26 hours total if you continue through both phases (~8 hours calibration, ~18 hours main labeling), spread across roughly 2 to 3 weeks. Hourly, billed for all work completed. REQUIREMENTS: 1. Proven experience rating speaking proficiency (IELTS, TOEFL, OPIc, CEFR, or similar) 2. Native or near-native English proficiency 3. Familiarity with standardized speaking-assessment rubrics 4. Strong attention to detail and comfort applying a rubric consistently at volume