AI Evaluation Specialist — Video Sourcing & Adversarial Q&A

We’re looking for someone experienced in AI model evaluation, red teaming, or multimodal annotation to source videos and create precise question-answer pairs designed to stump and challenge AI models. What You’ll Do Source videos that meet specific criteria Analyze video content closely Create precise question-answer pairs based only on what is shown Design difficult questions that test the model’s reasoning and attention to detail Identify edge cases or details that models are likely to miss Make sure every answer is accurate, clear, and objectively supported by the video Follow detailed guidelines and maintain consistent quality What We’re Looking For Experience with AI evaluation, red teaming, RLHF, benchmarking, or data annotation Experience working with video or multimodal AI tasks Strong reasoning and attention to detail Ability to create challenging but fair questions Excellent written English Ability to follow precise instructions and quality standards To Apply Please share: Your experience evaluating or testing AI models Examples of video-based or multimodal annotation work Any experience creating adversarial or difficult Q&A designed to expose model weaknesses Relevant projects or companies you’ve worked with Samples of previous work, if available Please do not apply if your experience is limited to basic data entry or general virtual assistant work. We are specifically looking for someone who understands how to create challenging evaluation tasks for AI models.