Upwork home
Sign up
Project Catalog
DesignDesign
Video & AudioVideo & Audio
Development & ITDevelopment & IT
MarketingMarketing
Writing & TranslationWriting & Translation
MoreMore

You will get an LLM evaluation rubric and annotation guidelines your team can run

Zengyu Z.Status: Offline
Zengyu Z. Zengyu Z.
Preview image

Let a pro handle the details

Buy Generative AI services from Zengyu, priced and ready to go.
Zengyu Z.Status: Offline
Zengyu Z. Zengyu Z.
Preview image

Let a pro handle the details

Buy Generative AI services from Zengyu, priced and ready to go.

Project details

"How do we know our AI feature is actually good?" If your team answers that with vibes, this project fixes it.

I will design a complete evaluation framework for your LLM feature: scoring dimensions that match YOUR product goals, clear rating scales with accept/reject thresholds, and annotation guidelines with worked examples that real annotators can follow consistently. The Standard tier adds a Label Studio configuration ready to import - your annotators can start labeling the same day. Premium adds a 50-sample pilot annotation with inter-annotator agreement analysis, so the rubric is calibrated against reality before you scale it.

I do this professionally for a large-scale AI copilot product: rubrics through multiple versions, LLM judges validated against human raters, and annotation ops end to end. You get the same methodology, scoped to your feature, in about a week.

Works for: chatbots, AI agents, copilots, RAG/search answers, content generation. English, Chinese, or mixed-language data.
AI Algorithms
Large Language Model, Multimodal Large Language Model, Transformer Model
AI Applications
AI Chatbot, Conversational AI, Natural Language Generation, Natural Language Understanding
AI Development Language
Python
AI Tools
Hugging Face
AI Models
ChatGPT, GPT-4, LLaMA
What's included
Service Tiers Starter
$350
Standard
$600
Advanced
$950
Delivery Time 5 days 7 days 10 days
Number of Revisions
122
AI Model Integration
-
-
-
Batch Normalization
-
-
-
Database Integration
-
-
-
Detailed Code Comments
-
-
-
Image Upscaling
-
-
-
MLOps
-
-
-
Model Deployment
-
-
-
Model Documentation
-
-
-
Model Monitoring
-
-
-
Model Testing & Optimization
Model Tuning
-
-
-
Natural Language Processing
-
-
-
NLP Tokenization
-
-
-
Pre-Training
-
-
-
Prompt Engineering
-
-
-
Setup File
-
-
-
Source Code
-
-
-
Zengyu Z.Status: Offline

About Zengyu

Zengyu Z.Status: Offline
LLM Evaluation & Annotation Specialist | Bilingual EN-CN
Sunnyvale, United States - 7:57 am local time
I own the quality bar for three shipped LLM agent products at a Global Technical Company - defining what "great" looks like as written rubrics, running the end-to-end human evaluation pipeline behind them (annotator guidelines, golden sets, calibration, disagreement arbitration), and building the automated judges validated against human raters that make quality continuously measurable. My work turns "this answer feels wrong" into a failure-mode taxonomy engineering can act on, fast enough to change the current release.

Here, I help teams stand up the same rigor in days, not quarters:

• Evaluation frameworks - scoring dimensions, rating scales, accept/reject criteria, and golden sets tailored to your product, not generic checklists.

• Annotation operations - guidelines with worked examples real annotators can follow consistently, Label Studio configurations ready to import, and inter-annotator agreement analysis so you know your labels are trustworthy.

• Conversation quality audits - I read your AI agent's real transcripts, build a badcase taxonomy, cluster root causes, and hand you a prioritized fix list instead of a vague "quality score".

• Intent taxonomies - multi-level classification schemas with decision rules precise enough for both human annotators and LLM classifiers.

• Bilingual delivery - native Mandarin Chinese, professional working English. I evaluate and annotate in both languages, including mixed-language data.

How I work: async-first and documentation-heavy. Fixed-price milestones with clearly defined deliverables. You get artifacts your team can keep using - rubrics, guidelines, configs, reports - not just my opinion.

If your team is shipping an AI feature and the honest answer to "how do we know it's good?" is "we eyeball it", that's exactly the gap I fill.

Steps for completing your project

After purchasing the project, send requirements so Zengyu can start the project.

Delivery time starts when Zengyu receives requirements from you.

Zengyu works on your project following the steps below.

Revisions may occur after the delivery date.

Kickoff review

I study your product context, output samples, and any existing quality docs, then confirm scope with you.

Draft rubric for your review

You get the scoring dimensions, rating scales, and accept/reject criteria - with your feedback folded into the next pass.

Review the work, release payment, and leave feedback to Zengyu.