Data Annotation Analyst

Overview We're building a labeled test corpus to evaluate a PII (Personally Identifiable Information) detection system. We need a precise, detail-oriented annotator to open files — images, PDFs, spreadsheets, code files, logs — and identify exactly what personal information is present, recording the exact values visible in each document. You receive pre-organized batches of files. Your job is to fill an annotations.csv accurately. Precision matters far more than speed — mislabeled examples degrade our evaluation benchmarks. What You'll Do - Open each file in the batch and identify which PII labels are present from a 50-label taxonomy (we provide the full reference with examples and qualification rules) - Record the exact value as it appears in the document — no guessing, no inference - Classify each file by: domain (medical / financial / HR / technical / generic), context level (isolated field / sentence / paragraph / full document), co-occurrence count - For hard negative files (documents with PII field labels but no values filled in): identify which labels a scanner might incorrectly fire on - Flag ambiguous cases in the notes column rather than making a judgment call What You'll Be Annotating - Images: Driver's licenses, passports, insurance cards, scanned W-2s and pay stubs, portal screenshots — you'll visually read what's on the document - PDFs: Tax forms (W-2, W-9, 1040), medical records, discharge summaries, EOBs, HR contracts - CSVs / XLSX: Employee rosters, patient lists, transaction logs, payroll exports — one representative row sampled - JSON: API responses with user data, config files with credentials, auth token exports - Code & logs: Source files with hardcoded secrets, nginx/auth logs with IPs and usernames PII Categories You'll Be Labeling - Personal identifiers: first/last name, date of birth, email, phone - Government IDs: SSN, driver's license, passport, EIN - Financial: card number, card expiry, CVV, bank account, routing number - Medical: MRN, insurance ID, diagnosis, procedure, medication, provider NPI - Secrets & technical: API keys, passwords, auth tokens, IP addresses, hostnames, URLs - Location: street address, city, state, postal code, country Required Skills - High attention to detail — this is a precision role, not a volume role - Ability to read and navigate diverse file formats (images, PDFs, spreadsheets, JSON, code, plain text) - Understanding of what PII is and isn't — must distinguish real values from placeholders, field labels, and generic references - Comfort producing clean, consistently formatted CSV output - Prior data annotation, content labeling, or data entry experience Nice to Have - Experience annotating NLP or ML training datasets (NER, document understanding) - Familiarity with US document formats: IRS forms, insurance EOBs, medical intake forms, bank statements - Understanding of GDPR, HIPAA, or CCPA — improves judgment on edge cases - Ability to read code and log files (for SECRETS and TECHNICAL label annotation) Deliverables Per Batch - annotations_batch_NNN.csv — one row per file, all columns filled - Turnaround: ~50 files per batch, expected within 2–3 business days Scope: ~500–700 files across phased batches (Phase 1 is image-heavy — you'll be reading a lot of document photos). Ongoing engagement.