Terminal Bench Expert

Terminal Bench Expert Role- Terminal Bench Expert Employment Type - Remote 3-10 years of experience 3–10 years of experience in software engineering or relevant domains. Strong debugging, reasoning, and analytical skills Full-time. 40 hours per week with an overlap of 4 hours with PST. What does day-to-day look like: • Design high-quality Terminal-Bench task ideas and specifications. • Develop complex tasks requiring reasoning, investigation, and debugging. • Write clear task descriptions, solution approaches, and verification logic. • Define deterministic, outcome-based evaluation criteria. • Identify realistic failure modes, edge cases, and operational constraints. • Create tasks that challenge AI systems while remaining solvable by experts. • Collaborate with reviewers to refine task quality and difficulty. • Contribute expertise across one or more specialized domains. Required Skills: • 3–10 years of experience in software engineering or relevant domains. • Strong debugging, reasoning, and analytical skills. • Good understanding of system design, workflows, and dependencies. • Ability to analyze complex systems across multiple layers. • Experience with production systems, pipelines, or large-scale workflows. • Strong technical writing and documentation skills. • Exposure to LLMs, agentic systems, or AI evaluation frameworks. • Experience reviewing technical specifications or designing validation logic. Domains (Any of the following): • Software Engineering & Code Operations • Debugging & Codebase Navigation • System Administration & Shell Workflows • File & Text Processing Pipelines • Data Engineering (ETL & Data Pipelines) • Database & SQL Operations • Machine Learning Pipelines & MLOps • Post-training & Model Finetuning Workflows • AI Evaluation & Benchmarking Systems • Retrieval, Search & Ranking Systems • GPU / Systems Performance Optimization • Distributed Systems & Infrastructure • Cloud & Platform Engineering • DevOps & CI/CD Systems • Build & Dependency Management • Scientific & Numerical Computing • Simulation & Optimization Systems • Formal Methods & Theorem Proving • Document & Structured Data Processing (PDFs, Excel, etc.) • Media Processing (Video, Audio, Images via CLI tools) • Programmatic Graphics & Design (SVG, layout, rendering) • Data Visualization & Reporting Workflows • Geospatial & Spatial Data Processing • Time-series & Forecasting Systems • Security, Forensics & Reverse Engineering • Cybersecurity & Vulnerability Analysis • Networking & API Integration Workflows • Automation & Multi-step Toolchain Orchestration • CLI Tooling & Developer Tool Workflows • Version Control & Git Workflows • Observability, Logging & Monitoring • Storage Systems & File Systems • Finance & Accounting Workflows • Quantitative Finance & Risk Modeling • Legal & Compliance Workflows • Healthcare & Clinical Data Processing • Supply Chain & Logistics Operations • Marketing & Growth Analytics • CRM & Sales Operations • HR & Recruiting Analytics • Consulting & Strategy Modeling • Investment Workflows • Operations Research & Decision Optimization • Benchmark Infrastructure, Adapters & Harness