Uma Recruiter: How We Built an Agentic Solution to Talent Matching and Hiring

Today we're giving a peek under the hood at how we built Uma Recruiter, a core feature of Uma, Upwork’s AI work agent, and our first agentic hiring solution designed to meet the need of growing SMBs to quickly find skilled professionals who can plug in when and where they’re needed most.

Hiring on Upwork involves more than finding the right candidate in a database. A good match emerges from a multi-step process: understanding what the client actually needs (often more than what they write in a job post), exploring a large and diverse global talent pool, engaging candidates, and evaluating them for fit. Recent investments in search infrastructure, like Upwork’s acquisition of Objective, Inc.’s search technology and team, have significantly advanced our ability to support fast, accurate retrieval and interactive search sessions. But even state-of-the-art semantic search built on dense embeddings, vector databases, and fine-tuned ranking systems handles one query at a time. That speed comes at a cost: the system can't reason across multiple steps, revisit assumptions, or act on engagement signals mid-process.

Recruitment doesn't need to happen in milliseconds. A hiring process unfolds over hours, days or weeks. That different time horizon opens the door to a different class of AI system — one that can take a few extra seconds to reason more carefully, explore more broadly, and evaluate more thoroughly. This is what agentic AI makes possible, and it's what Uma Recruiter is built on.

What follows is a look at how it works, the engineering challenges we solved to make it reliable at scale, and what we've seen since launch.

How Uma Recruiter works

Uma Recruiter is built around three core components: an Agentic Brain powered by large language models (LLMs) that orchestrates the process end-to-end; Agentic Tools that execute actions across Upwork's platform; and Agentic Memory that carries context across steps. These components operate in concert across a structured, iterative pipeline, from job understanding through candidate sourcing, evaluation, outreach, and shortlisting.

When a client posts a job and requests AI assistance, Uma Recruiter constructs a structured representation of the hiring task. Clients often leave optional fields incomplete; for example, if a description mentions a time zone preference without filling in the location filter, the agent picks this up and applies it as a constraint. Similarly, it infers which skills are required and which are nice-to-have, even when the client hasn't formally categorized them. This structured understanding drives the next steps.

The agent then explores the talent pool using its Agentic Tools, which interface with Upwork’s freelancer search and recommendation systems. Candidates are sourced against the constraints identified, both those the client stated and those the agent inferred. Then they are evaluated across multiple dimensions: profile description, demonstrated expertise, past project performance, portfolio depth, current availability, Job Success Score, and more. This multi-dimensional representation allows the agent to reason over candidate fit in a way that more closely mirrors how a skilled human recruiter would evaluate a profile.

As it explores and evaluates, the agent draws on its Agentic Memory to further enrich its reasoning. When a client has previously hired a particular freelancer on the platform, that signal is factored in. For example, prior successful collaborations are a strong indicator of fit. The agent also uses talent job history selectively; rather than scanning a professional’s entire Upwork history, it identifies which past jobs are most similar to the current opening and uses those as the relevant signal.

Once a strong set of candidates is identified, the agent actively invites them to apply to the job post. Incoming proposals are evaluated by Uma’s proposal evaluation tool against the client's requirements, including the proposal text itself and answers to preliminary screening questions, and Uma Recruiter produces a ranked shortlist for the client. 

The Agentic Brain orchestrates this entire process, reflecting and deciding at each step whether to expand the candidate pool, refine sourcing constraints, or finalize the shortlist. The process terminates when a high-quality shortlist has been produced, that is, a shortlist that meets the requirements of the job post, or when further iteration is unlikely to improve outcomes. The result is not a single-pass retrieval, but the output of a structured, iterative reasoning process.

Engineering an agent for the real world

Building Uma Recruiter required solving several engineering challenges inherent to agentic systems. Unlike deterministic search pipelines that involve a small number of predictable operations, agentic workflows consist of dozens or even hundreds of calls to different models and services. Failures are inevitable: a tool may time out, a model may throttle, and an API may return transient errors. Instead of halting the entire process, the system must treat these failures as expected and recover gracefully. We built retry strategies, fallback paths, and partial-result tolerance directly into the agent’s execution layer so that a single failure does not collapse the reasoning loop.

Another challenge lies in aligning the agent’s decisions with human expectations. Match quality is inherently subjective, and early versions of the system sometimes selected candidates that looked strong on paper but didn’t resonate with clients. To address this, we created feedback mechanisms that incorporate client behavior, freelancer engagement, shortlist adoption, and eventual hiring outcomes. These signals feed directly into prompt refinement and model improvement, progressively closing the gap between the agent's judgments and the preferences revealed by real hiring decisions.

The complexity of the reasoning process also necessitates deep observability into system behavior. Engineers need to understand not just what the agent did, but why. We developed distributed tracing and step-by-step logging that record the agent's decisions, the outputs of each tool call, and the internal state transitions across the entire workflow. This visibility is what makes the system improvable at pace. Failures can be diagnosed at the reasoning level rather than inferred from downstream symptoms, and performance bottlenecks can be isolated to specific steps in the pipeline rather than attributed to the system as a whole.

Running a pipeline of this complexity at scale also demands careful attention to cost. LLMs are well-suited to tasks that require broad reasoning and flexibility, but not every step in the pipeline requires that level of capability. For well-defined subtasks, such as constraint extraction, smaller specialized models can match or exceed the performance of larger ones at a fraction of the cost. Selectively routing pipeline steps to the most cost-efficient model that meets the quality bar is an active area of research and development for the team.

Measuring system quality presents its own distinct challenge. Standard information retrieval metrics do not map cleanly onto an agentic pipeline with multiple interdependent steps. Our evaluation framework operates at two levels. At the component level, we use LLM-as-judge methodology: structured rubrics assess intermediate outputs like constraint extraction, candidate ranking, and proposal evaluation, enabling scalable quality measurement without manual annotation. At the system level, we ground evaluation in platform outcomes: hire and engagement data links agent behavior to the downstream results that matter most, enabling both offline benchmarking and ongoing monitoring of live performance.

What we’ve seen so far

Uma Recruiter initially launched in October 2025, and the technology is already improving key recruiting outcomes for Upwork Business Plus clients, including generating shortlists within six hours. Initial testing from last year showed a 30% increase in hires by using the shortlist, and overall time to hire down 11%. As we’ve continued to improve the feature, these gains have strengthened, with the number of jobs filled within seven days increasing 10% between November 2025 and March 2026, meaning more clients are finding the right talent on Upwork, faster.

These gains suggest the agent’s iterative reasoning, its ability to explore, evaluate, engage, and adjust, is meaningfully improving match quality, and point to a healthier, more efficient marketplace for both clients and talent. Higher-quality, more targeted invitations reduce noise for freelancers, cutting down on irrelevant outreach and increasing the likelihood of finding the right talent for the right job. For clients, the impact is equally meaningful, with less time spent searching and sifting through candidates and a faster path from need to hire, while preserving the client's ability to select and direct the freelancers of their choosing. As match quality improves, both sides spend less time navigating the marketplace and more time getting work done.

What’s next for Uma Recruiter

Uma Recruiter represents a broader shift in how we think about search and matching on Upwork. Instead of treating a job post as a static query to be answered once, we treat it as the beginning of a dynamic reasoning process. As we continue to strengthen the agent’s planning, expand the tools it can use, and enrich the signals it learns from, we expect it to play an even greater role in helping clients and talent connect with speed and confidence.

If you're looking to hire more efficiently and effectively, we encourage you to try Uma Recruiter, which we’ve started expanding access to in our Upwork Basic plan. And if you're an AI researcher or engineer excited about high-impact, cutting-edge work at the intersection of search, recommendation systems, and agentic AI—we're hiring!

‍

Acknowledgements: Thanks to Pablo Mendes, Leandro Cohn, Allie Blaising, Tim Smith, Gus Figueiredo, Caroline Williams, Laura Grando, Jacques Bernier, Santiago Pinczinger, Karthik Gomadam, Mihail Mihaylov, Yasir Mehmood, Pallavi Gudipati, Dimitris Douras, Geoffrey Anderson, Eren Ozturk, Bianca Bartel, and Andrew Rabinovich for their work on Uma Recruiter.