Building Safe and Transparent AI Solutions at Upwork

In AI research, achieving a system that works 90% of the time can be groundbreaking. But in real-world applications, that same 90% success rate often isn’t enough. As artificial intelligence (AI) reshapes industries and redefines how we live and work, ensuring it operates safely and reliably at scale is essential. Trust in AI depends on more than just its capabilities — it hinges on its ability to function transparently, ethically, and mindfully.

That’s why we believe the power of AI lies not just in its ability to perform tasks but in its capacity to do so responsibly. Whether it’s mitigating bias, protecting user data, or ensuring transparency in decision-making, prioritizing safety enables us to deliver AI solutions that not only act appropriately  but also address core customer needs. Building safe and transparent solutions is critical to establishing Upwork as a quality destination that customers can rely on with confidence. We are optimistic about AI’s potential to supercharge this approach.

Last year, Upwork launched its first AI solution, Uma^TM, Upwork’s Mindful AI. Uma powers our platform’s newest, most advanced AI-enabled experiences acting as an always-on companion that helps freelancers and clients be even more efficient and productive. Uma fuels the next steps toward realizing Upwork’s vision for how to serve customer needs using AI and is a core component of how we’ll help businesses and freelancers accomplish their goals faster than ever before. We intentionally designed Uma with safety at its core, enhancing creativity and productivity while ensuring ethical and trustworthy AI interactions. Let’s break down our approach.

Upwork’s AI principles & governance

Our AI development work at Upwork runs through Umami (Upwork’s AI & Machine Learning) team. The vision for this team is to empower every interaction on Upwork with robust, intelligent, and scalable technologies. 

As part of the integration of AI into our work marketplace, we established a set of principles to provide a foundation for trustworthy, ethical, and effective AI development and deployment. As AI becomes more pervasive, our principles help address the right balance between AI automation and human expertise, prevent bias across diverse global talent pools, and ultimately deliver ethical, high-quality solutions. Grounded in these principles, our approach to AI is transparent, accountable, and human-centered – both in how we shape and build models. 

In conjunction, we established Upwork’s AI Governance Committee (the "AIGC") tasked with providing oversight, guidance, and accountability in the development, deployment, and use of AI technologies across the company. This cross-functional team is charged with aligning Upwork’s AI initiatives with our ethical AI principles, compliance requirements, and values and mission. In doing so, the AIGC ultimately aims to allow for the benefits while avoiding the unnecessary risks associated with AI utilization.

Key AI safety measures in our products 

So what does all this look like in practice as Upwork ships new AI products?

Let’s walk through a few specific ways Upwork’s AI teams address AI safety across the standard LLM workflow – Data, Train, and Evaluate – before deploying a product like Uma. 

Collecting data responsibly 

Data responsibility is the cornerstone of AI safety. The quality and ethical use of data that goes into a model directly impacts AI output including its reliability and fairness. Uma, for example, is trained on vast amounts of data, including Upwork platform data, synthetic data, and human-generated data (we dive into these data types and how we scale AI models here). 

Our process for data collection is centered on transparency: clearly communicating what data is collected, why it’s needed, and how it will be used. This includes collecting only the data that is necessary and implementing strong privacy and security protections to safeguard sensitive information, and involves multiple rounds of data collection and vetting.

Lets focus on one key dataset, human-generated data, as an example. How do we make sure what Uma says and how it says it are consistent with our principles? We started by creating a “personality” for Uma by engaging experts from our platform talent and User Research teams, who helped us define guidelines and guardrails for Uma’s personality. For all our human-generated data, we worked with Top-Rated screenwriters or copywriters from Upwork who crafted compelling scripts for Uma interactions. From these engagements, we collected thousands of written conversations that covered dozens of relevant scenarios, like Uma helping to hire a freelancer for building a website or for project management of a go-to-market campaign. 

Our team then evaluated hundreds of these conversations across diverse topics, guided by clear rules on tone, inclusivity, accuracy, and harm prevention, based on the personality we created for Uma. Each conversation was rigorously assessed for quality, accuracy, and potential biases and then carefully reviewed by humans to ensure it adhered to our rules before we used the data to train our models. For both helpfulness and accuracy, our evaluation benchmarks show marked improvement above off-the-shelf solutions like ChatGPT.

Training Uma with a safety harness

Once we have a safe and high-quality dataset to train our models, there are additional checks we take to ensure AI systems operate safely, predictably, and ethically. One important internal tool is our safety harness. Just like a physical safety harness protects individuals from harm, an AI safety harness is a service that helps us catch unintended consequences and risks during development, rather than discovering them in production. For example, this could include robust fairness checks to ensure Uma recommendations are unbiased or assessing harmfulness in model responses (e.g., racism, sexism, inappropriateness). 

Our safety harness model allows us to compare the outputs of our Uma model relative to baseline models (e.g. Llama) for large numbers of potentially unsafe user prompts. Given the subjective nature of "harmful," not only does our safety harness model provide a score between 1 and 5 (with scores above 3 considered safe), but it also provides a more fine-grained description of response harmfulness.

Here’s an example response where our Uma model was prompted to tell a mean joke.

Our safety harness evaluation is included below. 

In this example, our safety harness scored the Uma output as safe, while the “vanilla” output received a much lower safety score. Although this is just one example, we can easily send thousands of inputs through our system to test our safety. Across 30,000 inputs, we found that our Uma model showed a 41% relative improvement in safety score compared to off-the-shelf models like baseline Llama3 and ChatGPT. These checks are crucial in establishing our confidence that Uma will be a critical component of our customers’ workflows.

Evaluating with proactive discovery and continual learning

Finally, when an LLM is ready for deployment, our team continues to monitor and evaluate its output via proactive discovery and continual learning. We are starting to develop systems so that across millions of potential Uma interactions, we can constantly monitor conversations for those rare cases where Uma may be uncertain or provide suboptimal responses.  

Below are examples of high uncertainty versus low uncertainty phrases or questions that Uma may receive. We would expect that Uma would struggle or potentially provide an unusual response to high uncertainty questions or phrases, and those are the ones that we want to protect against. 

High uncertainty

• I don’t have enough money to give a refund
• Will I get my Connects back if the job post resulted in a scam?

Low uncertainty

• Where can I post a job?
• How can I find top talent?

While we can try to anticipate high uncertainty inputs and plug gaps before production, it’s impossible to consider every scenario. As a result, we test uncertainty post-hoc through our live interactions with customers who use Uma. Anytime Uma converses with a customer, it is a live interaction that has the potential to present highly uncertain scenarios, and this increases as we scale Uma to more customers and surfaces. For instance, if a customer prompts Uma with an odd question, Uma may respond with “I'm not certain” or “I don’t have an answer for that.” When this happens, these potential anomalies are efficiently collected, trigger automated mitigation protocols, and log the outcome. This means that the model has identified an issue and will look for ways to solve it going forward whether it’s by retraining or fine-tuning. In this way, our model can truly improve itself with very little human intervention and continue to develop better responses to uncertainty. 

Committed to safer AI

While many technology companies embrace the mantra of “move fast and break things,” at Upwork, we balance innovation with a deep commitment to AI safety, ensuring we never compromise the well-being of our clients and freelancers. This dedication pushes us to go beyond the current state of AI, leading to innovations in inclusive and unbiased data collection, the development of novel in-house algorithms that power safeguards within our custom LLMs, and feedback loops that continuously enhance Uma’s safety and response to uncertainty with every interaction. Innovation fuels our technology and business, but safe innovation is what ensures lasting, sustainable benefits for the future of work.