The AI-Driven Future of Translation Work
Listen to this article
13:47
/
19:15
1x

By Victor Maricato, Ted Liu, and Yanyi He

As generative AI models become more powerful, much anxiety exists about which types of work will be negatively affected. Few work categories have come as close to translation in being perceived to be at risk of being displaced by AI. Unfortunately, such fear-based views often dominate headlines.

Such views have two problems. One, this perception of risk is short-sighted. This is not the first time that innovations in AI technologies have generated uncertainty regarding the future of translation work. For instance, back in the deep learning boom during the 2010s, neural machine translation systems were so impressive in accuracy and efficiency that some predicted this technology would dominate translation work. Yet translation work is still here. In fact, data from the Bureau of Labor Statistics (BLS) shows between 2021 (pre-ChatGPT) and 2023 (post-ChatGPT), the median hourly rate for translation work has in fact increased by 16% (and over 3% when inflation-adjusted), suggesting translation work is growing in monetary value.

The second problem with solely examining whether AI will replace certain work is that it is backward-looking and risks overlooking emerging new work. If AI is truly going to fundamentally change the way people work, it’s crucial to watch for new demands created by AI developments so that people can develop new skills and obtain higher-value work from this fundamental shift.

On the surface, it may seem daunting to completely rethink what translation work looks like years down the line. But we can start with what we know from the Upwork marketplace using our own platform data and showcase with data, the new emerging opportunities that benefit both freelancers and business.

Early impact of generative AI

In February 2024, the Upwork Research Institute released a white paper about the early impact of generative AI on the Upwork platform. In this study, we note the dual impact of new technologies such as generative AI: the displacement effect (work being replaced) and the reinstatement effect (new work being created).

Overall, generative AI has had a positive impact on the Upwork labor market, both in terms of job demand and freelancer earnings per contract. Categories of work, like Writing and Translation, tell an especially interesting story and are instructive in terms of looking at the nuances of the opportunities AI presents in the marketplace. What we observe is that despite the loss in low-value work (those between $251-$500 per contract), high-value contracts (those greater than $1,000 per contract) gain 8% in freelancer earnings in translation.

The results invite an additional question: Why is generative AI improving earnings in high-value translation contracts? Equally important: What makes a high-value translation work high value?

In this blog post, we take an exploratory approach to provide preliminary insights into these questions by analyzing Upwork translation job post data. In general, we find that AI is becoming more infused into translation work, especially high-value work. We see cases where coworking with AI is prominent, exemplified by emerging examples of content co-creation using AI tools and of freelancers as contributors to the development of multilingual models.

Types of high-value work: AI-related and non-AI specialized translation work driving higher value

To establish a baseline, we have conducted exploratory, heuristic analysis through text prompts using ChatGPT Enterprise (details of the methodology are in the appendix). In short, we use ChatGPT to analyze the textual data of a large sample of high-value job posts from the Upwork platform. More specifically, we’re interested in understanding the essential ingredients of a high-value job post, based on how the post describes the key skills, tasks, and deliverables of the contract.

We find that there are currently two broad categories in high-value translation contracts: AI-related work and then non-AI but complex, specialized translation work. Non-AI job postings often detail comprehensive and specialized tasks that require deep understanding and expert handling of specific nuances, indicating a demand for human expertise. These roles often involve localization translations, understanding contextual nuances, and managing large-scale text translations across different languages. Examples of such job posts include translation for video game text and legal translation. Despite the rise of AI, these non-AI roles still represent a significant portion of the translation work on the Upwork platform, accounting for approximately 65% of the high-value job posts analyzed in this category (this figure is a rough estimate based on the ChatGPT analysis). This suggests a continuing reliance on specialized human skills.

AI-related job posts typically highlight the need for skills in data annotation, model training, and familiarity with various AI integration tools. Many of these job postings come from companies actively developing AI models. For instance, an example job post might be hiring a freelancer with computational linguistics skills who would extend and improve a model in Dutch.

Diving deeper into the AI-related post description in translation, we see three main groups: 1) model training and development, 2) quality control and content correction, and 3) teaching and consulting (we used a zero-shot approach with GPT4 for classifying the groups of translation job posts). More specifically, in model development work, freelancers complete tasks related to creating and refining translation tools or models; such posts account for about 48% of such AI-related posts. The second group of quality and content correction contracts pertains to evaluating, auditing, or proofreading translation work generated by AI, contributing to about 48% of AI-related posts. The third group, about 4% of such AI-related posts, involves education or consultancy services to improve or optimize the use of translation tools for clients’ workflows.

Characteristics of high-value translation work

Having obtained broad results from the ChatGPT analysis, we now use specific natural language processing (NLP) tools to conduct a more targeted and interpretable analysis of translation post data on Upwork. The goal is to identify the exact skills that correlate with job value. To assess this relationship, we collected all the translation job posts on the Upwork platform, ranging from low to high values on the Upwork platform for our analyses.

The Need for Specialized Talent in Low-Resource Languages Generates Higher Value

We begin by testing whether large-scale and/or highly specialized translation projects predict higher monetary value. We hypothesize that less common language pairs in a job post indicate that the client seeks specialized skills. To test this, we extract the language pair needed for each translation job post. Then, we calculate and rank the average projected value for each language pair across all the posts. We find that it is indeed the case that rare language pairs (as opposed to, say, English-Spanish) tend to rank higher in terms of value (such pairs tend to involve low-resource languages). These low-resource languages often have text and data scarcity and can lack standardization, which makes machine translation of these languages lower in quality compared to human translation.

The top-value language pairs include examples such as Mandarin-Hebrew, Filipino-Hindi, and Thai-Portuguese. This also makes sense economically, as when a client can hire a freelancer who can do direct translation between Mandarin and Hebrew, they don’t need to hire for the translation work of Mandarin-English and English-Hebrew separately. In other words, freelancers with such specialized and complex translation skills incur lower transaction costs, and clients are willing to pay a premium for the reduced efforts and costs.

Top Translation Skills: Training, Auditing, and Teaching Clients How to Work with AI

More importantly, we want to detect meaningful keywords in high-value job posts. We define meaningful keywords as a function of how often they appear in high-value jobs compared with low-value jobs, defined as lift. Based on the predefined value threshold, we split the sample into high-value versus lower-value posts and compute the difference in the values of the jobs with that word against jobs that do not have that word. The score is thus a ratio that shows how much more likely a word is to appear in high-value job posts compared to low-value ones.  When the value is greater than 1, a term is more likely to appear in a high-value job post.

In Figure 1, we show the terms that are most meaningful and interpretable; in other words, terms that have scores greater than 1 (we exclude some significant terms that may contain confidential details of contracts and also terms that are difficult to interpret). For example, the term "audit" is about 34% (score = 1.34) more likely to appear in high-value jobs. The first four keywords presented in Figure 1 are intuitive when considering the previous analysis of the different types of AI-related translation work.

The importance of “audit” as a term in high-value posts indicates that Upwork clients value quality checks that freelancers can do to enhance the work done by AI. “Teaching” seems to refer to high-value posts that entail freelancers showing clients how to best integrate AI tools into translation work. Finally, the importance of “training” as a keyword (here the ratio is calculated using AI-related posts) could refer to the aforementioned teaching work. Still, most importantly, it indicates humans' critical role in training multilingual AI models. Conversely, with values below 1, terms such as “team” and “categorize” are more likely to show up in low-value job posts than in high-value posts—job posts with “team” likely refer to a smaller bundle of translation projects, and “categorize” likely refers to routine aspects of the projects.

To summarize, the AI-related translation work seems diverse, ranging from content correction to teaching and to model development and training. More importantly, we find that compared to low-value posts, high-value translation job posts are more likely to have AI-related terms, indicating that clients hiring are willing to pay a premium for AI work.

Translation AI


Figure 1: Ratio of term importance (high-value vs low-value job posts). Ratio >1 means a term is more likely to appear in high-value posts.

Future of translation work driven by AI

The results in this study suggest that AI is beginning to play a reinstatement role in translation work, despite the prevailing assumption that AI will largely displace such work. More specifically, we find that traditional high-value translation work will continue to be important; such work requires human expertise to understand the cultural and contextual nuances of translated texts, and some translation work is so specialized that it is not yet replicable by AI. Given such findings, it now makes even more sense that the initial impact of generative AI favors high-value translation work. We expect that the growth in high-value translation will continue to increase so that gen AI's net impact on translation will likely be net positive.

AI model training in foreign languages, a form of data annotation work, will become a more common, new translation work. Our estimation suggests that such AI labeling work is only 4% of all translation work in terms of freelancer earnings on the platform today. We expect demand for this work to increase as more AI companies need human talent to help with model localization. Such demand will not be one-off; as such models become more powerful, they will also require continuing development and maintenance in different language contexts. This development is also beneficial for minimizing biases of AI models solely trained in English (a high-resource language), ensuring the impact of AI models is more equitable. In fact, the recent advancement in multilingual translation technologies can help revitalize traditionally under-resourced languages and help with more equitable access to information.

While these translation models can do some of the tasks, there will be emerging needs for human translation using such tools to work on specialized, complex texts in language pairs that previously did not exist. Translators aren’t working themselves out of future jobs by helping to improve AI, either. As AI models become proficient in more languages, it can increase the global adoption of other products (e.g., websites of online marketplaces; software tools), opening up even more need for skilled translation consultation and business localization. This could include skilled translators serving as advisors to companies on linguistic and cultural nuances, effectively ensuring the continued need for human translation input. In a world of thousands of languages, this is a key opportunity to service the previously underserved markets.

While the insights of this blog come from our platform data, the infusion of AI into translation work seems consistent with freelancer AI tool usage and perceptions as well—the majority of freelancers are ready for this new AI-enhanced work. From our survey-based research from 885 freelancers conducted in early 2024, we find that freelancers in translation are among the most mature in their adoption of these new tools. Over 75% of workers in the translation category use AI frequently to complete their work, and 83% of freelancers who have worked on translation projects feel positive about generative AI.

There are many exciting frontier questions related to the future of translation work and the role of AI. One area worth continuing exploration is how the nature of translation work is changing. By that, we mean at the levels of tasks and deliverables, what will freelancers produce beyond documents of translated texts? Such a question is answerable by analyzing not only job posts but also communication and project deliverables data. Moreover, assuming that freelancers take on more multifaceted translation tasks, whether they perceive AI and AI tools as assistants, agents, companions, or something else will be important to examine as well. Whichever questions one looks at, it’s likely that translation work, similar to many other work categories, will see a future of cocreating with AI.

Appendix: Methodology

For the ChatGPT analysis, we randomly sampled 100,000 high-value job posts (>$1,000 per contract) in translation from the Upwork platform. We then used structured text prompts in the ChatGPT enterprise version to analyze the data, using non-prescriptive questions such as “Can you tell me the typical characteristics of these jobs, especially in terms of skill and task content?”

For the NLP analysis, we collected comprehensive job post data (including high and lower values) in the translation category, specifically variables including post description and predicted post value. The Upwork Umami team’s value prediction model generates the predicted job post values, and has up to 4% error within the 95% confidence interval. We use broad time coverage instead of just post-ChatGPT public release, since the effects of AI on translation may have materialized even before 2022. We then preprocess the job post descriptions through standard procedures, including tokenization, stopword removal, and lemmatization. To test the correlation between language pair and value, we have created a custom NLP function (integrating off-the-shelf Named Entity Recognition capability of spaCy). Additionally, we have standardized language names and corrected misspellings (any potential errors are detected by lexical similarity scores).  

To analyze key features and values, we first create a feature matrix using Term Frequency - Inverse Document Frequency (TF-IDF), resulting in measures of the relative importance of each term in the corpus. Then, we iteratively use univariate linear regression to estimate the correlation between each term’s importance and the predicted value of a job post, and select the top 10 features with the highest correlation scores. We acknowledge the limitations of this simple method, which can generate noisy results since we treat each key term as a feature without control variables. However, our goal here is to focus on the qualitative ranking of the features instead of the quantitative differences in the scores.

To assess the performance of the prior results, we conduct a lift analysis that compares the significance of each term’s occurrence in high-value versus low-value job posts. The selection of terms used for lift analysis is based on the top terms (at an upper bound of 10,000 terms) based on the prior correlation analysis. After calculating the lift ratios, we bootstrap these values to obtain and reduce standard errors. Finally, we estimate the confidence interval at the 95% level through a normal approximation to the binomial and derive the statistical significance. We also used the ChatGPT enterprise version to classify a sample of translation job posts into the three AI-related use cases. While we use structured prompts to classify these job posts, the classifications may be subject to error. Based on the categorization, we have conducted additional lift analyses that reveal more insights at the sub-group level.

About Victor Maricato

Victor Maricato is a machine learning engineer at Upwork in the Algorithms & Research squad. He focuses on applying state-of-the-art large language model (LLM) techniques to improve user experience. He studied computer science (artificial intelligence) at Heriot-Watt University.

About Ted Liu

Ted Liu is the research manager of the Upwork Research Institute, where he focuses on how work and skills evolve in relation to technological progress such as artificial intelligence. He received his PhD in economics from the University of California, Santa Cruz.

About Yanyi He

Yanyi He is a senior engineering manager (ML) at Upwork, where she leads teams to leverage ML and AI to uplift business metrics at Upwork and solves pain points at scale by using machine learning. She received her PhD in operations research (Excellent Research Award) and a minor in electrical engineering from Iowa State University.

Download your copy

We may communicate with you about the information you've requested and other Upwork services. The use of your information is governed by Upwork's Privacy Policy.

Success Image
Thank You

Your copy of The AI-Driven Future of Translation Work will be delivered to your inbox soon.

Oops! Something went wrong while submitting the form.
Join the world's work marketplace

Recommended research

Join the world's work marketplace

Find great talent. Find great work. Are you ready to move your business or career forward?

Find talent
Find work
Get Started