Behind the Scenes: Modernizing Upwork’s Infrastructure Takes Time and Patience
Making large-scale changes to any system takes a lot of planning and work—and it doesn’t always go smoothly. Following on the heels of last year’s upgrade, when we updated Upwork’s application server, we decided to migrate Upwork’s databases—the key infrastructure that helps support the 3 million+ projects posted on Upwork every year.
We know the frequent maintenance needed over the past few months to do the migration has been disruptive and we appreciate your patience. Here’s a look behind the scenes at what we’ve been doing and what you can expect from Upwork in the near future.
Migrating the production environment
Last April, Upwork did a massive server migration to improve site speed and stability. We were leaning hard on a production environment that was split between an aging data center and Amazon Web Services (AWS); our aging production database environment simply could not support Upwork’s business.
The results have made a better experience possible:
- We’ve doubled Upwork’s processing speed: The modernized application server can process requests in less than 200 milliseconds. Before the move, it took more than 500 milliseconds. That means a faster, smoother experience when you’re on the site.
- We also made a few minor architectural changes that provide more resiliency in case of server failures. Tech is always changing and these changes help keep Upwork ahead of the curve.
To continue building Upwork’s capacity to deliver a great experience to the freelancers and clients that use our site, we started to improve our database capacity and reliability. Then we ran into a few challenges.
Upwork’s databases have been running on an outdated environment, and we felt a more modern service—Amazon Relational Database Service (Amazon RDS)—would give us the technology and capacity needed to support the Upwork site.
The challenges? A lot of the configurations we had were no longer supported by AWS RDS. As a result, before we could even start the migration, our engineers needed to implement new methods and workarounds.
We’ve taken more than six months to identify, analyze, plan, and test our database upgrade. At the same time, we’ve taken steps to improve different aspects of our infrastructure, such as security, data protection, and system reliability. And we’ve also been cleaning up by isolating the more critical databases so we can provide those with better monitoring and performance optimization.
We realize there is a disruptive element that comes with any site update, especially for a large marketplace like Upwork. We also know that many clients and freelancers around the world rely on our site—and the data that supports daily activity—to get work done which makes updates crucial and necessary going forward.
We’re very close to finishing the database maintenance, and we anticipate maintenance will be less frequent in the second half of this year. Keep an eye on Upwork’s status page, where we’ll share the upcoming maintenance schedules to give you as much advance notice as possible.
Wilson Chong, director of Site Reliability and Database Engineering, is in charge of database infrastructure and site reliability for Upwork. He has been building and leading infrastructure development, automation, and site reliability for the past 10+ years.View Wilson Chong’s other articles