The Way We Work
August 1, 2011 by Brian McDonough

When employers hire remote contractors, the same fear they face with in-house hires — Am I hiring the right candidate? — is often magnified, because hiring and managing remotely is a new skill for some. Remote work, however, offers one solution that generally isn’t available with an in-house candidate: the test hire.

We’ve been advocating the idea of “test driving” a contractor relationship for years. Here’s the nutshell:  Before you hire someone for a job that will span 100 hours, hire her to a two- or three-hour test job that lets you assess her skills, timeliness and communication style. Or test several top candidates and then hire the best for that major project. In general, you get the best return on that investment if the test job is work of actual value — something you can use, rather than merely a skills test.

But there are a lot of ways execute that idea. Since oDesk has about four contractors for every in-house employee, we thought we’d turn to our own managers for tips on how to effectively test prospective workers. After the jump, we’ll talk about web designers, software developers, project managers, product analysts and customer service reps.

How to Test a Web Designer

Shipra Kayan is an interaction designer at oDesk, and she hires web designers and user interface prototypers for long-term relationships.

Shipra Kayan, oDesk

“For those hires, I’ll usually have a project in mind. I break off a chunk of the project that I feel like the candidate should be able to do in five hours,” she says. “I give them the small task and a 10-hour limit. Anything less and you’re really not giving them a chance. I don’t do fixed-price test projects because I want to understand how fast they are at turning things around, what times of the day they work, etc.”

Kayan says she relies heavily on oDesk’s management tools during the test process.  “I stalk the work diaries, and have sometimes already made up my mind even without talking again to the candidate or seeing the end results,” she says. “If things are not going well, I contact them and stop the project immediately, paying them for the hours worked to date.”

How to Test a Project Manager

Matt Cooper, oDesk’s vice president of marketplace operations, says the key to test work is to “closely replicate the work that will be done on the main job.” He uses this technique to hire project managers:

Matt Cooper, oDesk

“A prospective project manager is given a scenario and asked to create a 30-minute presentation on how they would drive the program,” he says. After his typical interview process, which involves a detailed job posting, emails following up on promising applications, and Skype interviews, he hires his top three candidates for the presentation assignment. “In some cases, where it’s a lower-level job, I may test more. I’d say three times the number you need to hire is a reasonable ratio.”

The scenario he gives is actually the one he’s hiring for. This not only lets him identify the best candidate, but to take the best ideas from unsuccessful candidates — he paid for ’em, after all — and incorporate them into the project.

“I usually do the test projects as a fixed price assignment, paying $100 for the presentation, setting the expectation that it should be three to four hours of work,” Cooper says.  “At the hourly equivalent, it’s equal, if not a slight premium, to what they would have earned in the full job.”

How to Test a Software Developer

Matt Cooper isn’t the only person at oDesk hiring developers, but you can’t beat his technique:

“We hire developer candidates on a real two-week project, with one of our existing developers looking over their shoulder,” Cooper says. “They work on real projects under supervision, and at the end of the two weeks, the other developers review their work and either vote them in or out.”

This has the collateral effect of building esprit de corps, making other developers — in-house and remote contractors — feel like they have a real stake in building a successful team, but mainly it gets quickly to the heart of whether a candidate is worthy.

“It’s been very effective,” Cooper says. “You don’t vote to hire someone if you think you’ll need to clean up their mess. And it has a ‘Lord of the Flies’ vibe that I find entertaining.”

How to Test a Customer Service Rep

Mike Barnett runs oDesk’s Customer Service department, and he says that a traditional, discrete test job doesn’t quite do the trick, but a well-governed probation period covers similar ground.

“We have a trainee program,” Barnett says. “Before we hire any rep we conduct an extensive interview over Skype, at least voice and ideally video.”

The customer service manager usually does the first interview, and a team leader — a senior rep out in the field managing the day-to-day workflow — handles the second interview.

Mike Barnett, oDesk

“We’re looking for the following skills: customer-service experience, strong writing and typing skills, a customer-centric mindset, empathy, strong listening skills, problem solving skills, and technical competency,” he says. “We ask them to demonstrate these abilities, or we look for them, during the interview.”

New reps are hired as trainees for 90 days. They’re limited to 20 hours per week and, after initial training, are deployed to help customers. Their team leaders are constantly monitoring and coaching them, and a group Skype chat lets everyone offer help as questions arise. After that 90 days, a trainee is either promoted to CSR Level 1, with the weekly limit bumped up to 30 hours, or the contract is ended. Barnett notes that the strong interview and training processes have ensured that he has never had to replace a trainee at the end of the trial period.

How to Test a Project Analyst

Subha Shetty, our Director of Project Management, hires contractors for a number of jobs, including product analyst, lead generation and documentation. She recently hired someone to evaluate our time-tracking tools.

Shetty notes that she starts with a simple assignment and a low hourly limit, but makes sure to give the contractor everything necessary to succeed. “That includes a list of specific tasks for the short term, a clearly agreed-upon time frame, and a consistent communication pattern.”

To pick someone for the fairly large job of evaluating our time-tracking tools, Shetty assigned a leading candidate to do a portion of the project before deciding to hire him to complete the work.

“I broke down the overall project into chunks and gave him the first chunk as a test,” she said. “It was to select the vendors to do the competitive analysis. I then judged his methodology, analytical ability and actual choice of vendors. I gave him five to 10 hours for this job. This helped me determine the quality of work and deliverable.”

Because she was satisfied with the results, Shetty increased the contractor’s weekly limit and assigned him the rest of the job. She mentions that how many candidates she tests depends on how many strong contenders she finds.  “If it’s a job for which the talent pool is more readily available, like lead generation,” she says, “I test with two to four candidates, giving them the same task and comparing their results.”

Tests for All Tasks

Every job requires a slightly — or dramatically — different test job, but there’s still no better way to see whether an apparently qualified candidate can handle the job than to observe her doing the job. If these test-hire tips aren’t enough for you, come back on Wednesday. We’ll pick the brains of oDesk marketing guru Erica Benton as she tells us the best way to use test jobs for key marketing positions, such as writers, public relations specialists and social media experts.

Meanwhile … tell us about how you test-hire candidates, or if you’re a contractor, tell us about the best experiences you’ve had with test jobs. The comments are open!

Brian McDonough

Freelance Writer

Brian McDonough has been a writer and editor for more than 15 years, and has managed teams of in-house and freelance writers for newspapers, magazines and web sites.