AI Expert Jennifer Davis on Why We Always Need Humans Behind AI

Freelance professional Jennifer Davis is sort of a golden AI unicorn to organizations—but she didn’t intend to pursue a career in AI. Not long ago, Davis was progressing successfully in her dream career as an academic in cancer research. She was bitten by the supercomputing bug during her postdoctoral research fellowship, which led to a fellowship in systems and computational biology. Then, after a long time in academia, she decided to broaden her experience.

Jennifer left full-time academia to apply her skills to practical business approaches in real business scenarios. She “dabbled” in consulting, as she put it. Soon, Accenture noticed her work and asked her to join their data science team. Her skills caught the attention of the IBM Watson team before long, and they convinced her to join their efforts.

The upside of working for large consultancies was that she worked on interesting projects. The downside was that she traveled 80% of the year and it was wearing her down. Eventually, she left the corporate track to become a full-time freelance professional through Upwork.

“Upwork is perfect for me,” Davis said. “I get to work on so many different projects and work on diverse teams so I don’t get bored, and I don’t have to travel six days a week. I also like the ability to take on projects that are a good fit. I want to make sure that I have a unique skill set that will fit a client’s unique problem.”

The value Davis delivers can be sizable. One client was using a computer vision platform for tumor detection, which required a high volume of training data. The company implemented a distributed computing technique to reduce training time from weeks to hours. Although they set up the system for speed, it was still taking too long to train, so they brought Davis in to figure out the problem.

“They handed over a hundred files of Python code to me,” she said. “I discovered they were missing one line of code in one of their files, which exponentially increased the time to train the algorithms.” She showed the company how to correct the problem, saving them over $5 million a year.

Avoiding shiny AI traps

Her wide technical knowledge and business experience enable her to see when companies are chasing after shiny AI objects that won’t help them stay competitive with the rest of the market.

“I teach clients about categories of AI that might make sense to them. Like how to use AI to automate billing and HR processes and use elegant tutoring to make sure employees are regularly upskilling. AI touches a lot of different areas and I try to help them think through what's actually going to be helpful and practical, and avoid the bad stuff like ChatGPT hallucinations.”

That was the case on a project for a major pharmaceutical company that used language models for computational chemistry to develop drugs. Davis made an assessment of the existing neural network and found that a large language model (LLM) wasn’t necessary. The lesson here: “Although LLMs are the most popular form of generative AI, they’re not actually the most effective in every scenario,” said Davis.

How AI makes companies vulnerable

After speaking with many organizations across industries from defense to finance, Davis observed that most companies rely too heavily on AI. They think that AI will solve all of their problems. And when a product’s really good, such as facial recognition, people think the product can run by itself without a human in the loop.

“But we’ll always need humans behind the AI. No product is 100%,” Davis cautioned. “That just doesn't happen because it's overtrained or overfit. And when a product is overtrained, it will perform very badly on new data. So, you always need humans to monitor the algorithm.”

She said a perfect example of why humans should always monitor AI has occurred with companies that provide soft credit pulls. A soft pull is when someone is authorized to check your credit report. The check doesn’t affect your credit because it’s not tied to a specific application.

She explained, “What companies providing soft pulls are actually doing is they’re taking a variety of publicly available information and creating a credit scoring algorithm based on that.

“Let's say you have only one class or category of person for your initial training data on this algorithm that's going to score people's credit. Soft pulls are really helpful because the more data you get in, the greater the diversity in that data. As more areas of the country are represented, the greater your gender, cultural, and ethnic representation. That’s when data drift occurs.

“What do I mean by data drift? Your algorithm no longer predicts correctly because your data has changed.

“So if you're not monitoring your algorithm for data drift, which is part of what MLOps (machine learning operations) is about, you have not only started off with a skewed, somewhat unfair algorithm, but it also gets worse with time. And then whoever's paying you for these soft pulls, like mortgage companies, will eventually figure out that your soft pull credit scoring algorithm is worthless.”

She knows of situations where over-relying on AI resulted in heavy fines or led to a lawsuit. And she’s determined to help businesses prevent those risks.

Leveraging the galactic potential of AI—safely

To protect against the vulnerabilities that can result from using AI, Davis believes every business should have a healthy MLOps program. What scares her the most is that most organizations jump into AI without having an MLOps function in place because they either don’t know how to do MLOps well, or they don’t understand the importance of it.

Jumping into AI without the safeguard is dangerous because, in addition to maintaining the integrity of your data, MLOps keeps AI ethical.

Davis recounted an article where the authors showed how easily someone could break into a pathologist's computer vision program that was used to score and grade tumors. And how a bad actor could use a machine learning algorithm to do the opposite of generative AI by removing a tumor from an X-ray, so it looked like there was no tumor. They could also do the opposite by putting a tumor on an X-ray when one didn't exist.

The story is a sobering reminder that “there are a lot of bad actors out there. And what’s scary is they’re not just breaching systems, but they also have reverse engineering control,” emphasized Davis.

Stories like these fortify her belief that responsible AI is essential for its long-term success and positive impact on society. “I aim to use my work in AI to contribute to developing and implementing ethical, trustworthy, and explainable AI solutions.” To that end, she focuses on creating AI systems that are transparent, accountable, and aligned with human values.

“I aim to use my work in AI to contribute to developing and implementing ethical, trustworthy, and explainable AI solutions.”

Before that can happen, she needs to get key business stakeholders on board and help them strategize the next steps. In typical Davis fashion, she educates clients by conveying the information in a way that’s non-threatening and understandable for all audiences.

“Having soft skills is really important when you're bringing up stressful or difficult topics. I try to understand the person I’m talking to by asking good questions, uncovering their motivations and their interests, and noticing who wants more technical details and who wants the business implications.”

What you should know before hiring an AI specialist

Davis suggests that before hiring an AI professional, make sure you’re familiar with your flavor of AI. “Generative AI has become a misnomer in a sense. For that reason, you need to be super careful about whom you're hiring,” she advised.

“A lot of people think that ChatGPT and DALL-E are all there is to generative AI, but GenAI crosses a lot of types of machine learning. Some of it has to do with reinforcement learning. Some of it is sequence-to-sequence language models or adversarial networks.” Of course, if you’re not sure who you need for a project, you could schedule a consultation with Davis for guidance.

Depending on your internal resources, you may not need to contract an external AI specialist to do all of the work. Davis found that “many companies have excellent employees who are eager to learn. If you want to be more cost-effective on a project, it’s good to bring in an expert like myself to give someone a bit more junior some coaching and guidance on their project. It’s a win-win situation because they get to flourish in their careers as well.”

Inspired to solve the impossible

“I like solving problems because I believe there’s always a solution,” said Davis. Even more exciting is seeing her clients succeed. “I love getting through a really difficult project and having them say, ‘Wow, Jennifer, I didn't think this would ever work, but it does!’ That's the reward in itself.”

She’s also fueled by the potential AI opens up for human creativity and speed in solving urgent issues. “There are problems that we really can't afford to wait decades to solve, like the drought problem in Austin, Texas, where I live. AI is helpful because it accelerates modeling so we can find solutions faster.”

Concentrating on the present to create a better future

Davis applies her problem-solving mindset to squeeze the most fulfillment from her personal life too. “Many of us get caught up in what we think we need and we just start accumulating stuff. But stuff doesn’t make me happy. What makes me happy is bringing a little happiness to someone else’s life. They could be little things like caring for the tree in my front yard because my neighbors say they get pleasure from staring at it.

“Believe it or not, I Airbnb the two extra bedrooms in my house. I was looking at them one day and thought, what am I going to do with them, rotate rooms every night? So it made sense to do something fun with them. I just love having guests. I get to meet all these different people, make new friends, learn about new cultures, make someone feel welcomed and loved in my home, and hear their stories.”

Davis spreads her uplifting influence further by teaching a post-graduate course on deep learning and neural networks at Columbia University. “Teaching provides me with a unique opportunity to empower and inspire the next generation of AI practitioners. The most satisfying aspect for me is witnessing their transformation as they grasp complex concepts in deep learning, neural networks, and machine learning.”

In addition to software engineers, many of her students are C-level executives. She said, “Seeing them apply these principles to real-world scenarios is truly rewarding. I love knowing that they’re contributing meaningfully to the AI landscape and driving innovation positively.”

Her dream project

Davis wants to further the positive potential of AI by helping large organizations set up and run an innovation center for AI. “I love my startup clients, but working with a technical team and senior leadership on a large project like that would be a dream come true.”

She had a taste of that dream when helping a company’s data scientists level up their technical skills. “It was nice because my past experiences enabled me to translate problems back and forth between technical and business aspects. I had a mix of really doing the hands-on, technical, and coaching work in MLOps. Then I translated all of that information to senior leadership and answered their questions, listened to what they wanted to do, then translated it back to the data scientists.”

Until her dream project arrives, she wants to help more companies institute safe practices with their machine learning and AI. “Organizations must be very mindful of the ethical side of all of this because machines don’t have ethics; humans do. There should always be a human in the loop, no matter which AI you're using.”

Davis offers companies several ways to work with her. Through Upwork, you can book a consultation. purchase a preset project through Project Catalog™ or invite her to work on one of your projects.