Five Critical Questions to Ask When Buying or Building AI Solutions

Buy or build? Where's the data coming from? Potential customers of AI and machine learning applications must first do their due diligence by asking the right questions.


Article Key

Image credit: Forrester/Mike Gualtieri 

Emerging technologies such as artificial intelligence have two challenges: Don't let the marketing promises get ahead of actual deliverables; and clarify for customers how to incorporate new, complex capabilities into the enterprise fold. Forrester analyst and VP Mike Gualtieri addressed both issues at a recent industry conference and provided some tips for integrators on how to get started with buying and building AI solutions.

"AI is the future of every enterprise and is a fundamental technology that every enterprise will use in one form or another," says Gualtieri. "Nearly 100 percent of enterprises will have AI by 2025," noting that some will develop AI themselves while others will buy next-gen applications containing AI. Many, he adds, will do both, leading to a $37 billion AI market by 2025, according to Forrester research and projections he shared at the event.

According to Gualtieri, machine learning (ML) is the most popular approach under the broad umbrella of AI applications. And while buying AI applications is similar to buying regular applications, it's not an identical process. "When there's a hot topic [like AI], companies want to get in on it and say they have it in the new version of their software," he says, noting they may sometimes stretch the truth. To cut through some of that opaque marketing, Gualtieri encourages solution integrators to put these questions to their AI vendors:

1. What's the Business Value?

The query may sound incredibly obvious, but Gualtieri said it was surprising how often this central issue gets overlooked when technology buyers start crushing on a technology. Companies need to discern how they'll use ML to make better decisions. Will they use predictive analysis to see who might launch a cyberattack, or which customers are likely to churn? Where will it save time or automate a process to scale? "Evaluate AI apps the same way you evaluate any other software," Gualtieri recommends.

2. Is This Really Productized?

Perhaps it was inevitable that AI would suffer from some vaporware syndrome. And as tough as it may be to hear this, software application manufacturers sometimes overstate their capabilities. They may advertise their new wares as AI-ready or AI-compliant. Gualtieri warns that sometimes these pitches are covertly marketed consulting services, where a provider may be looking to build a custom implementation tailored to highly specific needs. "Both software developers and consultants offer AI solutions. Find out before you buy if what they really mean is that they can build a solution and charge you (more) for it," Gualtieri cautions. "Many of them have lots of good services, but confirm that it's a real AI application first."

Diagram of build vs buy advantages

Image credit: Forrester

3. How Sophisticated Is the AI Technology?

The development phase for ML software is fundamentally different than for traditional software, according to the Forrester analyst. "If you're writing code, you're writing code to analyze data. So find out, does it analyze data to create a machine learning [software] model?" While the vast majority are using a machine learning model, it's then worth investigating the source(s) of the data being used. Did the developers use their own data, external data, or some combo? "Dig in and ask about where the data is coming from. Ask which algorithms they're using, since algorithms analyze data to create the model." These questions will help reveal the level of the technology's maturity and whether they're actually doing ML, Gualtieri says.

4. What Data Is Used to Train the AI Model?

Consider this a followup to the previous questions. Be sure to clarify whether the data that the software vendor used to create the AI model is your data or the vendor's own customer data. Is it open source or proprietary? There's not a right or wrong answer here; what ultimately matters is the customer's requirements for the AI application. "It's important to look at what data is needed for the use case, how it's connected, and where the data comes from," Gualtieri says. Those answers will vary, depending on the application, even when it’s used in the same organization.

5. How Are AI Models Monitored?

The AI model that emerges from data analysis isn't static or intended to last in perpetuity. AI software models are trained on historical data and what's likely to occur in the future, Gualtieri explains. "But models are imperfect and must be monitored since they decay over time," he adds. "You have to watch and re-train them periodically. So make sure the vendor incorporates methods to retrain the model with KPIs."

Answering these questions may lead integrators to reassess whether they're better off buying an AI application or building it themselves for a customer, the Forrester analyst says. "Buying it may feel more risky, but in some cases, you may get a strategic advantage if you build it yourself." In the end, AI scales and automates enterprise intelligence, which has a big payoff. "When you buy AI, understand what it might take to do it yourselves and act accordingly," Gualtieri says.


Intel, the Intel logo, and other Intel marks are trademarks of Intel Corporation or its subsidiaries.