26/10/2017 Planning for AI - O'Reilly Media

ON OUR
RADAR
AI BUSINESS DATA DESIGN ECONOMY OPERATIONS SEESECURIT
ALL

AI FOLLO W THIS TOPIC

Planning for AI
What you need know before committing to AI.
By Mike Loukides. October 17, 2017

Winding road (source: PublicDomainPictures.net)

For more, check out the Implementing AI track from the Articial Intelligence Conference
2017 in San Francisco.

Do you have your AI strategy? If you don't, be prepared to lose. Or at least, so say the
consultants, tech journalists, and pundits. You can't possibly be a competitive modern
company without an AI strategy.

We are the last people to say that AI isn't important, or that having an AI strategy isn't
a good thingor even that, if you dont start thinking about AI initiatives now, youll
end up behind. Articial intelligence is a game changer: its a revolutionary cluster of
technologies that has the potential to make fundamental changes in how we live and
work. However, much of what we read gets the cart before the horse. An AI strategy, if
https://www.oreilly.com/ideas/planning-for-ai?imm_mid=0f75dc&cmp=em-data-na-na-newsltr_ai_20171023 1/19
26/10/2017 Planning for AI - O'Reilly Media

it's just an AI strategy, doesn't get you very much. An AI strategy thats just an AI
strategy is a weird managerial superstition: pour magic AI sauce over everything, and it
will be awesome.

It needs to be said, though it should go without saying: don't build an AI strategy

without rst thinking about your business objectives. Better: incorporate AI into your
business strategies rather than building an AI strategy. Think about how AI can help
you achieve your goals; dont make it a goal in itself. On her blog Quam Proxime,
Kathryn Hume counsels established enterprises against the temptation to become AI-
rst. Instead, they should be something-else-rst-with-an-AI-twist. Enterprises
need AI systems to work smart, to take advantage of their data, to learn about and
improve on their past performance. Enterprises dont need AI to become something
new that they dont yet understand. They need AI to build on the strengths they
already have, and become what they already are, but better. Thats how to be
innovative and transformational; and if, along the way, you come up with ideas and
products that disrupt your current practices (and your industry), so much the better.

Get O'Reilly's AI newsletter

Email Address Subscribe

We protect your privacy.

Start building that strategy by asking what, precisely, you want to accomplish. Do you
want to improve customer experience? Do you want to improve internal processes like

https://www.oreilly.com/ideas/planning-for-ai?imm_mid=0f75dc&cmp=em-data-na-na-newsltr_ai_20171023 2/19
26/10/2017 Planning for AI - O'Reilly Media

HR? Do you need some insights into product design, or are their features in your
product that could take advantage of AI? Are there specic processes and tasks that
are error-prone, repetitive, and just plain boring, or where some assistance can make
your sta more eective? AI can help you accomplish all of these things and much
more, just like a hammer can be used to build any kind of house. But to succeed, an AI
strategy has to be part of an overall business plan. Whether youre improving your
current business or building out a new business, AI should serve your business plan,
not the other way around.

As youre building AI into your business plan, youll need to think about what AI can
and cant do, the foundational work that will allow you to build a successful AI
program, the problems youll encounter as you build your application, and more. AI
can do great things; but before you and your team can make it do great things, you
need to look at it realistically and understand the challenges. For example, AI has
proven to be really good at classifying things (for example, tagging pictures); but it
really cant tell you much about whats in those pictures. In The Seven Deadly Sins of
AI Predictions, Rodney Brooks points out that, while AI is really good at identifying
pictures of people playing Frisbee, it cant tell you whether Frisbees are good to eat,
whether infants can throw them, and many other questions. In a business context, an
AI application might be able to tell you whether customers look happy when they see
a new product design, but they cant answer a larger question, like whether the
product will be a success.

James Cham, in Machine Learning and AI: An Investment Perspective, says the
biggest risk is that we as managers will make really bad decisions about where to
invest, and we'll end up wasting billions of dollars on stupid projects that nobody ends
up caring about. Your goal is not to be that manager.

What is AI, anyway?

You can read hundreds of articles dening articial intelligence and machine learning.
All the denitions will be somewhat dierent, and all will overlap. Were not
interested in that debate; John Pavlus notes that we use articial intelligence and
machine learning interchangeably, for better or for worse, and that we would be
better o abandoning the term altogether and just talking about automation. Its a
good point; if youre automating a process eectively, it doesnt matter whether
youre using a neural network, a rule-based system, or an older but simpler technique.

https://www.oreilly.com/ideas/planning-for-ai?imm_mid=0f75dc&cmp=em-data-na-na-newsltr_ai_20171023 3/19
26/10/2017 Planning for AI - O'Reilly Media

Nevertheless, its still necessary to start with a (brief) denition of AI. Heres mine. AI
is characterized by output that isnt strictly dependent on the input or on the
algorithm: the output of an AI system depends critically on a training process, in which
the program learns how to perform its task. Training dierentiates AI from traditional
software applications and data analysis.

What does AI do well?

Its easy to be misled by almost apocalyptic statements about what AI can (or might)
do. But youre planning a business strategy for now, or a year from nownot decades
from now. So, you have to think about the reality of what AI can do.

S A FA R I

Learn faster. Dig deeper. See farther.

Join Safari. Get a free trial today and nd answers on the y, or
master something new and useful.

Learn more

Solving specic problems

AI is very good at solving well-dened, specic problems. AlphaGos victory over Go

expert Lee Sedol was impressive because playing Go well is extremely dicult, and
the best guess was that we were a decade away from machines that could beat a
player of Sedols calibre. But forget about statements like There are 10170 potential
Go games. The number of possible games is unimportant. Whats important is the
problem itself: win a game with very simple, explicit rules. Thats a well-dened and

https://www.oreilly.com/ideas/planning-for-ai?imm_mid=0f75dc&cmp=em-data-na-na-newsltr_ai_20171023 4/19
26/10/2017 Planning for AI - O'Reilly Media

very specic problem. Were not asking for a system that can win any board game, or
that can transfer its knowledge of one game to another, or that can choose whether it
would prefer to play Go, Chess, or Checkers. Building a system that can win is neither
simple nor easy, but for the purposes of business planning, thats less important than
the statement of the problem itself.

Think about autonomous vehicles, under development by Waymo (Google/Alphabet),

Uber, Tesla, and others. Driving a car is not a well-dened, specic problem. But it can
be broken down into well-dened, specic problems: planning a route, identifying
road signs and signals, identifying obstructions (including other vehicles and
pedestrians), detecting skids, managing the brakes, and so on. None of these
problems are easy to solve, though theyre solvable. Whats more important is that
theyre well-dened, and thats why theyre solvable.

The same principle applies if youre a small startup. Prospera analyzes images of
tomatoes to detect insect infestations, disease, and other problems that reduce yield.
This is a specic, well-dened problem. Prospera knows what sick tomato plants look
like, so it can label images for training; it knows where to place the cameras to get
good images of the plants; it knows how to collect data from the cameras reliably. The
results may seem like magic, but that magic only happens because Prospera chose a
well-dened problem, studied that problem carefully, and did the hard work that
enabled them to solve it.

Similarly, Chorus.ai listens to sales calls, transcribes the calls and annotates the calls
with action items and important topics that come up. Chorus isnt trying to build a
machine that makes sales, a goal thats neither well-dened or specic. Its
transcribing a conversation (a dicult but fairly well-understood problem), and
looking for specic signals that indicate action items in that transcription. Its an
assistant to the salespeople, performing routine but necessary tasks. Again, the
problem theyre solving is well-dened and specic.

Its not surprising that breaking a problem into smaller parts makes it easier to solve;
thats what engineering is all about. Lets say you want to build an AI application to
help customer service agents; after all, your high-level business goal is to improve
customer satisfaction. What might that look like? Improving CS is a big problem thats
hard to dene precisely. Here are some smaller, more specic steps to an AI solution:
the system needs to:

transcribe questions from the customers

https://www.oreilly.com/ideas/planning-for-ai?imm_mid=0f75dc&cmp=em-data-na-na-newsltr_ai_20171023 5/19
26/10/2017 Planning for AI - O'Reilly Media

transcribe answers given by customer service representatives

index and tag the questions and answers

store the questions and answers in a database

transcribe new questions and decide which features of those questions are
important

retrieve appropriate answers from the database

present the answers to customer service agents, who in turn present them to the
callers.

These are all simpler problems that can be solvedperhaps not solved easily, but
solved nonetheless. In real life, you might need to break several of these problems
down into even simpler, more specic tasks. Regardless of your problem, or the path
you take to solve it, building an AI solution will require you to back o from bold,
futuristic plans and break the problem down into well-dened and specic problems.

Augmenting humans

The key to using articial intelligence eectively is to augment and assist humans, not
to replace them. In The Fatal Flaw of AI Implementation, Jeanne Ross writes,
"Companies that view smart machines purely as a cost-cutting opportunity are likely
to insert them in all the wrong places and all the wrong ways." If you try to use AI to
replace humans, you're bound to make poor decisions about how to use it.

As Ross points out, using AI for fraud detection will probably increase the number of
fraud cases you discover. You don't need as many people to sift through the data
looking for anomalies, but you do need people to handle the cases you nd.
Furthermore, the additional cases an AI application discovers are likely to be the
hardest and more dicult than the cases you found using your older methods. AI
eliminates the dull, repetitive part of the job, but you may require more sta (and
more highly skilled sta) to deal with the less routine, more creative parts.

What makes AI hard?

If youre going to build an AI strategy, you have to think about what makes AI dicult.
Otherwise, youre starting up a major project without understanding the costs.

https://www.oreilly.com/ideas/planning-for-ai?imm_mid=0f75dc&cmp=em-data-na-na-newsltr_ai_20171023 6/19
26/10/2017 Planning for AI - O'Reilly Media

Data practices

First and foremost: AI products are data products. Training your AI product requires
dataand probably a lot of data. The somewhat harsh reality is that youre unlikely to
have useful data if you dont already have solid data practices in your organization. In
The AI Hierarchy of Needs, Monica Rogati spells out the steps needed to build an AI
practice: identifying data sources, building data pipelines, cleaning and preparing
data, identifying potential signals in your data, and measuring your results. And
Andrew Ng, in his OReilly AI Conference keynote, says that organizations that succeed
with AI will master strategic data acquisition: they wont just have data; they will know
how and where to get more.

An organization that is using data eectively probably has most of the necessary
infrastructure in place. Even if your AI aspirations arent directly related to your
current data eorts, youve gured out what needs to be done and have the
infrastructure ready. However, many organizations with AI aspirations dont have good
data practices, even if think they do. Take a careful and skeptical look at how you
currently use data as part of building your AI strategy.

Training and retraining

AI projects need to be trained before they can do any useful work. Training is
essentially running the program on a set of known data and letting it build a model
(essentially, tweaking internal parameters automatically) until it gets adequate results
on your test data. Then you run it on another set of test data, one that it hasnt seen
before, to see whether the results are still satisfactory. Training goes hand-in-hand
with writing and debugging the softwareand its quite likely that more work will go
into training than the rest of the development process.

Therefore, if youre planning an AI project, you need to account for training time. You
also need to be aware of how training can go wrong. The pitfalls include:

Overtting: If your AI application achieves 100% accuracy on your training data, are
you done? Is it time to break out the champagne? Unfortunately, no: if you get
100% accuracy on training data, your application has probably memorized the
training data, and will most likely perform terribly on real-world data. This is called
overtting; its a constant problem. On the other hand: since your application wont
be 100% accurate on training data, it will never be 100% accurate on real-world
https://www.oreilly.com/ideas/planning-for-ai?imm_mid=0f75dc&cmp=em-data-na-na-newsltr_ai_20171023 7/19
26/10/2017 Planning for AI - O'Reilly Media

data. Thats life. Some applications require much more accuracy than others: 99% is
not good enough for an autonomous vehicle, but 60% accuracy may be all you need
for a consumer app thats recommending additional purchases.

Bias: If your training data is biased, your applications results will be biased. That
seems obvious, but its easy for bias to sneak in unnoticed. Digital cameras are
heavily optimized to perform well on light skin, and do poorly with dark skin, so
applications like automated face recognition frequently produce incorrect results.
There are fewer women in technical jobs, so AI systems tend not to show technical
job postings to women. Researchers recently discovered that natural language
processing software couldnt process text that contained Afro-American slanga
signicant problem if youre trying to automate demographic research. And so it
goes. Bias can have legal consequences, and the AI did it isnt likely to play well in
court.

Retraining: Its easy to think that training is a one-time, set and forget thing. It
isnt. Customers change, business conditions change, products change: any change
in your environment can have an eect on your application. You may not notice, but
its performance will gradually degrade over time. If youre planning an AI project,
you need to account for retraining.

Liability issues

AI systems have a deserved reputation for being inscrutable. They can give you
results, but frequently can't tell you why they gave a specic result. You often don't
care, but in many applications, such as medical imaging, the why is as important as the
what. In some situations, inability to explain a result can be a legal liability.
Developing AI systems that can explain their results is an area of ongoing research.

If the diculty of doing AI hasn't scared you away, youre ready to start planning your
AI project. SeriouslyAI presents some big challenges. But, as Hume says, dont let
the perfect be the enemy of the good. These are all challenges that you can meet,
provided you dont start your project blind. Build a good data team, and develop good
practices for working with data. Allow adequate time for training. Be aware of the
problems youll encounter along the way. Theres nothing magic about AIand
likewise, theres nothing magic about what can go wrong.

Getting ahead of the curve

https://www.oreilly.com/ideas/planning-for-ai?imm_mid=0f75dc&cmp=em-data-na-na-newsltr_ai_20171023 8/19
26/10/2017 Planning for AI - O'Reilly Media

AI is moving forward at speeds that seem almost inconceivable. Every week, if not
every day, there are articles about new developments: new techniques, new hardware,
new optimizations, and more. Can you aord not to get on the AI train?

You cant. But dont get on the train blindly; know where youre headed before buying
that rst-class ticket. Treating articial intelligence as a magic sauce that will make
everything better will lead to expensive mistakes. Understand what youre doing, why
youre doing it, and the limitations you face: both the limitations of AI itself, and the
limitations of your organization.

If you do that, youll be prepared to take advantage of AI.

For more, check out the Implementing AI track from the Articial Intelligence Conference
2017 in San Francisco.

Article image: Winding road (source: PublicDomainPictures.net).

Share Tweet Share 22 Share 600

Mike Loukides
Mike Loukides is Vice President of Content Strategy for O'Reilly Media, Inc. He's edited many
highly regarded books on technical subjects that don't involve Windows programming. He's
particularly interested in programming languages, Unix and what passes for Unix these days, and
system and network administration. Mike is the author of System Performance Tuning and a
coauthor of Unix Power Tools. Most recently, he's been fooling around with data and data
analysis, languages like R, Mathematica, and Octave, and thinking about how to make books soc...
more

https://www.oreilly.com/ideas/planning-for-ai?imm_mid=0f75dc&cmp=em-data-na-na-newsltr_ai_20171023 9/19
26/10/2017 Planning for AI - O'Reilly Media

AI

Highlights from the O'Reilly AI Conference in New York 2016

By Mac Slocum
Watch highlights covering articial intelligence, machine learning, intelligence engineering, and
more. From the O'Reilly AI Conference in New York 2016.

AI

https://www.oreilly.com/ideas/planning-for-ai?imm_mid=0f75dc&cmp=em-data-na-na-newsltr_ai_20171023 10/19
26/10/2017 Planning for AI - O'Reilly Media

How AI is propelling driverless cars, the future of surface transport

By Shahin Farshchi
Shahin Farshchi examines role articial intelligence will play in driverless cars.

AI

Untapped opportunities in AI
By Beau Cronin
Some of AI's viable approaches lie outside the organizational boundaries of Google and other large
Internet companies.

AI

https://www.oreilly.com/ideas/planning-for-ai?imm_mid=0f75dc&cmp=em-data-na-na-newsltr_ai_20171023 11/19
26/10/2017 Planning for AI - O'Reilly Media

Small brains, big data

By Jeremy Freeman
How neuroscience is beneting from distributed computing, and how computing might learn from
neuroscience.

ABOUT US SITE MAP

Our Company Ideas

Teach/Speak/Write Learning

Careers Topics

Customer Service All

Contact Us

https://www.oreilly.com/ideas/planning-for-ai?imm_mid=0f75dc&cmp=em-data-na-na-newsltr_ai_20171023 12/19
26/10/2017 Planning for AI - O'Reilly Media

2017 O'Reilly Media, Inc. All trademarks and registered trademarks appearing on oreilly.com are
the property of their respective owners.

Terms of Service Privacy Policy Editorial Independence

https://www.oreilly.com/ideas/planning-for-ai?imm_mid=0f75dc&cmp=em-data-na-na-newsltr_ai_20171023 13/19
26/10/2017 Planning for AI - O'Reilly Media

https://www.oreilly.com/ideas/planning-for-ai?imm_mid=0f75dc&cmp=em-data-na-na-newsltr_ai_20171023 14/19
26/10/2017 Planning for AI - O'Reilly Media

https://www.oreilly.com/ideas/planning-for-ai?imm_mid=0f75dc&cmp=em-data-na-na-newsltr_ai_20171023 15/19
26/10/2017 Planning for AI - O'Reilly Media

https://www.oreilly.com/ideas/planning-for-ai?imm_mid=0f75dc&cmp=em-data-na-na-newsltr_ai_20171023 16/19
26/10/2017 Planning for AI - O'Reilly Media

https://www.oreilly.com/ideas/planning-for-ai?imm_mid=0f75dc&cmp=em-data-na-na-newsltr_ai_20171023 17/19
26/10/2017 Planning for AI - O'Reilly Media

https://www.oreilly.com/ideas/planning-for-ai?imm_mid=0f75dc&cmp=em-data-na-na-newsltr_ai_20171023 18/19
26/10/2017 Planning for AI - O'Reilly Media

https://www.oreilly.com/ideas/planning-for-ai?imm_mid=0f75dc&cmp=em-data-na-na-newsltr_ai_20171023 19/19