Product Policy Chapter 8 The Concept/Evaluation System Pooja Israni

1

Chapter 8. The Concept Evaluation System
Introduction.
Previously, we have studied in the part two various methods of generating new product
concepts. Therefore, now a system of evaluation is needed. In the new product development
process, we are doing evaluations, and there are evaluation techniques. Presenting models
such as the cumulative expenditures curve and the A-T-A-R model that helps us decide which
evaluation technique to use.
Whats going on in the new product process?
Goods that appear complex are just collections of metal shapes, packaging material, fluids,
prices, etc.
The Quality control evaluating each part before releasing it to the next step, you have the idea
of a new product evaluation system. The new product appears first as an idea, a concept in
words or pictures, and we evaluate that first. As workers turn the concept into a formed piece
of software (good), or a service, that good or service is then evaluated. We evaluate the
product and its marketing plan as separate and divisible pieces let us telescope the
development process.
The Evaluation System for the Basic New Products Process.

The overall purpose of evaluation is to guide us to profitable new products.
In the following chart the ideas become concepts; concepts get refined, evaluated, and
approved, development projects are initiated; and products are launched. Throughout,
Product Policy Chapter 8 The Concept/Evaluation System Pooja Israni
2

different questions need to be asked, and different evaluation techniques provide the required
answers.
First evaluation precedes the product concept. Phase I, when an opportunity or threat is
identified, a judgment was made that if the firm tried to develop a new product in a given
area, and it would probably succeed. Where should we look, what should we try to exploit,
what should we fight against? The tool is opportunity identification and evaluation. In phase II
(concept generation), ideas begin to appear, and the purpose of evaluation changes: now the
goal is to avoid the big loser or the sure loser. We want to take them out and spend no added
time and money on them. This step is essential if we are focus limited resources on the worth-
while concepts.
The initial review segment tries to spot the potential big winners. Most good concepts are just
good concepts. A few are great, and we recognize them as soon as possible. These get added
effort, usually in the form of complete concept testing and development program.
Phase III (concept/project evaluation) and the decision on whether to send the concept into
full-scale development. If the amounts to be spent make it an important decision, will benefit
from a very thorough scoring model application.
Phase IV (development) technical and marketing activities are done. Have we got what we
want? Is this part ready? Is that system subset cleared for use? Does it not only work, but
produce what the customer needs? A protocol check tells us whether we are ready to develop
a product for serious field testing.
Phase V (launch) able to make, and market, the item on a commercial scale. Resolved by some
form of market testing.
Product line considerations in concept evaluation.
At the early phases, of the new product process, there are risks involved in making project
selection decisions. Depending on the evaluation, the firm may let through too many bad ideas
or reject good ideas.
The cumulative expenditure curve

Product Policy Chapter 8 The Concept/Evaluation System Pooja Israni
3

New product evaluation system flows with the development of the product. What evaluation
occurs at any one point depends greatly on what happens next. A gradually upward-sloping
curve represents the accumulation of costs or expenditures on a typical new product project
from its beginning to its full launch.
The early expenditure is representative of product development in technical fields. R&D is the
big part of the cost package, and marketing costs are relatively small. The lower curve shows
opposite type of firm.
The risk/payoff matrix




Cells AA and BB are correct decisions.
Cells BA and AB are errors, but they have different cost and probability dimensions.
Usually BA (the go error) is much more costly but dont forget opportunity costs!
Consider how new-to-the-world the product is as that has an impact on the risk
level.
At any single evaluation point in the new product process, four situations are shown. Two
outcomes (success or failure) and two decision options (move or drop off/kill the project).
There are four cells in the matrix.
The AA cell and the BB cell are fine; we drop a concept that would ultimately fail. Or we
continue on a concept that would ultimately succeed. AB is a drop error: a winner is
discarded. BA is a go error a loser is continued to the next evaluation point.
So error AB is much worse than BA because the ultimate profits from winning product are
bound to be much greater than all of the development cost combined.
The exception, is the opportunity cost, a manager must think of the matters (if there are
enough funds to enhance the project) when deciding what evaluation to do. If the net costs of
the next step in any situation are low, then a decision will probably be made to go ahead,
perhaps with very little information.
Example: P&G supported both Febreze (an odor elimination) and Dryel (dry clean clothes at
home) with extensive market testing, including lengthy test marketing, as they were seen as
risky, new-to-the world (also new to P&G). P&G has been in the sector of detergents and
knows about the market it supports detergents line with much less extensive testing, as that
would be considered a much less risky launch.
New products team should consider four generic risk strategies:
Decision A
Stop the Project Now
B
Continue to Next Evaluation
A. Product would fail if
marketed


AA

BA
B. Product would succeed if
marketed


AB

BB

Product Policy Chapter 8 The Concept/Evaluation System Pooja Israni
4

Avoidance: Eliminate the risky product project altogether, through an
opportunity cost is uncured
Mitigation: reduce the risk to an acceptable, threshold level, perhaps through
redesigning the product to include more backup systems or increasing product
reliability.
Transfer: move the responsibility to another organization, in the form if a joint
venture or subcontractor.
Acceptance: develop a contingency plan now or deal with risks as they come
up.
The Decay Curve.

The risk matrix decisions lead to the idea of a decay curve, the percentage of any firms new
product concepts that survive through the development period, from the 100 percent starting
out before concept testing to the 2 percent going to market. Discarded 98 percent drop off at
various times during the process, and when they drop off is primarily determined by the
analysis of the risk matrix.
Decay curve C is from a leading company in the paper industry that wanted to kill off all
possible losers early and spend time developing only tose proposals worthy of marketing. Their
evaluation system implemented it faithfully. Decay curve A represents a service firm that had
very low development costs and wanted to drop a project only when there was solid evidence
against it.
The decay curve is partly a plan and partly a result. The two should be sychronized. Its value as
a managerial concept in helping the manager see the need for thinking through the stream of
development costs and the risk/payoff matrix for each new product concept as it starts its
journey through development.




Product Policy Chapter 8 The Concept/Evaluation System Pooja Israni
5

Planning the Evaluation System
Everything is tentative
Even up through marketing. Form can usually be changed, and so can costs, packaging,
positioning, and service contracts; so can marketing date and reactions of government
regulators.
This means two long-held beliefs in new product are actually untrue. One is everything should
be keyed to a single Go/No go decision. One decision can be critical.
The financial analysis should be done as early as possible to avoid wasting money on poor
projects. This leads firms to make complex financial analyses shortly after early concept
testing. The financial analysis is best built up piece by piece.
Rolling evaluation is another tentative matter is the marketing date. Marketing begins very
early in the decelopment process. Rollouts are now so common ir is hard to tell when all-out
marketing begins.
The project is being assessed continuously, premature closure is avoided, participants avoid
mindsets of good and bad. Dealing with risk via acceptance or mitigation. Product
development projects are risky, so we evaluate, continuously upgrade the quality of
information available throughout the process to minimize the chances of failure and deal with
them as they come up.
Potholes.
One critical skill is the ability to anticipate major difficulties, the potholes of product
innovation. Are always a problem, but they only become costly when we fail to see them
coming in time to slow down or steer around them, carefully scan for the really damaging
problem and keep them in mind when we decide what evaluation we will do. If it is deep
enough, the development team seriously consider the risk avoidance option: drop the project!
The people dimension
Product developers dealing with people, people cause problems.
Evaluation system should contain early testing that is supportive. Concept testing is sometimes
called concept development to reinforce the idea of helping the idea, not just killing it off.
Shows how difficult it sometimes is to kill off marginal products late in development. Another
people problem relates to personal risk. All new product work has a strong element of risk
risk to jobs, promotions, bonuses. A good evaluation system, built on a through understanding
of the new item will follow as its way through development, protects developers from these
pressures. Be supportive of people and offer the reassurance that players need.
Surrogates
Surrogate questions give us pieces of information that can substitute for what we want to
learn but cant.
Product Policy Chapter 8 The Concept/Evaluation System Pooja Israni
6


Surrogates often change at different times in the evaluation process.
Only when we know our final cost and the competitions cost can we answer the original
question. But the surrogates helped us whether we were headed for trouble.
The last tool is based on how we forecast sales and profit on a new item. Mostly it is an income
statement, an array of figures allowing us to see what the profits will look like based on where
we are at any one time in the development. The marketing fiel as the A-T-A-R concept
(awareness-trial-availability-repeat)
The A-T-A-R model

Diffusion of innovation, a person or a firm to become a regular buyer/user of an innovation,
there must first be awareness that it exists, then there must be a decision to try that
innovation, then the person must find the item available to them, and finally there must be the
type of happiness with it that leads to adoption, or repeat usage.
Units purchased by each adopter, and the economics of the operation.
A mathematical formula and run it through one set of data. Only a few figures are known at
the start, but estimates can be plugged into the other spots and the whole thing set up for use
down the line.
Product Policy Chapter 8 The Concept/Evaluation System Pooja Israni
7

A-T-A-R model gives us guidance on evaluation system design. Importance of awareness, trial
and so on. Tests will have to be run where customers are checked out for rheir interest in
trying, their reactions after trying, and whatever else contributes to the formula.
Each factor is subject to estimation, in every development phase we are trying to
sharpen our ability to make the estimates.
An inadequate profit forecast can be improved only by changing one of the factors.
Qualitative changes (such as a new advertising theme) can be made in addition to the
quantitative. The proposed changes are then run through the formula again, which yields
another set of results.
A-T-A-R is a term that came from consumer products marketing.
A consumer buying unit may be a person or a home. The target users were selected partly
because we know them well.
Awareness, we want to know if the buying unit has been sufficiently informed to stimulate
further investigation and consideration of trial.
For trial of our new product, we might imagine an in store situation where the prospective
customer tries out the product and sees if the product is satisfactory. Potential buyer could try
the product, that is, try it in risky situations, waiting for a thief to challenge it. The trial is not
perfect, but it is close enough for real customer learning. Vicarious trial where a person or firm
who did try something shares results with someone who cant try it.
In a trial, we want two things to happen:
1. The buying unit went to some expense to get the trial supply if there was no cost,
then we cant be sure there was evaluation of the product message and interest
created
2. The buying unit used the new item enough to have a basis for deciding whether it is
any good.
For availability, we want to know whether the buyer can easiily get the new product if a
decision is made to try it. It is usually the percent of those outlets where our target buyers
shop where the firm has stocking of the new item. If the firm sells direct, there is always
availability.
All commodity volume ACV is another measure, which is the percentage of the market that
has access to the product in local distribution channels. Business-to-business often uses
distributors of some type, usually under some franchise or semifranchise agreement. But small
firms cannot be sure of availability and spend much of their marketing money on trying to get
it.
Repeat is easy for consumer packaged goods. It means the trial was successful. We have to
decide what statistic will tell us that. Some people use the direct one were you satisfied?
sometimes the indirect one is used too have you had occasion to recommend the product to
others?
Product Policy Chapter 8 The Concept/Evaluation System Pooja Israni
8

Where do we get figures for the A-T-A-R model?
The evaluation techniques will provide the data needed for the A-T-A-R model. Various
evaluation events can help on several of the key factors, that make the biggest contribution
noted as best in the figure. We spend our limited funds first on the best steps and then on
others if funds are available. If we have to skip a step we know we are leaving open the
question of whether users are likely to try the item when it becomes available.
Buying Unit: Purchase point (person or department/buying center).
Aware: Has heard about the new product with some characteristic that differentiates
it.
Available: If the buyer wants to try the product, the effort to find it will be successful
(expressed as a percentage).
Trial: Usually means a purchase or consumption of the product.
Repeat: The product is bought at least once more, or (for durables) recommended to
others.
Further uses of the A-T-A-R model
It is useful at this early point, as it provides an early sales and profit forecast based on
estimates specific to the new products. It calls for numbers that usually can be researched, and
it uses them in a managerial way. A-T-A-R is implicit throughout the discussion of market
launch planning important for the marketing effort to achieve awareness, trial, availability and
repeat use. As a tool to assess the launch, identify where the problem areas are, and steer it
back on course.
Summary
This chapter looked at the factors that aid in designing an evaluation system for the basic new
products process, designed to provide pieces of information that guide the project in its
journey to the market. First came the cummulative expenditures curve, risk/payoff matrx, and
the decay curve. The primary one was that almost everything about a process situation is
tentative. The product itself is still evolving, at least until it sells successfully; the actual date of
marketng is increasingly unclear as firms adopt limited marketing approaches; evaluation
actually begins with the innovation charter well before ideation; and a product is an
assemblage of many parts, each requiring its own evaluation.
A-T-A-R model, tells us some of the critical steps, how our information about them can be used
to forecast sales and profits, and how to design an evaluation system accordingly.
What are the specific tools, what can each do, and what are their weaknesses?


Product Policy Chapter 8 The Concept/Evaluation System Pooja Israni
9

Multiple choice questions
1. A new product first appears in the form of a ______.
a) Prototype
b) Idea
c) Model
d) Code
e) Design
2. When does the first use of evaluation usually occur?
a) It occurs after a product concept has been developed
b) It precedes the product concept
c) It follows the development of a prototype
d) It immediately follows the product launch
e) It occurs after a concept is fully screened
3. Which of the following is the goal of evaluation during the concept generation stage?
a) Encouraging the development of all concepts generated
b) Eliminate the big/sure loser (s)
c) Assessing the usability of the product
d) Ensuring that the developmental budget obtained is as large as possible
e) Identifying the opportunities
4. According to the risk/payoff matrix, which of the following types of errors occurs
when a product which could have been successful is discarded during the
concept/project evaluation stage?
a) Precision error
b) Expectation error
c) Go error
d) Fuzzy gate error
e) Drop error
5. Identify the risk strategy which is most likely to incur an opportunity cost.
a) Mitigation
b) Avoidance
c) Acceptance
d) Transfer
e) Inclusion
6. Firms that develop contingency plans prior to launching a new product have opted
for a strategy of _______-
a) Active acceptance
b) Passive avoidance
c) Active mitigation
d) Passive transfer
e) Passive acceptance
7. Apex Corporation uses rolling evaluation to assess its new product development
project. Based on this information, which of the following combinations of risk
strategies is most likely being used by the organization?
a) Avoidance/transfer
Product Policy Chapter 8 The Concept/Evaluation System Pooja Israni
10

b) Acceptance/avoidance
c) Acceptance/mitigation
d) Mitigation/avoidance
e) Transfer/mitigation
8. A company wants to determine whether a potential new product will be preferred
by customers, even before it is launched. Since the customer reactions cannot be
gauged at this stage, the firm can make use of ______ to gather pieces of
information that can substitute for what it wants to learn.
a) Morphological matrix
b) Perceptual mapping
c) Fundamental analysis
d) Lexical analysis
e) Surrogate questions
9. Identify the surrogate question that can help know customer reactions early on,
even before the product has been developed.
a) Will customers prefer the product?
b) Will the cost of the product competitive?
c) Does the concept match our manufacturing skills?
d) Will competition leap in?
e) Will the product sell?
10. The A-T-A-R model is most akin to a ____.
a) Morphological matrix
b) Pro forma income statement
c) Risk/payoff matrix
d) Decay curve
e) Cumulative expenditures curve