Vous êtes sur la page 1sur 32




Development of a Technical Innovation Audit

Vittorio Chiesa, Paul Coughlan, and Chris A. Voss

Measuring performance is helpful, but it's only part of the story. To learn from
our past successes and failures, we need to understand how they carne about. To
continually improve, we must examine not only our innovation performance, but
the processes with which we develop and exploit these innovations.
Vittorio Chiesa, Paul Coughlan,and Chris Voss present aframeworkfor auditing
technical innovation management. Their auditing methodology goes beyond perfor
mance measurement by highlighting problems and needs, and providing infor
mation that can be used in developing action plans for improving performance.
The foundation of their audit methodology is a process model of technical
innovation. The model addresses the managerial processes and the organizational
mechanisms through which innovation is performed. Underlying this method is
the notion that success in innovation is related to good practice in the relevant
management processes. The model identifies four core processes: concept gen
eration, product development, process innovation, and technology acquisition.
Supporting these core processes are three enabling processes: the deployment of
human and financia! resources, the effective use of appropriate systems and tools,
and senior management leadership and direction. The outcome from these core
and enabling processes is performance in terms of innovation and the resulting
competitiveness in the marketplace.
This model provides the basis far a detailed audit of current innovation prac
tice and performance. The audit has two dimensions: the process audit assesses
whether the processes necessary far innovation are in place and the degree to
which best practice is used; and the performance audit focuses on the outcomes
of each core and enabling process and of the overall process of technological
innovation and its effect on competitiveness. The performance audit helps identify
needs and problems, but it doesn 't explain why gaps exist between current and
required performance and it doesn't provide an action plan far closing these
gaps. The process audit meets these needs.
The audit methodology uses a twolevel approach: a rapid assessment based on
innovation scorecards and an indeptli audit. These scorecards provide an over
view of the company 's strengths and weaknesses with regard to technical inno
vation management, highlighting those areas that require indeptb examination.
The indepth audit ideruifies not only the processes, but the areas within each
requiring attention.

Address correspondence to Chris A. Voss, Lonclon Business School,

Sussex Place, Regent's Park, Lonclon NWI 4SA, Unitecl Kingclom.

J PROD INNOV MANAG !996;13:105-136

1996 Elsevier Science !ne. 0737-6782/96/$15.00
655 Avenue of the Americas, New York, NY 10010 SSDI 0737-6782(95)00109-3

lntroduction able instrument for auditing technical innovation man-

agement. The objectives of the development of an au-

easurement of a firm's innovation perfor- dit for technical innovation lay in the U.K. Department
mance has focused traditionally on macro- of Trade and Industry's (DTI) mission to improve the
level indicators of input and output. Yet, competitiveness of U.K. industry. It saw technical in-
increasingly in other areas, in particular quality man- novation as one of the drivers of national competitive-
agement, it is being stated that indicators of input and ness and sought means of getting companies to de-
output, whether macro- or micro-, indicate rather than velop and improve their innovation management
explain performance. If we are to understand innova- processes and performance. One means of doing this
tion performance more profoundly, we must look at was by developing an audit that could be used by
innovation capability and the processes involved in companies for self- or third-party assessment, which
developing and exploiting innovations. We feel that could then act as trigger for improvement in practice
the widely applied and influential quality awards, such by the company. The DTI contracted a team based at
as the European Quality Award [27] and its U.S. London Business School to develop and test such an
equivalent, the Malcolm Baldrige National Quality audit. The goals were to develop an audit of technical
Award [57], account for much of their support through innovation, drawing on published research and other
their use as audits of the processes of quality rather sources of best practice; to test this in a representative
than as the basis of awards. Auditing goes beyond sample of organizations; and after this to develop and
measuring: it builds on this to identify gaps between disseminate the audit throughout the United Kingdom.
current and desired performance, to identify where An audit of technical innovation capability has to
there are problems and needs, and to provide informa- address not only the process of formulation of inno-
tion that can be used in developing action plans to vation strategies, it should also address the firm' s ca-
improve performance. pabilities to implement such strategies and to adapt
This article describes the development of an action- innovation practices in response to changing contexts.
In this article, we propase a model for auditing a
firm' s innovation capability, based on a process model
BIOGRAPHICAL SKETCHES of technical innovation. This model has four core
Vittorio Chiesa is Senior Researcher at the National Research processes; the identification of new product con-
Council of Italy and a lecturer in Business Organization and Eco- cepts-concept generation; taking the innovation from
nomics at Politecnico di Milano. He obtained a Master's Degree in
electronic engineering at Politecnico di Milano and has been Vis-
concept to launch-product development; the devel-
iting Researcher at London Business School in the Centre for Op- opment of innovation in production=-process innova-
erations Management. His main research interests concern R&D tion; and the development and management of tech-
and technology management, internationalization ofR&D, innova-
tion management, and benchmarking. He is the author of severa!
nology per se-technology acquisition. The model
articles published in international journals. links the core to enabling processes and outcomes that
together constitute the innovation process in the firm.
Paul Coughlan is Lucas Professor in Manufacturing Systems and It facilitates two ways of assessing an organization: a
Management at the School of Business Studies, Trinity College,
process audit and a performance audit. The process
Dublin. He holds a Ph.D. from the University of Western Ontario
and an M.B.A. and first degree in engineering frorn the National audit focuses on the individual processes necessary for
University of Ireland. His current research interests include the innovation and the extent to which best practice is
introduction and evaluaton of multifunctional approaches to man- applied. The performance audit focuses on the effec-
aging manufacturing and product development.
tiveness of the individual processes and of the overall
Chris Voss is BT Professor of Total Quality Management and Di-
process of innovation, in terms of their impact on com-
rector of the Centre for Operations Management at London Busi- petitiveness.
ness School. He previously worked at the University of Warwick In this article we draw on existing research in in-
and the University of Western Ontario, having gained his doctorate
at London Business School. His current research focuses on oper-
novation management to develop the content of the
ational practices and performance in manufacturing, service, and audit of technical innovation; we then describe the use
technology management. His work has appeared in the Journal of of the audit in terms of process and performance au-
Product Jnnovation Management, Research Policy, and the Inter diting, the results of field testing and refinement, and
national Journal of Operations and Production Management. He is
currently chairman of the European Operations Management present two case studies of its use. Finally we discuss
Association. the lessons learned and issues for the future of auditing
technical innovation.

Auditing-Background anee management), resources (intellectual property,

funding, facilities, and equipment), and linkages
Audits of management processes have been developed (structure, interfunctional linkages, externa! linkages,
in many areas. We have already highlighted the use of regulatory compliance). They give a number of exam-
quality award models for auditing in both Europe and ples of successful innovative companies and associ-
the United States. For example, in describing the use ated with them the practices adopted.
of the European Quality A ward framework, the Euro- This theme, relating success in innovation to good
pean Foundation for Quality Management describes a practice in the relevant management processes, has
self-assessment audit as: been supported by many writers. For example, this
a comprehensive, systematic and regular review of an relationship has been found in many different specific
organization's activities and results referenced against aspects of the innovation process (the product devel-
a model of business excellence. [The] process allows opment process [16]; R&D management and the tech-
the organization to discern clearly its strengths and nology acquisition process [70]; implementation in
areas in which improvements can be made and culmi- production innovations [80]).
nates in planned improvement actions which are moni- Audits seem to have a number of characteristics.
tored for progress. The process offers the organization They embrace a model that sets out the scope of what
the opportunity to learn [27]. is to be audited, they develop a set of detailed ques-
tions around this model that enable the auditor to de-
A number of writers have addressed the challenge of
termine whether good practce is in place, and they
evaluatinga firm's technical innovation capability and
will have a process for its use. We decided to follow
performance. For example, Burgelman et al. devel-
this pattern and proposed that to audit a firm' s inno-
oped an innovative capability framework to audit tech-
vation capability we should develop a model that
nological and functional capabilities, formulation and
views innovation as a business process. This model
implementationof innovation strategies, together with
was to address the managerial processes and organi-
supportive organizational mechanisms [12]. This
zational mechanisms through which innovation is per-
framework includes five main dimensions:
resource availability and allocation (as indicated
by level of R&D funding, breadth and depth of
skills, distinctive competencies, allocation of Constructing the Audit Model
R&D resources);
Before constructing the model, it was necessary to
understanding competitors' innovative strategies define the scope of innovation to be considered. This
and industry evolution; we took as industrial technological innovation, as de-
understanding the technological environment; fined by Freeman [31]. He sees this as a process that
includes the technical, design, manufacturing, man-
structural and cultural context (as indicated by agement, and commercial activities involved in the
managing R&D projects, transferring them from marketing of a new or improved product or the first
R&D to manufacturing, integrating functional use of a new or improved manufacturing process or
groups); equipment. Our next step in developing a model for an
strategic management capacity to deal with entre- innovation audit was to identfy the key processes of
preneurial behavior. innovation. This task required a framework within
which we could distinguish between different types of
For Adler et al. [4], technical functions need a way to processes. We chose one based on conversion of in-
benchmark not only their products but also their stra- puts to outputs. In this framework, there are three cat-
tegic management process. They provide a framework egories: core processes, through which the firm con-
for assessing the overall functional strategies of tech- verts its substantive product or process concepts into
nical units. They set out three main elements of stra- deliveries to externa! customers; enabling processes,
tegic management: direction setting, policies, and ad- through which the firm supports the core processes
justment mechanisms. Within the policy element, they with the conversion of resources and strategic visions
analyze the role of processes (such as personnel man- into guidance anda foundation for innovative activity;
agement, technical project management, quality assur- and the outcome is performance in terms of both in-

novation and the resulting competitiveness and perfor- Enabling Processes

mance in the marketplace. These processes cross func-
tional, hierarchical, and organizational boundaries. To the deployment of human and financial re-
identify the core and enabling processes of innovation, sources-resources
Roberts' [65] definition of technological innovation
provided a useful basis. He stated that the overall man- the effective use of appropriate systems and
agement of technological innovation includes the or- tools-systems and tools
ganization and direction of human and capital re- providing the top management leadership and di-
sources towards effectively: (1) creating new knowl- rection=-leadership
edge, (2) generating ideas aimed at new and enhanced
products, manufacturing processes and services, (3) Finally, the process of technical innovation should re-
developing those ideas into working prototypes and sult in improved innovation performance, which in
( 4) transferring them into manufacturing distribution turn leads to increased competitiveness. The above are
and use. Based on this, our model comprises the fol- combined in our overall model, which is outlined in
lowing core and enabling processes: Figure l.

Core Processes
From Model to Detailed Audit
identification of new product concepts-concept
generation Having developed a model, we next needed to trans-
late it into a detailed audit. A comprehensive approach
taking the innovation from concept, through de- to auditing a firm' s technical innovation capability
velopment and transfer to manufacturing and should encompass a means for:
use-product development
the development of innovations in manufacturing assessing the current innovation practice and per-
processes=-process innovation formance;

the development and management of technology identifying the gaps between current and targeted
per se-technology acquisition practice and performance and the reasons for
Audits such as the European Quality Award also stress
the importance of those activities that support the defining the action plans needed to close these
core-the enabling processes. gaps.

Figure l. The process-based model of innovation.




As such, an audit can have two dimensions: a process number of elements or subprocesses. The full set of
audit and a performance audit. processes and subprocesses or elements, together with
the key literature and implications for audit, is shown
Process audita process audit focuses on such ques- in Table 1. These formed the basis of both the inno-
tions as whether the individual processes necessary for vation scorecards and the in-depth audit.
innovation are in place and the degree to which best
practice is used and implemented effectively.
Performanceaudita performance audit focuses on
the outcomes of each individual core and enabling Innovation Scorecards
process and of the overall process of technological
innovation and the impact of this on competitiveness. Innovation scorecards provide a rapid overall assess-
A performance audit produces quantitative results that ment of the practices adopted with respect to the
facilitate comparison both between and within organi- known best practice and whether or not the required
zations and monitors trends. Its weakness, however, is managerial processes are in place. The basis of the
that it is insufficient in itself to be a basis for learning scorecard approach is a description, for each process
and improvement. Although it indicates where needs of innovation, of the characteristics of good practice
and problems exist and the extent of the gap between and poor practice. This description can be translated
current and required performance, a performance audit into scales against which companies can review them-
does not suggest why there are gaps or an action plan
to cover gaps. For this we require the process audit.
We carried out an extensive literature review to
identify, for each process, the characteristics associ-
ated with success and failure in innovation. The num-
Process Audit
ber of different processes involved in technological
In general, auditing implies assessment (either self or innovation precludes discussing in this article the de-
external) of the practices used through comparison velopment of an audit scorecard for each core and
with known best practice. The framework developed enabling process. However, we can demonstrate this
in the first part of this article provided the basis for development through examining part of the audit-the
doing this. We propose a two level approach: a rapid generating new product concepts element of the con-
assessment approach based on innovation scorecards cept generation process. In attempting to describe the
and an in-depth approach based on that applied in characteristics of best practice, we examined writings
quality awards. in the area, which are summarized in Table l. Por
A process audit of a firm's innovation capability example, Maidique and Zirger [52] and Rochford and
requires that practices adopted to manage the innova- Rudelius [68] stress the importance of systematically
tion process are reviewed to assess: monitoring new product needs, Cooper [19] and Tush-
man and Moore [73] noted the importance of identi-
the degree to which there are appropriate business fying customer needs and matching them with tech-
processes in place; nological capabilities. Further, von Hippel proposed
that direct links with customers and exploiting lead
the deployment of good practice-the breadth of
users as a source of innovative concepts [79]. Tush-
use in the company;
man and Moore noted the use of broadly based teams
the degree to which each practice meets known drawn from a variety of functions in developing new
best in class or world class standards. product concepts [73]. Finally, Clark and Fujimoto
described the importance of early involvement of dif-
Having defined a model, the next step was to identify, fering functions in the process (16]. We took these
from literature review, the subprocesses and factors statements as indicators of the characteristics of good
that made up each of the processes. In sorne of these practice, and we constructed a hierarchy of good pro-
areas there was a strong research literature; in others cess characteristics for this element.
there was little and we had to rely more on the asser- In order for companies to use these statements to
tions of experts and practitioners. Por the core pro- assess their own processes, we ranked the character-
cesses, this review and the development of the detailed istics on a 4-point scale: 1 being unsatisfactory, 4 be-
underpinning of the audit are described in Appendix 1. ing good. Table 2 shows such a hierarchy for the con-
In doing this we disaggregated each process into a cept generation activity.

Table l. Processes and Subprocesses of Innovation

Process Elernent Source Implications for Audit

Concept generation process:

Generating new product Maidique and Zirger [52]; systematically monitoring market needs
concept Rochford and Rudelius [68]; putting up mechanisms for functional groups to meet the
Moenart and Souder [54]; customer
Moenart et al. [54] use of feedback frorn functions that rneet the customer
Von Hippel [55] building long-term relationships with customers and
especially lead users
cross-functional screening of new product concept ideas
Cooper [18]; Tushman and matching technological capabilities to market needs
Moore [73]; Johne and Snelson [74]
Product innovation planning Utterback [77]; Crawford [22] linking the product innovation plan to the corporate plan
market led planning process
prioritizing product development projects
Twiss [75); Wheelwright and establishing procedures for selecting new or enhanced
Clark [83] products
Adler [3] integrating processes for generating new product concepts,
planning product innovation, and realizing new products
Innovativeness and creativity Twiss [75]; Felberg and DeMarco eliciting and supporting new product ideas and initiatives
[28] from employees
rewarding entrepreneurial behavior
supporting unplanned product initiatives
circulating new product ideas
Kaplan [45]; Twiss [75] structuring organization for favoring creativity and
Maidique [51]; Allen [5] Felberg choosing the appropriate people for critica! innovative
and DeMarco [28] roles
Exploiting innovation Maidique [51]; Burglernan [11]; evaluating alternatives for developing new business
Roberts and Berry [67] opportunities
Roberts [66] selecting venture alternatives for entering a new business
Burgleman et al. [12] assessing the relatedness of enterpreneurial initiati ves to
core competencies
using governmental funding mechanisms
Productdevelopment process:
Product development process Cooper [ 19, 20] managing product development projects from the concept
to launch, establishing the scope of the process, phases,
gates, reviews, sign-off procedures
Teamwork Cooper [19]; Clark and Fujimoto integrating al! relevant functions in the product
[16]; Thamhain [721 developrnent process
Clark and Fujimoto [16]; Pinto early involvement of the key interna] functions and
and Pinto [61] externa! organizations
Clark and Fujirnoto [ 16] facilitating communication arnong the different groups
involved in the development process
degree of parallelism, integration of steps and task
interdependence built into the process
establishing role and priority of project progress reviews
Transfer to manufacturing Wheelwright and Clark [83] linking manufacturing and engineering
and distribution handling engineering changes
Hayes and Clark [38]; Coughlan rapid feedback frorn manufacturing to design and
[21] engineering
Tearnwork and organization Hise et al. [42] Use of cross-functional teams
Defining the states of project managers in the organization
Ciccantelli and Magidson [15] Use of organizational integration mechanisms at the initial
Industrial design Dumas and Minzberg [26] Incorporation of industrial design into product development
Use of inside or outside design consultancy tearns
Bailetti and Litva [7] Creating mechanisms for customer requirement inforrnation
to be integrated into product design

Table l. Continued
Process Element Source Implications fer Audit

Production process innovation

Formulating a manufacturing Hill [41] evaluating the capabilities of existing production processes
strategy establishing a formal procedure for generating a
manufacturing strategy
Rayes and Wheelwright [39] matching process capabilities to the requirements of the
linking process innovation to product innovation
allocating resources for developing new process technologies
monitoring sources of process innovations
Implementation of new Voss [81]; Leonard-Barton [47]; matching technology complexity to the capability to adopt
processes Gerwin [32]
Voss [81]; Adler [2]; Tyre [76] managing the links with suppliers in the development and
Voss [81]; Adler [2]; Tyre [76]; accompanying production process innovations with the
Drazin and Kazanjian [25] appropriate changes to the organization
modifying performance measures to reflect the capabilities
of new processes
Continuous improvement Rayes and Clarke [38]; Bessant identifying opportunities for improvement in processes
Deming [23] integrating process improvement with quality control
Camp [14] benchmarking production process performance
Wood and Elgie [85] involving manufacturing process developers in improvement
after installation
Technology acquisition
Formulating technology Little [50], Mitchell [53]; Hax systematically monitoring trends in existing and future
strategy and Majluf [37] technologies
Little [37]; Burgleman et al. assessing competitors' technological capabilities
[12]; Roussel et al. [70]; Foster identifying emerging technologies
[1986]; Papas [59]; Prahalad understanding core technologies and competencies of the
and Hamel [62] firm
Hax and Majluf [37]; Prahalad building the required core competencies based on the
and Hamel [62] technological capabilities
Mitchell [53]; Roussel et al. relating technology to business objectives and strategies
Selection, generation, and Little [50]; Twiss [75] choosing sources of technologies (in-house), R&D, licensing,
sourcing of technology partnering, externa! alliances)
use of both quantitative and qualitative methods to evaluate
R&D projects
Roussel et al. [70] corporate procedure for selecting R&D projects
choosing a portfolio balancing risk and reward, project
Identifying key issues in R&D organization supporting the
firm's technology policy
favoring communication, creating structural interfaces of
R&D with other functions, optimizing resources corporate
Management of intellectual Twiss [75]; Roberts and Be1Ty protecting intellectual property rights (patenting, trade
property [67] secrets)
exploiting intellectual property (licensing out)
Leadership process:
Human resources Adler et al. [3] defining the firm mission in technology and innovation
Burgleman et al. [12]; Quinn building innovation strategies into corporate strategies and
[63] plans
Prahalad and Hamel [62] identifying the core distinctive competencies
including representatives of innovation and technical
functions on the board

Table l. Continued
Process EleITient Source Implications for Audit

Process for innovation Quinn [63] evaluating processes for generating and implementing
Camp [14] benchmarking processes for innovation against best practices
Van de Ven [78]; Burger [13] making innovation processes visible to top managernent
Climate for innovation Tushman and Nadler [74] encouraging new idea development, risk taking, and
Nadler and Tushman [56]; making innovation policies shared and understood in the
Tushman and Nadler [7 4]; Van organization
de Ven [78] defining performance measurement system encouraging
Resource provision process:
Goals for innovation Adler et al. [3]; Allen [5]; identifying the key roles needed for managing the innovation
Tushman and Nadler [74]; process
Adler et al. [4] recruiting, developing, evaluating, and rewarding human
establishing career development paths for technical people
(dual ladder, international development, cross-functional
Funding Twiss [75]; Wheelwright and stability of funding of R&D activities and technology
Clark [83] acquisition
flexibility of funding of product and process development
Prahalad and Hamel [62] sharing risks and reducing costs of innovation through
alliance networks
Systems and tools provision
Systems Orlando [58] information and product systern used to support the
processes for product development
information systerns enhancing communication in the
innovation process
Tools Adler [3]; Rosenthal et al. [69] use of tools for capturing customer needs
use of tools for design of new products
Rickards [64] use of tools for promoting creativity
Quality assurance Clausing [ 17]; Hauser [35] managing quality in the design process
use of methods to analyze and improve the quality of
innovation processes
integrating process improvement and product innovation
with quality management

We repeated this for each one of the detailed ele- islands of good practice and process that, unless fully
ments of the audit framework. From this, we con- deployed across the company, may not be fully effec-
structed individual hierarchies for each element. In the tive.
final audit document, these were edited to produce a The scorecard approach can provide companies
set of scorecards. The full set of scorecards is shown in with an overview of their strengths and weaknesses
Appendix 2. with regards to technical innovation management. It
In the terms commonly used by companies in can enable them to highlight the areas that they should
benchmarking, the difference between the actual and examine in more depth.
ideal score is the gap between current and good prac-
tice. This gap is represented diagrammatically in Fig-
ure 2. As with the Baldrige Award [57], an organiza- InDepth Audit
tion can be rated on both use and deployment-the
breadth of use of a process in the company. The re- To enable organizations to assess in more detail their
sulting profile can reflect that a company may have management of innovation, we developed an in-depth
1996; 13: 105-136

Table 2. Example of Hierarchy for Innovation Scorecard Concept Development

2 3 4

Generating new Ad hoc Product concepts New product ideas Direct links with
product concepts development of new developed within sought in the customers and Jead
product concepts. single functions. marketplace and users to identify
Ideas internally research into expressed and latent
based. customer needs. needs.
Limited customer Involvement of Broad range of
contact. marketing and functions involved
technical functions in concept
in developing and development and
screening new screening of
product concepts. opportunities.
Early analysis of
new concepts.
Product innovation No product Planning for next Planning for up to Longterm planning
planning planning. generation of two generations of for three +
products. products. generations of
products, 5-15 years
innovation planning.

audit to identify not just the processes, but also the deficiencies in current practices, indicate where good
areas within each where attention should be focused. processes might be found, and provide a rich set of
To facilitate an in-depth audit, we drew on the litera- data for learning and managerial action for improve-
ture review outlined in Table 1. For each process, we ment.
then specified at an extra level of detail, the areas
where attention should be focused in auditing each
innovation process. To illustrate this, in Appendix 3 Performance Audit
we give an example of the areas specified in the prod-
uct development process. The resulting in-depth pro- As in the process audit, a firm's innovation perfor-
cess audit can provide information on strengths and mance can be measured in two ways:

Figure 2. Diagram of gaps between current and best practice (82].


... .

. . .! " e. . ... ..

... 1e . e! . ~
c = .. ~ : . .
i .~ ~
... a:!!3 .
> .!
~ .H .5 l
:1 ; ~ .i .E! li
p -;nE
; '! A e
.. .
~ -~ 5 .;.:
si 1i '"E e e;
e .l

. ,,
r o .1:!!
;; E;
tl .ie c o E 11.!
i! .E
e -~ ! .b~o - l 'li
H n H e~",, >~~
~J : .e ; i; -e
e<'.! .! o
: d: P i el ~"' o
::E E-. ; g .li
:z: : ! ;
J! >

E ;,
~ J
.s i o 1 i~
e= ::E
,,~ A.
J! ci o :
"',_ ! i!,,
::e e u
o o
et ~
"' e
::e j d:
"' "' ,_ .E e
::e .s l
4 .E 4
1996; 13: 105- 136

the performance of each core and enabling pro- signs and the manufacturing cost, and these measures
cess that is relevant for the firm; are objective. In contrast, industrial design perfor-
mance includes ergonomic and aesthetic aspects and is
the global result of the innovation process, that is
both highly subjective and difficult to measure.
how it impacts on the competitive capability of
Production process innovation. Production pro-
the firm.
cess innovation performance can be measured in terms
of its effectiveness, development speed, and cost [80].
Measuring the Performance of the Measures of effectiveness include the numbers of new
production processes and significant process enhance-
Innovation Process
ments per year. Process innovations often result from
Concept development. Measurement of perfor- continuous improvement activities [8] reflected in
mance in this area focuses on efficiency of the inno- such measures as: the number of suggestions per em-
vation process [ 16], that is the number of innovations ployee, the percentage of them that have been imple-
developed. Related measures include: the number of mented, and the average annual improvement in the
innovations (that can be split into product enhance- process parameters (quality, cost, lead time, work in
ments and new products); the number of new product progress, reliability, down time, capability).
ideas generated, evaluated, and developed; number of Technology acquisition. Performance of the tech-
new product-based ventures or business areas started; nology acquisition process can be measured in terms
the average product life cycle length; and the product of efficiency and effectiveness [29,30,75]. A measure
planning horizon (years, number of product genera- of efficiency is the R&D productivity, that is the ratio
tions). Measurement of concept development also ad- of technical performance to the R&D effort. A second
dresses the effectiveness of the process in terms of efficiency measure is the R&D/technology acquisition
customer satisfaction. Related measures include: num- cost per new product. Effectiveness can be measured
ber of designs meeting customer needs and the degree by the ratio of profits to technical progress or technical
of product variety and product range. Other measures performance improvement. Other measures of effec-
of product innovation effectiveness, such as quality of tiveness include: the percentage of R&D projects that
an innovation and functional performance, were in- led to successful new or enhanced products, licenses
cluded under product development. or patents; and the number of licenses in/out and pat-
Product development. Product development per- ents in a certain time span.
formance can be measured on three dimensions: Leadership. The performance of leadership related
speed, product performance, and engineering/design to innovation should reflect top management involve-
performance [16,81]. The concept of time-based com- ment in ensuring that the innovation process is effec-
petition [71] links speed to competitive advantage. tive. However, by its nature leadership impacts on the
Measures of speed include time to market (from con- whole process of innovation, and thus finding mea-
cept to launch) and time taken for the various stages sures of its effectiveness that are not too broad is dif-
(concept, design, initial production, launch). As inno- ficult. Surrogate measures can be used such as board
vations sometimes occur on the basis of customer membership of technical/product development people;
stimuli, speed is reflected also in the response time to the degree of awareness, at each level of employee, of
a customer request. Griffin [33] suggested three time the company innovation policies; and the number of
variables for measuring cycle times in new product pages in the annual report devoted to innovation/
development: development time, concept to cycle technology.
time, and total ( cycle) time. Resourcing. Performance of the resourcing pro-
Product performance can be measured on three cess reflects the appropriateness of the resources de-
main dimensions: cost, including unit cost, production voted to the innovation efforts. Measures include the
cost and development cost broken down by stage; percentage of activities that have been delayed or can-
technical performance, including how well the product celed due to lack of funding or human resources and
functions); and product quality. the number of people at the various levels of the or-
Engineering/design performance can be measured ganization playing key roles in innovation such as
by the manufacturability, testability, and industrial de- sponsors, champions, and gatekeepers.
sign of a product. Measures of manufacturability and Systems and tools. Systems and tools used are
testability include the number of the required rede- likely to be specific to the company's nature and en-

vironment. For example, engineering companies might sons between past and present performance and iden-
use computer-aided design (CAD) in product design, tification of trends. We may also wish to compare
whereas process industries might use simulation tools performance against the goals set for innovation by
in process design. Performance measures thus should companies and against the competition, in order to
be related to the appropriateness of systems and tools find whether there are gaps between the current and
and systems used in the organization, and the depth the required performance. These comparisons can be
and breadth of use of tools. Measures include the per- used as the basis for setting performance targets for the
centage of products developed using CAD and the future.
percentage of engineers trained in simulation tech-
Our approach to measuring performance of the in- Measuring the lmpact of Innovation on
novation process is summarized in Table 3, listing a Competitive Performance
set of metrics and indicators of performance in each
core and enabling process. Ultimately for innovation performance to be beneficia!
In using these measures, as well as measuring per- to the firm, it has to lead to increased competitiveness.
formance at a single point in time, we may wish to "New product introductions provide the potential for
measure how performance evolved, through compari- innovating firms to gain a preferred market position in

Table 3. PerformanceMeasures
Concept generation average annual improvement in process parameters (quality
Number of new product ideas, product enhancement ideas cost, lead time, work in progress, reliability, down time,
evaluated in the last year capability)
Number of new product-based business area/ventures Technology acquisition
started in the past 5 years R&D/technology acquisition cost per new product
Customer satisfaction (design meeting customer needs; R&D projects that lead to new or enhanced products,
produce range and variety) process innovations, Iicenses, patients (% number of
Product planning horizon (years, product generations) projects, % R&D expenditure)
Average product life cycle length number of licenses in/out over the Iast 3 years
Product development number of patients over the Iast 3 years
Time to market: cost/benefit performance of completed R&D projects
average concept-to-Iaunch time Leadership
time for each phase (concept, design, initial production, number/percentage of members from technical
launch) functions/product development in the main and
average overrun subsidiary/divisional boards
average time of product enhancement percentage of employees aware of, sharing the innovation
average time of redesign policies and values
Product performance: number of pages in the annual report devoted to innovation
cost (unit cost, production cost, development cost) and technology
technical performance (e.g., case of use, operating cost, Resourcing
serviceability) personnel in product development and technical functions
quality who have worked in more than one function
Design performance: percentage of projects delayed, canceled due to lack of
manufacturing cost funding
manufacturability percentage of projects delayed, canceled due to Iack of
testability human resources
number of product redesigns Systems and tools
Process innovation percentage of designers/engineers with access to CAD
number of new processes and significant enhancements
percentage of products on CAD database
per year
Speed percentage of projects on which specific tools are applied
installation lead times (start to trouble-free working) (FMEA, QFD, Rapid prototyping, Taguchi methods)
Development cost percentage of designers/engineers trained to design far
Continuous improvement: manufacture
number of improvement suggestions per employee percentage of team leaders trained in creativity techniques
percentage implemented certified processes

relation to rivals and realize more durable returns than successful innovation may have benefits that can be
would otherwise be possible" [46]. Measures of how exploited afterward in other innovation projects in the
an individual innovation (new or enhanced product or same product range). These measures concern the im-
process) contributes to enhancing the firm's competi- pact of an individual innovation on a firm's competi-
tiveness can be based on whether the innovation has tiveness. In addition, measures of the impact of the
had market impact or been financially successful. The innovation process on competitiveness over time are
market impact of individual innovations can be used as needed. Such measures can include percentages of
indicators of the competitive impact of the overall in- sales and profits from new or enhanced products in-
novation process. troduced in the time span considered and total market
The impact of an innovation on competitiveness can share. Table 4 summarizes a sample set of metrics that
be measured by sales and profits generated from that can be used to measure the impact of innovation on
innovation and by the market share gained. These cornpetitiveness.
measures can be compared against both competitors
and expected results. The impact on the profitability
and sales of the product portfolio can be measured by Testing and Using the Audit Tool
comparing sales and profits of the portfolio before and The testing and issuing of an audit can go through a
after the innovation has been completed. Finally, per- number of phases. The first is field testing, resulting in
formance measures should reflect the degree to which modifications and irnprovement; the second is imple-
an innovation enhances a firm's capability to produce mentation in the field; the third is testing in the longer
innovations. Proxy measures could include sales, mar- terrn when firrns have used it, have taken actions as a
ket shares, and profits of a series of innovations linked result, and these actions have been evaluated as to
to one another (among which the innovation consid- their long-terrn effectiveness. For this innovation au-
ered). dit, the first two phases have taken place; the long-
However, there are sorne limitations to measure- terrn effectiveness will only be evaluated after sorne
ment at the level of the individual innovation. For years of use.
example, it <loes not take into account factors such as
the influence of one innovation on financia! results of
other products belonging to the same category [10] or
Table 4. Performance Measures Competitiveness
whether and how the innovation changes a firm's re-
source base or influences the capability for further
Type of Impact
innovations. on Competitiveness Metrics
As such, measures of impact on competitiveness
Impact on firm's Sales:
should address three main areas:
competitiveness of an domestic market
individual innovation (to be regional market
the performance of the innovation m financia! compared against global market
terms; competitor's and/or expected Market share:
results) domestic market
the impact of the innovation on the competitive regional market
performance of the product portfolio to which the global market
innovation belongs; Impact on firm' s product Profits
portfolio of an individual Sales of the portofolio befare
the contribution to the firm's process of learning. innovation and after innovation
Profits of the portfolio befare
The first two areas are concerned with the analysis of and after innovation
Impact on firm' s Sales, market share, and profits
the impact of an innovation on competitiveness at a of a series of innovations to
competitiveness of a series
given point. The second area takes into account that an of innovations which the innovation belongs
innovation can be successful considered by itself but Impact of the innovation % Sales/Profits from products
may have a negative impact on the financia} perfor- process on competitiveness introduced in the last 3/5 years
mance of other products in the portfolio. over time
% Sales/Profits from products
The third area views each as an element within a
with significan! enhancements
series of innovations of which results should be mea- in the last 3/5 years
sured globally and not just individually. (Even an un-

Frequently, academic research provides the founda- Users are heterogeneous. An important objective
tion for tools for practitioners to use in diagnosing and for the DTI in using such a framework was that it
improving business practice. Such tools are often de- had to be generic. That is to say it should be
rived from research that has been through a thorough usable by a wide range of companies in the
academic testing process where attention has been United Kingdom, and it should cover the full set
paid to issues of validity [17). However, we propose of processes involved in managing technical in-
that to be of value to practitioners they must also pass novation.
tests of both usefulness and usability, sometimes
Potential applications are not fully understood.
known in industry as beta testing.
There was no tradition in the United Kingdom of
such audit tools, and companies were not experi-
enced in self-assessment.
Beta Testing
Alpha testing is unable to guarantee a bugfree
product. There are limited opportunities for alpha
Dolan and Matthews [24], in a thorough review ofbeta
testing the tools before testing in the field. Tests
test design and management, state that it has a curious
with academic colleagues and students cannot
status in new product development. They define it as
properly replicate industry use.
"one of a variety of procedures by which potential
users 'try out' a product.'' beta testing refers to testing Limits the potential sample size due to complex
with a small number of potential adopters, not ran- ity. Whereas a questionnaire can be tested easily
domly chosen, where complexity, cost, and time pres- by mailing to an initial set of users, testing an
sures mean that large sample, statistically significant audit tool requires recruiting collaborating com-
testing cannot be used. It is widely used and its utility panies and working with them on a case basis.
has been established, yet it is often informal. This limits the ability to work with a large sample
Dolan and Matthews see beta testing as including size.
the following steps: pre-Beta activity leading to afro-
zen design and purpose, site recruitment, and data col- All four factors pointed toward beta testing as an ap-
lection. The core purpose of beta testing is the testing propriate process in this case. In describing the beta
of the system, product, or service. All 21 programs testing of the audit tool, we will follow the structure
studied by Dolan and Matthews did this. suggested by Dolan and Matthews [24]: objectives,
The process of testing starts with the recruitment of site recruitment, test design, test implementation (data
users. Typically, the sample starts at about five test collection), and outcome.
sites and may be expanded to more. The next step is
data collection from the site and the feedback of this
data to the testing firm. This will typically involve a
full functionality test in a variety of application envi- In translating academic work into a tool for managers,
ronments and debugging and appropriate redesign of we identified three requirements that we had to test in
the product or service. The output will be a validated the field: functionality, usability, and usefulness.
core product/service and support material. Other out-
puts will depend on the objectives but will include Functionality, A primary objective of field testing
testimonials, ideas for the next generation of products, was to test both the basic functionality of the tool
and success stories/reference sites for use in market- and the functionality of the support process. The
ing. second objective was to test the degree to which
They suggest a number of conditions under which the tool was generic: that it was appropriate for
beta testing might be appropriate. The audit tool companies from different sectors, different sizes,
closely met four of these: 1 and with different technologies.

Usability. The tool was designed for stand-alone

use by practitioners. This placed a high premium
on testing for usability. Usability includes a num-
The other conditions put forward by Dolan and Matthews were that
the decision-making unit for purchase is complex and the opinion leader ber of areas. One is the degree to which users are
phenomenonis operative. Neither of these applied to the audit too!. able to use the tool properly without support from
1996; 13: l 05-136

academics or consultants. Another is the clarity of unable to participate leaving a sample of six. This is
language. Academics are not renowned for their similar in size to that of the typical reported beta test
ability to write in plain English. Overuse of aca- group [24]. The characteristics of the firms are shown
demic terminology might result in difficulty in in Table 5. Firms A-F were used for beta testing. They
using the tool. represent a full range of size and production processes
and cover all four of the categories suggested by Pavitt
Usefulness. Any tool is only worth using if it
[60]. After the launch of the tool in the United King-
provides value to the user. In the short-term, use-
dom, its use was audited in two further firms, G & H.
fulness could be seen in terms of companies' per-
ceptions of whether they found it of use and
whether it led to effective action plans to improve
Test Design and Implementation
innovation management in their organizations. In
the longer tem, this can be evaluated through Testing was designed to take place in three phases.
measuring the effectiveness of the programs that The first two involved individual meetings with the
resulted from the assessment. companies and analysis by the companies. The final
phase involved a benchmarking meeting of participat-
ing companies.
Sample Recruitment
Phase one was designed to test user understanding
As the audit tool was to be made available to all U.K. of the tool and its terminology. As expected, there
manufacturing companies, it was important that test- were many areas where companies did not understand
ing was done on a sample that reflected the heteroge- the language or the concepts. In general, they found
neity of industry. Pavitt argues that factors such as the the initial version of the framework too complex. Un-
development trajectory and the source of ideas are less it could be understood, then it was not likely to be
contingent on the type of business [ 60]. He stated that used properly. Language problems included both use
businesses can be classified into four areas: science of academic phraseology and use of U.S. terminology
based, scale intensive, information intensive, and spe- that had not yet crossed the Atlantic. An example of
cialized suppliers. Selection of companies from each the latter was the phrase ''product realization pro-
area would provide a breadth of testing and in particu- cess"; none of our sample knew what this meant.
lar help validate the generic nature of the audit. In These comments were reviewed, and a considerably
addition, the DTI was concerned that the tool should revised framework and tool were produced. In addi-
not be applicable solely to large and sophisticated tion, we reviewed the functionality of the framework
companies. lt was therefore felt important to involve and the tool. Companies commented on areas that they
firms of different sizes. Sample firms were recruited thought were missing. This highlighted the area of
through the U.K. Confederation of British Industry. metrics, and additional attention was paid to determin-
An initial group of 10 firms was invited to take part in ing jointly what metrics might be usable and incorpo-
testing the audit tool. Four firms declined or were rating them into the framework.

Table 5. Firms Testing the Audit Tool

Firm lndustry Processes Size Type"

A Petrochemicals High volume process Very large Scale based

B Electrical fittings High volume line Medium Specialized supplier
e Systems integration Low volume batch Medium Information based
D Fibers and chemicals High and med volume process Large Science based
E Adhesives Medium volume batch Medium Specialized supplier
F Specialty paper Medium volume process Small Scale intensive
G Ink jet printers Low and medium volume batch Small Specialized supplier
H Chemicals (multibusiness company)" High and medium volume process Large Scale based and science based
and batch
Small, <200 employees; medium, 200-499 ernployees; large, 500-5000 employees; very large, >5000 employees.
h Based on Pavitt [60].
e Company H implemented the audit too! in ali of its businesses, which included a wide range from high to low volume, from scale-intensive bulk chemicals
to science-based specialty chemicals.
1996; 13: 105-136

In phase two, the companies were asked to use the felt that the tool provided a proper evaluation of their
audit tool unaided by the development team. Two ver- processes. Feedback carne in two specific areas. First,
sions of the tool, one simple and the other more thor- companies focused on particular rather than all areas
ough, were tested in different companies. This test in their use of the tool. Second, all found difficulty in
indicated that (as might have been expected) the sim- collecting data for the metrics.
pler version was much easier to use. The focus of innovation will differ between firms
As it was envisaged that the tool would be used not and can change over time. For example, Abernathy
just for interna} self-assessment but as a framework to and Utterback [l] stated that the relative focus of in-
guide benchmarking, in the final stage of the testing, novation will depend on the firm's position in the
we tried to emulate the latter. This was done in the product life cycle. During earlier stages, the focus is
format of a benchmarking club. Although benchmark- on product innovation; as the cycle develops, process
ing is normally done against best in class, in countries innovation plays an increasingly important role. As
smaller than the United States, this may limit its use as noted earlier, Pavitt [60] has proposed four distinct
there may not be a world class company to benchmark groupings of companies each having different techno-
against. In addition, something useful can be learnt logical trajectories and focuses for innovation. We re-
through benchmarking against most companies, even viewed the pattern of use made by companies, cate-
if they are not world class. As a result, benchmarking gorized into Pavitt's four types of firm (see Table 6).
clubs are widely used in the United Kingdom. They We found that the focus diff ered depending the nature
are groups of companies seeking to benchmark an in- of the company. In science-intensive companies, the
dividual of set of processes and are formed through a focus was as expected to be on technology acquisition;
variety of mechanisms, including industry associa- in scale-intensive companies, on process innovation;
tions, sponsorship by the U .K. DTI, and facilitation by in information-intensive companies, on product inno-
universities or consultants. In our club, the group of vation and development; and in specialized suppliers,
companies,having conducted self-assessment, met to on product innovation and development. There was
share knowledge of their own performance and pro- surprisingly little difference based on company size:
cesses in a series of meetings. In addition, we used small companies were as concerned with technology
these meetings to gain feedback on the audit tool and management and innovation and were as capable of
its use for benchmarking. This proved successful, with using the audit tool as large companies. Company F
each company having used the benchmarking frame- had better systems in place for process innovation than
work to analyze their own processes and being able to many larger companies, and company G, which we
contribute to the exchange and to learn from the oth- review later, was as sophisticated in its technology
ers. acquisition as companies 100 times its size, with a
During this stage, we debrief ed the companies to complex network of technology linkages.
determine how well the tool met its functional goals These observations were consistent with the indi-
and also met our goals of usability. In addition, we
sought feedback on the processes used, difficulties en- Table 6. Use of the Tool by Firm Type
countered, and the organization for self-assessment Company Type Focus
and benchmarking. The feedback from each of these is
A Scale intensive Technology acquisition,
described next.
process innovation
F Scale intensive Process innovation,
technology, acquisition,
Test Outcome product innovation
e Information intensive Product development,
technology acquisition
Functionality=Generic Application of the Audit Tool B Specialized supplier Product innovation and
Of particular concern was whether the tool was ge- E Specialized supplier All
neric, i.e., the degree to which a single set of tools and G Specialized supplier Technology acquisition,
frameworks could be used by a wide range of compa- product innovation, and
nies. The feedback from companies indicated ali felt development
D Science based Technology acquisition,
that they could use the tools equally well. That is, in
product development
the areas that they wished to evaluate themselves, they
1996;13: 105-136

vidual discussions with the firms which indicated that from processes to be the best auditors. Leaders of audit
the tool was equally usable by a wide range of users teams were drawn from a wide range of functions,
but that their use would focus on those areas appro- including R&D, marketing, and product development.
priate for the technology and management concerns. When a cross-functional team was used for the audit,
Our conclusion was that the audit tool was generic and the functional origin of the team leader was not seen to
robust, but as firms would have different foci, the be important.
supporting documentation should indicate that compa- Time to perform the audit. The audit tool con-
nies may wish to focus on particular areas of concern tained two parts, a scorecard for the initial assessment
or relevance to themselves. and a detailed audit for in-depth assessment. The ini-
Metrics created much debate. The underlying phi- tial audit using the scorecard approach could be done
losophy of the tool was that it should be used to audit over 2 to 8 weeks, depending on the size and corn-
the processes of innovation and suggest metrics that plexity of the company. The typical time involved
companies should use in reviewing and comparing the ranged from 4 to 20 days. The time for detailed audit
performance of individual processes. At the outset of varied greatly, as companies could choose to do a
the testing, most companies had no metrics associated detailed audit on a narrow or a wide range of areas. In
with any of the processes. Sorne measures suggested general, the time required was two or three times that
in the literature, such as patents, were not seen by the for the initial audit.
companies as valid measures of performance. As a Our conclusion based on the feedback from the beta
result, all companies in the testing found difficulty in testing was that auditing where possible should be
developing and collecting data for the metrics during performed by a tearn, which should bring a broad set
the test program. It was generally agreed that the eas- of perspectives. The audit leaders' function is less im-
iest areas in which to collect data were measures of portant than their ability to manage a team and their
product innovation such as percent of sales from prod- credibility with both the team and senior management.
ucts that have been introduced in the last 3 years. This
difficulty was not always seen as negative. Company
A stated that one of the biggest benefits to them of Usefulness
using the audit tool was that it provided a framework
for them to think about and to define the metrics that Our final objective was to test the usefulness of the
were needed in the future. One of their actions follow- tool. At a company level, an indicator of usefulness is
ing the beta testing period was development of routine whether it led to effective action plans to improve
data collection for a number of key metrics. innovation management in the organization. It could
also be viewed in terms of subjective assessment of its
use to companies. Por the DTI, usefulness ultimately
Usability will be tested in the longer term, through performance
of U.K. industry, in the short-term by the degree of
W e were particularly interested in the usability of the adoption of the tool by companies.
tool and questions such as what resources were As the effectiveness of the tool will only affect per-
needed, how long it took, and how best companies formance over the long-term, testing usefulness is by
should organize for use of the audit. We found a wide its nature difficult and qualitative in the short-term.
variety of answers to these questions, together with a W e have therefore reported on the initial views of the
number of underlying patterns of use that were found participating companies. At the end of the final work-
to be particularly successful. shop, questions concerning success were asked of each
Organization. Half of the audits were conducted of the participating organizations. The responses were
by an individual, in one case the managing director. very positive. All companies stated that they found the
The other companies conducted the audit with a team. tools and frameworks challenging and useful. The
The feedback on team use was very positive. A team chief executive of one stated this effect to a public to
drawn from different functions was able to give a far a meeting of 500 companies at the Confederation Brit-
more objective view of the process as it brought to- ish Industry. The managing director of another com-
gether strengths of different perspectives and viewed pany stated that he had used many tools and question-
processes not just from within a function but also from naires but that this one had really made him think. On
outside. Senior management in larger firms was too far a more concrete level, most of the companies took

specific action as a result of using the audit. Sorne its dissemination, a full set of training materials was
examples are given: developed by the DTI. lts dissemination philosophy
was not to go to companies direct, but to use service
Company A-The development of metrics to mea- providers such as consultants, training organizations,
sure innovation performance and educational establishments. Training was pro-
Company C-Revisions to be simultaneous engi- vided for these groups, who in turn provided wider
neering programs dissemination and training for managers and compa-
Company E-Creating a debate at the senior man- nies. As of early 1995, managers from more than 500
agement level about the role of innovation in the companies had been trained in the use of the audit tool.
firm Further support material was under development in-
Company G-A prioritized set of actions to im- cluding laser discs.
prove the product development process and the
control by senior management
Company H-Actions to improve the teamwork and
Case Studies
cross-functional management in the organization
We felt further feedback could be gained through re-
A frequent, though minor output of beta test programs
viewing the final workbook version of the audit tool in
is testimonials, and the willingness of the participants
use. We developed case studies of two users of the
in the beta testing to give testimonials can be seen as
tools in contrasting companies and use: one a small
an indicator of usefulness. The sponsor of the tool
company, using the tool with outside facilitation; the
development, the DTI, decided to contact sorne of the
other, a large multinational, independently adapting
companies for testimonials for use in marketing the
the tool for companywide use. Case studies of the two
tool to industry in the United Kingdom. They used a
users are shown in Exhibits 1 and 2.
third party to contact companies. All beta test compa-
A number of tentative conclusions can be drawn
nies contacted gave testimonials.
from these two cases. First, the audit process works in
Despite the qualitative nature of the data, both we
diverse environments, the two companies are different
and the DTI concluded that feedback from the com-
in many dimensions. The companies have used it with-
panies indicated that at the initial stage, the usefulness
out undue difficulty to audit their innovation and tech-
was satisfactory. The ultimate test of usefulness will
nology management processes, and this audit has led
come only in the longer term, once the impact of the
to actions for improvement. lt does not indicate yet
actions chosen by the companies can be evaluated.
whether these actions will successfully improve inno-
vation performance. The two cases illuminate the or-
From Test to Use ganization for the audit. In both cases, team ap-
proaches were used and certainly in case H were seen
DisseminationWorkbook and Training to be essential. Both cases also used facilitators-one
external, one internal. The facilitator's role, particu-
Once the beta testing had been completed, the feed- larly in case H, focused not on providing technical
back was reviewed for improvements that might be expertise but with helping make the team processes
needed. On the content side, it was agreed that in its work. Both were championed and pushed by very se-
initial form, it had not addressed the marketing area nior managers, who had the support of other senior
directly. A set of market-related questions were con- colleagues.
structed by the DTI and added to the audit under the Case H was also illustrative of the way in which the
heading market focus. This and other necessary audit could be tailored. The company adapted the audit
changes were made to the content of the audit tools. tool's use to fit the needs of differing companies in the
The feedback from the companies on their use of the group. There has been sorne feedback from the cases
tool was used to develop a set of guidelines for its use that indicates sorne details may have been missed in
both for self-assessment and for benchmarking. A six- the beta test. Both companies fed back the results to
step approach was developed. This final version of their managers using a 10- or 100-point scale, not the
tools and guidelines for use was translated by the DTI 4-point scale used in the scorecard. Both used stronger
into a workbook that was launched with a national language in their description of scoring. Company G
program for its dissemination and use [82]. To support called a full score world class; company H called it

Exhbt l. Case Study G

Background of the teams and the interviews conducted by the facilitators.

The summary of the audit is shown in Figure 3.
The company is a small high-technology company based on ink The audit and subsequent discussion led to identification of
jet technology. From its start-up in the 1980s, it has expanded gaps between current and desired processes and performance.
to become a leading world player in its field with a stream of Just as valuable was validation of many existing programs and
innovative products. It operates in virtually every market in the activities. Examples of the issues discussed included:
The board of the company is extremely concerned with de- ldentification of key opportunities for improvement
veloping and maintaining world class processes and perfor- The processes for designing for manufacture and transfer
mance in ali of its processes. One area vital to its long-term from design to manufacture were poor and led to low re-
competitive position is innovation. liability and delivery problems.
There was insufficient attention paid to the voice of the
The Audit customer; in particular, there was a lack of formal feedback
The audt was initiated by one of the directors who had heard processes.
Gaps recognized and programs in place
about the audit tools developed by the research team. He askcd
the team to facilitate an initial audit of the innovation and In a number of areas the audit revealed gaps that had al-
manufacturing processes at the company. The audit consisted ready been recognized and where the current programs of
of the following stages: improvement and trajectories of change were correct and
should lead to high levels of practice and performance.
Set Up These included industrial design and the product develop-
ment processes.
Two facilitators from the research team met with senior func- ldentifving mismatches
tional managers and directors of the company to determine the The company had a wide range of improvement programs
scope of the audit, and to instrnct the company in the use of the in place. Sorne of these did not reflect the real needs as
audit tools. The technical director was made responsible for identified by the audit. Gven their limited resource, this
conducting the innovation audit. The quality manager was led to consideration of whether the portfolio of improve-
made responsible for all coordination within the organization. ment actions should be changed.
Validating current areas
An often missed output from an audit is the confirmation
The technical director formed a small team drawn from various that much existing activity is excellent and should be rec-
parts of the company to audit the company's processes, using ognized as such. In this company, a number of processes
the scorecard part of the audit tool. Over 2 weeks, these data such as the technology acquisition were generally of a very
were collected by the team. This was followed by a review high standard.
meeting between the team and the technical director to discuss Dilemmas
the initial findings and to explore the issues behind these, Audits can provide data showing conflicting facets of an
Shortly afterward, the outside facilitators spent a second day at issue. Despite being an innovation led company with ex-
the site interviewing employees involved with the innovation ceptional leadership, there were also elements of the com-
process. (In addition, an audit tool concerned with world-class pany's culture and style that reduced risk taking and inno-
manufacturing was administered). The facilitators collated the vation at the middle management level.
interview data and the data from the innovation and manufac-
These issues and others were debated at the review meeting. At
turing audit tools.
the end, a set of priority areas for improvement was identified.
Review Meeting Responsibilities were assigned for implementation.
Six weeks after the review meeting, a separate meeting was
One week later, a review meeting was held at the company, led held with the chairman of the company. The objectives were to
by a facilitator. The meeting consisted of a broad cross-section explain to him the process, to brief him on the outcome and the
of those from the functions involved, including a number of key actions required, in particular those actions which involved
board members. The outcome from the audit was presented by him personally, and finally to get his commitment to making
the facilitators. This was as numerical data in bar chart form the improvement actions in the company. Implementation is
together with a commentary based on the feedback now taking place.


to mfg.,__ __,k, ,
Resourcing1-- __,"',., +---GAP_,..
Teamwork and Orgn.r------------1<::.''I
!nnovn. & creativity1----------4;.::
Leadership,__ _.,c ,
Management processes1-----------'"-'-''---~
o 20 40 60 80 100
(100= WORLD



Systems .._--GAP_,..I


Sourcingof Techno!ogy

& Regln.


o 20 40 60 80 100
(100= WORLD

Figure 3. Summary of audit of Company G.

ideal and best practice. Any revisions of the tool self-audit tool have frequently developed simpler
should take these into account. Finally, both compa- tools, based on the framework. This question needs to
nies restricted themselves to using the scorecard part be examined through longer term study.
of the tool, finding it sufficient for their needs. If wide-
spread, it may indicate that the detailed audit tool is
not necessary for every company or that the detailed Discussion
tool is too complex for the average company to use in
most situations. This is consistent with the experience The preceding sections have described the develop-
of the Baldrige award. Companies using Baldrige as a ment of an audit that has two parts; a performance

Exhibit 2. Case Study H

Background Organization for the Audit

This company is the regional entity of a large multinational The audit is mandatory for ali divisions. The audit is carried out
chemical corporation. It consists of eight separate operating in each operating division by a panel. This panel normally
divisions. These divisions are diverse both in size, processes, consists of technical people, business managers, and where
and type of business. In 1993, the company wished to develop possible, sales people. The objective of the broad composition
an interna! self-assessment capability to be used as part of its is twofold: first to provide honest dialogue, second to provide
technology management standards. They decided to use and a broad perspective. "If the dominant technology management
adapt the audit too! to do this. Before doing this, they studied view is not challenged, learning will have failed." The panel
the too! and decided to review it in terms of its applicability to gets the audit too! in advance, then meets in an intensive 3- to
the needs of the company. A small team was brought together 4-hour session to discuss and to rate their practices and perfor-
by the chief scientist and spent 1 day in a brainstorming ses- mance. Ali but two of the divisions used a facilitator to help
sion, reviewing the areas of the audit to see if they were ap- them with their audit. The view of the company was that the
propriate for the company and whether anything should be process went much better with a facilitator.
added. They identified a number of areas that they wanted to For divisions that operated in more than one market, there
add, such as management of know-how, and removed or reti- were significant differences between markets, and it was more
tled sorne parts of the existing audit. relevant to audit the processes associated with each market
In addition, they examined the suggested metrics and se- rather than try to average them. Differences between divisions
lected a subset of these that they felt most appropriate to them- were not seen as a problem; where a category of the audit was
selves. The resulting changes are summarized in Table 7. Al- not relevant, it was ignored. An example is product develop-
though new areas such as as management of know-how were ment in a division whose sole focus was chemical processes
added, much of the rest represents retitling and reordering to and did not develop new products per se.
match the company's focus. These reflect the science-based Gap Analysis
nature of the company, with a particular focus on the manage-
ment of the technology base. Most existing audit questions The data were analyzed by using a gap analysis format. Divi-
were kept the same, but the scale was modified to a 1-10 scale. sions were asked to identify target as well as actual-the dif-
ferences being the gap. An example of a gap analysis from a
The Audit particular division is shown in Figure 4. The overall average of
the gaps for the eight divisions is shown in Figure 5. Whereas
Built in to Company Processes
the gaps in individual divisions varied greatly, an underlying gap
The company has an explicit technology policy of taking ful! across the whole of the company was in the area of teamwork.
advantage of technology-based opportunities to generate
growth and enhance long-term competitiveness and to develop
its technology capability by building on existing skills and The identification of gaps leads to the next step, which is the
establishing long-term relationships with leading technology debate of the issue in the division. This needed tact, using a
organizations in areas of strategic interest. This policy is ac- "rnaybe we are not quite as good as we think we are" approach
companied by a set of technology management standards, The rather than direct "we are bad." Once agreement on the gap
audit was designed to be incorporated into the company's tech- and problem has been reached, the company culture supports
nology management standards as a technology self-diagnostic. rapid deployment of resource and action. The process has been
This diagnostic matched closely eight sections of the com- through its first annual cycle successfully and is expected to be
pany' s existing standards and provided a means of auditing repeated again, as it is embedded in the technology manage-
these on an annual basis. ment standards.

audit to provide simple but revealing performance to find comparators, whereas the process audit can
measures for each core and enabling process and a more clearly indicate potential problems. Combining
process audit to evaluate the processes of technical both the performance and process audits can lead to a
innovation. The two can be considered complemen- thorough understanding of a firm's innovation perfor-
tary. The performance audit, although it might identify mance. Further, a virtuous cycle can take place when
gaps between a firm' s actual and desired practice, does performance measures and process audits are used to-
not identify the causes of gaps and thus does not help gether. Performance measures offer synthetic and ef-
define action plans for improvement. However, this is fective indicators of results of innovation and help
addressed by the process audit. In addition, perfor- focus the attention on critica! areas. A process audit
mance measures are inherently difficult to define and enables a deeper understanding of what occurs in the

lnnovation ~~~~~~~:Z2S~s:sss:sss:ssza
Teamwork ~~~~~~~~~~b;sz;sz;~QQQJ
Planning ~~~~~~~~~:zzsz;:zz~::zz;;s:i
Gap Analysis~~~~~~~~~~~m~szi
Sourcing ~~~~~~~~~~~~~~2221

Mgt of Knowhow

o 2 4 6 8 10 12
Division Average

Figure 4. Example of audit results for a division in Case H.

innovation process and the good and bad characteris- scope of the audit is outlined in Figure 1. There
tics of practices adopted. Therefore, they help define may be other areas outside the scope of this audit.
where actions should be taken and what to do. At this Best practice evolves. Management does not stand
point, performance measures can help to assess the still, and new approaches are developed and new
extent of improvement due to the actions taken and research sheds light on new factors contributing
so on. to success and failure. As with case study H,
The testing of the tool has indicated that it can be users of the audit may wish to add or modify
used successfully by a range of organizations. How- factors.
ever, we see four potential limitations: An audit asks questions=it does not provide all the
answers. Users should not expect an audit to pro-
Scope. The audit was developed with a clear tech- vide them with a complete set of answers to po-
nological innovation orientation. This may limit tential problems. Audits ask questions and iden-
its applicability to product innovations that do not tify gaps and problems. As described in case G,
rely on technology as an input. lt is based around this can lead to identification of gaps between
product innovation, and though conceptually ap- current and desired processes and performance,
plicable to service innovation, the language does identifying key opportunities for improvement.
not fit the service environment very well. The Just as valuable can be validation of many exist-
ing programs and activities. The individual solu-
Table 7. Changes Made by Company H tions and actions required may very well be con-
Original Audit Areas Revised Audit Areas
Figure 5. Average Gaps by Category in Case H Divisions (n =
Concept generation Innovation and creativity
8 divisions ).
Product development Product innovation and
development lnnovation~--------------;
Teamwork and organization
Resourcing innovation
systems and tools
Process innovation Chemical process innovation Gap Analysis~~ssss:~s:ss~~'J
Technology acquisition Technology gap analysis Sourcing ~SS~~
Sourcing of technology MgtofKnowhow~SSSS~~~
Management of know-how Leadership/people~~s=:s~s:ss~~
Technology planning process Product Dev~~:s:;ssssss~~
Leadership Technical leadership and people Process Dev~c __ s==~~~~
development o 2 3 4 5
Increased competitiveness
Average gap
1996; 13: 105-136

tingent on the nature of the organization. One levels (product line, business unit, corporate) and
audit user described the process as ''intelligent helps look at innovation as a cross-functional and
signposting.'' cross-hierarchy process. Although the scope of the
Expertise required. The two levels of audit, seor- work has been industrial technological innovation, the
ecard and in-depth, have been developed and proposed approach would be broadly valid in services,
tested. The experience from the field indicated but sorne modification might be needed particularly in
wide use of the scorecards but more limited use the terminology and the focus. Much of service and
of the in-depth audit. This raises questions as to business process innovation is information technol-
whether the in-depth audit may be too complex ogy-based, and its scope often extends beyond just the
for stand-alone use and requires greater training process or service to changing the nature of the busi-
and expertise in the auditor than the typical com- ness or even industry.
pany can provide.

The authors wish to acknowledge the support of the U.K. Depart-

Conclusions ment of Trade and Industry, the Engineering and Physical Science
The use of a process-based approach has led to an Research Council who funded the research on which this article is
based, and the Confederation of British Industry and their mem-
organic tool for auditing a firm's innovation capabil- bers who collaborated in the work.
ity. The use of this tool has the potential to help com-
panies in a number of ways:
to identify the processes relevant to innovation, References
bringing together the core processes (product l. Abernathy, William J. and Utterback, James M. Patterns of industrial
concept generation, product development, pro- innovation. In: Readings in the Management of lnnovation, 2d edition,
duction process innovation, and technology ac- Michael Tushman and Willliam Moore (eds.). Harper Business, 1988,
pp. 25-36.
quisition) and the enabling processes (leadership, 2. Adler, Paul S. CAD/CAM: Managerial challenges and research issues.
human, and financia} resource management, the IEEE Transactions on Engineering Management 36(3):202-215
adoption of systems and tools for innovation); (1989).
3. Adler, P. Managing DFM: Learning to coordinate product and process
to develop performance measures for each pro- design. In: G. I. Susman (eds). lntegrating Design and Manufacturing
for Competitive Advantage, New York: Oxford University Press, 1992,
cess of innovation and to assess the overall im- pp. 140-156.
pact of innovation on competitiveness; 4. Adler, Paul S., McDonald, D. William and Mac Donald, Fred. Stra-
tegic management of technical functions. Sloan Management Review
to allow companies to audit their innovation ca- 33(2): 19-37 (Winter 1992).
pability through measuring the overall innovation 5. Allen, Tom J. Managing the Flow of Technology. Cambridge MA:
performance and the performance of each inno- MIT Press, 1976.

vation process, auditing innovation processes and 6. Athuatene-Gime, K. lnward technology licensing as an alternative to
interna! R&D in new product development: A conceptual framework.
benchmarking the practices adopted against the Journal of Product lnnovation Management 9(2):156-167 (1992).
world class practice. 7. Bailetti, Anthony J. and Litva, Paul F. Integrating customer require-
ments into product designs. Journal of Product Innovation Manage
through combining the performance and process ment 12(1 ):3-15 (1995).
audit approaches, to allow a broad audit with both 8. Bessant, John and Buckingham, Johanna. Innovation and organisa-
tional learning: The case of computer aided production management,
qualitative and quantitative measures. Britisn Journal of Management 4(4):219-234 (l 993).
9. Brassard, M. The Memory of Jogger Plus. Methuen, MA: Goal/QPC,
The tool has been through initial field testing. It is 1989.
designed to be generic, covering a range of business 10. de Bretani, U. K. Success and failures in new industrial services. Jour
and administrative systems within technological inno- nal of Product lnnovation Management 6(4):239-258 (1989).
vation, and applicable across a wide range of busi- 11. Burgelman, Robert A Managing the interna! corporate venturing pro-
cess. Sloan Managemeru Review 24:33-48 (Winter 1984).
nesses. It can be used for auditing the capability of a
12. Burgelman, R. A., Kosnik, T. J. and van den Poel, M. Toward an
single area and/or process of innovation on the basis of innovative capability audit framework. In: Strategic Management of
the past performance and of comparisons against com- Technology and Innovation, Robert A. Burgelman and Modesto A
Maidique (eds.). Homewood, IL: Irwin, 1988 pp. 31-34.
petitors' and/or known best practices (the importance
13. Burger, P. C. A report on the development of a research agenda for the
of each area can be different in different industries and product development and management association. Journal of Product
in different companies). It can also be used at various lnnovation Managemeru 6(1 ):51-60 (1989).

14. Camp, Robert C. Benchmarking. Milwaukee, WI: Quality Press, 1989. wright (eds.). Restoring our Competitive Edge, New York: Wiley,
15. Ciccantelli, Susan and Magidson, Jason. Customer idealized design: 1984.
Involving consumers in the product development process. Journal of 40. Henke, J. W., Krachenberg, A. R. and Lyons, T. F. Cross-functional
Product Innovation Management 10:341-347 (1993). teams: Good concept, poor implementation. Journal of Product Inno
16. Clark, Kim B. and Fujimoto, Takahiro. Product Development Perfor vation Management 10(3):216-229 (1993).
mance. Boston, MA: HBS Press, 1991. 41. Hill, Terry J. Manufacturing Strategy, The Strategic Management of
17. Clausing, Don P. Total Quality Development. New York: ASME the Manufacturing Function, 2d edition. London: Macmillan, 1993.
Press, 1994. 42. Hise, Richard T., O'Neal, Larry, Parasuraman, A. and McNeal, James
18. Cook, T. D. and Campbell D. T. QuasiExperimentation: Design and U. Marketing/R&D interaction in new product development: Implica-
Analysis lssues for Field Settings. Boston, MA: Houghton Mifflin, tions for new product success rates. Journal of Product Innovation
1979. Management 7(2):142-155 (1990).
19. Cooper, Robert G. New products: What distinguishes the winners? 43. Horwitch, Mel (ed.). Technology in the Modern Corporation: A Stra
Researc and Technology Management 33(6):27-31 (1990). tegic Perspective. New York: Pergamon Press, 1986.
20. Cooper, Robert G. Third generation new product processes. Journal of 44. Johne, Axel F. and Snelson, Patricia A. Success factors in product
Product Innovation Management 11 :3-14 (1994). innovation: A selective review of the literature. Journal of Product
Innovation Management 5:114-128 (1988).
21. Coughlan, P. D. Engineering change and manufacturing deployment in
new product development. In: Integrating Design and Manufacturing 45. Kaplan, N. Sorne organisational factors affecting creativity. IEEE
for Competitive Advantage, G. l. Susman, (ed.). New York: Oxford Transactions on Engineering Management (March 1960).
University Press, 1992, pp. 157-177. 46. Lawless, M. J. and Fisher, R. J. Sources of durable competitive ad-
22. Crawford, Merle C. Defining the charter for product innovation. Sloan vantage in new products. Journal of Product Jnnovation Management
Management Review 21:3-12 (Fall 1980). 7(1):35-43 (1990).
23. Deming, W. Edwards. lmprovement of quality and productivity 47. Leonard-Barton, Dorothy. Implementation as mutual adaptation of
through action by management. National Productivity Review 1(1): technology and organisation. Research Policy 17:251-267 (1988).
12-22 (Winter 1982). 49. Liker, J. K., Fleischer, M. and Arnsdorf, D. Fulfilling the promises of
24. Dolan, R. J. and Matthews J. M. Maximizing the utility of customer CAD: Integrating technology and organization. Sloan Management
product testing: Beta test design and management. Journal of Product Review 33(3):74-86 (1992).
lnnovation Management 10:318-330 (1993). 50. Little, A. D. The Strategic Management of Technology. European
25. Drazin, R. and Kazanjian R. K. Implementing manufacturing innova- Management Forum, Davos, Switzerland, 1981.
tions: Critica! choices of structure and staffing roles. Human Resource 51. Maidique, Modesto A. Entrepreneurs, champions, and technological
Management 25(3):385-403 ( 1986). innovation. Sloan Management Review 21(2):59-76 (1980).
26. Dumas, Angela and Minzberg, Henry. Managing form function and fit 52. Maidique, M. A. and Zirger, B. J. A case study of success and failure
in design. Design Management Journal 3:26-31 (1991). in product innovation: The case of the U.S. electronics industry. IEEE
27. European Foundation far Quality Management. SelfAssessment, 1995 Transactions on Engineering Management 31 ( 4):192-203 ( 1984 ).
Guidelines. Brussels: EFQM, 1995. 53. Mitchell, Graham R. New approaches for the strategic management of
28. Felberg, Jeff D. and DeMarco, David A. New idea enhancement at technology. In: Technology in the Modern Corporation: A Strategic
Amoco Chemical: An early report from a new system. Journal of Perspective, Mel Horwitch (ed.). New York: Pergamon Press, 1986.
Product lnnovation Management 9:278-286 (1992). 54. Moenart, Rudy K. and Souder, William E. An analysis of the use of
29. Foster, Richard N. Timing technological transitions. In: Technology in extrafunctional information by R&D and marketing personnel: Review
the Modern Corporation: A Strategic Perspective, Mel Horwitch (ed.). and model. Journal of Product Innovation Management 7(3):213-229
New York: Pergamon Press, 1986, pp. 215-228. (1990).
30. Foster, Richard N., Linden, Lawrence H., Whiteley, Roger L. and 55. Moenart, Rudy K., Souder, William E., De Meyer, Arnoud and De-
Kantrow, Alan M. Improving return on R&D. Researcn Management schoolmaster, D. R&D-Marketing integration mechanisms, commu-
28( l ): 12-17 (1985). nication flows and innovation success. Journal of Product lnnovation
Management 7(3):31-45 (1994).
31. Freeman, Christopher. The Economics of Industrial Innovation. Lon-
don: Penguin Modern Economic Texts, 1974. 56. Nadler, David A. and Tushrnan, Michael L. A model for diagnosing
32. Gerwin, Donald. A theory of innovation process far CAM technology. organisational behaviour. Organisational Dynamics 9(2):35-51 (Au-
IEEE Transactions on Engineering Management 36(3):90-100 ( l 988). tumn 1980).
33. Griffin, Abbie. Metrics far measuring product development cycle time. 57. National Institute of Standards and Technology. Malcolm Baldrige
Journal of Product Innovation Management 10(2):112-125 (1993). National Quality Award1992 Award Criteria. Washington, DC:
U.S. Department of Commerce, 1992.
34. Grindley, Peter. Turning technology into competitive advantage. Busi
ness Strategy Review 2(1):35-48 (1991). 58. Orlando B. CALS/EDI integrating speed and product delivery. Man
ufacturing Systems 12(10):56-58 (1991).
35. Hauser J. R. and Clausing, D. The house of quality. Harvard Business
Review 66(3):63-73 (1988). 59. Pappas, Chris. Strategic management of technology. Journal of Prod
uct lnnovation Management 1:30-25 (1984).
36. Haw, Amoldo C. and Majluf, Nicolas S. Strategic Management: An
Integrative Perspective. Englewood Cliffs, NJ: Prentice Hall, 1984. 60. Pavitt, Keith. What we know about the strategic management of tech-
nology. California Management Review 32(3):17-26 (1990).
37. Hax, Amoldo C. and Majluf, Nicolas S. The Strategy Concept and
Process: A Pragmatic Approach. Englewood Cliffs, NJ: Prentice Hall, 61. Pinto, Mary Beth and Pinto, Jeffrey K. Project team communication
1991. and cross-functional cooperation in new program development. Jour
nal of Product lnnovation Management 7(3):200-212 (1990).
38. Hayes, R. and Clark K. B. Exploring the sources of productivity dif-
ferences at the factory leve!. In: K. B. Clark, R. Hayes, and C. Lorenz 62. Prahalad, C. K. and Hamel, Gary. The core competence of the corpo-
(eds.). The Uneasy Alliance: Managing the ProductivityTechnology ration. Harvard Business Review 68(3):79-91 (1990).
Dilemma, Boston, MA: HBS Press, 1985, pp. 151-188.
63. Quinn, James B. Innovation and corporate strategy: Managed chaos.
39. Hayes, R. H. and Wheelwright, S. C. Matching process technology In: Technology in the Modern Corporation: A Strategic Perspective,
with product/market requirements. In: R. H. Hayes and S. C. Wheel- Me] Horwitch (ed.). New York: Pergamon Press, 1986.

64. Richards, T. Innovation and creativity, trees and pathways. R&D Man concept generation,
agement 21(2):97-108 (1991).
65. Roberts, Edward B. Managing invention and innovation. Research and product development,
Technology Management 31(1):11-27 (1988).
66. Roberts, Edward B. Entrepreneurs in High Technology. New York: production process innovation,
Oxford University Press, 1991.
67. Roberts, Edward B. and Berry, Charles A. Entering new businesses: technology acquisition,
Selecting strategies for success. Sloan Management Review 26(3):3-
17 (Spring 1985). and review the subprocesses and the key literature
68. Rochford, L. and Rudelius, W. How involving more functional areas assertions concerning practice and performance.
within a firm affects the new product process. Journal of Product
lnnovation Management 9(4):287-299 (1992).
69. Rosenthal, Stephen R. and Tatikonda, Mohan V. Competitive advan-
tage through design tools and practices. In: lntegrating Design and Concept Generation
Manufacturing for Competitive Advantage, G. I. Susman, (ed.). New
York: Oxford University Press, 1992, pp. 15-35. Concept generation concerns the process for identify-
70. Roussel, Philip A., Saad, Kamal N. and Erickson, Tamara J. Third ing customer needs and matching these with techno-
Generation R&D. Boston, MA: HBS Press, 1991. logical capabilities to generate potential new product
71. Stalk, George, Jr. and Hout, Thomas M. Competing against Time. New
York: The Free Press, 1990.
concepts and enhancements of existing products. It
72. Thamhain, H. J. Managing technologically innovative team efforts to- includes four main activities:
ward new product success. Journal of Product Innovation Manage
ment 7(1):5-18 (1990). generating new product ideas-identifying cus-
73. Tushman, Michael L. and Moore, William L. (eds.). Readings in the tomer needs and matching these with technologi-
Management of lnnovation, 2d edition. Harper Business, 1988.
cal capabilities to generate potential new product
74. Tushman, Michael L. and Nadler, David. Organizing for innovation.
California Management Review 28(3):74-92 (1986). concepts and enhancements of existing products;
75. Twiss, Brian. Managing Technological lnnovation. London: Pitman, planning product innovation-generating the
76. Tyre, Marcie J. Managing the introduction of new process technology:
product innovation strategy, including scope and
International differences in a multi-plant network. Research Policy rate of innovativeness and timing of market entry;
20:57-76 (1991).
77. Utterback, James M. Environmental analy sis and forecasting, In: initiating and supporting creativity;
Readings in the Management of Innovation. 2d edition, Michael L.
Tushman and William L. Moore (eds.). Harper Business, 1982, pp. exploiting innovation through new businesses
299-310. and supporting new product initiatives in existing
78. Van de Ven, Andrew H. Central problerns in the management of
innovation. Management Science 32(5):590-607 (May 1986).
79. von Hippel, Eric. The Sources of lnnovation. New York: Oxford Uni-
versity Press, 1988.
Looking first at generating new product ideas, suc-
80. Voss, Chris A. Implementation: A key issue in manufacturing tech- cessful innovations often arise in response to a recog-
nology, the need for a field of study. Research Policy J 7:53-63 (1988). nized need, and the customer plays an integral role in
81. Voss, Chris A. Measurement of innovation and design performance in the development of innovative ideas. Critical pro-
services. Design Management Journal 31 (l ):40-46 (1992).
cesses include how market needs and opportunities are
82. Voss C. A., Chiesa, V. and Coughlan P. Innovation Your Move, Self
Assessment Guide and Workbook. London: DTI, 1993. monitored and uncovered-especially, how functional
83. Wheelwright, Steven C. and Clark, Kim B. Revolutionirlng Product groups that meet the customer (such as marketing,
Development. New York: The Free Press, 1992. field service, and support) are involved in building
84. Wood A. R. and Coughlan, P. Northern Telcom: The gating process. such knowledge and how innovative customers (lead
In: Managing the DesigtiManufacturing Process, J. E. Ettlie and
H. W. Stoll (eds.). New York: McGraw Hill, 1990, pp. 223-242. users) are identified and involved in developing and
85. Wood A. R. and Elgie R. J. Early Adoption of Manufacturing lnno testing new product concepts and enhancements
vation. Ontario, Canada: University of Western Ontario, 1976. (7,54,55,79]. Hise et al. [42] show that marketing
should have a high level of involvement in new prod-
uct development particularly at the design and evalu-
Appendix l. Content of the Core and ation stage. Identification of customer problems is an
Enabling Processes essential starting point for successful concept devel-
opment. An important cause of failure has been iden-
tified as the absence of a perceived need (19]. Cic-
Core Processes
cantelli and Magidson (15] suggest that "design ...
In this section we examine each of the four core pro- could be greatly enhanced by focusing more attention
cesses: on understanding customer behavior and needs, espe-
l 996;13:105-136

cially in the initial stages of product development.'' project management of product development, in-
The marketing information system is the meaos by cluding tasks and procedures for taking a new
which many organizations identify changes that lead product from concept to launch;
to the development of the new product concept.
teamworking and project organization;
Researchers have highlighted the need for multi-
functional teams in product idea generation and transfer from design to manufacturing and distri-
screening to ensure that there is the diversity of infor- bution;
mation required [44,73]. However, marketing inputs industrial design.
alone cannot identify majar developments in the tech-
nical environment. Technical inputs also play a role Researchers have studied project management and the
and need to be balanced with marketing inputs. For organization needed to bring a new product concept to
example, product concept generation needs to be the marketplace [13]. For example, Cooper [20] and
shaped in such a way that both demand-pull and tech- Wood and Coughlan [84] described the stage-gate sys-
nology push are taken into account. A second activity tem for driving new product projects from idea
concerns product innovation planning, that is defining through to launch. Others have shown how successful
a product innovation charter including target business product development requires a certain degree of
arenas, the objectives of product innovation, and spe- integration and parallelism of steps, the concurrent en-
cific programs of activities. Adopting a formalized ap- gineering approach [ 16, 19]. In the area of cross-
proach to product innovation planning helps managers functional teamworking and related activities, Tham-
coordinate and integrate the activities involved. More- hain [72], for example, studied the characteristics of
over, it helps prioritize product development projects innovative team performance and identified a range of
in the face of multiple product opportunities, different tasks, people, and organization-related factors associ-
rates of innovativeness and timing of market entry, ated with successful innovation. Effective teamwork-
and limited resources [22,58,63]. Product concept gen- ing requires a high degree of cross-functional coop-
eration requires innovativeness and creativity. Mech- eration and the early involvement of key internal and
anisms such as eliciting new product ideas from em- external functions [16,40]; projects are more success-
ployees, supporting unplanned product initiatives, ful when project leaders are responsible from the start
circulating new product ideas, and rewarding entrepre- to the finish and their status, influence, and responsi-
neurial behavior have been recognized as part of the bility are clearly defined within the organization.
most innovative organizations. Innovativeness and Teamworking also requires that the input of functions
creativity can be achieved through both the adoption to project teams and the weight and phasing of func-
of more open organizational structures and the ere- tional inputs to the definition of the product concept
ation of individual roles critica! to innovation such as and of the functional specifications are clearly estab-
idea generators, product champions, gatekeepers, and lished [61,63]. Mechanisms used to facilitate and en-
product development program managers [5,28,45,51, hance two-way communication between the different
75]. groups involved in the product development process
Exploiting innovation requires new business devel- include meetings, liaison roles, sharing of common
opment techniques and procedures, and strategic and technology development systems, and use of elec-
organizational activities such as venture capital invest- tronic communication networks [72].
ments, internal ventures, sponsored spin-offs. The use Transferring from design to manufacturing and dis-
of each alternative may depend upon the familiarity of tribution is a key activity of the product development
the firm with the new product market and technology process. This activity includes linking manufacturing
[12,66]. and engineering (through prototype testing, rapid feed-
back from manufacturing to design and engineering,
handling of engineering changes, production ramp up)
Product Development and linking manufacturing and distribution. Again, the
Product development is the process whereby new early involvement of the different functions (design-
product concepts are taken through the stages of de- ers, engineers, manufacturing people, marketing
velopment, testing, and manufacturing to successful people) and the use of integration mechanisms allow a
launch and support of the product. Four main activities reduction in the number of late product changes and
may be observed: the ehange costs [21].

An often neglected aspect of product development Technology Acquisition

in the United Kingdom and the United States is indus- Technology acquisition involves monitoring, selec-
trial design. In many European companies, such as tion, and acquisition of technologies; the development
Braun, and Japanese companies, such as Toshiba, in- of new or improved technologies through R&D or
dustrial design is central to their product innovation externa! acquisition; and the exploitation of technical
strategies and is built in as an integral part of the knowledge. The main activities include:
development process [26].
formulating the company's technology strategy,
setting technological goals and plans;
Production Process Innovation R&D management and organization-including
the processes for R&D project management, the
Innovations of production processes are crucial as both use of externa! sources and relationships for tech-
a direct source of competitive advantage and indirectly nology acquisition, licensing, and building tech-
in association with product innovations. The main ac- nology alliances;
ti vities include:
management of the intellectual property-the
policies for protecting and exploiting the property
generating production process innovations-the rights.
process for generating a manufacturing strategy
matching manufacturing capabilities and market Formulating a technology strategy sets long-term tech-
needs and for developing new production tech- nological plans and strategies [36,43,53]. This activity
nologies; includes external technology intelligence, scrutiny of
interna! technological capabilities, technology selec-
implementing new production processes; tion, technology sourcing (make or buy in R&D [6]).
The critical areas are the identification of core and
continuous improvement of production processes. complementary technologies, the evaluation of a
firm's capabilities in core technologies, and the defi-
Formulating a manufacturing strategy and ensuring nition of the processes leading to a firm' s core com-
continuing innovation in production processes requires petencies that make it distinctive from competitors
mechanisms to understand the capability of existing [62]. Roussel et al. have suggested that in successful
processes, to ensure that the manufacturing processes firms, a partnership is established between business
support the business needs and objectives, to monitor managers and R&D people. This partnership ensures
and tap into externa! sources of process technologies, that decisions are taken considering all the factors rel-
to link product and process innovations [39,41]. Pro- evant to technology development programs, such as
duction process implementation is a critica! area and risk, reward, timescale, strategic relevance, and option
field of study [32,39,47,76,80]. Effective implementa- creation for further technology developments [70].
tion relies on the use of multifunctional teams, involv- The R&D management system should identify the
ing technology suppliers, the capability to match the key issues in organization and make the trade-offs
production organization to the production process in- explicit. Typical organizational trade-offs include the
novation [25,80,85]. In this way, the firm can match extent to which the control of technology development
technology complexity to the capability to adopt and should be centralized, the extent to which acquired
reduce the associated risk. A third area that has at- technological capabilities should be integrated into a
tracted the attention of researchers is that of continu- firm's R&D, the changes in the R&D organization that
ous improvement in process innovations [38]. These can reduce product development lead time, and the
improvements frequently result from a series of incre- extent to which there should be market research capa-
mental innovations. A number of practices facilitate bilities in R&D. Structural choices should be taken as
continuous improvement in manufacturing processes: answers to these key issues. In terms of R&D man-
creating work teams to identify opportunities for im- agement, successful organizations stress aspects such
provement; using process control data, customer feed- as communication, linked structural interfaces, trans-
back, and competitive benchmarks; and involving pro- parency and shared uncertainty, creation of a sense of
duction process developers in improvement after importance and urgency in individual researchers,
initial installation [14,23]. willingness to kill projects, and corporate-wide opti-

mization of resources [70]. Finally, technology acqui- Human resources represent a key asset for successful
sition requires explicit definiton of a policy for pro- management of innovation. In particular, the availabil-
tecting and exploiting intellectual properties and ity of appropriate people in the critical innovation
knowledge through policies for patenting, trade se- roles [73] can lead to a sustainable competitive advan-
crets, and licensing out [36,37,43]. tage [34,62]. These roles relate to different issues in
innovation: the entrepreneur, who exercises the con-
trol of the venture and assumes the risks of the busi-
Enabling Processes ness; the product champion, who adopts an idea for an
In this section, we examine the supporting or enabling innovation and make possible the innovation' s suc-
processes of innovation: cessful implementation; sponsors, who are the execu-
tives channeling resources to innovative projects; and
leadership-leadership from top management in technological and market gatekeeperes [ 5 ,51]. The
strategy, process, and setting and maintaining a relative weight and importance of these roles vary ac-
climate for innovation; cording to the firm' s organizational development, in-
resource provision-provision of appropriate or- cluding size and degree of diversification. Funding
ganizational and financial resources; innovation appropriately requires funding stability ( of
the total amount) and flexibility (so that short-term
systems and tools-provision and use of appro- opportunities can be exploited) [70,75,83].
priate systems and tools to support the core pro-
cesses of innovation.
Systems and Tools

Leadership This area addresses the relevance of methodologies,

systems, and tools for supporting the innovation pro-
The relevance of top management leadership in inno-
cess. There is a very wide range available. Which of
vation has been pointed out by many authors. For ex-
these are appropriate toan individual organization will
ample, the role of leadership is illustrated by Burger
vary greatly with the context, but in all areas there will
[13] in developing a research agenda for the Product
be a particular set that can support the core processes.
Development and Management Association. Here,
We see this area as comprising:
leadership was number one in the ranking of perceived
importance of major issues. Leadership looks at the the systems used to support the processes of prod-
top management involvement in setting goals and pri- uct development and the communication between
orities for innovation [ 4], championing the corporate the functions involved;
effort to achieve best practice for each of the core
processes of innovation, and setting stretch goals to the contribution of tools to achieve faster and
the organization [62]. Central to leadership is creating more effective product development;
a climate that encourages and supports innovation and the management of quality in the design process
entrepreneurship. This would include encouraging and the methods used to analyze and improve the
new idea development and risk taking, having a per-
quality of the innovation process itself.
formance measurement system that encourages this,
and disseminating the company's policies on innova- There is a wide-ranging literature on systems and tools
tion within the organization [56,74,78]. [69]. Systems include computer-aided logistics sys-
tems (CALS) [58], computer-aided design (CAD)
[2,49], and simulation. Tools include rapid prototyp-
Resource Provision
ing, K-J analysis, and quality function deployment
Resourcing innovation includes the mechanisms and (QFD), design for manufacture, and creativity
organizational processes for: [3,9,35,64]. These systems and tools are continuously
evolving, and sorne are specific to particular contexts.
recruiting, developing, evaluating, and rewarding In addition to specific quality tools, quality manage-
human resources required for innovation;
ment stresses the continuous improvement of pro-
funding innovative projects, product develop- cesses, which is equally as important in the process of
ment, R&D, and technology acquisition. innovation as anything else.

Appendix 2. Full Set of Scorecards

2 3 4

Generating New Product Concepts
New product development Product concepts developed New ideas sought in the Direct links with customers
unplanned. in one department with marketplace with research and leading users to identify
limited customer contact. into customer needs and expressed and latent needs.
with marketing and technical A broad range of functions
functions involved. involved in concept
development and screening
and with early analysis.
Product Innovation Planning
None! Next generation of products Up to two generations of Long-term planning for three
planned. products planned. or more generations of
Inventiveness and Creativity
Control systems and New ideas encouraged, but Risk taking encouraged and Employees' innovative and
organization discourage risk avoided. champions for new ideas entrepreneurial behavior
creativity. sought and supported. encouraged and rewarded.
Mechanisms available to
fund unplanned activities.
The Product Development Process
No product development Simple procedures applied to Project development on Established processes and
procedures. ali projects but no parallel major products planned in objectives with flexibility to
activities. phases with reviews. allow small projects to move
through quickly. Parallel and
integrated activities.
Team work and Organization
No teamwork and little Sorne use of functionally \Videspread use of Wide use of multidiscipline
communication between based teams but with weak multidiscipline teams. Clear teams with early
functions. project management and no project authority, interna! involvement by ali. Strong
involvement of other cross-functional review prior team leadership and with
functions prior to start-up. to development but limited team and leader empowered
involvement from purchasing to make decisions.
and suppliers.
Transfer to manufacturing and distribution
No transfer process. Designs Manufacturing-engineering Strong links between Manufacturing has effective
"thrown over the wall" to communication prior to manufacturing and design. capability to test prototypes
the next department. transfer. and ramp-up new products.
Effective handling of
engineering changes.
Industrial Design
No consideration of Design introduced at a late Use of interna! designers or Industrial designers involved
industrial design. stage in the process. externa! design as core part of project team
consultancies. from concept stage.

Appendix 2. Continued
2 3 4

Process Innovation
Generating Process Innovations
Serious differences between No manufacturing strategy: Manufacturing strategy Strong Iinks between product
process requirements and process technology bought ensures that process and process development.
technology available. off the shelf capabilities support market Information on new process
needs, lnvestment in technology actively sought
improving existing and and new processes tested to
developing technologies. gain experience.
lmplementation of New Processes
No attention to Implementation seen as Cross-functional Implementation teams stay
implementation. installation. implementation teams, together into full production
to ensure learning and
improvement. Active
involvement of suppliers,
Continuous lmprovement
If it isn't broken, leave it Focus on maintenance of Need for continuous W ork teams encouraged to
alone. processes, not improvement. improvement of processes identify opportunities for
recognized-primarily the improvement. Use of wide
responsibility of process range of interna! and externa!
engineering function. data to support improvement.

Technology Acquisition
Technology Strategy
No technology strategy and Inward-looking technology Understanding of technical The company understands its
no mechanisms for strategy identifies needs on a needs in each function with core competencies in
understanding technology. project-by-project basis, monitoring of trends and technology and innovation
product-driven joint ventares and has policies for al\ocating
and technical alliances. resources to build and
strengthen them. Monitoring
of the technologies use by
Selection Generation and Sourcing of Technology
"Not invented here Participation in industry Ongoing contacts with Explicit policies for sourcing
syndrome"-No R&D technical associations but little universities, government technologies, including
sourcing plan. externa! technology sourcing, agencies, industry consortia, in-house R&D, licensing in
etc. and close relationships and out, partnerships and
with leading suppliers and externa! linkages.
Environment and Regulation
No policies or controls-get Formal policies and Active management to Proactive, anticipating trends
away with what you can. procedures to deal with promote compliance and with line responsibility for
environmental and regulatory improvement. compliance. Products and
issues but passive general processes designed to
management. minimize environmental
impact and health and safety

Appendix 2. Continued
2 3 4

Innovation Goal.1
No management involvement No innovation goals and lnnovation and technology Explicit and challenging
in innovation. technical functions not capability seen as a means goals set for innovation with
represented at board leve!. of gaining competitive edge a long-term corporate
and incorporated in the understanding of how it can
mission statement. shape business strategy.
Processes for Generating and lmplementing lnnovation
Management not concerned, Management encourages Innovation management, Management is proactive in
good practice in innovation product realization, and ensuring best practice in
management. technology acquisition innovation and product
presented to and discussed at realization.
board leve!.
Climate for lnnovation
Management encourages General encouragement for Performance measures for Management ensures that
short-terrn profitability and innovation, but no innovation reviewed risk taking is encouraged
risk minimization by measurement or reward. regularly by board with a rather than penalized and
managers and employees at customer-led climate new ideas rewarded. It
the expense of innovation. encouraged. ensures that the technology
mission of the company is
shared and understood
throughout the company.

Human Resources
No human resource planning The human resources needed The skills required for Career structures support
for innovation; key skills for innovation generally innovation are identified and innovation through
missing. known and available, but are fully resourced through development in ali functions.
usually slow to be applied. recruitment and training.
Last year' s spend adjusted Industry average levels. Policies on how R&D Related to potential business
up for inflation and down R&D and innovation budgets should be funded. Sorne contribution over short- and
for cash availability. subject to sharp fluctuations mechanisms to ensure that long-term with mnima!
from year to year. capacity is available in fluctuations despite cash
suppliers, manufacturing, flow variation.
and support functions.

Systems and Tools

Limited use of information Information system usage Widespread information Systems geared to improving
systems of CAD. within functions. system usage, primarily for design effectiveness and to
one-way information flow shortening product
including CAD, CAD/CAM, development lead times.
and process simulation on a
functional basis to improve
design effectiveness.
Systems links with suppliers
and customers.

Appendix 2. Continued
2 3 4
Tools for lnnovation
No significant usage of Ad hoc too! usage, with no Sorne use of design tools to \Videspread use of
management and design clear objectives. improve product and process appropriate tools to capture
tools. design effectiveness and/or customer needs and to
creativity. ensure the effectiveness of
product and process design.
Established protocols such as
design for manufacture,
design for test, design for
customer use.
Quality Assurance
Limited quality management. Quality control in Quality practices and TQM program in place
manufacturing, but little procedures in place for including a focus on
involvement by engineering. quality assurance of products achieving improved
ISO 9000 possibly in place, and processes. innovation performance.
but focus on procedure only.
lncreased Competitiveness
Measurement and Goals
No measures of innovation Measures of financia! and Operational targets are set Customer satisfaction
performance or customer sales performance of new for sorne aspects of feedback surveys initiated
satisfaction. products and measures of innovation at departmental with feedback into the
product quality. leve!. innovation process.
lnnovation performance
Anecdotal evidence only. Positive trends in most Good-to-excellent results in Excellent results in major
areas. major areas with evidence areas with sustained results.
that results are caused by Results clearly caused by
active management of active management of
innovation. innovation.

Appendix 3. Example of Areas to Consider for In-Depth Audit of Processes

Product Development Areas to Address
Product development process: Describe the processes and a) The product development process; the scope of the process;
procedures for taking a new product from concept to launch. the phases, gates, the phase reviews, and sign-off procedures.
The balance between documentation and bureaucracy, the
extent to which the process is replicable and compliance is
mandatory, the degree of uniformity across the firm.
b) The degree to which simultaneous engineering approaches
are used; the degree of parallelism and integration of steps,
degree of task interdependence/overlapping working and
integration between the phases, which can take place and is
built into the process.
e) The flexibility of the procedures to allow for small projects
to move through fast.
d) The relationship between objectives of the development
process objectives (e.g., development cost, product quality,
development time, process predictability, product cost) and
the key development activities (e.g., specification,
integration, evaluation).
1996; 13: 105-136

Appendix 3. Continued
Product Development Areas to Address
Teamwork and organization: Describe how the development a) The location, dedication, and cross-functionality of project
process integrates ali relevant functions; ensures that product teams. The degree to which teams and their leaders are
concepts are properly defined prior to the start of development; responsible for the product from start to finish and are
communication between the different groups involved in the empowered to make decisions.
development process is facilitated. b) The status, influence and responsibilities of project managers
in the organization (e.g., spending authority, leadership,
budgetary control, conflict resolution, team selection, input to
performance reviews, dedicated full time or part time).
e) The procedures to ensure early involvement by all key
internal functions (including engineering, design,
manufacturing, customer interface functions, marketing, and
purchasing), and external organizations such as key suppliers.
d) The input of functions to project teams, the weight and
phasing of functional inputs to definition of the product
concept and creation of the functional specification. The
inclusion of project performance goals in functional
managers' performance reviews.
e) How major risk areas are identified up front, the mechanisms
for early identification of problems and how resources are
put into managing these areas.
f) The mechanisms for ensuring rapid two-way communication
between individuals' teams, organizations (inside and outside
the company). (1) Integrating mechanisms such as meetings,
physical location, secondment, liaison roles, matrix
organization. (2) Information systems such as shared
databases, CAD/CAM.
Transfer to manufacturing and distribution: Describe the process a) The strength of the links between manufacturing and
for ensuring the successful transfer from design into engineering. The capability of manufacturing to test
manufacturing and subsequent distribution to the customer. prototypes, to identify key risks and problems, and to give
rapid feedback to engineering and to ramp up new products
into fullscale production.
b) The involvement of line production workers to ensure that
new products can be manufactured easily.
e) The procedures for the effective handling of engineering
d) The degree to which product teams take products into
manufacturing and through to the marketplace and feed
information back into the innovation process.
e) The feedback from manufacturing on quality,
manufacturability, etc. to design.
Industrial design: Describe how industrial design is built into a) The use of internal expertise and external design groups to
the innovation process. ensure high quality industrial design.
b) The involvement of designers throughout the innovation