Components of Science-Based Innovation Measurements and Their Links to Public Policies

S. SURESH KUMAR

ABSTRACT This paper dwells on the measurement of effectiveness and impact in respect of innovations in science and attempts to correlate their links to public policies for funding. It is argued that these links are not often straightforward, making it necessary to analyze and assess how science relates to society more quantitatively. Based on a survey and analysis of relevant statistics, this paper seeks to identify signicant measures to indicate effectiveness in science and outline their relations and links to policy. This is necessitated in the context of a more cogent argument for public support of science. 2000 Elsevier Science Inc.

Introduction The traditional way to analyze trends in R&D performance is to examine the allocation of funds devoted to basic research, applied research, and development. The objective of the basic search is to gain more comprehensive knowledge or understanding of the subject without specic applications in mind, although it may be in elds of present or potential commercial interest. Applied research is aimed at gaining knowledge to determine the means by which a specic, recognized need may be met. It includes investigation oriented to discovering new scientic knowledge that has specic commercial objectives with respect to products, processes, or services. Development denotes the systematic use of knowledge gained by research directed towards the production of useful materials, devices, systems, or methods, including the design and development of prototypes and processes. These denitions correspond to popular models that presume that innovation occurs in a linear manner, straight-line progression through the stages. Performance of basic research produce scientic breakthroughs that lead to applied research, and in turn, to development and diffusion of commercial products and processes. Although the simplicity of the above linear model has been appealing to policy makers, these traditional categories of basic research, applied research, and development have been found to be not always ideal in describing the complexity of relationship between science, technology, and innovation in the real world.
Address correspondence to S. Suresh Kumar, Head, Planning, Regional Research Laboratory (CSIR), Trivandrum 695 019, Kerala, India.

nological Forecasting and Social Change 64, 261269 (2000) 2000 Elsevier Science Inc. All rights reserved. 655 Avenue of the Americas, New York, NY 10010

0040-1625/00/$see front matter PII S0040-1625(00)00087-1

262

S. S. KUMAR

Alternative models may provide a more realistic depiction of the innovation process. But they are also probably quite complicated to be used in collecting reliable data for policy-making purposes. During the celebration of the 50th Anniversary of science, all the commentators recognized the report The Endless Frontier, known more familiarly as the Bush report (in honor of Vannevar Bush), as the centerpiece of post-World War IIs U.S. Science Policy [1]. Some have expressed regret about its legacy for our present time, across national boundaries, because U.S. leadership in science has inuenced others to emulate this. J. R. Steelman [2] brought out a document Science and Public Policy: A Program for the Nation, which discussed at length the nature and scope of sciencegovernment relations in the transition to peace-time science. A comprison of the Steelman report and the Bush report reveals a good deal about the political and scientic context and concerns of the late 1940s. Both the documents were aimed at providing a national system to enable it to manage its own R&D programs and effectively coordinate diverse research activities of government, industry, and universities. One of the MIT students of Bush, F. Terman, adopted Bushs ideas about academicindustrial collaboration to inspire a new generation of scientists and engineers at Stanford University. The ultimate result was silicon valley in California. The Steelman report revealed data on the steadily declining share of national research outlays going to universities since the 1930s. Hence, it also recommended a doubling of the nations R&D expenditure within a decade. It proposed that R&D spending should be linked to national income, and noted that the target for 1957 would amount to 1% of GNP for that year.With 20% for basic research, 14% for applied research in health and medicine, 44% for development (nonmilitary), and 22% for defense. Recent Japanese science policy initiatives seemed to have derived their stimulus from this approach. These documents, taken together, eventually led to an informal social contract with regard to science policy implementation in the United States and elsewhere [13]. But the relevance of this contract is now in question. Recent changes in the context for science have revealed an underlying policy problem on account of funding, together with demands for greater accountability, emergence of complex environmental and economic problems, etc. There is, at present, an urgent need for a model to demonstrate the correlation of science funding to societal benets [4, 5]. The concept of the social contract, promulgated through the Bush report and the Steelman document, assume that scientic knowledge is essential to meet national needs, that a simple linear model describes how science meets national needs, and that the scientic community requires relative autonomy from political and societal concerns. Both the documents assume that societal benets and proportional to the support of basic research even though the pay off is unpredictable and may be distant. Policy makers, funding agencies, and legislators seemed to have rejected the documents on account of the implicit assumptions that science assumes no responsibility to apply knowledge for social benet, and society does not set scientic priorities. With this assumption implicit in the social contract, conicts are bound to occur because democratic politics demand that there should be no expenditure of public funds exempt from socio-political accountability. The old contract needs to be renegotiated. In the new contract it is important to analyze and assess how science relates to society [5, 6]. There is a vital need for science to improve its relation to society. We have to explore complex explanation of new technology [7, 8]. D. Stokes suggests a new model in which there are inherent overlaps between basic and applied research, citing numerous major scientic advancements as

SCIENCE-BASED INNOVATION MEASURES TO PUBLIC POLICIES

263

examples [810]. D. Stokes model describes several different categories of research: pure basic research, purely applied research, and strategic research. OECD denition now differentiates between pure basic research and oriented basic research. But the question still remains as to what extent government should support different categories and how are the benets from such funding support should be assessed for effectiveness [11, 12].

Pilot Innovation Studies The National Science foundation seeks to provide an answer through studies in association with the U.S. Bureau of Census. The proof of the effectiveness of civilian science policies of the government is obtained from the performance of innovative rms within the country. This suggests a need for an evaluation of the industrial innovation process. The study conducted in 1993 revealed several characteristics of U.S. innovating companies. The U.S. innovators obtained most of their information from internal sources and clients. Government labs were found to be insignicant sources, highlighting the need for greater focus on oriented basic research than pure basic research or developmental work in government labs. This is also corroborated by G. H. Daniels [10], D. E. Stokes [19]. R. Byesly, and R. A. Pilke, Jr. [16]. In fact, an NSF report by the division of science resources studies, while commenting on the technological importance of Asian patents, observes that the information in patent documents reveal that many Asian patents represent several advances in technology [13]. Although U.S. invention has strong ties to basic research, increased patenting activity in NICs (newly industrializing countries) in Asia is more correlated with enhanced rates of high-tech patent acquisition. U.S. high-tech products constitute 25% of all merchandise purchase from the United States by Asia. Intraregional trade in high-tech products especially from Japan is also a primary source of foreign technology products. Singicantly, these developments are accompanied by S&T restructuring and increased allocation of public funds for science in countries like south Korea and Taiwan, which have registered the greatest support in patenting activity among the NIC. The most rapid growth in U.S. patenting among Asian inventors was recorded by those from Asias newly industrializing countries, particularly Taiwan and south Korea. NIC patenting in the United States quadrupled during the 70s, and increased tenfold during the 80s, with the most dramatic growth registered after 1987. This closely tracks the rapid growth of the industrially funded R&D spending in these two countries, but also government support for science. During this period these NICs have gone through a process of S& T restructuring, with emphasis on government support for science [13, 14]. Of course, the international trends in industrial R&D reveal that industrial sector is the main source of new technologies, and the United States leads the world in the sale of intellectual property [14]. The high-tech performance of Japan and NIC could also be related to industrial R&D intensity in their own countries as well as acquisition of technological know-how from abroad, particularly in the United States. R&D Intensities OECD indenties six industries as being high-tech based upon their high R&D intensities (R&D spending as a percent of production) relative to other manufacturing industries. Following are the six high-tech industries and their R&D intensities in the 90s. Aerospace 20%, ofce and computing equipments 17%, communication equipment 10%, drugs and medicines 5%, scientic instruments 5%, electrical machinery 4%, motor vehicle 3%, chemicals 2.5%, equipment for other manufacturing industries 2%. In

264

S. S. KUMAR

constant dollar terms, production of high-tech manufactures by the major industrialized nations more than doubled in a decade, while other manufactured goods grew by just under 30%. In the early 80s, output by high-tech industries represented under 14% of global production of all manufactured goods. By the 90s, it rose to 24%. High-tech manufactures represented 25% of the total U.S. production of manufactured output (15% in the 80s), and in European Union countries it rose from 12 to 18%. In Japan, it increased from 16 to 30%. Thus, by the late 90s, high-tech manufacturing output was estimated to be 30% of the U.S. manufacturing output, 34% of the Japanese, and 20% of the European Union countries. Throughout the 80s and early 90s the United States was the worlds leading producer of high-tech products. While the United States has been maintaining its share since the 80s (except for a mild decline in the mid-80s, from which it soon recovered), Japan has moved up by 6 percentage points since the 80s, mainly at the expense of EU high-tech producers: Germany, France, and Italy.

Market Competitiveness The market Competitiveness of individual U.S. high-tech industries shows that while scientic instruments, drugs and medicines, and aerospace gained, U.S. computer and ofce equipment industries experienced a sharp drop in global market initially but gained most of the loss. The United States maintained this leadership by means of intensive industrial R&D supported by cooperative linkages with knowledge production in Federal Laboratories. In high-tech exports, the United States share of the foreign markets dropped from 23% in the 80s to less than 20%. Japan surpassed the United States and Germany in overall high-tech exports by the late 80s, and accounted for nearly 25% of the OECD member countrys high-tech product exports by the mid-90s compared with 18 for the United States and 12% for Germany. EU manufactures have been responsible for about 50% of the OECD high-tech exports. The export market share of the Untied States is highest in aerospace (40%), whereas it is also highest in communication equipment, nearly (35%), and also computer and ofce equipment, nearly (30%), whereas for Germany it is high in pharmaceuticals, electrical machinery and instruments, at (20%). These also broadly represent the R&D intensities in the respective countries in these areas not only in industry, but also in terms of government funds to support oriented basic research as well as applied research. Indirect Impact Recently, increasing attention has been paid to indirect benets and the measurement of competency benets to relate outputs to R&D investments by government and industry. Specic methodological advances include renements of the notion of incrementality and attribution. It was found that quantifying the direct impact of a selected number of R&D projects using the techniques of benet-cost analysis is not adequate [14, 15]. A study was carried out by the ARA Group (consultants) with SPRU UK to review the state of the art in R&D impact assessment, which concluded that methods are available that provide a qualitative picture of the impacts of R&D patent analysis citations, science linkages, R&D expenses, pattern of funding, high-tech components, etc., and can be effectively used as partial indicators for impact assessment [1517]. These indicators help to assess indirect impacts like acquisition of new knowledge of the latest R&D ndings, advice, and expertise based on this knowledge, which lead to

SCIENCE-BASED INNOVATION MEASURES TO PUBLIC POLICIES

265

economic benets and training of new researchers who eventually contribute to increased innovative activity. The basis for recognizing the importance of indirect impacts of R&D is the notion that most innovations do not follow the classic linear model. The stability of cumulated known knowledge is often the rst line of reference for innovation process, and not current research.

Economic and Social Signicance of Science & Engineering Research Academic research plays a key role in enabling technological advances in the private sector, especially in medicine and electronics in the United States. According to one survey, approximately 10% of new products and processes depend on recent academic research. Another survey provides evidence that the association between academic and industrial research has been strongest in medicine and electronics. For example, in 1990, universities were responsible for 18% of all U.S. patents dealing with genetic engineering and recombinant DNA, 16% of patents dealing with natural resins/peptides or proteins, and 12% of patents dealing with chemicals involving microbiology and molecular biology. There is often a signicant delay (1015 years) between dissemination of fundamental knowledge and its eventual effect on industrial processes. Measurements of the returns to investment of R&D have long been studied by economists. Determining economic value of S&T output is a difcult proposition. Economic value is not congruent with ethical or aesthetic value, which underlie social decision making. The effect of private R&D could be divided into two groups: (1) direct impact through productivity enhancement in the R&D performing industry, and (2) indirect impact through productivity enhancement in other industries interacting with the industry performing R&D. The indirect imports have been founded to be larger [18]. This is best revealed by the computer industry, which has the highest productivity growth and the highest ratio of R&D to sales. The relationship between R&D and productivity is not as strong for other areas, though singicant. Nevertheless, given the complexity of production processes, there are difculties in calculating productivity changes attributable to R&D involving analytical and philosophical issues. But such limitations tends to produce understimates of the importance of R&D [19, 20] (Nordhaus, 94, Center Foundation Discussion paper No. 1078, Sept.). The focus of the public R&D budget in the United States has been on defense and health primarily (75%), whereas Japan has focussed more on energy and advancement of knowledge (70%). Government funding of academically related research has been high in the national program of health in the United States, and the United States has maintained its high-tech industrial leadership in drugs and medicines. Moreover, the United States share of world S&T articles by eld has been highest in space sciences, clinical medicine, biomedical research, and biology. At the same time, focal areas of interest for Japan are physics, engineering, and technology, explaining to some extent its high-tech ascendancy in electronics, communications, instrumentation, and advanced materials. Indicators Based on Patenting Trends International patenting trends in critical technologies of advanced manufacturing, biotechnology, and informatics reveal that Japan, with a share of 44%, lead in advanced manufacturing through the 80s, and the United States share declined from 72 to 60% in biotechnology during the same period. In informatics, Japan has increased its share

266
TABLE 1 Patent Families Country United States Japan United States United States Japan 8185 10% 0.5 70% 1.6 66% 36% 54% 8590 40% 2.0 40% 1.0 1.8% 39% 31% Cost & increase in funds 30% 20%

S. S. KUMAR

Area Advanced Manufacturing

Biotech 40% 30% Informatics

by 10% from 23 to 33%, whereas the United States share has decreased by 5% from 38 to 33%. At the same time, Japan has restructured its R&D policies to orient more towards basic research, particularly to support innovation in robotics and communication equipment, in addition to industrial R&D in these areas [20, 21]. The basic science links of industry has also increased during the period. For example, in the United States, academicindustry coauthored S&T articles has increased from 22 to 35%, with pronounced growth in clinical medicine and biomedical areas (from 30 to 45%) [22, 23]. This reveals the industry interest in pursuing basic research. Trends in U.S. patent ownership suggest Japans share has gone up from 11 to 21% during the 8095s, whereas the U.S. share has come down from 62 to 55%. But Japanese patents have been in the nature of process/product improvements in the 80s, though in the 90s the picture has altered to reect greater science linkages. Japanese investments in basic research have also gone up since late 80s [24]. An analysis of statistics on patent families, in terms of highly cited ones and citation ratios with reference to critical technologies show that the United States could increase its share of highly cited families in advanced manufacturing and informatics as well as citation ratios by enhanced R&D funding while maintaining its lead in biotechnology (Table 1). Patent activities have been evaluated on the basis of technology cycle time (TCT) and science linkages. Indices like current impact index are used to ascertain the degree to which a countrys patents contain important technological advances. In 80% of commercially important industries Japanese patents were cited more often, suggesting the effect of higher investments in industrial R&D. But Japanese patents trailed in respect of science linkages. Japans scores also lead in TCT, suggesting that their patents have a greater impact on advancement of new technologies. But U.S. invention tends to be more fundamental.

High-Tech Production Take a look at the gures for global production and exports of high-tech industries (Table 2).
TABLE 2 High-Tech Exports 1983 Country United States Japan 3 2 (in lakhs) 6.5 5.0 1993 Share of all mfg. 83 23% 10% 93 20% 30% Const US$

SCIENCE-BASED INNOVATION MEASURES TO PUBLIC POLICIES

TABLE 3 R&D Investment Shares Area Computing M/c. Electrical M/c. Comm. equipment Early 80s 50% 90% 95% Mid-90s 90% 120% 105%

267

This coincides with the period of S&T restructuring in Japan to focus more on basic research, particularly in computing m/c, electrical m/c, and communication equipment.

Japans Share vis-a ` -vis the U.S. Figure The data on R&D performance by the United States and Japan manufacturing during 19731993 show that in constant price terms the U.S. R&D investment has increased by 80%, whereas in Japan, in increase has been over 450%, with the highest being in computing machine and equipment and motor vehicles (Table 3). Changes in Government Policy in Japan Key points in Japanese S&T policy changed with the times. The issue of S&T in the 50s was survival, including war rehabilitation. The country was, of course, more ravaged than the United States by war. The administrative structure for S&T were government organizations. The STA was establised in 1956 to drive government S&T policy. Also, the council for S&T was established in 1959 to make recommendations for S&T. But by the mid-60s, the issue was growth, to expand society and economy through science. The policy decision was taken to create a national R&D program called the Large Scale Program. But in 70s the issue was energy and environment, which called for strategic research, and by the 80s, the issue was basic research, as such [24]. The government policy made a transition from technology to science [24]. These are reected in programs promoting basic research like ERATO in 1981 of the STI. The Frontier Research Program (86) of RIKEN (Institute of physical and chemical research) and the R&D project on Basic Technology for Telecom, and Agriculture and Forestry have been operating their programs related to basic research, almost competitively. The government aims to encourage basic research, and increase collaborative research. The Japanese government has pushed for research association and cooperative R&D. This policy of transition to basic research has been strengthened since the mid-80s, which coincides with Japan high-tech ascendency in OECD since the 90s, in respect to a variety of factors, as already examined [24]. There are many joint programs between universities and society in the United States, including an enterprise forum between California Institute of Technology and MIT and CONNECT of the University of California in San Diego, which is a partnership strengthening ties between the uiversity and rms, and supporting the development of local high-tech industries [23]. Needless to say, the U.S. high-tech performance and leadership has been abetted through these strategic alliances in knowledge production [25]. It was found that overseas-based facilities perform basic R&D activities better than Japan-based facilities, though in developmental activities Japan scored better. Conclusion Public funds for research and their socio-economic impact have to be correlated through measures of effectiveness. This will help to delineate to what extent governments

268

S. S. KUMAR

should support different categories and how are the benets are to be assessed for effectiveness. As argued in this paper, indicators that can be used as a measure for this purpose are (1) R&D intensities (i.e., R& D spending as a percent of production); (2) market competitiveness or increase in the market share of high-tech products. In addition to such direct measures, indicators for indirect impact are also important. An assessment of such impacts can be obtained through patent analysis, patent citations, science linkages, R&D expenses, funding patterns, and high-tech components. The basis for recognizing the importance of indirect impacts of R&D is the notion that most innovations do not follow the classic linear model. Moreover, quanitatively determining the economic value of S&T output is a difcult proposition. Indirect impact through productivity enhancements in the R&D performing industry and other interacting industries can be effectively used as indicators. Indicators based on patenting trends also serve a useful purpose. It is found that from an analysis of patenting trends and scientometrics that basic science links of the industry is correlated with its patenting activity. These have been found to closely follow increased allocations of public funds for strategic research. The case of Japan is highlighted to reveal how S&T restructuring with an increased focus on publically funded research in Japan has yielded commensurate results in terms of the above-cited indicators of research effectiveness and its impact measures. I gratefully acknowledge the valuable suggestions and assistance given by K. P. Sadasivan, Library Ofcer at the Regional Research Laboratory, Trivandrum, in accessing the relevant literature. I am also grateful to the unknown referee who, by offering his critical comments, has helped to improve the nal report. Analysis is based on data from Science Indicators, 93 & 96, the National Science Board, Washisngton, DC.

References
1. Bush, V.: ScienceThe Endless Frontier. A report to the President on a program for post-war scientic research, July 1945 (reprinted by NSF, Washington, DC, 1990). 2. Steelman, J. R.: Science & Public Policy. Government Printing Ofce, Washington, DC, August 1947 (reprinted by Arno Press, New York, 1980). 3. Manseld, E.: Academic Research Underlying Industrial Innovatioan: Source, Characteristics and Fnancing, Review of Economics and Satistics 77(1), 5565 (1995). 4. Terleckyj, N.: Effect of R&D on Productivity. Natl. Plg. Association, Washington DC, 1974. 5. Terleckyj, N.: Direct and Indirect Effects of Industrial R&D on the Productivity Growth of Industries, in New Developments in Productivity Measurement & Analysis. University of Chicago Press, Chicago, 1980. 6. Levy, D. M., and Terleckyj, N. E.: Effects of Govt R&D on Private R&D Investment and Productivity: A Macro-Economic Analysis. Paper presented before the Annual Meeting of the Southern Econ. Assocn., Atlanta, Nov. 11, 1982. 7. Leydon, D. P., and Link, A. N.: Why are Govt R&D and Private R&D Complements 2, Applied Economic 23, 16731681 (1991). 8. Nadiri, M. I.: Innovation & Technology: Spill Over. Working paper No. 4423. Harvard University Press, Cambridge, MA, 1993. 9. Stokes, D. E.: Pasteurs Quadrant: Basic Science & Technological Innovation. Bushings Institution Press, Washington, DC, 1997. 10. Daniels, G. H.: Science, 156/699 (1967)Ofce of Technol. Assessment, Research Funding as an Investment: Can We Measure the Returns (1986). 11. Congressional Research Services, Linkages between Federal Research & Development, Funding and Economic Growth (1992). 12. Hee, C. H., and Hirasawa, R.: Changes in Jap. Govt. Policies to be a Front-Manner in S&T, Science & Public Policy 25, 4754 (1998). 13. Science Resources Research Division, National Science Foundation: Asias New High-Tech Competitors. NSF 95-309, NSF, Arlington, VA, 1995.

SCIENCE-BASED INNOVATION MEASURES TO PUBLIC POLICIES

269

14. Kiba, T., and Collinson, S.: R&D Performance in Japanese Companies: A Relative Evaluation, Science and Public Policy Aug., 227 (1998). 15. Albert, M. D., Areny, D., Navin, F., and McAllister, P.: Direct Validation of Citation Counts as Indicators of Industrially Important Patents, Research Policy 220, 251259 (1991). 16. Byrely, R., and Pielke, R. A., Jr.: The Changing Ecology of United States Science, Science 269, 1531 (1995). 17. Pavitt, K.: Patent Statistics as Indicator of Innovative Acivities: Possibilities and Problems, Scientometrics 7, 7779 (1985). 18. Navin, F., and Olivastroo, D. M.: Technology Indicators Based on Patents and Vitation, Handbook of Quantitative Studies of S&T. Elsevier Science Publishers BV, Amsterdam, 1988. 19. Trajtenberg, M.: Economic Analysis of Product Innovation: The Case of CT Scanner. Harvard University Press, Cambridge, MA, l990. 20. Scherer, M.: Research on Patents and the Economy: The State-of-the Art. Contribution to the 1st EPO-IFO Workshop, Munich, 1992. 21. Sounder, W. E.: The Evolution and Transfer of National Technologies. A Conceptual Model with Comparative Indices, Journal of Scientic & Industrial Research 54(April) 231242 (1995). 22. Griliches, A.: Productivity, R&D and the Data Constraints, American Economic Review 84, 123 (1994). 23. Manseld, P., and Lee, Y., Jr.: The Modern University Contributor to Industrial Innovation & Recipient of Industrail R&D Support, Research Policy 25, 10471058 (1996). 24. Preeman, C.: Technology Policy and Economic Performance: Lesson From Japan. Printer Publishers, London, 1987. 25. Abelson, P. H.: The Changing Frontiers of Science & Technology, Science 273(26), 445447 (1996). Received 5 March 1999; revised 14 July 1999; accepted 15 February 2000