Set based concurrent engineering: serious gaming and implications for practice

Endris Kerga1, Marco Taisch1, Sergio Terzi2, Monica Rossi1

1

Politecnico di Milano, Piazza Leonardo da Vinci 32, 20133, Milan, Italy, endris.kerga@mail.polimi.it, marco.taisch@polimi.it, monica.rossi@polimi.it 2 Universit degli studi di Bergamo, Marconi 5, 24044, Dalmine (BG), Italy, sergio.terzi@unibg.it

Abstract This paper presents a Serious Game (SG) about SBCE (Set-Based Concurrent Engineering) which is one element of Lean practice in Product Development (PD). The game is structured in two stages that simulate the traditional approach to product concept development called PBCE (Pont Based Concurrent Engineering) and SBCE processes. Although SBCE is becoming popular in academia, its understanding and level of adoption in practice are low. Thus, the game is designed to bring a hand on experience to practionaires to understand and reflect on the benefits and applicability of SBCE in a particular companys context. This paper presents some of the learning outcomes gained through running the game in a company which designs and manufactures products for HVAC/R (heating, ventilation, air conditioning and refrigeration) markets. Moreover, some practical and theoretical insights gained throughout the game play are discussed. Keywords Lean product development, set based concurrent engineering, point based concurrent engineering, serious gaming

1 Introduction
Set Based Concurrent Engineering (SBCE) is an element of lean thinking in product development (PD). It is effective in early stages of design when concepts are generated and selected [Sobek, Ward, Liker, 1999], [Ward, Liker, Cristiano, Sobek 1995]. In a traditional approach which is called Point Based Concurrent Engineering (PBCE), a single concept is selected as early as possible assuming that it will be feasible. However, PD is characterized by uncertainties due to changes in customer requirements, manufacturability issues, sub-system configurations and so on. Thus, projects suffer from design reworks due to the so called false positive feasibility, where project teams assume a concept is feasible, but will learn later in the development process that it is not [Oosterwal 2010]. Lean companies use SBCE approach to tackle such a problem by effectively utilizing product knowledge (lesson learned) to generate and evaluate alternative design concepts. Unless a concept is proven to be infeasible, designers wont eliminate it from a solution set. Moreover, in SBCE process, feasibilities of design sets are realized through integration events rather than gates reviews as in the traditional practice. The unique feature in SBCE process is that communication is based on proven data. Negotiation within multiple teams is facilitated by a pull event where teams can visualize risk and opportunities using trade-off and limit-curves. Finally, PD teams converge into an optimal design taking rough objective criteria (such as cost, quality and time), so as the process will continue to detail design and development stages. However, the extant literature presents SBCE process as a set of principles, and its performance benefits are primarily based on evidences from large Automotive and Aerospace industries. The prominent and successful cases of SBCE in Automotive are reported from Toyotas PD [Sobek, Ward, Liker, 1999]. However, Toyotas PD has peculiar characteristics that are necessary for effective implementation of SBCE process. For instance, Toyotas success partly can be attributed to its dominating power to leverage suppliers capabilities to facilitate SBCE process.

Suppliers of Toyota are required to explore multiple alternative muffler prototypes, test them into different working conditions and bring performance results in the form of trade-off curves to win a bid for a project. Toyota thus offsets the extensive portions of SBCE process to partners, and pulls knowledge to support its internal processes. Nonetheless, such kind of opportunities might not be available for many industries (e.g. small and medium industries, SMEs). In this regard, a successful implementation and desirable performance outcomes from SBCE cannot be guaranteed for all kind of industries for granted. A recent study on Swedish industries which are implementing SBCE reports a reduction of up to 75% product cost, 50% reduction in lead time, 50-75% improvement in product technical performances (innovation), and 50-100 % reduction in project risks, warranty cost and number of engineering changes [Raudberge 2010]. In the contrary, some industries are observed to have negative results applying SBCE. For example, in some industries there were a 25% increment in lead time and a 25% increment in development cost. Some firms obtained neither gains nor losses in adopting SBCE. Those companies who attain negative results and those who had zero loss/gain were asked about the rationale of the results. Most companies answered that SBCE is not the way they normally used to work [Raudberge 2010]. In another field study on 19 Italian manufacturing industries, it is reported that the awareness level of SBCEs principles as well as how to effectively apply it in PD process are not well understood yet [Rossi, Kerga, Taisch, Terzi, 2011]. These left both academic and practical gaps that need further investigations. First, a learning method to execute SBCE process need to be provided so as to introduce and bring a hand-on experience to practitioners. Such a method can bring the theoretical principles of SBCE into more tangible or concrete concepts for practionaires (product designers and project managers) to reflect on SBCEs applicability in their own real process before adopting it in real design practices. Second, SBCEs advantages, limitations and implementation barriers should be investigated from practical point of view in a particular companys context. Answering the above gaps helps to increase the awareness and successful implementation of SBCE process across industries, in particular, for small and medium industries (SMEs). The main aims of this paper are to try to fill the gaps and suggest a roadmap for successful implementation of SBCE process. Based on the SBCE principles [Sobek, Ward, Liker, 1999], an SBCE serious game (SG) using LEGO bricks has been developed. Moreover, it is validated in one case company to investigate the learning outcomes of the game as well as to collect feedbacks from to layout the necessary enablers needed for its success. The company designs innovative humidification and control systems in the HVAC/R market (www.carel.com). Mechanical, electrical, software designers, and project managers who have different years of experiences have participated in playing the game and in giving feedback. In section 2, introduction to the games features will be introduced. In section 3, a model to evaluate the learning outcomes of the game will be discussed. Brief summaries of the results found in the game play will be presented in section 4. In section 5, the methodology followed to develop and interpret an SBCE roadmap for this particular company will be discussed as implications of the game results for practice. Finally, in section 6, the conclusions of the paper and future possible researches are outlined.

2 Introduction of SBCE Game

To design the SBCE learning tool a serious game approach is used. In general, the application of games with the aim of education and learning is defined as Serious Games or SG [Wouters, van der Spek, Oostendorp, 2011]. In SG, players assume different roles and are involved in simple and complicated decision making processes, which makes it attractive for SBCE process where alternative design exploration and convergence involve multiple-view points. Moreover,

SG creates a safe and entertaining environment, so that players from the industry freely experiments SBCE process without interfering in an actual PD process. In the game, players have to design a simplified airplane structure as shown in Figure 1, using different type of LEGO bricks. The airplane has four sub-systems to be designed (body, wing, cockpit and tail). The game is divided into two stages: first stage, where players design an airplane for a given list of customer requirements following a PBCE process; second stage, where players are provided with the necessary instruments to execute SBCE process. The instruments will help players to explore alternative design concepts, communicate about alternative solutions within a team, and converge into a preferred (a high value) airplane structure. After each stages, players performances breakdown in terms of cost and time of development will be provided to facilitate discussion. The game is played with four players in a team, and each player represents sub-system departments (body, wing, cockpit and tail).
lc

LEGO Components

LEGO types

Number of Points 6

Body

10

L lb
3
wb

lw

Wing

Ws

lt
3

Tail

ts

Key "L" : Length of Airplane "lb" : Length of Body "lc" : Length of Cockpit "lt" : Length of Tail

"lw" : Length of Wing "wb" : Width of Body "ws" : Wing Span "ts" : Tail Span

Cockpit

Figure 1: A simplified airplane to be designed in SBCE game (using LEGO bricks).

Figure 2 shows the inputs, the steps and activities of the SBCE game. The main inputs to the first stage of the game are customer requirements and supplier component catalogue. The list of customer requirements to build an airplane structure were made intentionally to be vague, for example, the number of passengers might be from 90 to120 and the wing span could be 7 to 15. Such range based customer requirements (vague) reflect the reality, in which customers often suggest imprecise information, and force designers to explore their concept solutions wide open. In the game, these requirements can be handled in different ways in the first stage (PBCE process) and in the second stage (SBCE process). Thus, players will understand the advantage of following SBCE than PBCE process to achieve the requirements right the first time. In the game, there are five customer requirements: number of passenger (Np), airplane weight (W) (airplane structure (Wa) and passengers weights (Wp)), length of airplane (L), wing span (ws) and tail span (ts). The supplier components are LEGO bricks in different sizes and shapes that are used to build body, wing, tail and cockpit. Each brick has circular points on the top, and the number of points on the top of a brick define the characteristics of the component. A single point on a brick has the following character: cost (10), lead time or component ordering time (0.5), capacity (3), weight (100), length (1), and width (1).

(PBCE process)

Input to the game

Vague customer requirements
Supplier catalogue/ components

F A I L

Re-design the Airplane structure. Redesigning has penalty cost and time!

2.Testing
Stability, Geometric configuration, Aerodynamics

PASS

Evaluate performances

Input to the second stage SBCE process enablers

A. QFD (tradeoff curves) B. Set based Communication (checklists) C. Knowledge provision (limit curves) D. Rough cost and time evaluations

3. Stage two
(SBCE process)

4. Measuring the learning outcomes based on the results in the two stages

Figure 2: Inputs, steps and evaluation of learning outcomes in SBCE game.

2.1

Stage one and testing

Taking the customer requirements and the supplier catalogues, players will be asked to build an airplane in this stage. This stage simulates PBCE where players first design an airplane structure, build it and then test it to testing constraints. Design-Build-Test approach is what many non-lean organizations follow at early stage of design [Kennedy 2008]. Once players finish designing and building a prototype design in the first stage, they should submit it to testing department to check for stability, flying conditions and dimensional configurations as seen in Table 1.
Constraints type 1 2 Length Wing span Ratio of weight (RW) Constraints equation 2/3 L =< ws < L Rw = Wp/Wa < 1.25

Wa = weights of all airplane components or structure Wp = NP * 60 (Average weight of each passenger) 3 4 Airplane stability Alignment between body and cockpit lt < lw lc = wb

Table 1: Fictions passing constrains from testing department in SBCE game.

The facilitator of the game acts as a testing department. Players will not be given these testing constraints at the start of the game. If the design fails, the prototype should be redesigned. Redesigning has penalty costs and additional time to be penalized. After the first trial the testing constraints will be given to player. If the prototype passes the testing constraints, players will be given the breakdown of their performances in terms of cost and time. The determinations of cost and time are given as follows: Total development cost (C) = cost of components (cc) + cost of iteration (ci) + cost of penalty (cp) Where, cc = Total number of points*single point cost, ci =30% of cost of components (this is an additional cost if players fail to pass testing constraints), and cp (is an additional cost if players

Evaluate performances

Garris, et.al (2002) framework for measuring learning outcomes in serious gamming Questionnaire is prepared based on Likert five scale model (Likert 1932)

1. Stage one

4.

fail to meet customer requirements). cp is determined based on unsatisfied customer requirements following rules in Table 2.
Unsatisfied requirement cp Np 30%*cc L 40%*cc ws 10%*cc ts 5%*cc W 20%*cc

Table 2: Rules to calculate the cost of penalty (cp) in SBCE game.

Total development time (T) = Total ordering Time (tot) + Assembly Delivery Time (ADT) + Iteration Time (it) Where, tot = Total number of points * Lead time, it =30% of Lead Time, and ADT = ADTI * tot. ADTI (Assembly Delivery Time index) is a function of an airplane wing length. ADT variable represents the required time that is needed to assembly the airplane structure. To determine the ADTI values the following rules in Table 3 are used.
lw ADTI 3 4 6 7 8

0.8

0.7

0.6

0.5

0.4

Table 3: Rules to calculate Assembly Delivery Time index (ADTI) in SBCE game.

2.2

Stage two and supporting enablers

In the second stage, players will follow a structured SBCE process phases. This stage simulates a different approach than the first. Here, players follow a Test-Design-Build approach, and design decisions are made as late as possible until feasibilities are proven. In summary, players will explore, communicate set design solutions, evaluate feasibilities of set of solution and finally converge into a preferred one. The stage is automated in MS Excel program. Figure 3 shows the four steps players should follow. Moreover, they will be provided with tools and other SBCE enablers to conduct each phase of the game. A. Explore alternative set of designs: at this phase, players will be supported by QFD (Quality Function Deployment) tool to explore alternative sub-system solutions and able to ingrate customer requirements into an airplane parameters. QFD is a powerful tool in applying SBCE process, it helps designers to translate rough customer requirements into alternative sub-system solutions [Liker, Sobek, Ward, Cristiano, 1996]. Therefore, each player in a team will explore alternative body, wing, cockpit and tail solutions using its own QFD. This phase is the beginning of a SBCE process in the game. B. Communicating set of design solutions: players at this phase can eliminate airplanes subsystem solutions that are not incompatible. Here, players supported by simple checklists that enable the teams to communicate incompatible sets. C. Provision of knowledge from testing department: from step B players have complete alternative airplane solutions which are compatible, but need to filter them using physical constraints. In the game, the physical constraints come from the testing department. Limitcurves are used to generalize knowledge and visually depict solutions which are feasible from testing point of view, see [Oosterwal 2010] for more details about limit-curves. Therefore, at this phase, players eliminate those solutions which cannot pass the testing constrains listed in Table 1. D. Convergence to a preferred solution: once alternative feasible airplanes are identified, estimating the cost and development time of each solutions help to select the preferred solution. Refer section 2.1 to see the cost and time calculations used.

In summary, the second stage is to lead players through a step-wise phases of the SBCE process. The objective is to educate players how to delay decisions early in design phase, and facilitate test-design-build approach to avoid unnecessary design reworks and missing customer goals.

Figure 3: Sub-phases of stage two of SBCE game.

3 Framework for evaluating the learning outcomes of SBCE game

The comparison of performances between the two stages can be taken as a validation mechanism to roughly estimate the advantages of SBCE process (second stage) over the traditional process (first stage). However, the main purpose is not to measure the performance leverages of SBCE process using the game. Because, the game is a simplified version of the reality and cannot capture the real complexities of a PD that make a SBCE approach more advantageous (such as product complexity, innovativeness of the product, team size and so on). Therefore, in this paper, it is aimed at measuring the effectiveness of the game to translate the SBCE principles and its associated supporting elements. Given that, it is also aimed at measuring how practitioners have perceived the potential of SBCE process and its elements in improving PD performances. Garris et.al. identified three level of knowledge or learning aspects in order to measure the effectiveness of a SG[Garris, Ahlers, Driskell, 2002]: Declarative learning: the first of three aspects is the learning of facts or increasing one's knowledge about a subject. Frequently, SGs concern specific problems or real world situations, and are developed from a certain theoretical back-ground. Thus, one of the learning objectives is to increase a domains understanding of players. In this paper, the understanding of the SBCE theory and its supporting elements by players are parts of the declarative learning outcomes. Procedural learning: this aspect refers to the learning of procedures, and also to the understanding of patterns of processes and behaviour. In the SBCE game, procedural knowledge is related to players ability to associate the specific elements of SBCE process and the perceived performance benefits of using them to support a better decision making. The word perceived is used to underscore the limitation of SG approach to generalise the outcomes in reflecting the real achievable results. Instead, industrial players can be asked to consider the SBCEs potential in improving PD performance measures if applied in their development practices. Strategic learning: the third learning aspect is that of increasing intellectual ability. Within gaming this aspect has been explained as implementing knowledge from the game in new situations (real world). Gaming can also contribute to develop reflective competences. Within complex systems, as in PD, it does not only refer to implementing

what is taught in theory but also observing behaviour and adapting to new situations. In SBCE game, several complexities have been simplified but adequate intrications are added to keep players engaged. Since existing literature do not provide sufficient methodological approaches or guidelines to apply SBCE, reflecting on the possibilities of applying the full or part of the SBCE process using the game has been insightful. Based on the above framework, a structured questionnaire is constructed taking the Likert five scale model (strongly disagree (1), disagree (2), neutral (3), agree (4), and strongly agree (5)) [Likert 1932] to measure the learning outcomes. After playing the game with 49 highly experienced designers (mechanical, electrical and software) and project leaders of the Carel industries, players were asked to evaluate the declarative, the procedural and the strategic learning aspects of the game. The players have working experience ranging from 4-20 years and age from 27-54 years. Section 4 focuses in summarising these measurements. In section 5, the results of the measurement are reconsidered to generalize the implications on PD practices by developing an SBCE roadmap.

4 Learning outcomes of SBCE game

4.1 Declarative learning outcomes

In general, the game has increased the level of awareness of players. Nine declarative (D) questions (D1 to D9) were asked. They were aimed to ask if the game help players to understand the principles and enablers of SBCE embedded in the play. From the results, it is clear that, the game has been effective in transferring the theory of SBCE principle and enablers. It helped designers and mangers to understand how to explore alternative solutions, and communicate in sets rather than single solution. Furthermore, the game has been effective in transferring the concept of knowledge capturing and representing in the form of trade-off and limit-curves. From the discussions we had with players, SBCE process has been a unique approach compared with the one they follow in PD. Among the questions that pose doubts among the industrial players is if SBCE game makes them understand the problem of communication within a PD team. Communication among teams in SBCE process takes different form than a traditional point based approach, where designers have only one conceptual solution to communicate about. In traditional design approach, it is a norm that a functional team passes design information over the wall to a subsequent function once completed. In SBCE process, different functions pull together their multiple possible solutions and check sub-systems compatibilities in parallel and define feasible regions (this is called set based communication) [Liker, Sobek, Ward, Cristiano, 1996]. Set based communication is necessary when sub-system designers explore multiple solutions form their own perspectives, and need to evaluate the sets. In the game, players were provided with simple check-lists to support communication and negotiation among teams. Though players understand how to use these communication mechanisms, 33% of the players were either disagree or neutral on the effectiveness (=3.86, =0.87). This is due to two possible reasons. First, the simplicity of the airplane structure to be designed in the game, which might limit the importance of set based communication to be observed, and the need for using tools/mechanisms to facilitate it. The second and the key reason is related to the practical characteristic of the company and the nature of product types its designing. The company is an SME where most projects are done with few people, and sometime only one person designs a full product (if the design is an improvement from previous version). In this case, close collaboration between few and collocated team members fasten design cycles. Thus, communication in sets might not be as challenging as it would be if teams were large enough. Moreover, products in HVAC/R market do not involve several sub-systems with complex interactions as in Automotive/ Aerospace designs where SBCE adoption is prevalent. Therefore, set based communication tools seems to be more critical

in cases of having large and dispersed teams, large design sets, and complex sub-system interactions.

4.2

Procedural learning outcomes

Seven questions were used to measure the procedural (P) leaning outcomes or the perceived benefits of SBCE process. The result of the feedback is shown in Figure 4. From the chart, it can be seen that avoiding design risks is the most agreed advantage of SBCE as perceived by industrial players (88% agree or strongly agree, with =4.1). This is clearly obvious since in the first stage of the game (PBCE), players failed to pass the testing conditions at least one or two times. However, in the second stage since the game provides enablers of SBCE to explore all available solutions and frontload the design process with knowledge (testing constraints), there were less risks of failures. It is true in practice as well. If designers are able to frontload knowledge from downstream, many design risks can been avoided. Similarly, another major advantages of SBCE as perceived by industrial players are reduction of product/ process costs (73 % agree or strongly agree, with =3.8) and facilitating learning (71 % agree or strongly agree, with =3.8). In SBCE, reduction in product cost can be achieved by looking at either less costly alternatives in a design space or by adding the right features that customers want (i.e. avoiding over-design). Process cost can be reduced in SBCE effort by reducing late design changes, which might cause unnecessary reworks and missing market opportunities (as frequently observed in the first stage of the game). On the other hand, learning is perceived by many as the core essence of SBCE. Learning from failures, learning from others and learning from extensive testing/ experimentation are mentioned as part of the lesson-learned in the game play. Moreover, it is not only learning that makes SBCE a more sensible approach to PD, but also capturing knowledge in visual forms such as trade-off and limit curves to make future uses effective.

Figure 4: Perceived performance benefits of SBCE process. Looking at Figure 4, except improving product innovation, the majority of the players perceived SBCE as a process that can improve the rest of the performance measures if implemented in the company. However, 50% of them are either neutral or disagree about the potential of SBCE to enhance innovation. This result implies deeper insights for PD as pointed below. Theory Vs. practice Theoretically, in SBCE, there are direct and indirect means in which innovation can be improved. The exploration phase of the process might directly impact innovation, if new alternative solutions are discovered. Indirectly, SBCE efforts help designers (in long term) to reuse knowledge, and leave them more time for innovative tasks in subsequent projects. But, both means are extremely challenging to achieve in real design scenarios. For example, physiological inertia is a known phenomenon in design practice, where designers often tend to explore solutions within known design spaces [Kowalick 1998]. Thus, it often becomes unlikely

to observe out of the box solutions, and the innovation process depends on the availability of individual/ technological capabilities. Having these capabilities need expensive investments for SMEs to support SBCE process and foster innovation. Trade-off Vs. innovation In extant literature, SBCE has strictly been related to use of trade-offs. Toyota, for example, reported to use extensive trade-off analyses for exploring, evaluating and communicating sets. Thats why trade-off curves have been provided in the second stage of the game. However, using trade-offs restricts the level of innovation. Altshuller, in his prominent theory, underscores the limitation of accepting trade-offs in design [Altshuller 1984]. He argued that, trade-off means accepting compromise, and will not lead to innovative or inventive solutions. For example, if a designer is challenged to choose between two materials aiming to reduce weight and maximize strength, using trade-off analyses he/ she might come up with a material that has an average weight and an average strength (compromising the two requirements). However, Altshullers theory which is called TRIZ (Theory of Inventive Problem Solving) has different approach to find solutions [Altshuller 1984]. In TRIZ, trade-offs are not accepted, rather they are eliminated. In the above material choice example, TRIZ provides guidelines to lead the designer to think out of the box and come up with material alternatives which satisfy both requirements at the same time without compromises. From the above discussions, it is clear that the innovation potential of SBCE depends on the capabilities needed to explore and evaluate innovative solutions. Moreover, SBEC has to be integrated along with other established theories and practices (e.g. TRIZ) to bring new perspectives to enhance innovation in PD.

4.3

Strategic learning outcomes

This is primarily concerned with the adoption of what has been learnt from the game into real design scenarios. Industrial players were asked about ten strategic questions (S1-S10). In general, respondents reflect that, using trade-off curves, limit curves and the checklists have been taken as useful hints on how to make their design and development process better. However, most of the industrial players recognize that building such knowledge capturing/provisioning tools are challenging and time consuming. This shows how SBCE demands time to be effective before seeing considerable performance improvements after implementation. As many lean initiatives, SBCE should be taken as a process and as a journey.

5 SBCE game and implications for practice

Finally, it has been possible to draw some insights from the game to generalize the lesson learned from the game. The insights are aimed to discover the relationships between the SBCE enablers embedded in the game and their impact on knowledge building, performance improvements, and generally for SBCE successful implementation. In order to obtain such insights, summated scales [Spector 1992] are developed based on the same questions asked to players in the three learning outcomes (declarative, procedural and strategic). The questions are regrouped, and data sets are aggregated to build theoretically valid constructs. Constructing valid scales from Likert item questions (that have similar theoretical constructs) is an iterative process. Moreover, the reliabilities of the summated scales are checked using Cronbach alpha coefficients. In general, a Cronbach alpha () coefficient more than 0.70 is considered as reliable [Cronbach 1951]. Since the data are based on ordinal (Likert) items, the most appropriate and prevalent correlation coefficient, Spearman's rho (), is used to quantify the relationships between reliable scales [Norman 2010]. The scales that are developed and their explanations are given as follows:

Tool (T): is a scale that represents one of the SBCE enabler. Its data are aggregated from many similar questions asked to players. Tool scale in this analysis is defined as the understanding and use of tools/methods/technologies in SBCE process. Tools that are used to support the exploration, communication, and convergence phases of SBCE. Involvement (INV): is a scale that represents another SBCE enabler. In the game, there are two stakeholders which are simulated in the play (customers as external and the testing department as downstream function). In reality, several stakeholders might be key to be involved to support SBCE process. Therefore, Involvement (INV) scale can be defined as the understanding and integration of stakeholders in SBCE process. Communication (C): is another scale representing SBCE enabler. It is defined as the recognition, understanding, and adopting of mechanisms and techniques that facilitate set based communication in SBCE process. Knowledge building (K): is a scale that represents one of the most important enabler of SBCE process. It is defined as the identification, capturing, and representation of knowledge in SBCE process. Performance (P): this scale does not represent any SBCE enabler. But, it is the effect of the SBCE enablers on perceived performance improvements. It is defined as the potentials of the SBCE enablers such as Tools (T), Involvement (INV), Communication(C) and Knowledge building (K) to achieve better PD performances. Implementation (IMP): this scale is to measure the success of SBCE implementation. It is defined as the capability of a company to be adept in problem solving and becomes a learning organization. Figure 5 shows the correlations between the above scales and the corresponding significance levels (p-values). As shown from the figure, the SBCE enablers have strong correlations with each other and contribute significantly to performance improvements in PD, as well as to bring the design organization to a successful learning organization. Tool (T) for example, has significant correlations (p 0.01) with all other enablers INV, C and K, and with values of correlation 0.4581, 0.3866 and 0.5548 respectively. The results show that tools and methods (e.g. limit curve, trade-off curves, and checklists) as enablers contribute to facilitate set based communication (C), to have effective involvement with stakeholders (INV) and build knowledge (K) in SBCE process. Moreover, the existence, the understanding and the adoption of SBCE enablers are highly correlated to achieve better performance improvement. Except communication (C), tool (T) and involvement (INV) significantly contribute for knowledge building (K) with significant levels (p 0.01), and with values of correlation 0.5548, 0.4521 respectively. As discussed in section 4.1, set based communication has not been perceived to be key in this particular case company. Nevertheless, previous studies also showed mixed results about the relationships between communication (C) and knowledge building. For example, Sobek, et al. conducted a detail observation on the communication style between Toyota and its suppliers. They have found that, suppliers of Toyota communicate in ranges and also develop several prototypes for the same design concept. Toyota requests the suppliers to communicate in this style. The rationale behind is that, Toyota wants to understand the underlying trade-offs between alternative designs [Sobek, Ward, Liker, 1999]. This implies that the relationship is helping to build knowledge about design performances in the form of tradeoffs. Thus, set based communication has at least a positive relationship with knowledge building. In another survey study, however, set base communication is perceived as a negative practice by some industries [Liker, Sobek, Ward, Cristiano, 1996]. In particular, US and other western manufacturers consider communicating vague or unfinished design information as ineffective. Rather, these industries prefer to base their communication on precise and stable information (this is what is called point based communication). In addition, requirements and specifications are well-defined early in the beginning of a design process. Such kinds of relationships with

suppliers help manufacturer to control quality and cost targets. Otherwise, suppliers might abuse the relationship as a leap for bad performances. The above discussions show that set based communication is not always taken as a means of building knowledge, rather point based communication is referred to make faster and accurate decision in some industries.
Implementation (IMP)

= 0.78

**=

**=

0.5183

0.4304

Performance (P) = 0.79

**= 0.4833

*=

** = 0.5268

0.3635

** = 0.3866

Knowledge building (K) = 1

**= 0.3806

** = 0.5548

*=

*=

0.3378

0.3540

**= 0.4521

Tool (T) = 0.74

** = 0.3866

Communication (C) = 0.70

= 0.2452

Involvement (INV) = 0.78

** = 0.4581

Figure 5: Correlations between SBCE enablers, performance and implementation success SBCE implementation roadmap. Constructed based on the players feedback (Note: ()p 0.10, (*) p 0.05, (**)p 0.01). In summary, looking at Figure 5, one can easily understands that SBCE process is a journey. It takes time to get successful results and become a learning organization. The understanding and use of the tools necessary for communication, involvement and knowledge building are the primarily steps to start the SBCE journey. Once the tools are effectively embedded in PD, SBCE fosters communication and facilitates effective involvement of stakeholders in SBCE implementation process. Then, the company can identify, capture and reuse market and technical knowledge. Once knowledge are effectively shared across projects and across design teams, performance improvements can be seen as a result of the SBCE implementation efforts. Finally, one can say that, SBCE implementation is successful and a company become a learning organization, who can dynamically design products in a knowledge base environment.

6 Conclusions and future research

SBCE process has been in academic discussion for more than a decade. Now is the time to translate the outstanding principles to the industries. To gain wider acceptance in practice, the

SBCE Implementation Roadmap

SBCE game will significantly contribute to increase the awareness level in industries. Moreover, the benefits and challenges in adopting SBCE process in practice can be devised using the measurement framework suggested in the paper. The results from the game play in the case company showed important implications for the theory and practice of SBCE. However, more researches are required to make stronger validations in other industries. The following important research questions can be forwarded for further investigations: What impacts does team size has on the effectiveness of set based communication? What impacts does products complexity has on the effectiveness of set based communication? What are the factors that determine the rationale of using either set based or point based communication with PD stockholders? How to integrate SBCE with other innovation theories (e.g. TRIZ) to enhance innovation? What are the organisation, individual and technological capabilities required to sustain the SBCE implementation in PD?
Acknowledgement This work was partly funded by the European Commission through the Linked Design Project (FP7-2011-NMPICT-FoF, www.linkeddesign.eu). The authors wish to acknowledge their gratitude and appreciation to the rest of the project partners for their contributions during the development of various ideas and concepts presented in this paper. References Altshuller, G. (1984) Creativity as an Exact Science. Gordon & Breach, New York, ISBN 0-677-21230-5, 1984. Cronbach, L.J (1951) Coefficient alpha and the internal structure of tests. Psychometrika, 16(3), pp.297-334, 1951. Garris, R., Ahlers, R., Driskell, J. E. (2002) Games, Motivation, and Learning: A Research and Practice Model. Simulation and Gaming, 33(4), pp.441-467, 2002. Kennedy, M. N. (2008) Product Development for the Lean Enterprise: Why Toyota system is four times more productive and how you can implement it. The Lean Enterprise Institute, Cambridge, Massachusetts, 2008. Kowalick, J. (1998) Psychological Inertia. TRIZ Journal, June 1998. Liker, J.K, Sobek DK, Ward AC, Cristiano JJ. (1996) Involving Suppliers in Product Development in the United States and Japan: Evidence for Set-Based Concurrent Engineering. IEEE Transactions on Engineering Management. 43:165, 1996. Likert, R. (1932) A Technique for the Measurement of Attitudes. Archives of Psychology, 140: 155, 1932. Norman, G. (2010) Likert scales, levels of measurement and the laws of statistics. Advances in Health Science Education, DOI 10.1007/s10459-010-9222-y, 2010. Oosterwal, D.P. (2010) The Lean Machine: How Harley-Davidson Drove Top-Line Growth and Profitability with Revolutionary Lean Product Development. American Management Association, New York, USA, 2010. Raudberge, D. (2010) Practical Applications of Set-Based Concurrent Engineering in Industry. Journal of Mechanical Engineering 56(11), pp.685-695, 2010. Rossi, M., Kerga, E., Taisch, M., Terzi, S. (2011) Lean Product Development: Fact Finding Research in Italy, International Conference on Industrial Engineering and systems Management (IESM) Proceedings, Metz (France)-May 25-27, 2011. Sobek D.K., Ward A.C., Liker J.K. (1999) Toyota's Principles of Set-Based Concurrent Engineering. Sloan Management Review, 40, 67-83, 1999. Spector, P. E. (1992) Summated Rating Scale Construction - An Introduction. Sage University Papers Series on Quantitative Applications in the Social Sciences. Sage Publications, Inc. No. 07-082, 1992. Ward A., Liker J.K., Cristiano J.J., Sobek II D.K. (1995) The Second Toyota Paradox: How Delaying Decisions Can Make Better Cars Faster. Sloan Management Review, 36, 43-61, 1995. Wouters,P., van der Spek E.D. and Oostendorp H. (2011) Measuring learning in serious games: a case study with structural assessment. Educational Technology Research and Development, 59-6, pp. 741-763, 2011.