Proceedings of the ASME 2014 International Design Engineering Technical Conferences &

Computers and Information in Engineering Conference

IDETC/CIE 2014
August 17-20, 2014, Buffalo, New York, USA

DRAFT: DETC2014-34954
IDEATION METHODS: A FIRST STUDY ON MEASURED OUTCOMES WITH PERSONALITY TYPE

Pui Kun Choo

Singapore University of
Technology & Design

Zhi Ning Lou

Singapore University of
Technology & Design

Bradley A. Camburn
Singapore University of
Technology & Design

Ben Koo
Tsinghua University
Beijing, China

Francois Grey
Centre for Nano and Micro
Mechanics, Tsinghua
University, Beijing

ABSTRACT
The research reported here considers an experiment and
subsequent data coding and analysis to extract correlations
between personality type and ideation outcome from several
methods. This article presents the background theory,
research methodology, and empirical results associated with
the experiment. The experiment is based on observations of
designers developing a real product, and associated
assessment tools, where the goal is to correlate the quality,
quantity, and variety of design outcomes with respect to
personality type. This approach lays the foundation for a
tailored ideation method or a suite of ideation methods that
takes advantage of the preferences and strengths of
individuals. We find that there are significant correlations
between type and ideation metrics and that these correlations
are supported by related theory from psychology and business
management.

Since different personality types have been shown to

communicate differently [4,5]; and different ideation methods
are based on different types of communication [6,7], it is only
intuitive to infer that there may be differences in the
effectiveness of an ideation method for different personality
types. This information could potentially be used to develop a
structured ideation method or a suite of ideation methods that
is tailored for individuals or different compositions of teams.
This study follows the practice of designers engaged in a
sponsored project. Procedures are employed to extract
concrete data about their performance on a design task. These
data are then post processed to make inferences about the
correlation of personality type and ideation methods. The type
of research employed in this study is often referred to as an
empirical study [1]. Other studies have in fact contributed to
examining personality types during design. Individuals from
different fields tend to span a range of particular personality
types, and different personality types prefer particular stages
of the design process [8,9]. There has also been comparative
research on the various ideation methods used in this paper.
Design outcome metrics have been used to compare the
quantitative results of different ideation methods [6,7,9,10].
However, the authors believe this may be the first time that
different ideation methods and personality type are directly
compared for the ideation phase of design.
The objective of empirical studies is to verify and record
observable phenomena. This allows for the quantificaion of
what may previosuly have been intuitive knowledge. The
experimental cycle allows this at a high level of granularity.
Along this line, the following research questions were used to
guide and develop this study:
1. What are the statistically significant trends (if any) in
quality, quantity, novelty, and variety of design solutions

Keywords: ideation methods, brainstorming, mindmapping, Method 6-3-5, C Sketch, personality type, MyersBriggs Type Indicator, Six Thinking Hats, empirical study
1. INTRODUCTION
Structured ideation methods are critical for the progress of
many projects in engineering design. Technically accurate
information on the effectiveness of these methods is equally
critical [1]. It is important to assess the methods through
formal research methodologies and obtain insights on how and
why ideation methods can be made effective.
Design managers have been grouping or seeking to
understand teams based on personality type for a long time
[2,3]. One possible avenue for research is to elicit the
relationships between personality types and ideation method.

Kristin L. Wood
Singapore University of
Technology & Design

2.

To effectively maximize the results from brainstorming, one

can use mind-mapping. Mind-mapping is an intuitive semantic
and categorization technique that emulates a process similar to
how we organizes ideas in long-term memory. First, a key
idea is placed at the center of a piece of paper. Next, possible
solution categories are added, branching off the key idea.
Finally, specific solutions to the problem are added to these
key categories. Thus, each solution generated is related to the
original problem statement. Research has demonstrated that
mind-mapping may significantly increase the number of ideas
generated compared to the classic brainstorming approach.
This result is attributed to the categorizing of ideas, which
arranges concepts hierarchically, hence suggesting the
difference between the design avenue or category of ideas and
specific solutions. Moreover, mind mapping facilitates piggybacking and leap-frogging of ideas due to its a twodimensional graphical map structure. This structure opens up
the opportunity to identify and fill in gaps in the possible
design space; for example, upon creating the mind-map, one
may notice a certain branch of solutions is less complete than
another [14]. An example mind -map from the study is shown
in Figure 2.

produced by different personality types across different

ideation methods, and between personality types within
each method?
Are there statistically significant differences in how
participants of different personality types self-perceive the
outcome of different ideation methods?

We address these questions by creating a structured

ideation environment as part of an active design group, and
measure the outcome quantitatively using established metrics.
These metrics include quality, quantity, novelty, and variety of
the design outcomes as well as self-efficacy of the participant
designers. The following sections describe the ideation and
personality type assessments employed, the experimental
setup, the design problem and its context, the participants, data
encoding and analysis procedures, as well as results and
conclusions of the study.
2. BACKGROUND THEORY
2.1 Brainstorming
As an intuitive method [11] of idea generation, the
brainstorming method by Osborn [12] encourages divergent
thinking [11] in design problem solving. It can be employed
individually or in a group typically including five to fifteen
people. Individually, one may start by identifying a few
concepts to build on and generate as many solutions as
possible. In a group, members verbally communicate ideas to
one another for thirty to forty-five minutes. Ideally, the
members should not be inhibited in expressing any ideas to
achieve a comprehensive range of solutions. Development of
individual ideas result as members respond by making
connections to others ideas. These connections will vary as
each member differs in skill sets, experience and personality,
creating diversity in solutions. However, individual
brainstorming has been shown to be more productive than
group brainstorming. In a group, domination by a single or a
few group members may occur [13]. Inhibition could also
happen in the presence of an expert [13], or when groups are
unreceptive to new ideas, and may result in discussing only
existing solutions. A facilitator could be appointed to ensure
participation by all, while restricting negative criticism. A
complementary mind-mapping or 6-3-5 / C-Sketch session
combined with or held after initial brainstorming may lead to
greater effectiveness. An example brainstorming sheet from
the study is shown in Figure 1.

Figure 2 An individual Mind-map

2.3 C-Sketch
As an extension of the 6-3-5 method, which uses written
description for idea generation, the C-Sketch method is an
intuitive method that uses graphical descriptions instead of
written descriptions. It is usually employed after the problem
definition and clarification stage of design [15]. In a team,
each member is given a sheet of blank paper on which they are
to sketch three solutions with respect to the design problem
statement. After t minutes, the papers are to be passed on to
the next person on the right. Another t minutes will be given
to add modifications or additional ideas to each idea. The
process of passing repeats until all members have contributed
to every individual paper. The number of people in each team
is typically six, although a range of three to eight members
may work well. Likewise, the time duration is variable; it can
be fifteen minutes initially followed by ten-minute alteration
sessions. Each individual is usually given a uniquely colored
pen or marker to encourage no elimination of ideas, and allow
members to easily identify their contributions for later
discussion. There is no verbal communication allowed to
prevent domination of the session by a single or small group
of members, while encouraging participants to make
individual inferences of the sketches that may result in
unanticipated ideas. Labeling should also be kept to a
minimum, but instead focus only on main keywords. It is also
important to refrain from negative criticism, but instead to

Figure 1 An individual Brainstorming sheet

2.2 Mind-mapping


to be ISTJ, followed closely by ESTJ, then ENTP [9]. Jungs
theory of eight cognitive modes, representing problem solving
approaches, is related to the two dominant modes or sub-types
as shown in Table 3 [3].
MBTI can be used in team formation strategies
[2,3,16,17] to achieve diverse teams [2,3,16]. It ensures a
mixture of members with a variety of cognitive styles,
providing groups with a spectrum of viewpoints and problem
solving methods. Through identification of each individuals
type, it informs and encourages understanding amongst
members [3].

focus on further developing the ideas. Brainstorming may be

combined with a C-Sketch session, either before or after, for
development of ideas through verbal communication. CSketch facilitates leap frogging of ideas [7], and achieves
diversity in design [15]. Provocative stimuli [15] from
sketches of other members reduces design fixation, and helps
to develop new ideas. Research has also shown that C-Sketch
induces a forty percent increase in quantity of ideas produced
over a variety of comparable methods [11]. An example of a
solution produced in C-Sketch can be seen in Figure 3.

Orientation

TABLE 2: OVERVIEW OF MBTI

Extraversion (E)
Prefer working in groups and
through external interaction,
often taking a breadth-ofknowledge approach.

Introversion (I)
OR

Judgment

Perception

Sensing (S)

Style

Figure 3 A concept developed during C-Sketch

More comfortable working alone

reflectively, taking a depth-ofknowledge approach into ideas
and concepts.
Intuition (N)

Gather information through

practical experience, focusing
on observable phenomena,
facts and details.

OR

Thinking (T)

Perceive through imagination

and internal sensing, focusing on
the big picture, theories, and new
possibilities.
Feeling (F)

Analytical and logical, judging

objectively through impersonal
evaluations.

OR

Judging (J)

Subjective and weigh human

factors, often making decisions
based on personal values.
Perceiving (P)

Decisive and planners,

preferring structure and order.

OR

Keep options open, are flexible,

spontaneous and exploratory.

Each of the above ideation methods was deployed in the

experiment. A summary of the methods is provided in Table 1.
There are many useful methods, these three were chosen for
several reasons. First because they require relatively little
introductory training and allowed for the full deployment of
several methods in limited time. Secondly these methods
represent a mixture of structured versus open-ended
approaches to ideation, and individual versus group
formatting. Finally, there was a rich compliment of empirical
literature to evaluate the properties of these methods readily
available for review. The remainder of Section 2 details the
Myers Briggs Personality Type Indicators and Six Thinking
Hats.

2.5 Six Thinking Hats

The Six Thinking Hats model by Edward de Bono
distinguishes six modes of thinking, represented by six colored
hats [5] (Table 4).

TABLE 1: SUMMARY & COMPARISON OF METHODS

TABLE 4: OVERVIEW OF SIX HATS

Method
Individual
Brain-storming
Group
Brain-storming
Individual
Mind-mapping
Group
Mind-mapping
C Sketch

Communication
Written word only
More verbal than
written word
Written word only
Verbal and written
word
Sketching primarily

TABLE 3: JUNGS COGNITIVE MODES [3]

Information Collection
ES
EN
Experiment
Ideation
IS
Knowledge

Style

White Hat
Concerned with facts, and objective
information.
Black Hat
An analyst, the devils advocate
who gives negative but logical
criticism, identifying why something
might not work.
Green Hat
Creative, generates new possibilities
and solves problems through lateral
thinking.

Provocative stimuli, use of

analogies

Categories, structured and

organized
Imagery, graphical,
provocative stimuli

2.4 MBTI
The Myers-Briggs Type Indicator (MBTI) [4] assesses an
individuals level of preference in four categories that indicate
aspects or approaches to problem solving, decision-making
and communication of information or ideas. The categories are
based on C.G. Jung's theory of psychological types (Table 2).
A total of sixteen types result from the permutation of the
categories. Research has shown that engineers are more likely

IN
Imagination

Decision-Making
ET
EF
Organization
Community
IT
Analysis

IF
Evaluation

Red Hat
Utilizes emotions and intuition.
Yellow Hat
An optimist, giving logical positive
criticism on why something might
work.
Blue Hat
Often the leader facilitating,
overseeing and organizing thinking
processes to achieve the agenda.

Role-playing hats in a group facilitates group-thinking

processes, as multiple perspectives can be covered [18].
Otherwise, if each individuals hat is known, those of different
hats can be grouped together to achieve a balance of thinking
types within a group [2,3,16]. By identifying ones
correlations, association, or preferences for particular hats, it
focuses and amplifies the particular preferred mode of


thinking. This improves communication as thinking modes are
used deliberately, giving greater freedom to the thinker to
express thoughts through the mode chosen [18].
Research has shown that groups that are formed with just
either MBTI or six hats are more effective than groups that do
not meet the criteria of the respective team formation
strategies [2]. Groups that are formed based on both MBTI
and six hats results and team formation strategies are
significantly more effective than groups formed with just
either MBTI or six hats, under certain conditions [2].

concrete, students from local Chinese high schools were

invited to participate in the event, and were interviewed by the
Lego2Nano teams on the first day of the challenge, to
understand context of the high school students needs and
constraints.
The Lego2Nano challenge focused on teamwork. Teams
were selected based on carefully balancing age, gender,
nationality and their different technical backgrounds, as well
as personality traits such as whether individuals are intuitive
or critical, extrovert or introvert. An important focus in the
first couple of days was on activities that helped the team
members learn about each other and appreciate their
complementary skills.
Professional scientists and engineers have spent many
years trying to improve the AFM, so theres no reason to
expect that young scientists many of whom didnt even
know what an AFM was at the outset of the event - could
complete this challenge in just one week. Still, a significant
step forward towards a low-cost AFM was made. And, at the
same time, the event represented a radically new approach to
teaching science and technology, promoting teamwork and
encouraging internationalization and interdisciplinary as part
of a Chinese system.
This new approach is also called XLP, short for eXtreme
Learning Process. XLP is a learning activity design
methodology intended to explore the boundaries of cognitive
capabilities of groups of people with diverse talents. XLP
activities divide participants into two groups, namely
Challenge Designers (developers and organizers) and
Missionaries (participants).
In general, Challengers first play out learning related tasks
on themselves, months ahead of time, so that they can assess
how much time and resources are needed to accomplish
certain tasks. After trying out the tasks and identifying at least
one feasible solution, then, Challenge Designers will prepare
the event according to the necessary success factors for an
intended audience, called the Missionaries. Challenge
Designers will stand by Missionaries during the intensive
workshop, usually four to five days. The purpose is to guide
Missionaries when necessary, but not to do the design tasks
for them. Some times, Challenge Designers will serve as
technicians to help Missionaries perform certain
implementation tasks, but for a price, usually measured in
virtual currencies. A comprehensive XLP would consist of
simulated Banks, Courts, and Patent Offices. More detailed
explanation of XLP can be found in [21].

3. EXPERIMENT
Previous research shows that there are distinct differences in
the results of various ideation methods [6,7], and that a
designers personality type is correlated with different
behavioral preferences in problem solving and communication
[17,20]. Cross-correlating these two variables could help to
develop a tailored ideation method or suite of ideation
methods that reinforces the proclivity of each individual on a
team, or the team dynamics as a whole. However, to achieve
such a method, an investigation and development of quantified
model of correlations are needed.
An experiment is thus pursued to measure outcomes at the
intersection of personality types and ideation methods.
Personality tests are given before participants, composed of
designers working on a coordinated project, arrive on site.
These tests result in both MBTI and Six Hats type indicators.
During the design challenge, participants are provided the
opportunity to employ design ideation methods in a controlled
environment.
Ideation outcomes are recorded for post
analysis. Several methods of ideation are employed:
brainstorming, mind-mapping and the C-Sketch method.
Brainstorming and mind-mapping are employed both
individually and in groups. The following section describes
the experimental procedures and subsequent analysis in detail.
3.1 Context
This experiment involved deploying design ideation methods
and tracking results during a product design challenge. The
design challenge, known as Lego2Nano, was the third in a
series of China-UK Summer Schools between Tsinghua
University, Peking University and the University College
London, held on the campus of Tsinghua University. A
number of individuals from diverse educational backgrounds
were selected to work together for five days to design and
build a low-cost Atomic Force Microscope 1 (AFM) suitable
for use in Chinese high schools.
An aspirational theme of the challenge was to determine
how a low-cost AFM might transform science teaching in
schools. To make this part of the challenge much more

3.2 Participants

1 Traditionalopticalmicroscopesareunabletoresolvefeaturessmallerthan
aboutonemicrometerathousandthofamillimeterthewavelengthof
visiblelight.TheAFMusesdirectphysicalcontactbetweenasharptipanda
surfacetodetectfeaturesonasurfacethataremuchsmallerthana
micrometer.Suchmicroscopescanevensensesingleatoms.TheAFMwas
inventedinthe1980s.Ithasprovedveryusefulformanyfieldsofresearch,
includingstudyingnewmaterialsforenergystorage,measuringimportant
biologicalaspectsofDNAmolecules,andfabricatingnoveltypesofelectronic
devices.Butthistypeofmicroscopetypicallycosts$100,000ormorefora
professionalqualityversion,andevensocalled"educationalmodels"areat
least$20,000.

Figure 4: Breakdown of percentage of participants by their identified dominant type (rounded to the nearest integer). The
three type sets above are indepedent, each individual is represented once in each chart.

The challenge participants were composed of PhD and postdoctoral students in engineering and physics. Some of the
particiapnts specialize in the development and use of
nanotechnology equipment, while others focus on graphic and
industrial design. Additionally, some of the participants have
considerable experience building and using nanotechnology
tools. These nanotechnologists also had minimal experience
with design methodologies which is positive, as they remain
unbiased towards to any particular method due to previous or
personal experience. There were thirty-one (31) participants,
coming from top universities around the world. The designers
and nanotechnologists were distributed evenly between teams.
We were able to collect a complete set of data from twentyfive of the participants, as the process was voluntary, and
participants completed only those methods they chose to
engage their efforts in. Figure 4 depicts the breakdown of
personality types.

3.4 Description of the Workshop Tutorials

The format in which the researchers engaged in this activity
was to provide the participants with a series of instructional
videos and brief information sessions on applying ideation
methods, and then walk participants through completion of
each method. There were videos for brainstorming, mindmapping and C-Sketch methods. Each instructional period was
timed at ten minutes. Participants were then given fifteen
minutes to work through each of the methods.

3.3 Design Problem

The unique design problem for this challenge was developed
as part of a goal to create a novel research collaboration
between China and the UK. Participants are aware of the fact
that their results will become very real and are thus highly
motivated to produce a novel design. For the purposes of a
challenge event, the technical challenge was divided into five
aspects:
1.
2.
3.
4.
5.

Figure 5 Construction of a final prototype

3.5 Data Recording

The methods were deployed in sequence during the challenge,
and participant communication was controlled, as was the
amount of time allotted to each method. The sequence and
communication levels of each method are as shown in Table 1.
During individual methods, participants were not allowed
to talk. During team methods, teams were not allowed to
discuss technical issues with other teams but individuals were
allowed to discuss freely within their own team. The exception
to this approach is C-Sketch, in which communication only
occurs by passing the sheets of paper even though it is a team
exercise.
The participants were instructed to produce as many ideas
as they could during the fifteen minutes allotted to each
method. Data collection occurred at the beginning of the
session and after each method, consisting of a self-efficacy
survey and collection of all concept sketches. Each individual
was provided with a uniquely colored and coded pen so that

Resolution of the force scanning device

Creative engineering and technical design aspects of the
device
Scientifically meaningful applications as supported by the
device
Suitability of the device for use by high school students
Ingenuity in sharing and crowdsourcing the device
production and its applications

The participants were given one week in which to

conceive a design and produce a working prototype. They
were provided with a few basic prototyping components
including LEGO construction sets, MindStorms, a few piezo
crystals, and an AFM probe or cantilever tip. For context,
although prototyping will not be discussed in this paper, the
final prototype is shown seen under construction in Figure 5.


his or her solutions could be tracked anonymously. It was still
possible to correlate personality type to ideation solution as
individuals provided this same code on their personality
surveys.
The methods of self-efficacy assessment via surveys and
raw ideation solution collection provide for an analysis of the
effectiveness of each method not only in terms of design
content but also self-perceived ability or satisfaction. Previous
studies have shown that self-efficacy is positively related to
actual performance both in the past and future [19, 20]. Taking
a picture with a high resolution DSLR camera allowed for
later reference and analysis of the drawings.

can be employed to test for significance of these findings.

3.7.1 Quality. Quality is a measure of the feasibility of a
developed design or system in question to satisfy design
requirements. For example, the challenge assessment of
quality might be a normalized measure of the resolution of a
microscope, where a microscope with a higher relative
resolution is rated with a higher quality. Since the designs in
question were at a conceptual level, experienced
nanotechnology researchers provided input on the potential
quality of each solution according to the scale in Table 5.
3.7.2 Quantity. Quantity is a direct and basic measure of the
number of ideas produced (either in total for a single method
or by an individual). Quantity can be measured as either
unique ideas, that is, ideas that a rater determines to be unique
functionally with respect to other ideas; and raw quantity of
ideas, which is the total number of ideas listed during an
exercise, even including repeats. Repetition is identified
through Novelty and Variety.

3.6 Solution Encoding

The researchers worked to encode the solutions into a
common format to remove any biases due to handwriting
quality, or irrelevant aspects of the original drawing. The
solutions were described using a common format and re-listed
as entries in an excel table. Images were also translated into
descriptions of solutions. A link was maintained to the original
image file for reference. For those solutions that consist of
multiple aspects, each aspect is listed as a unique entry. This
approach provides an equal comparison between a solution
that only covered a single aspect of the problem and those that
combined multiple aspects in a single solution. For example, it
would not be equitable to rate a solution that covers probe
design and stage motion control in a single drawing
equivalently to a solution that only includes a probe design.

3.7.3 Novelty and Variety. Variety is a measure of the

explored solution space during the idea generation process [1].
The generation of similar ideas indicates a lower variety, and
therefore corresponds to a lower probability of finding better
ideas in the possible solution space. We calculate variety, in
this study using the equation adapted from Shah by Chan et al.
for evaluation of design ideation methods [1,10]. The equation
for applying this approach to a set of ideas is stated as

3.7 Metrics
The analysis of results utilizes a standard set of metrics
employed in design science literature for the purpose of
evaluating ideation outcome. These are: quality, novelty,
variety, and quantity as first introduced by Shah and adapted
by Chan et al. [1,10]. Self-efficacy was another metric used to
assess the design outcomes. In parallel both the Myers Briggs
Personality Type Indicator (MBTI) and Six Hats methods
were used to record personality type. Three raters with
background in the challenge and solutions encoded the data.
Pearsons correlation coefficient for inter-rater agreement for
solution binning by function was calculated as a 0.73 raw
score, and 1.0 after discussion and resolution of each
mismatched specific solution.
The chosen metrics were first introduced as a generic tool
to provide quantitative evaluation of creative results produced
in ideation sessions and for design research [1,6,7,10]. The
metrics provide information about the performance of the
individuals during an ideation session, and overall from
certain methods in a numerical form so that statistical analysis

(1)
is the novelty of specific solution i; is the
where
total number of times a specific solution was generated for
that sub-function of the problem in the given ideation method;
and is the total number of times the specific solution to be
evaluated was generated in the given ideation method.

(2)
where n, is the total number of solutions generated by an
individual with a particular ideation method.
Novelty is a measure of uniqueness of a solution [10];
and, in a complementary way, variety is a measure of the
uniqueness of a set of design solutions. Mathematically, it is
simply the average novelty for a set of solutions. In our case,
solutions were collected in concept variant bins. The solutions
in a bin all perform the same function with the same basic
principle. These are considered a specific solution. For

TABLE 5: QUALITY SCORING RUBRIC EMPLOYED DURING EVALUATION

Score

Level

Not a valid concern or idea

Valid idea but not implementable

2
3
4

Examples (Control of Probe Approach)

Use magic

Low

Accuracy is challenging

Medium

Probe angle is hard to control

High

Probe angle is hard to control due to atomic reaction forces

Low

Probe angle must be controlled

Medium

Use the same controller for probe approach angle and tapping mode scanning

High

Use a low power high voltage controller for probe

Low

Use Arduino to control probe

Medium

Use an Arduino with PID function to control probe

High

Arduino controller, linked with USB microscope and z-piezo as sensors for PID control

Valid idea that is implementable

Sublevel

Specific implementable solution


instance for the subfunction of stage scanning scheme, two
drawings from different participants depicting a probe that is
free to move in x, y, and z alongside a stage which is fixed
would be listed as the same specific solution. However, a
solution depicting a probe that moves only in z and a stage
that scans in x, and y would be a different unique solution
under this subfunction.

brainstorming, and group mind-mapping. This is clarified by

the New Solution Bins sub-chart in Figure 6.

3.7.4 Self Efficacy Surveys. A method is also required to

determine the participants own perception of the results of
each method. This is important not only to determine the
participants psychological reaction to the method but also as a
parallel test of the metrics. This is possible, as it has been
established that self-efficacy is correlated with actual
performance [19,20]. Paper surveys are used to establish selfefficacy. Each individual was given a single page multiplechoice survey across ideation methods, and five minutes to
complete the survey. We asked participants to rate the
effectiveness of each ideation session in terms of quantity,
quality, novelty, and variety of the ideas they generated, as
well as the over-all usefulness of the method from their
perspective. The questions are structured as a five-point Likert
scale that ranges from Strongly Agree to Strongly
Disagree.
4. RESULTS AND DISCUSSION
Over all we find the session to have been largely productive.
We found that the 25 participants who opted in to analysis for
the study produced 1095 individual solutions and 321 unique
solution bins. That means the average individual produced
more than 43 individual solutions and more than 12
completely unique ideas in the total 75 minutes of
brainstorming. The detailed results section consists of three
segments detailing overall trends of the ideation session,
MBTI results, and finally Six hat results. We applied paired ttest analyses for mean shift in the data results,
correspondingly, all p values reported in the results section
are the significance estimates reported from this test.
4.1 GENERAL IDEATION RESULTS
Before examining comparisons between types it is important
to review results of the ideation methods as a whole. The
average results across all participants can be seen in Figure 6.
As would be expected from the literature, mind-mapping and
C-Sketch were effective methods. Individual mind-mapping
scores were higher in quantity than individual brainstorming
(p = 0.008), and group brainstorming (p = 0.015). C-Sketch
also significantly outperformed individual brainstorming (p =
0.058), and group brainstorming (p = 0.064) in quantity. Csketch is more importantly known for permitting the
refinement and advancement of ideas, accordingly C-Sketch
saw a significant increase of the quality of ideas produced,
with ideas produced in C-Sketch having a higher mean than all
other methods in quality (p <= 0.0003).
We find a surprising result that individual methods
generally outperformed group methods. The explanation for
this may be that during sequential efforts of the same method
in individual and then group efforts, the participants saturated
the obvious results within a particular method. This explains
why there is a jump up in scores when introducing individual
mind-mapping or C-Sketch, but fewer ideas are introduced on
the second effort of using a similar method, e.g. group


Figure 6 depicts the average number of new solutions
introduced per person. Comparisons must carefully account
for this. For lateral comparisons, (between types within one
method), the comparison can be direct. For longitudinal
comparisons, (between different methods within one type) the
comparison is relative to the average of each method.

Quantity

4.2 MBTI COMPARISONS

For MBTI analysis, Jungs cognitive modes (Table 3) are
cross-compared as in Wildes Teamology. Studies have found
them to be the dominant indicators of individual performance
in team dynamics. There are eight possible modes, however
we had a small sample size of Introverted Feeler types
(Evaluation) and Introverted Sensing types (Knowledge);
therefore we do not report results on these types. This deficit
occurred as participants elected to join this project of their
own accord and we did not have the opportunity to screen for
an even number of each type.
The results of this comparison can be seen in Figure 7.
Each bar in Figure 7 is the average score for all individuals in
a given cognitive mode. Additionally, each plot represents an
independent data set. The ET-EF-IT results are separated from
the EN-ES-IN graphs. For instance, there are no Introverted
Thinkers that are also Extroverted Feelers, but there may be
some Extroverted Feelers that are also Extroverted Intuitors;
thus, those two sets are not directly comparable.
By examining the results seen in Figure 7, a number of
insights can be found that relate the cognitive modes to
ideation results and the related theory. Results of the decision
making types (ET-EF-IT) will be discussed first. It would be
expected that the ET or Organizer types would score highly in
mind-mapping. They do outperform IT or Analysts. IT or
Analyst types would be expected to perform most highly in CSketch as it is the most analytical method. Indeed they
significantly outperformed EF or Community types (p = 0.05).
This supports the theory that Analysts excel in the solution
refinement process of C-Sketch. Lastly, what would be
expected from theory on the decision making types is for EF
or Community types to perform well in group methods. In fact
they are the only type that actually did better in group
brainstorming than individual brainstorming, but not quite
significantly so (p = 0.15). With regards to quality, EF or
Community types produced the best ideas in group mindmapping. This would be expected as group mind-mapping
requires a lot of group integration, their scores were
significantly higher than for IT or Analyst types (p = 0.01).
Finally, for variety, IT or Analyst types had greater variety
than ET or Organizer types in GBS (p = 0.01), it may be that
Analyst types were more comfortable without an
organizational structure.

Quality

Variety

New Solution Bins

Figure 6 Total averages for all participants across the

ideation session. Error bars are 1 standard error. The
vertical axis is performance in each metric (see section
3.7) The horizontal axis is the method code: I =
indivudual, G = group; BS = brainstorming, MM =
mindmapping, and 635 = C-Sketch

Decision Making Personality Type Set

Quantity

Quality

Variety

Information Gathering Personality Type Set
Quantity

Quality

Variety

Figure 7 Comparison between type averages for Jungs Cognitive modes. Error bars are 1 standard error. The vertical axis
is performance in each metric (see section 3.7) The horizontal axis is the method code: I = indivudual, G = group; BS =
brainstorming, MM = mindmapping, and 635 = C-Sketch

Type

Similarly, comparisons of interest can be made to the

theory for information collection types (EN-ES-IN). The EN
or Ideation types would generally be expected to perform well
across the board in group or extroverted ideation methods.
There were no particular methods in which EN types did
better than other types, this may be because group methods
were placed after individual methods. IN or Imagination types
would be expected to perform well at individual methods.
They indeed have the highest performance in C-Sketch among
the information collection group comparisons, with
significance (p = 0.01) that they produce more ideas. This
indicates that the segmentation approach of C-Sketch allows
for individual introspective or imagination type ideators to
flourish. With regards to quality, some comparisons were
significant also. ES or Experimenter types outperformed EN
Ideation types in C-Sketch average quality (p = 0.02). This
could indicate that the C-Sketch process of iteratively
evaluating an idea has an aspect comparable to
experimentation. ES or Experimenter types similarly
outperformed EN or Ideation types in Variety of C-Sketch
ideas also.

BS

MM

635

MM

635

EN/ES 0.171

0.130 0.281

EF/ET 0.482 0.400 0.088

ES/IN

0.058

0.313 0.500

ET/IT 0.001 0.144 0.049

EN/IN

0.001

0.031 0.166

EF/IT 0.012 0.090 0.417

3.6

3.6

EF

3.7

3.6

3.04

3.5

3.3

3.1

ET

3.7

3.5

3.9

IN

2.8

3.1

3.1

IT

2.9

3.2

3.2

4.3 Six Hat Comparisons Similarly for the Six Hats type
indicators, significant differences in performance results were
found between different types. To ensure that the type sets
were independent, a similar process of separating groups was
employed before comparison. An individual can be evaluated
as having a high score for multiple hats. Each individual must
be assessed according to their strongest hat preference to
ensure that comparisons are independent for inter-type
comparisons. However, some individuals who participated
scored a strong but equal indication for several hats. These
individuals were removed from analysis in the six hats
comparison. Once those individuals were removed from the
set, there were only enough individuals remaining to make
statistically significant comparisons between strong Yellow
Hat thinkers and strong Blue Hat thinkers. The results of this
comparison are shown in Figure 8.

Decision Making
BS

3.8

ES

Self-efficacy results for the cognitive modes appear to

align with the quantitative results for EF and ET, Introverts
tended to self-assess results lower than extroverts, which may
be why IT reports a lower effectiveness of their ideation, the
same can be seen with IN types. Table 6 summarizes the
average score for self-assessed high quality, novelty and
variety of ideas in the method.

Table 6: Self efficacy for MBTI. Range is from 1 to 5, where

5 is strong agreement.
Information Collection

EN

Quantity

Blue hats are characterized by preference for process

driven problem solving. It would be expected that Blue hats
perform well in 6-3-5 or C-Sketch, because it is a very
systematic method. Indeed, the mean performance of Blue
hats is higher for quantity (p=0.49), and variety (p=0.25) in
635 than that of Yellow hats, but not significantly. This may
be due to the fact that Yellow hats are generally strong
ideators and thus comparison to the other hats is required in
the future when data is available. However, there was
significance (p = 0.036) to the difference of mean scores for
self-efficacy with Blue hats reporting an average indication
that that they ideated more effectively than Yellow hats in CSketch. On a five point Likert scale, the Blue hats listed an
average 4.0 equivalent to Agree that their ideas had high
quality, quantity, and novelty during C-Sketch. Yellow hats
only listed 3 or Neutral for C-Sketch performance. The
remainder of self-efficacy results can be seen in Table 7. Other
comparisons in Table 7 also agree with the quantitative
results.

Quality

Table 7: Self efficacy for Six Hats. Range is from 1 to 5,

where 5 is strong agreement.

Self-Efficacy for High Quality, Novelty and

Variety of Ideas in the Method

Variety
Brainstorming
Methods
Mind-Mapping
methods
C-Sketch

Yellow

Blue

0.360

3.1

0.220

3.2

0.035

5. CONCLUDING REMARKS
The results of this paper provide a clear indication that there
are significant differences in the ideation results of different
personality types across a set of ideation methods. It is shown
that these significant comparisons match what would be
expected from the theory of types. For example, Jungs
cognitive modes for decision-making and information
gathering can both be used to interpret the characteristics of
ideation results. This type of comparison allows a deeper
understanding of ideation suites. On one hand, an emphasis
on one ideation technique will not fully explore the potential
of a group of individuals with differing communication and
decision skills, and their ideation preferences. On the other
hand, the results open consideration of new ideation methods
or sequencing of methods that would leverage the
characteristics of each communication style simultaneously.

Figure 8 Total averages for all participants across the

ideation session. Error bars are 1 standard error. The
vertical axis is performance in each metric (see section
3.7) The horizontal axis is the method code: I =
individual, G = group; BS = brainstorming, MM =
mindmapping, and 635 = C-Sketch

It is clear that yellow hats are generally productive in the

brainstorming and mind-mapping techniques. As would be
expected from the theory, indicating that the characteristic of
yellow hats is to expand on existing ideas act as optimistic
ideators. It is also interesting to note that yellow hats produce
this high quantity in both individual and group methods. The
statistical significance of Yellow Hats produced a higher
quantity of ideas than Blue hats in group brainstorming was
only p = 0.069. This falls below the .05 threshold, but given
the small sample size, it is a suggestion that with a larger
sample size the difference may prove significant. Furthermore,
it was significant that Yellow hats produced a higher than
average quality in group brainstorming (p=0.047). This is also
in accordance with extant Six Hat theory on Yellow hats as
idea supporters, since group brainstorming permits piggy
backing and leap-frogging.

P value

6. LIMITATIONS AND FUTURE WORK

The personality type indicators have been applied in a variety
of team formation, management, and psychology contexts;
however, this study was only able to evaluate the
characteristics of MBTI and Six hats indicators as compared to
ideation method in a single design problem. In support of our
approach is the consideration that this design problem was a
highly multidisciplinary one, touching on nano-technology,
controls, kinematics, interaction design, and programming.
Additionally, the type indicators may have some imprecision
in evaluating personality type. The objective of our study was
to explore trends across a set of individuals and therefore
attempt to reduce any effects that might be a reflection of the
individual. In regards to participants, there were not enough

10


For Innovative Design: Ideation Performance Based On
Analogical Distance, Commonness, And Modality Of
Examples, Journal Of Mechanical Design, 133, 081004.
[11] White, C., Wood, K., and Jensen, D., 2012, From
Brainstorming to C-Sketch to Principles of Historical Innovators:
Ideation Techniques to Enhance Student Creativity, Journal of
STEM Education, 13(5), pp. 12-25.
[12] Osborn, A. F., 1957, Applied Imagination, Scribner, New York.
[13] Diehl, M., and Stroebe W., 1987. Productivity Loss in
Brainstorming Groups: Toward the Solution of a Riddle, Journal
of Personality and Social Psychology, 53(3), pp. 497-509.
[14] Otto, K., and Wood, K., 2001, Product Design: Techniques In
Reverse Engineering And New Product Development, Prentice
Hall, Upper Saddle River, NJ.
[15] Shah, J. J., Vargas-Hernandez, N. O. E., Summers, J. D., and
Kulkarni, S., 2001, Collaborative Sketching (C-Sketch) An
Idea Generation Technique For Engineering Design, The
Journal of Creative Behavior, 35(3), pp. 168-198.
[16] Jensen, D., Wood, J., and Wood, K., 2003, Hands-On
Activities, Interactive Multimedia And Improved Team
Dynamics For Enhancing Mechanical Engineering Curricula,
International Journal of Engineering Education, 19(6), pp. 874884.
[17] Shen, S. T., Prior, S. D., White, A. S., and Karamanoglu, M.,
2007, Using Personality Type Differences To Form Engineering
Design Teams, Engineering Education, 2(2), pp. 54-66.
[18] Carl III, W. J., 1996, Six Thinking Hats: Argumentativeness
and Response to Thinking Model.
[19] Bandura, A., 1971, Social Learning Theory, General Learning
Corporation, New York.
[20] Locke, E. A., Frederick, E., Lee, C., and Bobko, P., 1984,
Effect Of Self-Efficacy, Goals, And Task Strategies On Task
Performance, Journal Of Applied Psychology, 69(2), pp. 241251.
[21] Gu, X. Y., Wang, D, Y., Zhou, S. Y., Yang, F. F, Lu, D. R.,
2013, Distributed Learning Workflow: An Operating System
that Integrates Information Technologies and the Real Campus
Research in Higher Education of Engineering, 2013(2), pp. 7289.

individuals of certain types and a number of unanswered

questions remain regarding the properties of those types for
which we did not have sufficient data. Finally, there are
potentially a number of other analyses and cross-comparisons
which could be developed using data from this workshop such
as technical skill sets and other personality type assessments,
but this would exceed the space allotted to this paper to
properly examine.
As with any psychological study a primary objective of
future work is to increase sample size. Additionally we hope
to explore the inter-relations with other aspects of the design
process (such as prototyping) and personality type. It could
also be useful to allow participants significantly more time to
ideate. It has been shown that some ideation methods permit
continual production of ideas if given a longer span of time. In
general, we find that this study was a fruitful and intriguing
look into the comparison of personality and ideation and find
this research area to be open for continued efforts.
ACKNOWLEDGEMENTS
This work is supported by the Singapore University of
Technology and Design (SUTD) and the SUTD-MIT
International Design Center (IDC, idc.sutd.edu.sg). This
project is partially supported by Chinas Natural Science
Foundation, project number: 70971073. Additionally, this
work is made possible by collaboration with Tsinghua
University in Beijing, and Center for Nano and Micro
Mechanics. The authors would also like to thank Peking
University and University College London participants for
their design efforts and patience in this study.
REFERENCES
[1] Shah, J. J., Smith, S. M., and Vargas-Hernandez, N., 2003,
Metrics For Measuring Ideation Effectiveness, Design Studies,
24(2), pp. 111-134.
[2] Jensen, D., Feland, J., Bowe, M., and Self, B., 2000, A 6-Hats
Based Team Formation Strategy: Development And Comparison
With An MBTI Based Approach, In Proceedings of the ASEE
Annual Conference, Session 2425.
[3] Wilde, D. J., 2008, Teamology: The Construction And
Organization Of Effective Teams, Springer.
[4] Myers, I. B., McCaulley, M. H., and Most, R., 1985, Manual: A
Guide To The Development And Use Of The Myers-Briggs Type
Indicator, Consulting Psychologists Press, Palo Alto, CA.
[5] DeBono, E., 1985, Six Hats Thinking, Little, Brown and Co,
Boston, MA.
[6] Linsey, J., Green, M. G., Murphy, J. T., Wood, K. L., and
Markman, A. B., 2005, Collaborating To Success: An
Experimental Study Of Group Idea Generation Techniques,
DETC2005-85351, In ASME Design Theory and Methodology
Conference, pp. 24-28.
[7] Linsey, J. S., Clauss, E. F., Kurtoglu, T., Murphy, J. T., Wood, K.
L., and Markman, A. B., 2011, An Experimental Study Of
Group Idea Generation Techniques: Understanding The Roles Of
Idea Representation And Viewing Methods, Journal of
Mechanical Design, 133(3), 031008.
[8] Puccio, G., and Grivas, C., 2009, Examining The Relationship
Between Personality Traits And Creativity Styles, Creativity and
Innovation Management, 18(4), pp. 247-255.
[9] Durling, D., Cross, N., and Johnson, J., 1996, Personality And
Learning Preferences Of Students In Design And Design-Related
Disciplines, In IDATER 1996 Conference, Loughborough
University, Loughborough.
[10] Chan, J., Fu, K., Schunn, C., Cagan, J., Wood, K., and
Kotovsky, K., 2011, On The Benefits And Pitfalls Of Analogies

Additional Resources:
self efficacy
Bandura, A., & Schunk, D. H. (1981). Cultivating competence, selfefficacy, and intrinsic interest through proximal self-motivation.
Journal of personality and social psychology, 41(3), 586.
Bandura, A., & Schunk, D. H. (1981). Cultivating competence, selfefficacy, and intrinsic interest through proximal self-motivation.
Journal of personality and social psychology, 41(3), 586.
Bandura, A. (1982). Self-efficacy mechanism in human agency.
American psychologist, 37(2), 122.
Bandura, A. (1994). Self-efficacy. John Wiley & Sons, Inc..
Carberry, A. R., Lee, H. S., & Ohland, M. W. (2010). Measuring
Engineering Design Self-Efficacy. Journal of Engineering
Education, 99(1), 71-79.
Schunk, D. H. (1995). Self-efficacy, motivation, and performance.
Journal of Applied Sport Psychology, 7(2), 112-137.
Zimmerman, B. J. (2000). Self-efficacy: An essential motive to learn.
Contemporary educational psychology, 25(1), 82-91.
Pintrich, P. R., & De Groot, E. V. (1990). Motivational and selfregulated learning components of classroom academic
performance. Journal of educational psychology, 82(1), 33.
Tsenn, J., McAdams, D., Linsey, J., A Comparison of Design SelfEfficacy of Mechanical Engineering Freshmen, Sophomores, and
Juniors, In Proceedings of the 2013 American Society for
Engineering Education Annual Conference and Exposition,
Atlanta GA
ideation
Blair, B. M., & Hltt-Otto, K. (2012, August). Comparing the
Contribution of the Group to the Initial Idea in Progressive Idea

11


Generation. In ASME 2012 International Design Engineering
Technical Conferences and Computers and Information in
Engineering Conference (pp. 425-436). American Society of
Mechanical Engineers.
Fu, Katherine, et al. "The Meaning of Near and Far: The Impact
of Structuring Design Databases and the Effect of Distance of
Analogy on Design Output." Journal of Mechanical Design 135
(2013): 021007.
Linsey, J. S., Markman, A. B., & Wood, K. L. (2012). Design by
analogy: A study of the wordtree method for problem rerepresentation. Journal of Mechanical Design, 134, 041009
creativity metrics
Oman, S. K., Tumer, I. Y., Wood, K., & Seepersad, C. (2013). A
comparison of creativity and innovation metrics and sample
validation through in-class design projects. Research in
Engineering Design, 24(1), 65-92.
Shah, J. J., Kulkarni, S. V., & Vargas-Hernandez, N. (2000).
Evaluation of idea generation methods for conceptual design:
effectiveness metrics and design of experiments. Journal of
Mechanical Design, 122, 377.
interaction and personality traits
Carr, P. G., De La Garza, J. M., & Vorster, M. C. (2002).
Relationship between personality traits and performance for
engineering and architectural professionals providing design
services. Journal of Management in Engineering, 18(4), 158-166.
Feijs, L., Kyffin, S., & Young, B. (2005, November). Design and
semantics of form and movement. In DeSForM Workshop.
De Bono, E. (1995). Serious creativity. Harper Business.
De Bonos, E. (2012). Six Thinking Hats.
Meneely, J., & Portillo, M. (2005). The adaptable mind in design:
Relating personality, cognitive style, and creative performance.
Creativity Research Journal, 17(2-3), 155-166.
Richardson, A. L. (2008). Tinkering interactions on freshman
engineering design teams. In Proceedings of the 2008 American
Society for Engineering Education Annual Conference and
Exposition.
Ullman, D. G. (1992). The mechanical design process (Vol. 2). New
York: McGraw-Hill.

12