Académique Documents
Professionnel Documents
Culture Documents
a Survey
A PRIMER ON
SURVEY RESEARCH
BY DR. JAN G. WEST, PH.D.
NATIONAL BUSINESS RESEARCH INSTITUTE, INC. 15305 Dallas Parkway 3rd Floor, Addison, TX 75001 (800) 756-6168 www.NBRII.com
Youve been tasked with conducting a survey of your customers and/or employees, so the first thing youre doing
is searching the internet for how to information. Or, like most people, youve always thought that building a survey
meant pulling questions out of the air, slapping them on a piece of paper, and calling it a survey! But now that
youre trying to build one yourself, youve run into your first stumbling block: the survey questions themselves.
What do I ask? How do I ask them? Youve come to the right place! Not only will you learn how to conduct survey
research here, but you will also learn why. When the why rings true with good, common sense, you will have
great confidence in how you run your study, and you will be able to defend your actions and recommendations to
others. Most importantly, you will be obtaining fact and not fiction from your research, so that stakeholders
who use your information can be confident when basing important decisions and actions upon it.
The goal of great research is to drive business outcomes: improve Customer Satisfaction, Customer Intent to
Return or Willingness to Recommend, and Employee Engagement in order to improve Financial Performance.
Survey Research without Return on Investment may provide interesting information, but is truly a waste of time
and money. For over 3 decades, the National Business Research Institute (NBRI) has helped business leaders
understand the psychological constructs of attitude, opinion, and belief that determine employee and customer
behavior, and drive profitability. In this eBook, we share our knowledge of the following aspects of survey research:
1.
a. Topics
b. Questions
c. The Scale
2.
a. Paper
b. Online
c. Telephone
d. In Person
3.
4.
b. Mean Scores
c. Benchmarking Scores
5.
a. Descriptive Statistics
b. Inferential Statistics
NATIONAL BUSINESS RESEARCH INSTITUTE, INC. 15305 Dallas Parkway 3rd Floor, Addison, TX 75001 (800) 756-6168 www.NBRII.com
T / F
T / F
T / F
4. I have achieved Representative Sample at a minimum of 95% Confidence and 5% Error at the
T / F
T / F
6. Percent Favorable Scores are context, and are not used to interpret results or drive decision-making.
T / F
7. Mean Scores are context, and are not used to interpret results or drive decision-making.
T / F
T / F
T / F
T / F
11. One of the primary root causes of Customer Intent to Return is Employee Engagement.
T / F
12. One of the primary root causes of Financial Performance is Customer Intent to Return.
T / F
T / F
T / F
15. My Survey Vendor takes me from reporting my survey data to implementing solutions in 3 weeks.
T / F
16. My Survey Vendor has the ability to benchmark my data against my industrys data.
T / F
T / F
T / F
T / F
T / F
BONUS: The integrity of my data is absolute, so I have confidence tying improvement goals
T / F
B = 80-85 points
C = 70-75 points
D = 60-65 points
F = 55 or fewer points
NATIONAL BUSINESS RESEARCH INSTITUTE, INC. 15305 Dallas Parkway 3rd Floor, Addison, TX 75001 (800) 756-6168 www.NBRII.com
1 SURVEY CONTENT
The first step, of course, is deciding on survey content. Do you decide
this alone? Do you ask co-workers or other management personnel
for their input? Do you ask the audience what they think you should
be asking them? Never has it been more true than in survey research
that garbage in = garbage out. This can be a frightening and
daunting task, as you realize the critical nature of customer and
employee opinion as driving forces on your organizations financial
performance. What if customers do not intend to return to your
business? What if employees are apathetic, and are actually running
off customers? This makes it imperative that each study provide
management with pure, valid data fact and not fiction. While it
is easy to conduct survey research improperly, it is just as easy to
conduct it properly by adhering to a few, simple principles.
There are really only three sources of survey content: you, us, and them. You, most likely an Executive of a Major
Corporation; Us, Survey Research Consultants; and, Them, Your Target Audience. From the horses mouth
comes to mind about now because who better to tell you what you should ask on your survey than those who will
be responding to it? Your employees and customers know very well what you should include in your survey, as they
live the issues the survey will address every day. Clearly, your employees and customers are the best source of
input on what the survey should cover.
Great, you think. Now I have to do research before I can even begin the survey. And how does one go about
conducting Preliminary Research?
TOPICS
Beginning in 1982 and for over 3 decades now, NBRI has been conducting one-on-one telephone interviews of
small, stratified, random samples of individuals in employee and customer populations within virtually every
industry sector. As we have done so, we have discovered the issues that are important to all such populations.
Thirty years ago, we called these organizational dynamics because thats exactly what they were, and there were
no other names for them. Today, we call them topics. Topics are simply broad categories of survey question
types. Some of the most common Customer and Employee Survey Topics are shown on the following page.
NATIONAL BUSINESS RESEARCH INSTITUTE, INC. 15305 Dallas Parkway 3rd Floor, Addison, TX 75001 (800) 756-6168 www.NBRII.com
Ethics
Sales Assistance
Communications
Financing
Sales Process
Company Behavior
Company Image
Services
Competitive Position
Job Knowledge
Teamwork
Control Systems
Pricing
Technical Expertise
Product Delivery
Technical Support
Creativity
Products
Values
Customer Loyalty
Professional Conduct
Wait Time
Customer Service
Project Management
Warranties
Decision Making
Quality
Website
Employee Behavior
Safety
Handling Ambiguity
Professional Conduct
Benefits
Health
Profit Improvement
Career Development
Human Resources
Project Management
Change Management
Integrity
Quality
Climate
Interpersonal Skills
Recognition
Communications
Job Satisfaction
Respect
Company Image
Job Training
Results Driven
Compensation
Leadership
Safety
Competitive Position
Leading Change
Sexual Harassment
Control Systems
Learning Agility
Corporate Performance
Life Balance
Social Activities
Creativity
Management Style
Social Responsibility
Culture
Managing Performance
Strategic Agility
Customer Service
Supervision
Developing People
Morale
Teamwork
Diversity
Organization
Technology
Employee Commitment
Organizational Change
Values
Employee Values
Organizational Structure
Vision
Engagement
Performance Evaluations
Work Life
Ethics
Planning
Working Relationships
Flexibility
Policies
Fostering Relationships
Productivity
NATIONAL BUSINESS RESEARCH INSTITUTE, INC. 15305 Dallas Parkway 3rd Floor, Addison, TX 75001 (800) 756-6168 www.NBRII.com
QUESTIONS
Also beginning in 1982 and for over 3 decades now, NBRI
has been writing the questions to assess those topics;
questions that have become the very foundation of the
Survey Research Industry. So, as you see, you do not need
to interview your customers and employees first! You do
not need to discover the issues of importance to them.
And you do not need to write the survey questions. That
has all been done for you by NBRI.
It is important to reiterate that you are conducting psychological research of your customers and employees. What? I
just wanted to survey them! Right! And assessing the psychological constructs of attitude, opinion, and belief, those
human perceptions that drive behavior, is by definition, psychological research! As such, how each survey question is
phrased is very important. The wording of each question has the potential to bias responses artificially, either to the
positive or negative.
Consider, for example:
NATIONAL BUSINESS RESEARCH INSTITUTE, INC. 15305 Dallas Parkway 3rd Floor, Addison, TX 75001 (800) 756-6168 www.NBRII.com
instrument that has been tested on millions of individuals. Or would you rather reinvent the wheel and risk
conducting a survey that delivers misinformation?
The order in which questions are presented may also bias responses. It is important to randomize the order of
questions on employee surveys, but not customer surveys. Customers should be taken through the survey in a
thoughtful, logical order, with questions grouped by topic. But employees may have biases against certain topics,
so the order of survey questions to employees should be randomized to eliminate biasing the data.
THE SCALE
Even the scale may cause your data to be more positive or negative than data collected with an unbiased scale.
Consider, for example, a scale of Excellent, Extremely Good, Very Good, Good, Fair. This type of scale will produce
data that is artificially biased to the positive, rendering the results worse than meaningless the data will actually
be misleading.
The scale typically used in psychological research is a 6-point, balanced scale with 3 points of positive and 3
points of negative. This way, the respondent has high, medium, and low options of both the positive and negative.
And, there is no neutral point. After all, the intent of the survey is to gather opinions. If your survey is valid, defined
as representative of the issues facing your audience, then as psychologists, we know that an opinion exists for
each and every survey question. Yes, it is a psychological fact that if you have knowledge of an issue, you will
necessarily have an opinion about it. Even if it is as simple as vanilla ice cream. If you have experienced it, you
have an opinion. The goal is to gather opinions.
THE NORMATIVE SCALE
Strongly
Agree
Moderately
Agree
Slightly
Agree
6-point scale
Balanced
Slightly
Disagree
Moderately
Disagree
Strongly
Disagree
When NBRI has imported data for analysis from organizations that previously conducted their own research and
used a neutral point, we have found that respondents mark the neutral point about 20% of the time. This data
must be discarded. It is meaningless. This means that 20% of the survey dollars were just spent on no data at all!
When a respondent wants to skip a question, they can and they will. Why would anyone want to offer a no opinion
option on every single survey question?
NATIONAL BUSINESS RESEARCH INSTITUTE, INC. 15305 Dallas Parkway 3rd Floor, Addison, TX 75001 (800) 756-6168 www.NBRII.com
2 DEPLOYMENT METHODOLOGIES
PAPER
Paper surveys, while far less popular since the inception of the internet, represent
a valuable methodology for reaching the large numbers of employees and
customers without computer access, email addresses or telephone numbers.
When considering deploying a paper survey, outbound and inbound postage must
be considered, as well as ink, paper, printing, folding, stuffing, sealing, etc., and the
labor cost to do so. Furthermore, one can expect only about a 10% response rate
from customers asked to respond to a paper survey. If this is your only option, be
sure to check out Section 3 of this eBook to learn about the validity of data and
determine whether or not you will be able to have confidence in your results.
ONLINE
Online surveys have become the most popular means of deploying a survey because
of their time- and cost-effectiveness. In addition, surveyors can quickly respond to
poor scores and negative feedback from the respondent submitting the survey. This is
critical, indeed, as customers who experience issues are much more likely to patronize
an organization again if issues are dealt with promptly and satisfactorily, while those
whose issues are ignored or dealt with poorly tend to spread the word and cost the
organization not only their own business but that of many others, as well.
Because of the relatively low cost of online surveys, and the higher customer
response rate of 20% as compared to paper surveys, many Do-It-Yourself options
have sprung up on the internet in the last decade. Companies like Zoomerang and Survey Monkey make millions
of dollars per year by offering basic survey programming for the uneducated who would create their own surveys.
Completely ignored by these tools are considerations of biases in the wording of the questions, the validity of the
survey and the validity of the data gathered, reaching representative sample, employing appropriate statistical
analyses and data interpretation, benchmarking, and assistance with responding to survey results. This places
every user in the precarious situation of basing important business decisions on misinformation. Users of these
tools generally have no idea that their data is worse than worthless it is actually misleading.
There are no warning signs in research. You either know how to conduct research properly, or you dont. Most collegeeducated business people do not learn how to conduct psychological research unless they have participated in
doctoral-level courses that emphasize research methods. Companies like Zoomerang and Survey Monkey perpetuate
the myth that a survey can be created by pulling questions out of the air and sending it to a large number of people.
Research that drives your business when done properly will provide a return on investment that is huge compared
to the cost of a study done well. While cost is an important consideration, the choice between spending a few hundred
dollars for junk data versus a few thousand dollars for hard, objective data should clearly be no choice at all.
NATIONAL BUSINESS RESEARCH INSTITUTE, INC. 15305 Dallas Parkway 3rd Floor, Addison, TX 75001 (800) 756-6168 www.NBRII.com
TELEPHONE
The highest labor cost but also the highest response rate of all for customer surveys
is through telephone deployments. An online survey is still programmed, but instead
of deploying it online, telephone interviewers read the survey to participants over the
phone and enter the data online for them.
Some survey research organizations offer IVR (Interactive Voice Response) as an
inexpensive alternative to live telephone interviewers, but beware of the insult this
may cause to your valued customers! NBRIs research shows that ill-will is often
caused by the use of recorded telephone interviewers, and highly recommends
against it. On the other hand, customer loyalty actually increases when you put your
customer feedback in the hands of a highly skilled researcher. Deeper exploration of opinions and beliefs is only
possible between two humans, not between a human and a machine, and customers are keenly aware of whether
or not you feel they are worth it.
Due to the fact that interviewers are able to call a phone number numerous times if no one is reached the first
several times, and can do so without upsetting anyone, response rates of customer surveys deployed via telephone
are typically around 30%! Caller ID and leaving a call-back number also increase participation.
IN PERSON
The rarest of all survey deployments is in person. Due to the high labor cost,
usually out of the office and often out of town, thus requiring transportation and
accommodations, it is unusual to find a situation that is not better handled via
paper, online, or telephone. Indeed, only when extremely high response rates are
required due to the small size of the population should In Person Deployment even
be considered. With proper planning and scheduling, however, customer response
rates of 100% are possible through this methodology!
The response rates for employee feedback do not differ by deployment methodology: one can expect an 85% to
95% response rate when Best Practices are followed.
The response rates for customer surveys differ greatly by deployment methodology as discussed above.
NATIONAL BUSINESS RESEARCH INSTITUTE, INC. 15305 Dallas Parkway 3rd Floor, Addison, TX 75001 (800) 756-6168 www.NBRII.com
10
3 REPRESENTATIVE SAMPLE
Just as a valid survey truly represents the issues facing the target population, valid data truly represents
the thinking of the target population. The validity of psychological data is easily determined by measuring the
Confidence Level and Sampling Error.
Important! All surveys have Confidence Levels and Sampling Error, including one in which 100% of the population
participates. It is critical to measure and report these statistics in any form of survey research in order to prove the
validity of the data.
SAMPLING ERROR
So, if you have a population of 100 employees or customers, and only 10 complete the survey, does your data
represent how all 100 people think? What if you have 10,000 employees or customers, and 1,000 complete the
survey? Will a 10% response rate from any size population provide us with the confidence to claim that our data is
valid and truly represents the thinking of the entire population?
Sampling Error is basically the opposite of Confidence Level. Sampling Error is the probability that we have a
preponderance of people in our data that feel differently than the majority, such as extremely pleased or displeased
people. Data with large amounts of Error in it misrepresents the thinking of the population as a whole, and is not
valid. Error, then, must be kept to a minimum.
Reminder! All surveys have Sampling Error, including one in which 100% of the population participates, and since
it is critical to report this statistic in research, you need to know how to measure it. Do we pull numbers out of the
NATIONAL BUSINESS RESEARCH INSTITUTE, INC. 15305 Dallas Parkway 3rd Floor, Addison, TX 75001 (800) 756-6168 www.NBRII.com
11
air? Do we hope, think, wish our way to 50% being o.k. 75% being o.k. 90% being o.k.? Certainly not! This is
hard, objective, scientific research upon which our Clients often base multi-million dollar decisions. We have to
know, and so do you!
Based on 100 years of psychological research, NBRI depends on a proven Sample Size Chart!
99%
95%
99%
95%
90%
5%
Response
Rate
Sampling Error
5%
3%
3%
3%
50
46.5
44.2
88.4%
48.6
47.7
46.8
100
86.9
79.3
79.3%
94.8
91.4
88.2
500
285.1
217.2
43.4%
393.3
340.4
300.2
1,000
381.9
269.2
26.9%
604.7
488.2
409.5
2,000
498.2
322.2
16.1%
959.2
695.8
546.3
10,000
622.2
369.9
3.7%
1556.3
964.2
699.1
50,000
654.8
381.2
0.7%
1777.7
1044.8
740.5
100,000
659.2
382.6
0.4%
1809.9
1055.8
746.0
Population Size
The box around 95% Confidence Level and 5% Sampling Error highlights the fact that these levels are perfectly
acceptable in business and science. Note that with a Population size of 1,000, 269.2 completed surveys are
required to reach a 95% Confidence Level and 5% Sampling Error, but 604.7 completed surveys are required to
reach a 99% Confidence Level and 3% Sampling Error. It is not necessary to spend money collecting and analyzing
more data than is needed without an imperative to do so. Many of NBRIs studies are submitted to governmental
agencies, such as the Federal Drug Administration (FDA), and a 99.9% CL with less than 1% SE is required. In
general, however, NBRI recommends the 95%/5% to Clients because of the high validity of the data and the costsavings to do so.
So, does 10% from populations of 100 or 1,000 obtain valid data for us? As the chart shows, a 79.3% Response
Rate, or 79.3 completed surveys is required to obtain valid data at a 95% Confidence Level and 5% Sampling Error
NATIONAL BUSINESS RESEARCH INSTITUTE, INC. 15305 Dallas Parkway 3rd Floor, Addison, TX 75001 (800) 756-6168 www.NBRII.com
12
from a total population of 100 individuals, so 10% would be junk data. For a total population of 1,000 individuals,
a 26.9% Response Rate would be required for a 95% & 5% result, so 10% would be insufficient in this case, as well.
However, 10% would be more than enough to reach 95% & 5% for 10,000 people and 100,000 people, as only 3.7%
and 0.4% Response Rates are needed for these populations, respectively.
It is important to note that valid data must be obtained at the total population level and at each level of segregation
thereof. For example, if your total population in your Market Research Study is 1,000 people, and you want to learn
how the thinking of 500 males and 500 females differs, you will need a 43.4% Response Rate, or 217.2 completed
surveys from each subgroup. Likewise, valid data based on the Confidence Level and Sampling Error you select
must be obtained for each division, business unit, or department of a Company in an Employee Survey if you want
to compare the thinking of such groups.
Clearly, conducting Scientific Psychological Research that results in pure, clean, hard data that truly represents
the thinking of the population and its subgroups is best done by those with extensive education, training, and
experience in the field. Sources of error abound and are typically overlooked by the untrained eye. Defending the
quality and purity of ones research is a basic requirement for Doctoral Degrees, and represents a key value that
Ph.D.s at NBRI bring to every Clients Survey Research Study.
NATIONAL BUSINESS RESEARCH INSTITUTE, INC. 15305 Dallas Parkway 3rd Floor, Addison, TX 75001 (800) 756-6168 www.NBRII.com
13
In the upper left corner is the Topic: Employee Behavior. Then, there are 3 Survey Questions assessing this Topic:
Company personnel deliver what is promised.
Company personnel exhibit professionalism.
Company personnel are well trained.
Beneath each Survey Question are demographic segregations of the data:
Total Customer Base 2013.
Total Customer Base 2012 for historical comparison, also known as internal benchmarking.
Customer Type 1, 2, 3, and 4.
NATIONAL BUSINESS RESEARCH INSTITUTE, INC. 15305 Dallas Parkway 3rd Floor, Addison, TX 75001 (800) 756-6168 www.NBRII.com
14
The 2nd column shows the N, or Number Responding. The 3rd column shows the Mean Scores. The 4th column
shows the Benchmark Scores. Benchmark Scores are graphed and color-coded in the 5th column. Finally, the
Scale and entire Distribution of Responses (DOR) are shown in the last 6 columns.
In this way, the report user has all of the information in one place. This shows exactly how many individuals
answer in exactly which way, and rolls up all of the information to Total Customer Base benchmarked against the
Industry level via the Benchmark Score.
Now lets look at the 3 demographics outlined in red: Customer Types 2, 3, and 4, and lets consider each
statistic individually.
MEAN SCORES
In our example above, it is apparent that there are differences between the Customer Types 2, 3, and 4, with Mean
Scores of 4.77, 5.45, and 5.67, respectively. The Mean Score is the average of all scores, and is calculated by
weighting the six response choices. It is clear from the Mean Scores that true differences exist between these
groups, but what do the Mean Scores mean? Is Customer Type 4 significantly higher than Customer Type 3 and 2?
And are these good, average, or poor scores? Does the Vice President of Customer Type 4 qualify for a bigger
bonus than the Vice President of Customer Type 2?
Many people compare the Mean Score to the Scale itself to determine whether it is high or low, good or bad. Hint:
the mean of a 6-point scale is 3.5 (not 3, see below). Comparing the Mean Score to the scale (whether it is above
or below the mean of the scale, for instance) results in false information because the scale does not represent
NATIONAL BUSINESS RESEARCH INSTITUTE, INC. 15305 Dallas Parkway 3rd Floor, Addison, TX 75001 (800) 756-6168 www.NBRII.com
15
reality! That is, the average of all responses to all questions from all groups of all people is not 3.5! Consider the
two Customer Survey questions in the picture below, both of which receive a Mean Score of 3.99, for example.
BENCHMARKING SCORES
What if you knew how people normally answer any given question? NBRI does! This is called normative or
benchmarking data.
SCIENTIFIC PSYCHOLOGICAL RESEARCH
Strongly
Agree
Moderately
Agree
Slightly
Agree
Slightly
Disagree
Moderately
Disagree
Strongly
Disagree
3.99
Is this a high score or a low score?
I would recommend doing business with the Company to others.
Sales Representatives are responsive to my needs.
NBRI compares the data from your respondents with data from people in your industry who have answered
the same survey questions about their places of work (employees) or about your competitors (customers)!
Benchmarking Data is a mapping of mean scores from thousands of individuals to a Scale of the 1st to 100th
percentiles. The Average of all mean scores is at the 50th percentile; the 75th percentile represents Stretch
Performance; and the 90th to 100th percentiles represent Best in Class Performance. These are the Companies
with the highest Employee Engagement, Customer Loyalty, and Financial Performance!
When we look at Benchmarking Data, we may find that a Mean Score of 3.99 is a poor score for one survey
question, but a great score for a different survey question! The charts below demonstrate this principle.
For the survey question, I would recommend doing business with the Company to others, a mean score of 3.99
is a poor score at the 38th percentile of this particular benchmarking database, with the Industry Average at 4.56
(50th), Stretch Performance at 5.18 (75th), and Best in Class Performance at 5.64 (90th).
NATIONAL BUSINESS RESEARCH INSTITUTE, INC. 15305 Dallas Parkway 3rd Floor, Addison, TX 75001 (800) 756-6168 www.NBRII.com
16
Moderately
Agree
Slightly
Agree
Slightly
Disagree
Moderately
Disagree
Strongly
Disagree
5.64 5.18
90TH
4.56
75TH
50TH
3.99
MEAN
38TH
For another survey question, Sales representatives are responsive to my needs, a mean score of 3.99 is a Best in
Class score at the 96th percentile of the benchmarking database!
SCIENTIFIC PSYCHOLOGICAL RESEARCH
Strongly
Agree
Moderately
Agree
Slightly
Agree
Slightly
Disagree
Moderately
Disagree
Strongly
Disagree
3.99 3.64
MEAN
96TH
90TH
2.78 2.26
75TH
50TH
In short, it is impossible to understand the meaning of any score percent favorable, top box, mean score, or any
other without benchmarking data.
Another powerful benefit of benchmarking data is the ability to group results by quartile into a SWOT Analysis:
Strengths, Opportunities, Weaknesses and Threats. When the Total Company receives a benchmark score, each
Business Unit, Division, and Department receives benchmark scores, each Customer Type or other Customer
Demographic Group receives a benchmark score, and every Topic and Question on the survey receives a
benchmark score, it is immediately evident how each score ranks against the Industry (external benchmarking)
and against each other (internal benchmarking). In addition, because 5 or more benchmarking percentiles
is statistically significant (defined as true, factual, not due to chance), you know immediately if differences
between scores in the present study are significant or not, and you know immediately if differences between
scores in the present versus past studies are significant or not. The goal of great research is to drive business
outcomes: improve Customer Satisfaction, Customer Intent to Return or Willingness to Recommend, and Employee
NATIONAL BUSINESS RESEARCH INSTITUTE, INC. 15305 Dallas Parkway 3rd Floor, Addison, TX 75001 (800) 756-6168 www.NBRII.com
17
Engagement in order to improve Financial Performance. Survey Research consistently provides interesting
information about an organization but wastes time and money if no action is taken on the results to effect a return
on investment. When actions are taken, it is important to measure their effects. Benchmarking data provides an
immediate understanding of the differences between scores.
SWOT ANALYSIS
Category
Benchmarking Range
Strength
Opportunity
Weakness
Threat
NATIONAL BUSINESS RESEARCH INSTITUTE, INC. 15305 Dallas Parkway 3rd Floor, Addison, TX 75001 (800) 756-6168 www.NBRII.com
18
Number of Items
30
20
2012
2011
16
12
10
15
Strength
5
1
0
100th - 75th
10
Opportunity
75th - 50th
Weakness
50th - 25th
Threat
25th - 1st
In the First Annual Assessment in 2011, there were a preponderance of Threats and Weaknesses, shown in green. In
the Second Assessment in 2012, the distribution of Item Scores was more evenly spread from Threats to Strengths,
shown in light blue. And in the Third Assessment in 2013, there is a preponderance of Strengths and Opportunities,
shown in dark blue. Moving the scores at the item or survey question level ensures Total Company improvement, as
in the example on the following page.
NATIONAL BUSINESS RESEARCH INSTITUTE, INC. 15305 Dallas Parkway 3rd Floor, Addison, TX 75001 (800) 756-6168 www.NBRII.com
19
Benchmarking Percentile
90
75
63
72
75
ABC
2012
Stretch
79
50
50
25
0
Industry
Average
ABC
2011
ABC
2013
Best in
Class
In the chart above, ABC Company moved from the 63rd percentile of their industry in 2011 to the 72nd percentile
in 2012, and to the 79th percentile in 2013. This is a good example of the amount of improvement typically seen by
NBRI Clients. As important as it is to conduct survey research properly and obtain hard, valid data, it is simply an
exercise in futility without targeted action that dramatically improves the organization. You have two choices: work
on low scoring items that may only affect themselves; or, work on root cause perceptions that, when improved, will
effect dramatic and immediate widespread improvement across the organization.
INFERENTIAL STATISTICS
Those who work with psychological data on a regular basis, such as College Professors and NBRI Organizational
Psychologists, apply advanced analytics to the raw data in order to pull the root causes, or primary perceptions,
out of the thinking of the survey respondents. As important as the descriptive statistics described above are, they
cannot tell us the key factors driving customer and employee behavior. So, we put those aside for now, and return
to the Clients raw scores.
Most Survey Research Consultants offer their Clients correlations, which they call key factors or key drivers.
This is in large part a misnomer, as shown below. A very famous correlation is that of Ice Cream Sales and
Swimming Accidents. The numbers of both increase and decrease together: as one goes up, so does the other, and
as one comes down, so does the other. This is the essence of a correlation: items that move together.
CORRELATION
Ice Cream Sales
Swimming Accidents
NATIONAL BUSINESS RESEARCH INSTITUTE, INC. 15305 Dallas Parkway 3rd Floor, Addison, TX 75001 (800) 756-6168 www.NBRII.com
20
If your Survey Research Consultant were to tell you that these are your Key Factors or Key Drivers, what would you
work on? Unfortunately, the vast majority of Consultants simply lack the in-depth training and knowledge required
to properly analyze psychological data.
NBRI applies two additional analyses to Client raw data: regressions and path analyses. Following our example
above, when we analyze the data with a Regression Analysis, we find the Key Factor or Key Driver that is, in fact,
causing the Correlation is the Weather: Hot Summer Days cause both scores (Ice Cream Sales and Swimming
Accidents) to increase, and Cold Winter Days cause both scores to decrease.
REGRESSION
Swimming Accidents
WEATHER
Armed with this information, we now have a shortcut to improve a vast number of variables, from ice cream
sales to flip flop sales to sunglasses sales, etc. The same principle should be applied to psychological data from
customer and employee surveys. An example is shown below.
A major metropolitan city in the northeast United States was having high turnover; employees were dissatisfied on
several levels; NBRI was called in to pinpoint the source of dissatisfaction. We found the following correlation.
CORRELATION
I Trust Management
My Morale is Good
Had NBRI stopped here, Management would have been faced with a decision to work on employee trust in
management and/or employee morale. But advanced analytics provided Management with keen insight into the
thinking of the employee population by conducting a regression analysis and path analyses on the raw data.
NATIONAL BUSINESS RESEARCH INSTITUTE, INC. 15305 Dallas Parkway 3rd Floor, Addison, TX 75001 (800) 756-6168 www.NBRII.com
21
It was found that the employees overriding perception concerned job promotions being unfair. This was their Cold
Winter Days, driving down the scores of employee trust and morale.
REGRESSION
I Trust Management
My Morale is Good
Indeed, the perception that job promotions were unfair was driving down the scores of well over 40% of all survey
questions, having far-reaching effects into perceptions of Discrimination, Diversity, Policies, Culture, and more.
ROOT CAUSE MODEL
Item#
Topic
Item Text
Norm
24
Culture
36
Morale
53
Management
43
14
34
16
Diversity
46
25
Policies
43
43
Culture
42
54
Culture
37
When Management addressed the issue of favoritism that had contaminated the process of job promotions for
many years, the score of the Root Cause increased dramatically, but more importantly, the scores of all items
driven by this Root Cause increased dramatically as well, which in turn effected huge gains in Total Organization
Scores, Employee Satisfaction and Employee Engagement, greatly reducing turnover and improving productivity.
Clearly, the research had provided huge ROI and paid for itself many times over.
Once you have built a valid survey, gathered valid, representative sample data, described the body of data with
descriptive statistics, and cut to the chase with inferential statistics to pinpoint the most important issues to
address, it is time to turn FACT into ACT, with Action Planning!
NATIONAL BUSINESS RESEARCH INSTITUTE, INC. 15305 Dallas Parkway 3rd Floor, Addison, TX 75001 (800) 756-6168 www.NBRII.com
22
NATIONAL BUSINESS RESEARCH INSTITUTE, INC. 15305 Dallas Parkway 3rd Floor, Addison, TX 75001 (800) 756-6168 www.NBRII.com
23
The Action Planning Teams should also be creative about ways to build excitement and collect ideas. Sending
reminder emails and counting the days to the close, putting up posters in hallways and break rooms, putting
balloons on suggestion drop boxes, sponsoring departmental contests with movie or dinner ticket awards, and
other fun, celebratory ideas can be highly motivational to involve the workforce.
At the end of two weeks, each Action Planning Team meets individually. These meetings typically require no more
than a morning. When the meeting is over, it is typically not rescheduled, and a hard stop should be agreed to in
advance. Each Single Point of Contact will bring their long list of all recommendations received to improve the
Root Cause assigned to them, with a copy for every team member. The Single POC will not remove any suggestions
made to him or her without approval of their Action Planning Team.
The team goes through the list one item at a time. As each item is mentioned, the team votes to place each item in
one of the following categories:
1. unanimously agreed to be discarded, or
2. unanimously agreed to go on the short list for Senior Management approval, or
3. requires further discussion.
The team members then debate, defend, and discuss the remaining items requiring discussion in an effort to gain
consensus regarding their disposition. Depending on the number of items remaining on the list after the initial
vote, and the amount of time remaining for the meeting, teams may set a time limit of 5 to 10 minutes per item for
discussion. At the end of the discussions, the final vote takes place, and majority rules on any additional items to
be placed on the short list. The final short list is then ready to be presented to Senior Management.
Employees will provide innovative and valuable recommendations for Senior Managements consideration.
Recommendations from the Action Planning Teams should be given high priority, and approvals from Senior
Management should be provided as swiftly as possible.
Implementation of solutions to root cause issues should occur quickly. How it is done varies by subject matter,
organizational structure, culture, manpower availability, and a variety of other factors. Senior Management,
working closely with the Action Planning Teams, can lend guidance to the most efficient and effective means of
implementing these important solutions.
Regularly surveying and continuous improvement will ensure that your organization reaches Best in Class performance!
NATIONAL BUSINESS RESEARCH INSTITUTE, INC. 15305 Dallas Parkway 3rd Floor, Addison, TX 75001 (800) 756-6168 www.NBRII.com
NATIONAL BUSINESS RESEARCH INSTITUTE, INC. 15305 Dallas Parkway 3rd Floor, Addison, TX 75001 (800) 756-6168 www.NBRII.com