Assessment Center for Psychology Major Evaluation

1. What outcomes will you focus on?

a. Which outcomes are important assessment criteria for JMUs psychology major? Consider the
outcomes all majors should have when they graduate. What should majors be able to do when they
graduate? Make sure you explain why you have chosen those criteria.

All majors have objectives and goals that they hope each student will achieve upon graduation. These
goals should help each student understand what will be expected of them in their field and how to best
execute tasks required. Preparation for the job should only be part of the goals, however. Emphasis on
versatility is essential since career paths often change throughout life - many end up in fields they did not
get a degree in (Landrum, 149). Disciplined observation, problem solving, and communication are part
and parcel of education in psychology, as they are in many other disciplines (Landrum, 150).

Desirable outcomes that the JMU Psychology department reflect the standard international approach
brought forth by the Bologna Process and APA guidelines, expanding where they feel it is necessary.
According to the department website, their top ten objectives are as follows: Knowledge Base of
Psychology, Research Methods in Psychology, Critical Thinking Skills in Psychology, Application of
Psychology, Values in Psychology, Information and Technological Literacy, Communication Skills,
Sociocultural and International Awareness, Personal Development, and Career Planning and
Development (Psychology).

As we spoke about outcomes we felt were most important, we emphasized an overall interaction
between many criteria and predictors that should prepare people for the outcomes we will describe. The
specific criteria we seemed to emphasize most, however, were the last half of the ten goals of the
Psychology Department at JMU: Communication Skills, Sociocultural and International Awareness,
Personal Development, and Career Planning and Development. These were the most generalizable, so we
feel that every major should incorporate them into their curriculum if for nothing else but for versatility.

b. Which outcomes are important to predict for JMUs graduates? Which life outcomes does your
team think JMU students should have after they graduate? You will be using the performance in the
assessment center to predict certain life outcomes. Explain why you chose those outcomes.

For JMU graduates, we think it is most important that we develop ourselves to be the best humans we
can be. Ultimately, this will help us become the best in our perspective life paths. Important things that we
should strive for and look to predict are overall wellness, diversity of life experience, time management
and conscientiousness, and developing a growth mindset. Each of these will prepare us for graduate life
and ensure that we can be ready to take on a breadth of experiences that will be new to us.

Overall wellness and wellbeing are incredibly important when looking at the psyche. Should someone
fail to be content in any dimension of their life, they usually suffer the consequences of stress in the best-
case scenario and complete failure and a severe drop in self-esteem, efficacy, or self-worth at worst.
Having wellness in the dimensions of your life are therefore quintessential to achieving goals,
maintaining relationships, being successful, and staying motivated.
 Diversity of life experience is also important because it creates a person who is well rounded and can
adapt to many different situations. This is not just about work; life is unpredictable and often will throw
curve balls, so being experienced, or at least prepared to become experienced, will foster adaptability.

Time management and conscientiousness are essential to every aspect of life. Anything that is time
sensitive often needs careful consideration and planning. Being able to allot the time to work projects,
social events, relationships, and introspective work are vital to wellness. People who cannot allot time
properly often experience numerous downfalls. Part of being well rounded and happy is being able to
make the time to do the hard stuff and not leave things on the back burner.

Lastly, JMU graduates should develop a growth mindset, as this will help them in almost every aspect
of the above. Learning never ends and neither does change or growth. Understanding this and the need for
effort and work will prepare graduates to continually push themselves to be better and understand that the
best things come with effort.

2. What KSAOs will you measure?

Cultural Awareness

Cultural awareness is extremely important for all graduates to have acquired and improved upon
and is an ability that can constantly be worked on. This is the ability to see other people's point of view
and background which important in studying psychology because it studies the brain and environment.
When studying people, it is pertinent to recognize and value different or opposing beliefs than your own.
This ability can predict the success a person may have with working in a diverse environment and having
professional relationships. It is a criterion of having employees who can get along with others and can
work well with diverse ideas. This could also be a predictor of having a growth mindset by being aware of
different cultures as well as your own.

Leadership skills

A large part in different employment environments is leadership and being able to interact with
people and the ability to take the lead in a situation where unique strengths can be utilized. The skill to
lead without having a specifically identified leadership role is useful to companies because not everyone
can have a direct position. However, everyone can be a leader despite their position and work towards
better cooperation and production. This skill shows that a candidate is strategic and has a vision. It can
predict good critical thinking by working with difficult situations and shows initiative in improving the
current situation.

Strong Verbal Communication

Strong verbal communication is the ability to convey what you want and how you feel in a clear
and succinct manner. Relatively little confusion should be a result of someone utilizing this skill. Good
utilization of strong verbal communication would also have a heavy impact on ones overall wellness
being that emotional stability is a factor in wellness. Strong verbal communication would predict
leadership qualities and effective team players. It is also a criterion for several positions dealing with
giving out instructions and/or handling many emotions (i.e. teachers, counsellors, managers).
Critical Thinking

Critical thinking is essential in life. It is the analysis we use to form a judgment or decision. A
person demonstrating good critical thinking understands the interactions between different problems and
potential outcomes. This person would also demonstrate the ability to fill in gaps of missing information
with reasoning based on previous outcomes. Critical thinking should be a criterion for anyone looking to
be in any position but especially in those positions where leadership and decision-making takes priority. It
would be a predictor of application of knowledge and analytical skills.

Technological literacy

Technology is becoming evermore pervasive in the common workplace. Companies are trying to
enhance efficiency and cut cost through the implementation of more technology. Graduate schools are
pushing more technology into their curriculum to increase the competitiveness of their graduates. To stay
competitive in the work force students must be able to adapt and grow in a technological environment.

Work-Life Balance

College graduates who prioritize their well-being are more engaged at work and are better assets
to their respective companies.

Flexibility and Open Mindedness:

Flexibility and open mindedness are qualities that all graduating students should have.
This trait translates into many jobs and fields of study. Being flexible allows students to roll with the
punches of life while keeping your stress levels down. Having an open mind will allow students to view
the world in a grander scheme than their own and see cultures from different perspectives.

Information Gathering and Processing skills:

Much of college is spent gathering information and understanding what is appropriate for
whichever project the student is working on. Our graduates should be knowledgeable in the proper ways
to gather information and how best to utilize the sources available to them.

3. What data would you collect to measure student attainment of each of these KSAOs?

I. Cultural awareness and open-mindedness/ flexibility

i. Survey of behaviors (studying abroad, multiple language, sociocultural electives)

ii. Are you a member of some type of cultural/diverse group, job or organization?

II. Leadership skills and communication

i. Assess with escape room- Teamwork tests Barter test Survive test

III. Information gathering and technical literacy

i. Practical tests where real world documents/scenarios testing

IV. Critical thinking

i. Measured throughout the other tests and somehow complicated?

V. Work-life balance

i. Surveys

1. A self-report packet filled with surveys and questionnaires would be given to

each student to complete independently. One section would consist of questions
that assessed the individuals involvement in different activities, hobbies, clubs, etc.
For example, here a student would report any semesters where they studied
abroad, state if they can speak more than one language, and list any jobs or
extracurricular activities. It would also include some questions very similar to the
Cultural Competence Self-Assessment Questionnaire. Another section would ask
indirect questions regarding well-being. These questions would be related to the
findings of the Gallup Purdue index on the five different dimensions of well-being:
purpose, social, financial, community, and physical (Gallup 2014).
a. Cultural awareness
b. Flexibility and Open-mindedness
c. Work-life balance

2. A real life escape room simulator would also be used. A small group of 4-7 participants would
complete this portion together. Unlike a traditional escape room where death is the motivation to escape
quickly, participants are just informed to complete each task to the best of their ability, and exit the room
all together as quickly as possible. At the beginning, they are given a small task to complete before they
get a clue to find the next task to complete. Each task would be difficult and would require a team effort
to accomplish, such as a puzzle, problem, scenario, survival test, etc. During this process, multiple trained
professionals would be monitoring interactions between members, how quickly tasks were completed,
ideas generated between individuals, and so on.
a. Open-mindedness and flexibility
b. Critical thinking
c. Leadership
d. Strong communication skills
e. Informational gathering and processing skills.

3. A barter test would also been administered. This would be an individual test where each
participant would be given a small token of low value to start with, such as a small box, or a bouncy ball.
From there, the participants would be released and would have a set time (roughly an hour or two) to
return back with a different token of higher value. How they are supposed to get this higher valued
token would not be specified, which gives each participant the opportunity to come up with their own
bartering method among others. For example, one person may choose to just come out and try to trade
 with another person by saying the real purpose, where another person may try to create a story to aid in
persuasion.
a. Critical thinking
b. Information gathering and processing skills
c. Strong verbal communication
d. Flexibility and open-mindedness

4. A computer program portion would also be included. During this section of the assessment, a
couple different tasks would be included. The first part would consist of a document containing a scenario
that is filled with information. The student would have to pick out the important information and
complete different tasks using PowerPoint, Excel, etc to organize it. The second part would require each
student to sit through an online lesson that teaches them how to use a new computer program/software
system. After the lesson is over, the students would be given a task to complete using the system they just
learned about. Information on how many mistakes the students made, the time it took for each student to
complete the activities, the amount of effort put into each activity, and the accuracy of the final product
would be recorded.
a. Critical thinking
b. Information gathering and processing skills
c. Technical literacy

4. What criterion measures would you use to evaluate the effectiveness of the Assessment Center?

1. Explain the purpose of your assessment center (e.g., are you trying to decide who was successful at
passing the major?)

a. Our assessment center will determine of those graduating with a Bachelors degree in psychology,
who shows traits that will carry-over into the workplace. We will assess not only graduating students
ability to pass the required courses but also their critical thinking skills. Our center will show that JMU
students who graduate with a degree in psychology can apply what they have learned.

1. Describe the criteria measures you would use and why you chose them.

a. Our conceptual and actual criterion are as follows:

i. Intellectual growth grade point average

ii. Emotional growth advisor rating of emotional maturity

iii. Psychological growth application of knowledge on psychological tasks

b. We chose these criterions because they can be easily measured on a standard scale for all students.

1. Explain how you would analyze the validity of your assessment center (Daniel)
1. Concurrent validity or predictive validity? Why?
1. Our tests are more predictive in nature excluding the survey and self-report measures. They are predictive
because they are scenario/experiential tests where participants are making decisions based tasks that
weve given them.
2. Our surveys will be more concurrent in nature since they are self-report and based off of previously tested
measures of cultural awareness, flexibility and open-mindedness, and work-life balance.
2. Describe the validity assessment process.
1. For our surveys we will assess our tests validity using coefficients between one of our scores and a score
from a previously validated test.
2. Our behavioral tests will take previously validated tests that measure our criteria and design the tasks
around those tests so that higher scores indicate higher scores on their respective surveys.

5. What decision-making strategy would you use to determine which students are successful?

Related to chapter 5 (e.g., multiple cutoff, multiple hurdle)

We want our assessment to be fair for all students so we feel that taking the difference between the
average performance of the accepted group and the average performance of the total group will do so. The
first step would be determining whether the student has fulfilled all the required courses and credit hours.
From there we can give the student a final assessment to determine their knowledge of the field. Due to
the variability of the individual students, we feel having multiple cutoff scores would be better suited for
our assessing purposes. Grade point average would be banded by letter grade (i.e. A, B, or C) so that there
is no clear distinction between averages. This would allow students to apply their knowledge without
possibly missing out on a student due to their GPA level.

6. How should we motivate students to perform at their best on this assessment center?

1. Think about your experiences with test taking. What encourages you to take it seriously?
2. Use readings from motivation unit to support your recommendations.

Assessment is important for evaluating the adequacy of tasks and the performance on the assessment
gives insight into behavior and that is what the department needs to know in order to make changes. In
order to achieve motivated participants, having some autonomy involved can make the assessment more
valuable to the participant. One way of creating autonomy is to allow students to pick the time slot that
they can take the assessment. It would all be on the same day but they wouldnt have to do it at 8 AM for
example.

When taking senior assessments, the valence of the test is low because there are no incentives. There is
only the threat of having a hold on your account if the test is not taken. For the first assessment as an
underclassmen, there could be an incentive of getting into classes faster. If a higher score is determined
for the assessment, you could be allowed to take upper level classes usually barred by prerequisites.
Another incentive could be some recognition for a good score that students could put on a resume
potentially or at least be proud of.

Students are very attuned to getting a score back on everything that they do. That is why assessments
seem pointless and become something that they dont need to be motivated for. If there was some kind of
feedback, the first part of the assessment for underclassmen could understand the areas that they are weak
in and still have a couple years to make corrective adjustments (Muchinsky, 2016). For senior
assessments, it will show weaknesses, but it can also be a validation in how they have improved and their
general strengths to help with finding jobs.

Having some prior intervention to focus on developing intelligence would be beneficial for
underclassmen taking the first assessment. Students will better understand that the brain is highly plastic
and being smart means they take on challenges and follow through. Another aspect that could improve
motivation for assessments is to incorporate process praise throughout classes. Engagement, improvement
and perseverance should be noted more so than final products. The more students understand that their
brain can become stronger and that effort is prized, the more they will succeed.

7. How should we use the information from the assessment center?

Think about why we do Assessment Day in the department. How should we use the information?
With whom should we share the information? Why?

The whole reason assessment day is performed is to evaluate how things are working and if students are
learning what they are supposed to be learning. In a way, it is the validity of the department. The
information should be shared with the participants and with all of the faculty in the department for that
assessment. This is important because the faculty is who can make real changes in the classroom and they
need to see how their actions have affected learning.

It would be a useful tool to have the results available to the public in order to let new potential students
see how the department is working and how much the students are taking away from their classes. It
could be a draw for new students if we have good assessment results. This would also give an incentive
for the university to push for the best possible environment in every department, not just a few.
 References

Department of Psychology. (n.d.). Retrieved February 13, 2017, from

http://www.psyc.jmu.edu/undergraduate/curriculumobjectives.html

Landrum, R. E., Beins, B. C., Bhalla, M., Brakke, K., Briihl, D. S., Curl-Langager, R. M., . . . Kirk, J. J.
(n.d.). Desired outcomes of an undergraduate education in psychology from departmental, student, and
societal perspectives. Undergraduate education in psychology: A blueprint for the future of the
discipline., 145-160. doi:10.1037/12063-009

Lives, G. J. G. The 2014 Gallup-Purdue Index Report.(2014).

Muchinsky, Paul M., and Satoris S. Culbertson. "12." Psychology Applied to Work: An Introduction to
Industrial and Organizational Psychology. Summerfield: Hypergraphic, 2016. 367-83. Print.