Issue 5 June 2014

Performance

Assessment Newsletter

DIVISION OF ACADEMIC AFFAIRS

ASSESSMENT OFFICE

ASSESSMENT
INFORMATION

FACULTY MEMBERS,
WHAT DO YOU NEED TO KNOW ABOUT ASSESSMENT?

BCCC Culture of Assessment Vision

Through a college-wide culture of assessment, BCCC provides a quality
education for all who enter our doors. Assessment is by nature a goal-driven,
evidence-based, and improvement-oriented process that involves all
stakeholders working collaboratively. This ongoing process promotes
excellence in teaching and learning by assessing all elements of the educational
process. Our culture of assessment provides institutional resources, training,
and support. Continual assessment is an integral component of BCCCs
commitment to excellence as an institution of higher education.

Assessment Newsletter

Capstone Projects: A Model

In the May 2014 Assessment Newsletter, the

advantages and procedures of the capstone course
were discussed (link). Many researchers and
educators believe that the capstone course is an
excellent opportunity for a comprehensive direct
assessment of student learning. The capstone course
should be a multi-faceted course across different
academic areas and interdisciplinary connections as
an instrument of evaluation.
Since capstones are the most common assessment, in
many disciplines, such as sociology, political science,
engineering, and business, capstones are used as part
of an intuitions assessment program. In this article, a
case study model from a Sociology Program at
Skidmore College further illustrates how to assess a
capstone product and how to make program changes
based on results.
Case Study from a Sociology Program at Skidmore
College (Berheide, 2007)
Two sociologists that taught the required theory
course in 2003 examined one strong, one average,
and one weak paper. This first stab at assessment led
to three main conclusions:
1. All three papers, including the weakest one,
demonstrated "basic facility with many of the
crucial concepts in social theory."
2. The theory content goal needed revised.
3. The department needed to teach the connection
between theory and methods, not only in the
theory and senior seminar courses, but also in the
introductory courses and at least some elective
courses (Brueggemann, 2003).
The following year, the sociologists who taught
statistics and research methods evaluated how three

Issue 5 June 2014

more papers achieved the methodological goal and

concluded "students generally succeed in achieving
our methodological goals" (Fox and Karp, 2004, p. 7).
They made several recommendations "to strengthen
further an already effective program," including
suggesting that the program revise its goals.
In the third year, the sociologists decided to look at
how well students articulated what the discipline
contributed to understanding social life, concluding
that "senior sociology majors, at all levels of ability,
are applying sociological perspectives to issues of
concern to them" (Berheide and Walzer 2005, p. 4).
The 2005 assessment also identified two general areas
for improvement:
1. Encourage students to be even more explicit in
linking their specific concerns with implications
for sociological theory and knowledge.
2. Help students to improve their ability to move
from simply cataloguing findings to writing about
them in prose that reflects more synthesis.
(Berheide and Walzer, 2005)
The authors also found that overall, with relatively
little effort, the department had learned a remarkable
amount about what students knew and could do after
majoring in sociology. First, the faculty learned that at
least on these three goals they were doing a good job.
Second, they learned that the theory and methods
goals needed some revision. Third, they had learned
that they needed to create greater "sequencing"
within the major, especially around theory and
methods. Even their minimal approach to assessment
provided vastly better data than they typically drew
upon for making curricular decisions. In short, faculty,
according to the case study results, did not have to
spend a lot of time and effort to get very useful data.
Excerpts from C. W. Berheide (2007). Doing Less Work, Collecting
Better Data: Using Capstone Courses to Assess Learning. Peer
Review, 2007(9), 2, 27-30. Retrieved from
http://www.aacu.org/peerreview/pr-sp07/pr-sp07_research.cfm

Assessment Newsletter

BCCC FACULTY FOCUS FEATURE

Learning Outcomes Assessment and a Multiinstructor Course Offering
By Professor Linda Benjamin, MBA
Associate Professor: Business and
Technology Department

As far back as 2008, courses offered in the Business and

Technology Department began Student Learning
Outcomes Assessment. Reflective of the times, many
instructors selected their own outcomes, their own ways
of assessing, their own ways of reporting and their own
ways of making academic changes based on this data.
As the College evolved on these processes, so did our
department. My particular experience is with BUAD 112,
Introduction to Computers for Business and
Management. A typical semester will have from twelve
to fifteen course offerings of BUAD 112; some online,
some 12-week, and most 15-week in a face-2-face
classroom. For years, each instructor performed his or
her own assessments, reporting them back to the
department chair that uses this data for our own
program accreditation, ACBSP.
Beginning in 2010, I began working to bring this
disparate student assessment together so that coursewide decisions could be made. As our textbook series
offered two Capstone-type assignments, it was agreed
that all instructors would use the same capstone
assignment for assessing the common learning goals of
the course. It was a start. For several semesters the
gathering of data never quite worked out completely;
the wrong assignment was used, grading at the detailed
level was not completed, and some instructors did not
return their data (an issue that has since been rectified).

Issue 5 June 2014

In the fall of 2013, a new textbook bundle was

deployed with an accompanying online component.
To facilitate the change in textbooks and to make
sure that this move would support student learning
outcomes assessment, I created the instructors
course syllabus for the course, selected and set up
the matching course shell in the online software
(MyITLab) and prepared the common final exam for
all courses. Adjuncts were also invited to summer
training sessions to become familiar with the books,
software, and common selected assessments.
When fall 2013 assessments were completed, we
learned a great deal about using a software
component in the course; such as the variety of
student personal computer set ups, the variety of
responses when problems are encountered by both
students and instructors and how clicking on the
wrong thing at any given time generated a variety of
unanticipated results.
We also learned much about selecting and
implementing common assignments; such as how
instructor presentation. Although assignments may
be the same, instructor presentation may still vary
widely and impact student interpretation. We
further learned where student weaknesses fall
within the course and to what degree. In response
to the findings, course assignments were adjusted
for spring 2014 to reduce the total number of
assignments, but increase focus on key areas such as
MLA formatting.
This takes us to the end of the spring 2014 semester.
All course instructors will be meeting after grades
are submitted to complete an assessment on the
entire academic years work; books, software, and
student outcomes. As we have worked toward the
goal
of
common
assignments,
common
presentation, and common grading the question
remains; are our students learning what they need
to learn, and how can we improve their results.

Issue 5 June 2014

Assessment Newsletter

CAT OF THE MONTH

Misconception/Preconception Check
Estimated Levels of Time and Energy Required for:
Faculty to prepare to use this CAT

MEDIUM

Students to respond to the assessment

LOW

Faculty to analyze the data collected

MEDIUM

DESCRIPTION
Focused Listing and Background Knowledge Probes are simple techniques for gathering information on what
students already know in order to determine effective starting points for instruction. The
Misconception/Preconception Check also assesses students' prior knowledge, but with a twist. Its focus is on
uncovering prior knowledge or beliefs that may hinder or block further learning.
PURPOSE
The greatest obstacle to new learning often is not the student's lack of prior knowledge but, rather, the existence
of prior knowledge. Most college teachers know from experience that it's much harder for students to unlearn
incorrect or incomplete knowledge than to master new knowledge in an unfamiliar field. Consequently, teachers
can benefit from discovering early in the term which common misconceptions and preconceptions students have
that are likely to interfere with their learning in a given course. This CAT is designed to uncover specific instances
of incorrect or incomplete knowledge, attitudes, or values that represent likely barriers to new learning. Because
assessment activities such as this CAT identify misconceptions and preconceptions early on and help students
explicitly recognize and understand them, students stand a much greater chance of learning new material
correctly and integrating it into their "revised" and often "transformed" knowledge structures.
SUGGESTIONS FOR USE
Although there are common misperceptions or preconceptions about every field, they seem to be most
pernicious and common in those areas of the curriculum that have the greatest overlap with life outside the
university classroom. For instance, virtually all incoming first-year college students have knowledge, beliefs, and
attitudes about the phenomena they will study in political science, economics, anthropology, sociology, history,
and psychology courses. While these same students are likely to have fewer strongly held preconceptions and
misconceptions about mathematics, the natural sciences, and related fields, some of the wrongheaded, implicit
"commonsense" notions that they cling to can still effectively derail learning.

Assessment Newsletter

Issue 5 June 2014

Thus, this CAT can be particularly useful in social and behavioral science courses, especially those dealing with
controversial or sensitive issues. In the natural sciences and mathematics, the Misconception/Preconception
Check can help faculty uncover naive and sometimes magical beliefs that can act like filters: allowing
disconnected facts and principles through but blocking out a deeper understanding of method or worldview.
Although two of the three examples below concern beginning-of-term applications, this technique can be used at
any point in a course when students encounter new information or interpretations that they may find
counterintuitive or disturbing.
STEP-BY-STEP PROCEDURE
1. Start by identifying some of the most troublesome common misconceptions or preconceptions students bring
to your course. Brainstorming this question with colleagues in your department or field can be a very effective
way to generate such a list.
2. Select a handful of these troublesome ideas and beliefs-ones that are likely to interfere most with learning in
your course-and focus your Misconception/Preconception Check on them.
3. Create a simple questionnaire to elicit information about students' ideas and beliefs in these areas. You may
want to use a multiple-choice format or a short-answer format. Short-answer questions can uncover more useful
information, but they compromise anonymity. Multiple choice questionnaires are therefore safer, and the
responses are easier to analyze. If you need to know how strongly held the beliefs or ideas are, consider providing
Likert-scale responses such as those used by the biology instructor in the example above.
4. Have another faculty member read your questions to make sure they do not seem patronizing, threatening, or
obvious.
5. Before giving the questionnaire to your students, think through how you will respond to several likely
outcomes. Strike any questions or topics you do not feel prepared to deal with.
6. Explain your reasons for using this CAT to the students, make sure the anonymity of their responses is ensured,
and announce when and how you plan to respond to their feedback.
TURNING THE DATA YOU COLLECT INTO USEFUL INFORMATION
Depending on the way the questionnaire is designed, analyzing the feedback from this CAT can answer one or
more of the following three questions" What specific misperceptions or preconceptions do my students have
about the course material that might interfere with their learning? How many of the students have them? How
deeply embedded are these "problematic" ideas or beliefs?
To answer the first two questions, you can quickly organize the responses into rough categories by type of
misconception or preconception, and then tally them, as in the example above from the physics course. You can
best answer the third question if you have constructed the Misconception/ Preconception Check to provide
information on the students' degree of certainty or strength of beliefs, as in the example from the Human
Sexuality course. As you read through and tally responses to this type of question, look for patterns within and
across items. For example, watch for questions or topics on which the students' responses are clearly divided.
Looking across items, group the topics into four categories" those that elicit strong reactions at both the "correct"
5

Assessment Newsletter

Issue 5 June 2014

and the "incorrect" ends of the scale, those that elicit mostly incorrect or mostly correct responses, and those
that most students are unsure of or that elicit thoroughly mixed results.
IDEAS FOR ADAPTING AND EXTENDING THIS CAT
To encourage candid responses to sensitive topics, begin by asking students to identify common misconceptions
and preconceptions that they think other people have about the topic or field.
Have students work in teams to come up with "reasonable" explanations or justifications for the misconceptions
uncovered through this assessment.
PROS
Readminister the same questionnaire later in the term-after your instructional response-to see what, if anything,
has changed and how.
The Misconception/Preconception Check gives teachers a quick way to uncover likely barriers to learning and
thus to prepare to meet and overcome them.
Since this CAT is carried out anonymously, individual students are likely to reveal their own ideas and beliefs,
rather than providing "safe" answers.
Many students are both surprised and relieved to learn that they are not alone in being mistaken or unclear
about a given topic. The feedback session can provide that reassurance.
When students do explicitly recognize and question their own knowledge, beliefs, and attitudes, they gain a
measure of control over their own thinking. This technique can help students take one small step in the direction
of self-knowledge and self-awareness.
CONS
The first and most obvious disadvantage of using this technique is that virtually no one enjoys having his or her
certainties questioned. As noted above, unlearning - though often necessary - can be very difficult and even
painful.
A related problem is time. Changes in fundamental ideas, beliefs, and attitudes take time and rarely occur in a
linear fashion. For many students, the developmental changes involved in moving from a "magical," prescientific
view of the universe to an empirical approach will take much more than a semester to solidify.
CAVEATS
One person's unquestioned fact is often another's questionable theory-or pernicious falsehood. Tread lightly
when dealing with potentially sensitive issues if you want students to open up enough to risk having their
assumptions challenged. In general, do not use this technique to focus on issues that students may find personally
threatening until a climate of trust and civility has been established in the class.

Issue 5 June 2014

Assessment Newsletter

Thank Our Discipline Liaisons

Xitracs Training Updates

Last month we reported that training had occurred
with a number of Program Coordinators and
Discipline Liaison in preparation of developing the
Colleges Annual Student Learning Outcomes
Assessment Report. Throughout the month of May,
23 additional faculty members received professional
development hours by attending the Electronic Form
training conducted by the Assessment Office.
To date, ten programs coordinators have completed
his or her respective annual program assessment
report on Xitracs. An annual assessment report for
six programs has yet to be started. The remaining
programs have entered preliminary data as of May
27, 2014.
Our goal is to provide the most inclusive annual
assessment and tell BCCCs real story. The story can
only be told through our program coordinators and
faculty.

Implementing and standardizing the assessment

model requires support at all levels. The BCCC
assessment model provides support through the
Discipline Liaisons (DLs). They spend a lot of time
and effort assisting faculty colleagues with various
skill sets and support. Please join us in extending a
Big Thank You to the DLs.

BCCC SLO Assessment Training

This academic year, all eight DLs renewed their

contract. They are: Dr. Shawn Lane, Professor
Michael Kaye, Professor Terry Doty, Dr. Debra
Parson, Professor Beatriz del Castillo, Dr. Amrita
Madabushi, Dr. Rotimi Olojo, and Professor Paul
Long.

BCCCs Commitment toward Evidence

Assessment

Accreditation

Technical support is only a phone call away.

Accountability
Electronic Form Training for
BCCC Faculty and Staff

Issue 5 June 2014

Assessment Newsletter

FINAL CALL FOR SUBMISSION

2013-2014 Annual Student Learning Outcomes Report
Spring 2014 Assessment Documents Loaded into Xitracs
Program Coordinators - Click on Below Link to Complete the Annual Program Report
Faculty Verify through your Program Coordinator or Course Facilitator if assessment
findings and documents are due

https://bccc.xitracs.net/portal.htm
Please see the Program or Course Matrix for the due Spring
2014 Assessment Documents

Please see the Program or Course Matrix

Upcoming Assessment
for theConferences
due Spring 2014

Assessment Documents

4th Annual Association for the Assessment of Learning in Higher Education (AALHE) Conference
o Albuquerque
NM, June
2 4,Program
2014
Consult
with your
DL and
Coordinator to Complete this Task

Connecting Student Learning Outcomes to Teaching Assessment, Curriculum,

o Milwaukee WI, June 9 - 13, 2014

The Annual Institute on Best Practices in Institutional Effectiveness

o San Antonio TX, July 13 16, 2014

Assessment Conference 2014Myths & Movements: Re-imaging Higher Education Assessment

o Drexel University, September 10 12

Southern Association for Institutional Research (SAIR) 2014

o Destine FL, October 4 7, 2014
8

Issue 5 June 2014

Assessment Newsletter

ASSESSMENT OFFICE STAFF

Dr. Hsin-Yuan Chen,
Director of Curriculum and Assessment
E-mail: hchen@bccc.edu
Phone: 410-462-7442
Penny L. Weidner,
Coordinator of Assessment
E-mail: pweidner@bccc.edu
Phone: 410-462-7402

Assessment Office
Main Building Room 251
http://www.bccc.edu/accreditation

Assessment@bccc.edu