Running head: ASSESSMENT AT ETC

Assessment Proposal for Ezra Thomas College

Kathryn Block, Ryan Hathaway, Melissa Moore

University of Wisconsin La Crosse

ASSESSMENT AT ETC 2

Summary of Scenario

Ezra Thomas college is a private liberal arts college with a total of 3,200 students

currently enrolled. Ezra Thomas was founded 200 years ago in the Northeastern U.S. and is in

the middle of three major metropolitan areas. The college is currently governed by a Board of

Trustees composed of 20 members that are a variety of alumni, leaders, or individuals of

importance. Ezra Thomas is a primarily white campus with 80 percent of the students being

white. The campus demographic is 44 percent male, 49 percent female, 4 percent trans, and 3

percent are non-specified. Ezra Thomas pushes to diversify itself but most of the student

population have connections to the institution by familial bonds. Within six-years of being on

campus, there is a 71 percent graduation rate and 85 percent of first-to-second year students are

retained within the institution. Within the campus we represent the Department of Student

Support and Advising which is housed in the Division of Academic Affairs.

The Department of Student Support and Advising contains the Advising services,

Academic support, tutoring, and disability support services. The department’s leadership is

composed of Jonas Rockefeller (Academic Advising Coordinator), Elliot Schmidt (Academic

Support and Tutoring Coordinator), and Nora James (Student Disability Services Coordinator).

As the Director of the Academic Advising Center for Ezra Thomas we are tasked with the

oversight of Assessment efforts for the Department of Student Support and Advising. As the

director we are charged specifically with developing a detailed plan to assess student learning

within the Academic Advising and Tutoring Services. As well as assessing student learning, we

are to find evidence that our office’s work helps students commit to a major within their first

three semester at Ezra Thomas College.

ASSESSMENT AT ETC 3

As a department of Student Support and Advising we have four sets of core values which

are: the promotion of academic success, the collaboration between all departments to promote

student learning & success, the support of students of all backgrounds & abilities, and the

empowerment of our students and community to adequately prepare students for life after

college. These core values are derived from the department’s Mission statement which is “The

Department of Student Support and Advising at Ezra Thomas College recognizes that academic

and student support are key components of student learning and development. Ezra Thomas

College guides students towards choosing a major program of study early in their college career

to prepare each student adequately for life as a responsible and informed citizen post-graduation.

Through collaboration between academic advising, tutoring, and disability services, the

Department of Student Support and Advising strives to create strong, appropriate educational

plans to support each student’s career and personal goals. Our office will promote innovation

through the implementation of new programs and goals in accordance with students’ changing

demographics.”

Literature Review

Regardless of Higher Education Institution type, advising practices and models are key

factors that play an integral role in the success of student retention and graduation. Assessment

varies depending on several influences and making sure to select a method that best suits the

office environment and the student population served is paramount to making sure the strategies

used are a good fit for project conduction. Understanding department needs and how to address

them through data collection and evidence is necessary to make certain services provided by

advisors meet departmental goals. In this assessment proposal for the Department of Student

Support and Advising at Ezra Thomas College, we intend to convey the importance of
ASSESSMENT AT ETC 4

assessment to address our department’s goals, as well as suggest methods for gathering

information and using results. Our basis for this comes from research in the academic advising

field.

Delving into information about best practices, strategies and advising assessments that

subscribe to meeting the needs of all students is essential in order to achieve our office mission

and goals. It is important to consider the reasons for which students do not access specific

resources such as advisors and tutors. Working alongside Student Affairs Professionals who

work directly with students proves to ensure the dissemination of resource and services

information to students. Studies show that the collaboration of campus constituents increase the

number of students who seek resources from principal offices which include advising and

retention departments. As Zarges, Adams, Higgins, Eand, and, Muhovich (2018) mention, “An

institutional culture of assessment requires the collaboration of those committed to student

success” (p. 50). Knowing the barriers students face internally as well as those from campus

partner offices, is crucial to creating and sustaining a holistic centered advising unit. To address

collaborative needs, the National Association of Student Personnel Administrators (2004) and

the American College Personnel stated that, “both academic and student affairs administrators

should commit to holding all campus educators accountable for the contributions their learning

experiences make to overall student learning outcomes” (p. 55). This statement further

emphasizes the need for department collaboration in order to close the resource accessibility gap

that students with disabilities face when navigating institutions of Higher Education.

Another aspect of carrying out effective advising practices and models is ensuring the use

of resources that help advance students to degree completion (Martinez & Villareal, 2018). In

Martinez and Villareal’s study, instruments are proposed to assess and better understand tutoring
ASSESSMENT AT ETC 5

center utilization, study skills and other related factors in college students. The indicators used

are focused on the level of self-efficacy of the students participating in the study and relies on

their individual assessment to determine what the student needs are. Their findings proved to

offer insight into the specific student needs regarding their study skills and access to resources

and how to address them through strategic program implementation. Carlstrom, Hughey, and

Powers mention that, “Advisors may believe that students appreciate the contribution of advising

but may not view the affective outcomes as significant or have the means to assess them” (p. 72).

This statement further validates the need to assess whether or not students are aware of and

properly utilize advisor services and other student support services as needed.

As stated in our department goals, we are committed to creating an environment that is

inclusive for all and doing so by specifically focusing on and increasing our Equity, Diversity,

and Inclusion efforts to ensure the utilization of our services. Focusing on students with

disabilities and implementation of skills to serve this student demographic, Mole (2014)

mentions the gradual pace of practice implementation regardless of positive and receptive

policies. Mole’s study aims to identify ways in which to create inclusive learning environments.

The findings show “that the responsibility for access is widening” (p. 77). This proves that the

basis of our need to assess how we as an office can offer equitable sources is crucial to students

meeting their learning outcomes and our office meeting our department goals.

Although there is little research that focuses on assessment in advising, literature

available mainly focuses on resource utilization and student support department collaborative

efforts. After reviewing the literature, the question of what the specific barriers students are

facing that keep them from seeing their advisors and using services such as tutoring centers

remains. Because institutions and offices vary and differ due to several factors, it is difficult to
ASSESSMENT AT ETC 6

pinpoint an exact answer to this question. By taking into account the strategies and best practices

from leading institutions and researchers, we hope our assessment findings will contribute to the

betterment of the student experience by addressing student needs and making our resources

accessible to all students.

Assessment Recommendations

After carefully weighing the values and mission of our department with the expectations

of stakeholders, we have decided to propose assessing two student learning outcomes over the

next two years, both using mixed methods approaches with recommendations for both

quantitative and qualitative assessment. Assessment will be conducted on a 2-year cycle.

During the first year, the department will assess the first student learning outcome: “As a result

of meeting with an academic advisor throughout their freshman year, students will have the

knowledge and tools to commit to a major by the end of their third semester at ETC.” Firstly, this

will be assessed quantitatively using EAB Navigate, which is a software addition to a Student

Information System website. Navigate will allow us to track students’ specific course grades,

cumulative GPAs, declared major, advisors, advisor’s meeting notes, and course history. We will

track a random sampling of students who have and have not used our office’s services, and those

who have and have not declared a major by the end of their third semester at ETC. From there,

we will analyze for trends and differences between groups using chi-square tests and T-tests. The

Navigate software will be an additional resource required for this assessment recommendation,

but it will drastically cut back on the time and personnel needed to track many students’ progress

towards major declaration.

The second recommendation for assessing the first student learning outcome uses

qualitative methods, more specifically semi-structured informational interviews conducted by

ASSESSMENT AT ETC 7

trained student workers. Questions will focus on what led to each student’s decision to declare or

not declare a major, if and how they used campus resources to make that decision, and how we

can improve our services to help them more in the future. The main questions we want answers

for are: “Have students who have used our office chosen majors significantly more than students

who have not used our office? What factors influenced those students’ decisions? Do students

feel our resources are effective?” This portion of the research should be done after the

quantitative stage, in order to script more thoughtful and applicable questions. Completing the

qualitative phase will also be dependent on time and personnel resources, but at the very least we

see the quantitative phase as being absolutely necessary. Once we know how our resources are

impacting students’ abilities to declare a major early in their college career, we can take student

suggestions for improvement to tailor our advisor meetings, policies, and outgoing information

to better help students achieve this goal.

In the second year of the assessment cycle, focus will shift to the second student learning

outcome: “As a result of students meeting with our academic tutors and advisors, students will

have a stronger grasp on test taking and note taking skills.” While the first learning outcome was

based on a request from a stakeholder, this second learning outcome comes from our office’s

internal mission of promoting academic success. Again, both quantitative and qualitative

methods are recommended. The quantitative method will be able to use the same EAB Navigate

software already described, this time to analyze the academic success (as displayed by specific

course grades and cumulative GPAs) of students who have met with our tutors and those who

have not. We will also take into account which classes and majors most frequently require

tutoring.
ASSESSMENT AT ETC 8

The qualitative stage of assessing the second learning outcome will happen

simultaneously to the quantitative data collection. Qualitative assessment in this case includes

intake and outtake questionnaires before and after each student’s tutoring session. Questions will

include “What subject are you here for tutoring? What grade to you believe you are currently

receiving in the course? What type of assistance are you looking for (test taking skills, specific

subject tutoring, note taking skills, other)? Do you feel the tutoring session was helpful? How

confident do you feel using the following skills: Test taking, note taking, study skills, etc.?”

Student workers and other office personnel can enter the student’s answers into a computer, code

the responses, and analyze for trends. We also plan to conduct an open survey of students who

have not met with a tutor, to ask them about what factors stopped them from meeting with a

tutor, how confident they are in academic skills, and what they would like to see differently in

the academic advising and tutoring office. The quantitative and qualitative data collected on

student’s academic skills and tutoring preferences can concretely inform the subjects, skills, and

office structure that our tutors work with in future years.

ASSESSMENT AT ETC 9

References

American College Personnel Association & National Association of Student Personnel

Administrators. (2006) Learning Reconsidered 2: Implementing a Campus-Wide Focus

on the Student Experience. NASPA-052.

Fovet, F., Jarrett, T., Mole, H., & Syncox, D. (2014). Like Fire to Water: Building Bridging

Collaborations between Disability Service Providers and Course Instructors to Create

User Friendly and Resource Efficient UDL Implementation Material. Collected Essays

on Learning and Teaching, 7(1).

Powers, K. L., & Carlstrom, A. H. (2014). Academic Advising Assessment Practices: Results of

a National Study. NACADA Journal, 34(1).

Villarreal, V., & Martinez, A. (2018). Assessing Study Skills in College Students: A

Review of Three Measures. Journal of College Student Development, 59(5), 629–635.

Zarges, K. M., Adams, T. A., Higgins, E. M., & Muhovich, N. (2018). Assessing the

Impact of Academic Advising: Current Issues and Future Trends. New Directions for

Higher Education, (184), 47–57.

ASSESSMENT AT ETC 10

Detailed Assessment Proposal

Foundational Information

Mission Statement

The Department of Student Support and Advising at Ezra Thomas College recognizes

that academic and student support are key components of student learning and development.

Ezra Thomas College guides students towards choosing a major program of study early in their

college career to prepare each student adequately for life as a responsible and informed citizen

post-graduation. Through collaboration between academic advising, tutoring, and disability

services, the Department of Student Support and Advising strives to create strong, appropriate

educational plans to support each student’s career and personal goals. The office will promote

innovation through the implementation of new programs and goals in accordance with students’

changing demographics.

Core Values

● The promotion of academic success

● The collaboration between all departments to promote student learning and success

● The support of students of all backgrounds and abilities

● The empowerment of our students and community to adequately prepare students for life

after college

Internal Assessment Goals

1. Collect evidence that the office helps students commit to a major within their first 3

semesters at ETC.
ASSESSMENT AT ETC 11

2. Collect evidence to show that our academic advising and tutoring programs result in

student learning and development.

3. Collect evidence to determine how influential the office is on the College’s retention

efforts.

4. Collect evidence on how students of different demographic backgrounds utilize (or do not

utilize) the office’s services.

5. Collect evidence to analyze how effective the office’s disability services are at meeting

the needs of students who use them.

○ If there is no such evidence for the above goals, implement program changes to

promote student learning, and reassess. As a result of our innovative steps, our

department will change for the betterment of our student's needs and wants.

Student Goals

1. Students will commit to a major within their first 3 semesters at ETC.

2. Students will utilize academic advising and tutoring programs which will increase their

learning and development.

3. The utilization of our office services and programs will lead to the retention of students

within a four-year plan.

4. Our office will increase EDI efforts to ensure the utilization of our services by

marginalized groups and identities.

5. Students in need of accommodations for academics or housing will be able to contact our

office’s disability services.

Student Learning Outcomes

ASSESSMENT AT ETC 12

1. As a result of meeting with an academic advisor throughout their freshman year, students

will have the knowledge and tools to commit to a major by the end of their third semester

at ETC.

2. As a result of students meeting with our academic tutors and advisors, students will have

a stronger grasp on test taking, note taking, and retention skills.

3. As a result of the clear and consistent support found in our office for students, ETC’s

first- to second-year retention rate and 4-year graduation rate will increase.

4. As a result of students creating action plans within the Disability Services department,

students will have accommodations met and have stronger retention through graduation.
ASSESSMENT AT ETC 13

Curriculum Map
ASSESSMENT AT ETC 14

Informational Needs and Priorities

In order to maximize our resources, we decided to look into goals 1 and 2 over the next

two years, but hope to revisit goals 3, 4, and 5 in the coming years.

1st learning outcome - As a result of meeting with an academic advisor throughout their

freshman year, students will have the knowledge and tools to commit to a major by the end of

their third semester at ETC.

Information we need:

QUANTITATIVE: (Pull data from a random sampling of students after permission was
granted, track statistics on frequency/topic of discussion each time students use office)

Students who have used office (Random Sampled student)

● Have they committed to a major?
● When did they commit to a major?
● How long have they been on our campus?
❏ How many credits do they have?
● How many times they have met with an advisor (or utilized office)?
● How many times have they talked with their advisor about selecting a major (as opposed
to meeting for other services)?

Students who have not used office (Random Sampled student)

● Have they committed to a major?
● When did they commit to a major?
● How long have they been on our campus?
❏ How many credits do they have?

QUALITATIVE: (informational interview)

Declared major (Random Sampled student)

Students who have utilized office

● What led to their decision?
● Did our resources help guide them in that decision?
● Did they utilize other resources from our campus?
Students who have not utilized office
● What led to their decision?
● Did they utilize other resources from our campus?
● What stopped them from utilizing our resources/factors?
ASSESSMENT AT ETC 15

Non-declared (Random Sampled student)

Students who have utilized office

● What is holding them back from committing to a major?
● How do our resources help guide them in that decision better?
Students who have not utilized office
● What is holding them back from committing to a major?
● Have they used other resources on campus?
● What factors stopped them from utilizing our resources?

Priorities for this learning outcome information:

We need the quantitative data first for a basis to build the rest of our research on. We would like
to have qualitative data dependent on resources of time and personnel.

Overall Assessment Question for this learning outcome:

Have students who have used our office chosen majors significantly more than students who
have not used our office? What factors influenced those students’ decisions? Do students feel our
resources are effective?

2nd learning outcome - As a result of students meeting with our academic tutors and advisors,
students will have a stronger grasp on test taking and note taking skills.

Information we need:

QUANTITATIVE: (Pull data from a random sampling of students after permission is granted):

Students who have met with tutors and/or advisors (Random Sampled student)
● Who has met with tutors and advisors?
● How often?
● When do they come in throughout the semester (early, mid-terms, finals week)?
● What courses are students coming in for help with?
● What was each student’s previous semester GPA or Cumulative GPA?
❏ Did they also meet with tutor/advisor that semester?
● What was each student’s GPA after a semester meeting with tutor/advisor?
● What was each Student's GPA for the specific courses they received tutoring in?

Students who have not met with tutors and/or advisors (Modified cohort Sample student:
students from those courses)
● How do the grades of students not receiving tutoring compared to those receiving
tutoring in specific courses?
● How many credit hours do students who do not meet with tutors and/or advisors take in a
semester?
● How many hours do students who do not meet with tutors and/or advisors put in for work
and extracurriculars?
ASSESSMENT AT ETC 16

● What majors are students who do not meet with tutors and/or advisors committed to (if
committed at all)?
● What is the year to year (ex: first year to fifth year) comparison of students who have not
met with tutors and/or advisors?

QUALITATIVE: (intake questionnaire, outtake questionnaire, survey)

Students who have met with tutors and/or advisors (Random Sampled student)
● In what subject are they specifically looking for assistance? (iq)
● What grade does the student believe they are receiving in the course? (iq)
● What type of assistance are they looking for: test taking skills, tutoring for a specific
subject, note taking skills, other? (iq)
● Was the information the student received what they needed, and how helpful was it? (oq)
● How confident do they feel in using the following skills: Test taking, note taking, study
skills, etc.? (oq)

Students who have not met with tutors and/or advisors (Random Sampled student)
● Do they feel using tutoring services would be beneficial for them?
❏ Why or why not?
● What factors have been stopping the student from coming in to see tutors and/or
advisors?
● How confident do they feel in using the following skills: Test taking, note taking, study
skills, etc.?
● What would they like to see done differently in these services?

Priorities for this learning outcome information:

First and foremost, we want to know which specific courses students are most frequently coming
in for help, and analyze the GPAs of students in that course, both who have received tutoring and
those who have not. We also want to be sure to do intake and outtake questionnaires with each
student who comes in for tutoring.

Overall Assessment Questions for this learning outcome:

● In specific courses, do our students who utilized the office achieve better grades than the
median of a specific course?
● Do our students see an overall GPA increase from previous semesters after receiving
tutoring?
● Do students feel more confident in their academic skills after tutoring?

Priorities for our assessment:

We prioritize learning outcome one since the Stakeholders specifically are searching for that
information. Once we can state that learning outcome one is satisfied with the prioritization of
learning outcome two will start. With both learning outcomes, the quantitative data will give us
the basic answers, however, the qualitative information may give us formative information that
may lead to stronger student learning and development.
ASSESSMENT AT ETC 17

Methods

We propose to focus on the first learning outcome (major declaration) during the first

year and the second learning outcome (academic skills) during the second year, with an ongoing

alternating cycle. Both learning outcomes will be assessed by mixed methods with simultaneous

quantitative and qualitative approaches. For the quantitative data collection, we propose to use a

software system called “EAB Navigate,” which pulls all needed data from the student

information system and filters student data without the need for surveys or student participation.

Qualitative methods will vary for each student learning outcome.

Quantitative Data: EAB Navigate

Utilizing EAB’s Navigate software, all information needs are met for quantitative data

for both learning outcomes. A significant portion of our information is readily available through

our Student Information Services; however, Navigate provides an easy, user-friendly interface

that can then relieve time restrictions and human resources required for the quantitative

approach. With Navigate it will show the students information in an easy to read format that

includes students’ cumulative GPA, specific course grade, declared major, advisors, meeting

notes from advisors, amount of credits earned, and a history of courses. We do have a distinction

between those who have utilized our offices and those who have not which also is satisfied with

Navigate. Navigate provides a centralized location for any notes and meetings from any

department around campus; thus, we should be able to easily identify which student have and

have not used our office. We plan to utilize randomly sampled students to decrease bias. With

this, once our students have been selected, we will determine whether they fit in the category of

utilized our department or not. With that, we will pull the other information into a document and

employ trend analysis using chi-square tests, and T-tests to determine relevance and correlation.
ASSESSMENT AT ETC 18

Qualitative Data: First Student Learning Outcome

To collect additional information about how students are committing to majors, we also

propose to conduct informational interviews with a small random sampling of students, both

those who have declared their major by their third semester and those who have not declared a

major. The interviews will occur one-on-one between undergraduate students and work-study

students or graduate interns. We decided not to have academic advisors themselves conduct the

interviews in order to lessen the power difference between interviewer and interviewee, and

because advisors may be more familiar with students and therefore be more biased. Because

trained Student Affairs Professionals will not conduct interviews, the interviews will be semi-

structured, focusing on questions regarding what factors influenced students’ major

declaration/non-declaration and their use of our services in coming to that decision. Transcripts

of interviews will then be analyzed for patterns to determine if students felt that working with

our advisors truly did help them declare a major by their third semester at ETC, and what

changes were suggested to further benefit students.

Qualitative Data: Second Student Learning Outcome

Although varied, open-ended, and flexible, using qualitative data aims to gain a deeper

understanding from a human perspective. We propose to use qualitative intake and outtake

questionnaires to assess students’ knowledge of test taking and note taking skills. Using an

ethnographic approach, we will gather groups of students who both have met with their advisor

and those who have not met with their advisor. First, we will have students complete the

questionnaire that includes factors individual to their experience, then come together in the larger

group to have an open discussion about their experiences and tallying the listed factors. During

these focus group sessions, we will ask the students to discuss factors that have contributed to
ASSESSMENT AT ETC 19

their choice of either meeting or not meeting with their advisor. This method will help us identify

the reasons and narratives behind student advisor resource utilization.