Weber State University   

First Year Experience

Results of Assessment 

2004-2005 (submitted 11/17/05)

2004-2005 ASSESSMENT INITIATIVES

FYE achieved the following continuing improvement initiatives planned for this assessment year:

• The FYE Student Specialist conducted research on how other FYE programs evaluate their services and courses.  Most FYE programs use tools that are similar to those we use or used (e.g., course evaluations, national surveys, retention tracking, and focus groups).

• The FYE Student Specialist analyzed FYE enrollment trends to better estimate the number of sections FYE needs to offer to meet minimum enrollments of 15 students per section. Enrollment patterns in FYE stand-alone sections versus FYE cluster sections were studied to best select course pairings, days, and times of most interest to first-year students. The most popular course start times are between 8:30 and 11:00 a.m. Lower enrollment sections tend to be offered at night and at WSU Davis. Clusters including math or English yield the highest cluster enrollments, and clusters offered in Fall tend to have higher enrollments than those offered in Spring.

• The FYE Student Specialist explored the differences between Fall and Spring FYE students to better serve these two groups.  More Fall students were first- time freshmen (69%, n=258) than Spring students (31%, n=33). Last year’s data reflect a similar breakdown. This difference may explain some dissimilarities between some Fall and Spring evaluation responses.  FYE strives toward continued cognizance of its multiple populations’ needs in terms of course content and pedagogy.

• The FYE Coordinator helped develop the Academic Advisement Center’s Academic Self-Assessment Questionnaire that is currently being piloted on General Studies students on warning, probation, or suspension. FYE will administer this survey to Fall 2005 “Introduction to the University” (Education 1105) students to help FYE better gauge students’ perceived needs. Findings will be reported in next year’s assessment report.

• The FYE Coordinator continued implementing revisions to the instructor hiring and rehiring process to support the academic success and retention of first-year students.  This summer FYE is experimenting with a possible alternative to the full-semester shadowing experience required of newly hired “Introduction to the University” adjuncts.  The alternative is an intense, customized training program for new FYE adjuncts with college teaching experience and extensive knowledge of both Weber State University’s student services and educating first-year students.

• The FYE Coordinator made adjustments to the training of peer mentors.  For example, a shift in approach to the “Mentor Leadership Seminar” (Education 3170) was implemented by the Coordinator/instructor who assigned mentors the more active role of gaining knowledge on campus services through visitations to campus departments rather than the more passive role of obtaining knowledge presented by student service representatives visiting the “Mentor Leadership Seminar” classroom.

• To increase students’ sense of belonging to the WSU community, the FYE Coordinator communicated the need for adjuncts to incorporate campus engagement activities into course requirements due the beginning of the semester (especially in the Fall semester in which a larger percentage of first-time freshmen are enrolled).

EVALUATION OF COURSES AND SERVICES

First Year Experience evaluates its courses and services throughout the year.  Most data in this report pertain only to Fall 2004 because Spring 2005 course evaluation data were not yet available from Institutional Research and the Summer 2005 course was still in progress.

FYE standards of excellence were selected in advance of the data analysis. Most answer choices were formatted in a 5-point Likert scale ranging from least favorable rating (1) to most favorable rating (5), and achievement of FYE’s standard of excellence was defined as a mean rating of at least a 4.0. For answer choices of strongly disagree, disagree, agree, and strongly agree, achievement of FYE’s standard of excellence was defined as earned ratings of strongly agree or agree from at least 80% of the respondents. Analysis of responses asked to solicit additional feedback was focused on recurring themes. As decided before the start of the assessment year, the standard of excellence in the area of attendance at adjunct trainings and mentor trainings was five people in the target audience. For trainings to which fewer than five members of that group were represented, evaluations were analyzed but not reported.

ADJUNCT TRAINING

To promote the professional development of FYE adjuncts, FYE sponsored semesterly kick-offs and near monthly adjunct meetings.  To best plan future trainings, these learning opportunities were evaluated based on quality and content of the presentations.

Semesterly Kick-offs

The 2004-2005 adjunct training schedule [see Table 1] consisted of three semesterly kick-offs in which FYE adjuncts were strongly encouraged to participate to best prepare them for their upcoming semester of teaching. All Kick-offs are evaluated so FYE can use this feedback to plan effective future learning opportunities. Overall, the feedback on adjunct trainings surpassed the department standard of excellence with the exception of poor attendance to most.  The most frequent feedback from adjuncts indicated that the most effective training segments were those involving classroom management, discussing ideas, and meetings between paired adjuncts and mentors.  Adjuncts reported that what would have made the session more beneficial would have been detailed mentor responsibilities and group adjunct discussions; and that the topics that should be covered in future sessions are classroom management, instructors’ ideas, and mentors. In response to this feedback, FYE offered training on classroom management and on mentors.

Although pleased with the adjunct feedback on training, FYE was not pleased with the poor attendance at adjunct trainings, an issue that will be a focus of FYE’s late summer 2005 planning efforts. So we would be somewhat comfortable generalizing findings, reported analysis was limited to feedback on trainings five or more adjuncts both attended and evaluated.  From the Fall Kick-off, the only semesterly training that met the department’s minimum adjunct attendance standard, the evaluations from adjuncts (n=10) were favorable in terms of presentation content (mean 4.10) and presentation quality (mean 4.10).  To improve attendance at trainings, the Coordinator will try to build the training schedule a semester in advance based on GroupWise searches of adjunct availability.  RSVPs will be required so sessions with limited interest can be rescheduled or cancelled. Training topic selection will continue to be based on solicited and unsolicited input from FYE’s constituents.

Adjunct Meetings

In addition to semesterly adjunct trainings, FYE conducts periodic adjunct meetings [See Table 2]. Although the attendance record for these meetings was poor, overall feedback on adjunct meetings surpassed the department standard of excellence. For the November 30^th meeting, the only one that met the department’s minimum adjunct attendance standard, evaluations from adjuncts (n=7) were favorable in terms of presentation content (mean 4.57) and presentation quality (mean 4.43). To improve attendance at trainings, an effort to build the training schedule a semester in advance based on GroupWise searches of adjunct availability will be made and RSVPs will be required so sessions with limited interest are rescheduled or cancelled. Training topic selection will continue to be based on constituent feedback.

Student Evaluations of “Introduction to the University”

As of the date this report was submitted, we had not yet received Spring evaluation results from Institutional Research and the Summer course was in progress.  For Fall, student evaluations of Education 1105 instruction indicate that FYE exceeded its standard of excellence in most areas:

Instructor…

• clearly explained the course goals and requirements in a syllabus (mean 4.4, n= 189)

• utilized class time effectively (mean 4.2, n=189)

• used class activities to help fulfill the course objectives outlined in the syllabus (mean 4.3, n= 187)

• invited guest speakers who helped me understand the particular topic (mean 4.2, n=189)

• supplied helpful feedback on work (mean 4.1, n=188)

• was enthusiastic about the course (mean 4.4, n=189)

• demonstrated professionalism (mean 4.3, n=189)

• encouraged participation in class (mean 4.4, n=189)

• was accessible outside of class time (mean 4.3, n=188)

• increased ability to use a variety of campus resources such as the library, advisement center, testing center, and tutoring services (mean 4.1, n=189)

• assigned pertinent coursework (mean 4.0, n=189)

When asked which topics in the course students thought would most help them be more successful students, the most frequent responses were time management (32), campus resources (29), note taking (22), career exploration (17), test taking (16), study skills (16), and academic advising (10).  When asked what one change they recommend for future EDUC 1105 classes, the most frequent responses were less busywork (22), better organization of the service learning project (12), and more group activities (9). Recommended topics will be considered for future adjunct trainings and course content revisions.

* “n”s ranged from 220 to 223

**“n”s ranged from 187 to 189

*** indicates statement was not included in that semester’s evaluation instrument

Student evaluations of Education 1105 instruction indicate that FYE did not meet its standard of excellence in the following areas:

Instructor…

• made students feel course was worthwhile overall (mean 3.6, n=189)

• increased ability to identify a major and a career of interest (mean 3.8, n=188)

• increased ability to employ effective academic skills such as note taking, textbook reading, test taking, and time management (mean 3.8, n=188)

FYE will facilitate continuous improvements in all of these areas through a variety of means including adjunct trainings of the upcoming year.

Advising Module

For First Year Experience’s “Introduction to the University” course, Academic Advisement Center advisors conduct two-day modules that are a customized, extended version of AAC’s General Education Sessions. Data on Fall and Spring were kept separate, due to significant changes in how the module was delivered and due to the significantly larger percentage of Fall students who are first-time freshmen.

For Fall, 288 students were enrolled in FYE.  Of them, 57% (166) completed an Advising Module evaluation. This low response rate was not due to advisor error. It was due to an absence from class on one or both of the Advising Module days. Analyzed data only includes feedback from students who attended both sessions and completed both the pre-module instrument and post-module instrument. The FYE Coordinator will work with FYE adjuncts to facilitate increased attendance, perhaps requiring attendance to receive credit for completion of the module. 

The FYE Advising Module met or surpassed the department standard of excellence in 10 of 11 evaluated areas. The only area in which the module did not meet the standard was how to read and understand a graduation evaluation (3.95), a rating that almost met the standard. Advisors conducting these sessions are aware that this area needs improvement. The highest pre-session/post-session increases were in how to navigate the AAC website (1.65 rating increase), classes allowed to take as a result of College Tier/University Tier status (1.52 rating increase), and how to read and understand a graduation evaluation (1.41 rating increase).  Comparing this Fall to last Fall, we improved in 6 of 11 evaluated areas, our post-session means improved in 10 of 11 areas, and the pre-to-post rating differences improved in 6 of 11 areas. 

Note: Only students who completed both the pre-session and post-session portions of the instrument were included.

For Spring, 100 students were registered in FYE. Of them, 54% (54) completed an Advising Module evaluation. Compared to last Fall, rating means increased in all 11 post-session areas and pre/post rating differences increased in 9 of the 11 areas. The highest pre-session/post-session increases were in how to navigate the AAC website (1.83 rating increase), how to read and understand a graduation evaluation (1.81 rating increase), and classes allowed to take as a result of College Tier/University Tier status (1.63 rating increase).  Comparing this Spring’s post-session mean rating data to last Spring’s data, we improved in all 11 areas, and the pre/post differences increased in 10 of 11 areas.

FYE is pleased with the results of the Advising Module evaluations. Perhaps most noteworthy is that the pre-session to post-session mean differences indicate that students learn a great deal about important information and procedures. The mean ratings have consistently risen as improvements have been made to the content and presentation of the module.

Note: Only students who completed both the pre-session and post-session portions of the instrument were included.

Library Module

For FYE’s “Introduction to the University” course, the WSU Library conducts a one-day module.  In Fall [See Table 6], the Library Module surpassed the department standard of excellence (more than 80% of students rated each item favorably with a strongly agree or agree response) in all evaluated areas.  The most frequent student comment was that the library instructor was very helpful (32).  Library personnel indicate that Spring evaluations have been misplaced due to a personnel change.