Weber State University   

First Year Experience

Results of Assessment 

2005-2006 (submitted 06/15/06)

2005-2006 ASSESSMENT REPORT

FYE standards of excellence were selected in advance of data collection. Most answer choices were formatted in a 5-point Likert scale ranging from least favorable rating (1) to most favorable rating (5), and achievement of FYE’s standard of excellence was defined as a mean rating of at least a 4.0. For the Library’s 6-point scale [since revised to our standard 5-point Likert], achievement of FYE’s standard of excellence was defined as earned ratings of strongly agree or agree from at least 80% of the respondents. As decided before the start of the assessment year, the standard of excellence in the area of attendance at adjunct trainings and mentor trainings was five people from the target audience. For trainings to which fewer than five members of that group were represented, evaluations were analyzed but not reported.

2005-2006 RESPONSES TO 2004-2005 ASSESSMENT FINDINGS

In response to the findings of the 2004-2005 First Year Experience (FYE) Assessment, FYE achieved the following in 2005-2006 to continuously improve its programs and services. More details on most of these achievements and others are reported in subsequent sections of this report:

• To create a closer connection between FYE cluster faculty and the FYE program, as well as to provide an additional orientation service to FYE students, an Academic Advisement Center advisor visited Fall 2005 cluster classes on behalf of FYE during the first week of the semester. Feedback from advisors and cluster faculty was generally positive, so visitations will be conducted Fall 2006, yet only during the first (rather than the first and second) day of the cluster classes.

• The FYE Coordinator worked on improving the quality and convenience of instructor trainings and meetings, resulting in significantly better attendance.  Unlike last year, all instructor trainings and meetings met the department’s standard of excellence related to attendance.

• In an effort to formalize and standardize FYE students’ evaluation of mentors, the FYE Coordinator developed a mentor evaluation form subsequent to soliciting input from adjuncts and mentors.  Students’ formal evaluations of mentors are now considered in the mentor rehiring process.

• The FYE Student Specialist administered the Academic Advisement Center’s Academic Self-Assessment Questionnaire to FYE students. The survey was administered to Fall 2005 “Introduction to the University” students to help FYE better gauge and fulfill students’ perceived needs.  Summary findings will be shared with the FYE adjuncts and FYE cluster faculty so they have more information on the students they teach.

• To obtain more specific information on the areas FYE did not well surpass its standards of excellence in student evaluations, a focus group of Spring 2006 FYE students will be conducted during Fall 2006.

• To ensure that the needs of FYE students are being met in all FYE modules, the FYE Coordinator is working with Career Services and the Library to create an evaluation of the Career Services module and a revised evaluation of the Library module. The Career Services evaluation will be implemented Summer 2006.

2005-2006 EVALUATION OF COURSES AND SERVICES

2005-2006 Evaluation of FYE Promotions

To increase and improve marketing of “Introduction to the University” (EDUC 1105) and FYE student services, a structured marketing plan including dates, activities, and costs was created. The FYE Student Specialist also kept track of how people obtained information about FYE; she logged office visits, monitored hits to the FYE website, and collected information from the FYE cluster registration forms. The information collected from the cluster registration forms indicated that most of the 132 students heard of FYE from the following sources: a new student orientation (41%, 54), the FYE website (17%, 23), an advisor (17%, 22), or FYE’s new student mailing (11%, 14).  Some of FYE’s additional outreach and marketing initiatives included manning booths at the Scholarships Fair, Major Fest, and the WSU West Center Open House. Others included an FYE information session, two recruitment sessions for prospective mentors, as well as the production and use of FYE table tents and pocket folders for campus advertising.

2005-2006 ADJUNCT TRAINING

To promote the professional development of FYE adjuncts, FYE sponsored semester Kick-offs and near monthly adjunct meetings. To best plan future trainings, these learning opportunities were evaluated based on quality and content of the presentations.

Semester Kick-offs

The 2005-2006 adjunct training schedule [see Table 1] consisted of two semester Kick-offs.  All Kick-offs are evaluated so FYE can use this feedback to plan effective future learning opportunities. Overall, the feedback on adjunct trainings surpassed the department standard of excellence. Attendance was significantly higher than last year’s. To improve attendance at trainings, the Coordinator created the training schedule a semester in advance based on GroupWise calendar searches of adjunct availability.  RSVPs were required so sessions with limited interest could be rescheduled or cancelled. Training topic selection will continue to be based on solicited and unsolicited input from FYE’s constituents.

Adjunct Trainings and Meetings 

In addition to the semester Kick-offs, FYE conducted a Summer Training and six adjunct meetings [See Table 2]. Evaluations of the adjunct meetings met or exceeded FYE’s standard of excellence in terms of presentation content and presentation quality for each meeting. Attendance improved from last year. Attendance was improved by building the training schedule a semester in advance based on GroupWise calendar searches of adjunct availability and requesting RSVPs so sessions with limited interest could be rescheduled or cancelled. Training topic selection will continue to be based on participant feedback. The Summer training did not meet the FYE standard of excellence in terms of presentation content; however, it did meet the standard for presentation quality.

2005-2006 Student Evaluations of “Introduction to the University”

For Fall, student evaluations of Education 1105 instruction indicate that FYE met or exceeded its standard of excellence in 12 of the 14 evaluated areas [see Table 3]. When asked which course topics students thought would most help them be more successful students, the most frequent responses were time management (58), note taking (29), test taking (27), campus resources (26), personality/learning style assessments (21), study skills (16), and academic advising (11). When asked what one change they recommend for future EDUC 1105 classes, the most frequent responses were more group activities (11), less busywork (8), more in-class activities (7), and more challenging material (6). This feedback is always useful for course planning. FYE is pleased with the minimum mean rating increases over last year, acknowledging that in areas where our ratings are high, we can only hope for very minimum improvements.

 * “n”s ranged from 187 to 189

 **“n”s ranged from 176 to 178

Advising Module

For First Year Experience’s “Introduction to the University” course, Academic Advisement Center advisors conduct two-day modules for each course section.  These are a customized extended version of AAC’s General Education [now called General Studies] Sessions. Data on Fall and Spring were kept separate for two reasons: Significant changes to the Spring presentation were made as a result of the Banner system implementation. Also, there is at least one significant difference between Fall and Spring FYE students: a larger percent of Fall students are first-time freshmen.

In Fall 2005, 240 students were enrolled in FYE.  Of them, 62% (149) completed an Advising Module Evaluation. This low response rate was not due to advisor error. It was due to student absenteeism on one or both of the Advising Module days.  Students’ evaluations were only considered if they attended both sessions and completed both the pre-module evaluation and post-module evaluation. The FYE Coordinator will work with FYE adjuncts to facilitate increased attendance. 

FYE met or surpassed the department standard of excellence in all areas evaluated in the advising module. The highest pre-session/post-session increases were in how to navigate the AAC website (1.84 rating increase), how to declare a major (1.62 increase), how to access web-based information on majors (1.60 increase), and classes allowed to take as a result of College Tier/University Tier (1.55 increase). Compared to last Fall, our post-session to post-session means this Fall improved in all 10 areas and the pre-to-post rating differences improved in 7 of the 10 areas. 

Notes: Only feedback from those who completed both the pre-session and post-session parts of the tool was included. Also, the “I know how to read and understand a graduation evaluation” question was eliminated in Fall 2005 because we were transitioning between STAARS and Banner graduation evaluations and students had access to neither at that time.

In Spring 2006, 94 students were registered in FYE. Of them, 63% (59) completed an Advising Module evaluation. Compared to Fall 2005, all ten post-session ratings increased. Also, compared to Fall 2005, pre-session/post-session rating differences increased in 7 of the 10 evaluated areas. The highest pre-session/post-session increases were in how to navigate the AAC website (1.91 rating increase), how to declare a major (1.80 rating increase), and how to access web-based information on majors (1.58 rating increase).  Compared to last Spring, this Spring’s post-session mean rating data and this Spring’s pre-post differences indicate improvement in 7 of 10 areas.

In Spring, the knowledge area “I know which classes I am allowed to take as result of my College Tier/Univeristy tier standing” was changed to “I understand the importance of completing Math and English.” This change was made because the College Tier policy was eliminated and replaced with the Mandatory Assessment and Placement Policy.

Note: Only students who completed both the pre-session and post-session portions of the instrument were included.

FYE is pleased with the results of the Academic Advising Module evaluations. The pre-session to post-session mean differences reflect that in the Advising Module students are learning a great deal about important advising related information and procedures. The mean ratings continue to rise each semester as improvements are made to the content and presentation of the module.

Library Module

For FYE’s “Introduction to the University” course, the Stewart Library conducts a one-day module.  In Fall [See Table 6], the Library Module surpassed the FYE department standard of excellence in all evaluated areas. The most frequent student comment was that the Library Module was very helpful (29). At our request, the Library will revise this instrument to conform with the 5-point scale we utilize in our other evaluations.