Weber State University
|
|
First Year ExperienceResults of Assessment 2003-2004 (submitted
09/15/04) Changes in the Way FYE Evaluates and AssessesIn response to what was learned in the 2002-2003 assessment, significant changes were implemented in the way FYE evaluated and assessed in 2003-2004. FYE committed to accomplish the following: (1) Evaluate all FYE trainings. (2) Secure evaluations from all training participants. (3) Revise the evaluation instruments. (4) Revise the way course assessment tools are administered. (5) Revise the way FYE prepares the annual assessment report. Evaluate all FYE instructor/cluster faculty/mentor trainingsIn 2002-2003, only some of FYE’s trainings were formally evaluated by participants. In 2003-2004, all trainings were evaluated. This decision was made because FYE trainings are distinct from each other and, without evaluations of each, FYE felt ill equipped to decide on which trainings to continue, revise, or eliminate. Secure completed evaluations from all participants in these trainingsIn 2002-2003, attendance at trainings was sometimes low and response rates to evaluations of trainings were often low. During 2003-2004, FYE procured an evaluation from every training participant. FYE staff can now be confident that respondents were representative of participants. Revise the questions in the course assessment instrumentsIn 2003-2004, instruments used to evaluate “Introduction to the University” (Education 1105) were significantly revised to be similar to evaluations used by the academic colleges. Also, an evaluation of “Mentor Leadership Seminar” (Education 3170) was developed and administered. Revise the way the course assessment tools are administeredThe administration of Education 1105 evaluations was revamped. Prior to 2003-2004, instructors took their students to a computer lab to complete evaluations online. Instructors, mentors, and students were frustrated by lab scheduling restrictions and periodic technical problems. To utilize available computer lab times, some evaluations were being administered before the last week of the course. An FYE staff member now administers evaluations in the regularly scheduled classroom rather than a computer lab, during the last week of class, while the instructor and mentor are out of the room completing their evaluation instrument. Revise the way FYE prepares the annual assessment reportThe way FYE conducts its analysis of assessment-related material and its preparation of the annual assessment report was revised. The more thoughtful, year-round process incorporates use of a well-organized, frequently updated, resource binder. Achievement of Intended OutcomesGauges of success, referred to as FYE standards of excellence, were selected in advance of data collection. Most answer choices were formatted in a 5-point Likert scale ranging from the least favorable rating (1) to the most favorable rating (5). Achievement of the department’s standard of excellence was defined as earning a mean rating of at least 4.0. Fall and Spring findings related to FYE students were analyzed separately to acknowledge differences in the student populations. Feedback on FYE was solicited from four primary constituencies: FYE students, FYE instructors, FYE mentors, and WSU faculty who teach other courses within FYE clusters. When achievement of the department standard was gauged by the percentage who reported being satisfied with a service (gauge used when Likert scale not used), the department standard of excellence was set at 80%. “Introduction to the University” Instructors’ Evaluations of TrainingFYE sponsors instructor training opportunities of two kinds: once-a-semester trainings/kick-offs and regularly scheduled FYE instructor meetings. To best plan future trainings, FYE solicited feedback from instructors who attended 2003-2004 training sessions and considered the input when planning future trainings.
Instructors' Evaluations of Teaching "Introduction to the University"In addition to feedback solicited from instructors who participated in training activities, feedback was solicited from them through the Education 1105 evaluation. Most positive aspect of teaching EDUC 1105:
Ways the FYE department could provide better support to future EDUC 1105 teachers:
Cluster Faculty Evaluations of Teaching a Course Paired with Education 1105Most positive aspect of teaching within an FYE cluster:
Specific ways FYE department could provide better support for future cluster faculty:
Student Evaluations of “Introduction to the University”Planner Student evaluations of the FYE planner, an educational tool, met or exceeded the FYE standard of excellence.
Instructors Student evaluations of the Education 1105 instructors indicate that FYE met or exceeded its standard of excellence in the following areas:
“Introduction to the University” Instructors’ Evaluations of ThemselvesInstructor evaluations of themselves indicate that FYE met or exceeded its standards of excellence in the following areas:
FYE Coordinators’ Evaluations of “Introduction to the University” InstructorsThe FYE Coordinator conducted scheduled instructor evaluations in the form of class visitations. She rated instructors utilizing the student instruments. Mean evaluation scores met FYE’s standard of excellence (4.4, n=13). Most instructors’ overall ratings met the standard. “Introduction to the University’s” Mentors’ Evaluation of TrainingEvaluations of the Peer Mentor’s Fall Retreat (August 22) indicated that department standards of excellence were exceeded in the following areas:
Although responses to all other questions met or surpassed the FYE standard, FYE did not meet the standard in terms of the following:
Evaluations of “Mentor Leadership Seminar” (EDUC 3170), the course in existence to facilitate the skills peer mentors need to be successful in that FYE role, indicated a significant drop in student satisfaction from Fall to Spring. Although in Fall evaluations the EDUC 3170 instructor met or exceeded the FYE standard of excellence in all areas, Spring evaluations of the EDUC 3170 instructor were less favorable. Note that Education 3170 was not evaluated in prior years. In the following areas, the instructor met or exceeded FYE’s standard of excellence during both Fall and Spring:
In the following areas, the instructor met or exceeded the department standard of excellence during Fall but not during Spring:
In both Fall and Spring, when asked to list which topics in EDUC 3170 would help them be most successful as mentors, respondents listed mentor presentations, discussions with other mentors, and the resource library. In both Fall and Spring, when asked what one change to EDUC 3170 they would recommend, answers included increases in disseminated information, speakers on campus activities, campus department presentations, and mentors presentations. In Spring, several suggested more structure and fewer papers. At the end of the school year, mentors evaluated the activities they participated in as training for their role as mentors in EDUC 1105. FYE met or exceeded the department standard of excellence in most areas, except mentors’ evaluation of the Spring Academy of Leadership in terms of assisting them in their role as peer mentors (mean 3.8, n=6).
Mentors evaluated how their role in EDUC 1105 helped students. In all Fall responses, FYE department expectations were exceeded. Means were lower during Spring semester.
Faculty’s Evaluation of Their Teaching a Class in an FYE ClusterFaculty evaluated themselves as FYE cluster course teachers. FYE did not meet the standard of excellence in the Fall or Spring self-evaluations, although Spring evaluations were higher.
Other Noteworthy InitiativesIn addition to evaluations of instructor training and mentor training, as well as the Education 1105 and Education 3170 courses, FYE implemented other assessment-related initiatives. To get a sense of the utilization of FYE services in 2003-2004, FYE started documenting the purpose of people’s visits to FYE office (excluding the coordinator’s AAC advisement appointments) and the number of hits received at the FYE website. Approximately 54% of the FYE office visits were from mentors using the resource library (mostly to obtain information for the instructors for whom they mentor) and 30% of the visits were from prospective and new Weber students asking for help getting started at WSU. This information pleases the department that is interested in being a resource to instructors, mentors, prospective Weber students, and current Weber students. Due to feedback received through last year’s assessment, much of the FYE specialist’s time this year has been devoted to the expansion of the resource library and web site. In 2003-2004, FYE began tracking web page hits. In July 2004, the site receives more than 550 hits (a record).
Assessment related activities have resulted in the compilation, analysis and application of much useful information that will enable FYE to continue to improve its programs and services. Areas in Which FYE Excels
Areas of Greatest Improvement from Last Year
Areas Needing Improvement
Tentative 2004-2005 Plans
|
| ||||
