Weber State University

CHF - Early Childhood / Early Childhood Education

Results of Assessment

2006-2007 (submitted 09/04/2007)

Prepared by Jim Bird, Ph. D.

Three outcome measures were implemented during the Spring 2007, and one for Spring, Summer & Fall 2006.  These measures are designed to help determine the progress of students majoring in Early Childhood (EC) and Early Childhood Education (ECE), and assess the department’s curriculum.  The four measures were an exit interview questionnaire, a student teaching rating scale, an evaluation of student teacher’s work samples, and a comprehensive Capstone Essay Exam.

Synopsis of assessment information

The results from these assessments indicate that the program continues to provide an educational arena that produces ethical early childhood professionals who meet the NAEYC standards for Promoting Child Development and Learning; Building Family and Community Relationships; Observing, Documenting and Assessing; Teaching and Learning; and Becoming a Professional.

The assessments suggest the students can demonstrate in both written and applied format appropriate principles of guidance, assessment, and analysis of student learning.  Additionally, the information gleaned from these assessment tools indicate the students who graduate from the program believe they have the necessary skills to work with children and families, and the faculty were “helpful and supportive”.

On all of the assessment tools, the student either met or exceeded the established criteria.  Thus, looking at areas where the students scored low needs to be done with both caution and an understanding that they are only low relative to the higher areas – they still met the criteria.  Given this caution, the students’ lower abilities seemed to be in the area of articulating the theory and rationale behind some developmentally appropriate principles, and involving family and culture into the curriculum.  Additionally, they may benefit from more information about methods for deciding when a child requires outside consultation or referral.  As it pertains to the department, the students seem to want more contact with professionals in the field, more information on planning educational experiences for children, and more effective advisement.

Implementation of strategies based upon past outcome assessments

Changes that have been made in the department in response to past outcome assessments appear to be having positive results.  For example:

1. Past exit surveys indicated that the ECE majors did not feel they received adequate advisement.  In order to better assist the ECE students, the department’s advisor enhanced the communication between the departments of Teacher Education and Child and Family Studies’ faculty, and the students.  As a result, the ECE students responded positively to the question “I feel I received adequate academic advisement assistance” on the exit survey.  The ECE majors had not responded positive to this question in past surveys.  In addition, both the ECE and EC majors replied positively on “I would choose a major in Child and Family Studies if I could do it all over again.”
2. Both the 2004 and the 2005 outcome assessments indicated the ECE students do not believe they received opportunities to meet professionals within their field, or was exposed to professional organizations and publications within their field.  The department, through its student organization, implemented strategies to address this.  For example, the student organization is attempting to develop a student affiliate to the Utah Association for the Education of Young Children.  Additionally, the department was able to provide funds from a private donor to help send students to professional conferences.
3. The 2004 outcome assessment indicated the department needed to increase the student’s familiarity with anti-bias curriculum.  Because of this, the department implemented more direct instruction and modeling during student teaching.  Also, more information regarding anti-bias curriculum has been added to courses.  As evidenced by the Student Teacher evaluation, this curriculum change has been effective.

Summary of the results from the three assessments

1.  Exit Interview Questionnaire

The Exit Interview Questionnaire is sent to graduates in the department’s three majors; Family Studies (FS), Early Childhood (EC), and Early Childhood Education (ECE).  This questionnaire has been sent out since 2003.  Only the data for the Early Childhood and Early Childhood Education majors were tabulated for this summary.

The data listed below (Table 1) is taken from students who graduated in Spring, Summer and Fall 2006.  Thirty-one surveys were sent and 12 students returned them, which is a 39% return rate.  The students who returned the survey included 6 Early Childhood and 6 Early Childhood Education.

The Exit Interview Questionnaire consists of 15 questions.  The first question asks for the student’s major.  Questions 2-15 ask the students to rate their experiences on a 7-point Likert style scale (1=strongly disagree, 7=strongly agree).  Averages were complied on each question, overall by major, and the overall group of students.  The overall average was 5.95.  The average for EC was 5.93 and ECE was 5.95.

A review of each question indicates questions 5 and 15 rank the highest; 6.42 and 6.35, respectively.  Question 5 states, “My experiences in Child and Family Studies allowed me to gain skills needed to be an educator of parents and /or children.  Question 15 states, “The Child and Family Studies faculty has been helpful and supportive”.

The lowest scores (all falling 1 SD below the mean) are numbers 8 and 6; 5 and 5.09, respectively.  Question 8 states, “My experiences in Child and Family Studies taught me how to plan educational experiences.”  Question 6 states, “My experiences in Child and Family Studies gave me an opportunity to meet professionals in my field.”

A comparison of the 2006 responses to the 2005 reveal that question 6 still rates lower than the other questions; although, it needs to be noted that a score of 5.09 still reflects “agree”.

A review of the averages by major reveals the Early Childhood major to have concerns with item 14 (average = 5).  This questions states, “I feel I received adequate academic advisement assistance.  Question 14 was rated comparatively lower in 2005 (average = 5.67), 2004 (average=5.25), and 2003 (average = 5.1).

Table 1.  Exit Interview Spring & Summer 2006.  Early Childhood and Early Childhood Education Graduates.

Implications

In summary, overall it appears that both the EC and ECE majors “strongly agree” that they had positive educational experiences.  They particularly believe that their experiences allowed them to gain the skills they needed to be an educator of parents and children.  The also felt the faculty had been helpful and supportive.  The areas that were comparatively weaker were in “opportunities to meet professionals in the field”, and having experiences that exposed them to real world issues.  Additionally, the EC majors felt they needed more advisement.

The department has been trying to address the students’ need for more opportunities to meet other professionals by having a yearly “round table” luncheon where students are invited to have a free lunch and meet representatives from various early childhood professional fields.  Additionally, the students are encouraged, and can receive some financial assistance, to attend the National Association for the Education of Young Children’s national conference.  However, there are very few EC majors who take advantage of these opportunities.  Another method for getting the EC student familiar with professionals and real world opportunities is the requirement to complete a Cooperative Work Experience (CHF 4860) in the community.

As it pertains to academic advisement, the department provides an academic advisor who reviews each students program.  The advisor then assigns the student to an individual faculty member.  The faculty attempt to assist the student in identifying special concerns and selecting the appropriate courses.         

2. Student Teaching Rating Scale

The Student Teacher Evaluation rating scales has been used for six years.  During spring semester, 2003, the department revised this assessment to reflect NAEYC/NCATE’s preference for a three level rating scale.  In 2004, in order to obtain more information related to NAEYC’s professional standards, the tool was revised again by increasing the total number of items to 24.  In the summer of 2004, an additional 12 items were added.  This brings the current form to a total of 36 items.

The professional lead teacher in the Melba Lehner Children’s School administers this assessment based on the student teacher’s performance at the end of the semester. The questions are clustered into 5 areas; Guidance, Planning, Family, Teamwork, and Personal. The form uses a 3 point criteria; Exceeds Criteria = 3, Meets Criteria = 2, Doesn’t Meet Criteria = 1. A minimum score of 2 is required for passing.

Nineteen student teachers were evaluated in spring semester of 2007.  As indicated in Table 2, the overall average of the 19 students was 2.39, which meets the “Pass” criteria. This is slightly lower than the results from Fall 2006 when the average was 2.54.  However, it is the same as the average in 2005.  These scores indicate that the students continue to meet the overall competency requirements established by the department.  These scores were also examined by major, as shown in Table 2. The Early Childhood major obtained an average of 2.34.  The Early Childhood Education (ECE) major received an average of 2.48.

Looking at the averages for the five competency areas demonstrates that the students passed each of these areas.  The highest average was for “Guidance”.  This was true for both majors.  The “Guidance” area examines teachers’ abilities to build positive trust relationships, use developmentally appropriate guidance strategies, and articulate the theoretical and developmentally appropriate reasons for their decisions. The lowest average was on “Family”, however, it appears this was low because the ECE majors brought the average down.  “Family” was the lowest average for ECE.  The lowest for EC was “Planning”.  The “Family” criteria measure the student’s ability to involve family and culture into the curriculum. “ Planning” focuses on planning and articulating developmentally appropriate activities and anti-bias curriculum.

An examination of the averages for each of the 36 individual questions demonstrates that the students received a “Pass” for all of them.  The highest individual item was question #8 (AVG=2.74).  This item asks the supervising teacher to rate the student teachers’ ability to get on the children’s level physically and intellectually.  The lowest item was question #3 (AVG=2.05) that rates the student teacher’s ability to explain theoretical and developmentally appropriate reasons for guidance strategies.

Table 2 Student Teaching Rating Scale Spring 2007

Implications:

Overall the students continue to demonstrate competency in the established criteria.  All areas appear to be strong.   Within the areas, the teachers show strength in relating to the children.  The department should continue to increase the curriculum to enhance the students’ ability to articulate the theoretical and DAP reasons for guidance.  This item was the lowest in 2005 (AVG=2.0) and 2006 (AVG=2.21)

In summary, the 19 student teachers that were evaluated in spring 2007 received an average of “Pass” in the 5 areas.  This data appears to support the department’s instructional programs as it pertains to guidance, developmental planning, respecting and supporting families, cooperative and collaborative teamwork, and ethical and considerate teaching practices.  Additionally, this information indicates that the student teachers go beyond minimal requirements and/or are consistently prepared ahead of time, which is a strong complement to the EC and ECE majors.

3.  Early Childhood/Early Childhood Education Teacher Work Sample (TWS)

Each student teacher in the Children’s School submits Work Samples (TWS) that correspond to the following six rubrics: contextual factors; objectives/intended learning outcomes; assessment plan; design for instruction & instructional decision-making; analysis of student (child) learning; and reflection & self-evaluation. The TWS particularly addresses the students’ ability to meet NAEYC’s Standard 4c, understanding content knowledge in early education, and 4d, building meaningful curriculum.

The Teacher Work Samples are based on the educational activities the students had implemented during their student teaching. The instructor for the Advanced Planning and Guidance course (CHFAM 4710) reviews the Work Samples and rates them on a 3-point scale; exceeds criteria (3), meets criteria (2), and doesn’t meet criteria (see example below).  The Advanced Planning and Guidance course (CHFAM 4710) is taken by the student teacher concurrently with student teaching.

The Candidates Work Samples need to contain items that relate to the rubrics, as listed below:

1. Contextual Factors: The teacher uses information about the learning-teaching context and student individual differences to set learning objectives and plan instruction and assessment.

2. Objectives/Intended Learning Outcomes:  The teacher sets a measurable objective based on the Children’s Schools goals.

3. Assessment Plan:  The teacher uses assessment tools aligned with the objective to assess student learning before, during, and after instruction.

4. Design for Instruction & Instructional Decision-Making:  The teacher designs instruction from assessments, taking into account students needs and interests.  The teacher uses ongoing analysis of student learning to make instructional decisions.

5. Analysis of Student Learning:  The teacher uses assessment data to profile student learning and communicate information about student progress and achievement.

6. Reflection and Self-Evaluation:  The teacher reflects on his or her instruction and student learning in order to improve teaching practice.

Table 3 lists the average scores, and if the criteria were passed by the 19 students.  A score of 2 or higher reflects meeting or exceeding the criteria.

3 Spring 2007 Teacher Work Samples

As indicated in Table 3, the average score of the 17 student teachers resulted in an overall “Pass” (AVG=2.24).  Additionally, the overall scores on each of the two majors received a “Pass”.   Furthermore, The average scores for each of the 6 areas resulted in “Pass” for all 6 areas, and for both majors.

An examination of each area for both majors reflects the highest score is for the “Analysis of Student Learning” Area (AVG=2.44).  The lowest area is “Design for Instruction and Instructional Decision-Making” (AVG=2.08).  

Implications

The data in Table 3 indicates that the student teachers met the necessary criteria and standards for all of the areas.  This demonstrates they understand the content knowledge in early education and can build meaningful curriculum. 

It should be noted that in both 2004 and 2005 the students did not meet the criteria for “Design for Instruction & Instruction Decision Making”.  This year, they did (AVG=2.06), although it was the lowest of the 6 areas.  This suggests that the curriculum is enhancing the students’ abilities to design instruction from assessments, taking into account individual student’s needs and interests.

The highest area was in “Analysis of Student Learning” (AVG=2.44).  This area indicates that the student teachers are very good at using assessment data to profile student learning and communicate information about children’s progress and achievement.

4. Capstone Essay Exam

The Capstone Essay Exam consists of 6 questions.  This exam has been used since 2000, but was changed in 2005.  The previous exams included 5 questions, with slightly different ratings.  This comprehensive essay examination is given at the end of the term to the students registered in the Early Childhood Senior Seminar (CHFAM 4990A), which is a capstone course for Early Childhood and Early Childhood majors. The questions assess the students’ ability to describe, in written form, their understanding of pertinent early childhood principles and practices.  These include the goals, theories, and strategies of guidance; guidelines for constructing developmentally appropriate curriculum that includes an anti-bias strand relative to elements of diversity; using components of various developmental and learning theories for planning and evaluating instruction; describing the purpose of the NAEYC Code of Ethical Conduct and it’s major principles; explaining the process for deciding when a child’s behavior required outside consultation and/or referral; and describing an effective family involvement program in the school using the Epstein model.  This last question was added in 2005.

The Table 4 indicates the average percent, by year, which students received on each question of the comprehensive exam, and the overall average score by year.

Table 4 Comprehensive Exam Average Percent by Question from 2000-2007

Table 5: Comprehensive Exam Spring 2007.  Percent by Student and Question #.
Question # and Possible Points