Weber State University   

CHF - Early Childhood / Early Childhood Education

Results of Assessment

2003-2004 (submitted 09/30/04)

Five outcome measures were implemented during the academic year 2003-2004.  These measures are to help determine the progress of students majoring in Early Childhood (EC) and Early Childhood Education (ECE).  The five measures were a student teaching rating scale, a written capstone essay examination, an exit interview questionnaire, a self evaluated teaching portfolio, and pre-post teacher beliefs and practices survey.

Synopsis of assessment information

The results from these assessments indicate that the program continues to provide an educational arena that encourages ethical instructors who plan open ended, developmentally appropriate activities that support creativity and the development of the whole child.   Furthermore, the students’ overall competencies appear to be improving both as they progress through their course of study and because of the changes the department has made in the curriculum. [Note: information on portfolios and DAP/DIP still need to be added] The creation of a student portfolio system appears to be assisting students’ in identifying their areas of both strengths and weaknesses.  The pre-post assessment on students’ attitudes towards developmentally appropriate practices suggests the department is able to encourage this belief as the student progresses through the Early Childhood and/or Early Childhood Education major.

Graduates who responded to the Exit Interview Questionnaire appear to “agree” that their department experience prepared them for their career, taught them how to plan educational activities for children, and provided them with a very satisfactory experience. Furthermore, they appear to have enjoyed their educational experience as evidenced by “strongly agreeing” they would choose a major in Child and Family Studies if they were to do it all over again.

Based upon this data, the department has implemented the following strategies to improve students’ comprehension and application of the course information:

1.  Students continue to pass the required criteria during their Student Teaching.  These criteria include demonstration of appropriate guidance strategies, developmentally appropriate planning, working with families, teamwork, and ethical behavior.  The department will continue to focus on these areas in the preparatory courses.

2.  Feedback from the department’s Outcome Assessments initiated the inclusion of the Early Childhood Seminar (CHFAM 4990A) into the Early Childhood Education major.  An additional section of this course has been added to the course offerings to accommodate these students. Some of the ECE majors will begin to take this course Spring 2005, and will be included in the Capstone Assessment.

3.  The Capstone exam indicates the students continue to have difficulty knowing how to identify children who have special needs, and when and how to refer them.  Additional information regarding this topic has been included in two courses; Children at Risk (CHFAM 3500) and Advanced Planning and Guidance (CHFAM 3710).

4.  In order to increase students’ awareness of professional organizations and publications, the department is examining the possibility of requiring students to join an early childhood professional organization.  The department is discussing the possibility of providing some funding in order to assist the students with the cost of membership in NAEYC.

5.  A few years ago the department hired a professional student advisor to provide more in-depth student advisement.  This advisor coordinates carefully with the department of Teacher Education and with the department’s student organization.  Hopefully, these activities will increase the student’s feelings of departmental support.

6.  Because past Outcome Assessments have indicated the department needed to increase the student’s familiarity with anti-bias curriculum, the department has implemented more direct instruction and modeling during student teaching.  Also, more information regarding anti-bias curriculum has been added to courses.  This appears to be working as evidenced by the current Outcome Assessments.

7.  All of the students who completed their student teaching met the required levels of documentation in their portfolios. The portfolio is based on the NAEYC Standards for NCATE accreditation. These standards are in five areas: Promoting Child Development and Learning; Building Family and Community Relationships; Observing, Documenting and Assessing; Teaching and Learning; and Becoming a Professional.

Summary of the results from the five assessments

1. Student Teaching Rating Scale

The Student Teacher Evaluation rating scales has been used for four years.  During Spring semester, 2003, the department revised this assessment to reflect NAEYC/NCATE’s preference for a three level rating scale.  The professional lead teacher in the Melba Lehner Children’s School administers this assessment based on the student teacher’s performance.

The form was altered slightly again in November, 2003.  The new form added 3 additional questions.  Thus, this form contains 24 items, whereas, the previous one contained 21 questions.  The questions continue to be clustered into the 5 areas; Guidance, Planning, Family, Teamwork, and Personal. The form uses a 3 point criteria;  Exceeds Criteria = 3, Meets Criteria = 2, Doesn’t Meet Criteria = 1. A minimum score of 2 is required for passing.

Last year, 2002- 2003, the Early Childhood Education majors averaged 2.33 and the Early Childhood majors averaged 2.20.  The overall average was 2.27

During the academic year 2003-2004, at total of 36 students were assessed.  These included 13 during Fall semester, 2003, and 23 Spring semester, 2004.  The total population include 21 Early Childhood Education and 15 Early Childhood majors.

The form continued to use the 3 point criteria.  The overall average of the 36 students for both semesters was 2.4, which is slightly higher than the previous year of 2.27.  These scores indicate that the students continue to meet the competency requirements established by the department.  These scores were also examined by major. The Early Childhood Education majors totaled an average of 2.43.  The other students scored an average of 2.37.

Implications:

A comparison of the results between Spring 2003 with Fall 2003 and Spring 2004 reveals the students performed better during the last academic year.  The Early Childhood majors improved their scores in each of the 5 areas. As these areas pertain to ECE, the students’ performance improved in every area except Personal, where it declined by .12.  However, it is unlikely that this decline is significant given the high overall scores during both years.  It is felt that the scores improved because of our examination and feedback on the individual questions for this exam that were incorporated into teaching.

2. Capstone Essay Exam

The second Outcome measure was a capstone essay exam.  This comprehensive essay examination includes 5 essay questions given at the end of the term to the students registered in the Early Childhood Senior Seminar (CHFAM 4990A), which is a capstone course for Early Childhood majors. The questions assess the students’ ability to describe, in written form, their understanding of pertinent early childhood principles and practices.  These include the goals, theories, and strategies of guidance; guidelines for constructing developmentally appropriate curriculum that includes an anti-bias strand relative to elements of diversity; using components of various developmental and learning theories for planning and evaluating instruction; describing the purpose of the NAEYC Code of Ethical Conduct and it’s major principles; and explaining the process for deciding when a child’s behavior required outside consultation and/or referral.

This exam has been given for 5 years.  It was given twice in the 2003-2004 academic year because the department added another section of this course.  The purpose for the course addition was to accommodate Early Childhood Education (ECE) students, who are now required to take the Early Childhood Senior Seminar Course (CHFAM 4990A).  ECE students are required to take this course because past Outcome Assessments indicated it would be a valuable course for them.

The following chart indicates the average percent, by year, that students received on each question of the comprehensive exam, and the overall average score by year.  These scores only reflect students who are Early Childhood majors.  The Early Childhood Education majors will not be required to take this exam until the 2004-2005 academic year.

The department established the minimum competency level at 80%.  As can be seen by the chart, on the average, students met the overall competency requirements in 2001, 2002, Fall 2003, and Spring 2004 (85.94%, 83.82%, 82.17, and 82.64, respectively).  As a group, they did not meet this requirement in either the year 2000 nor 2003. 

Because of the students’ scores in 2000, in 2001 the Early Childhood Senior Seminar course (4990A) was changed to assist students in synthesizing the information that they accumulated during their studies.  This course change may have been reflected in the increase in scores during the 2001 and 2002 years.  However, it would not explain why students did poorer as a whole in 2003. The data does not reflect individual student differences on the exam.

An examination of the individual questions from the exam during Fall 2003 and Spring 2004 indicates the students did not pass question #5.  Additionally, students appear to have scored poorly on this question during most of the years.  This question asks the students to describe how they would decide when a child’s behavior requires outside consultation and/or referral.

 Implications

In order to enhance the students’ competency as it relates to consultation and referral (question #5), the department has incorporated more instruction in two courses regarding assessing and referring children.  These courses are CHFAM 3500, Children at Risk, and CHFAM 4710, Advance Planning and Guidance.  Hopefully, the improvement seen between Fall 2003 (69%) and Spring 2004 (73.94%) reflects this change.

During past Assessments, question #2 has also posed problems for students.  Question #2 poses questions regarding anti-bias curriculum.  In the Fall of 2003, the department started having the Supervising teachers work more directly with the student teachers through direct instruction and modeling.  Information has also been added to the Student Teaching Seminar course, CHFAM 4720.  Based on the last two assessments, it appears that the department’s strategies are increasing the student’s knowledge.

 3.  Exit Interview Questionnaire

The original Exit Interview Questionnaire which was sent in 2003 consisted of 13 questions (items 2-12, & 14) that the recipient rated 1 - 7 on a Likert Scale; 1=strongly disagree, and 7 = strongly agree.  Question 13, which asked about overall satisfaction of the respondent’s education in the department, was rated on a 10 point Likert Scale; 1=Extremely Dissatisfied and 10 = Extremely Satisfied.  In 2004, the questionnaire was revised to have question number 13 use a 7 point likert scale.  The questionnaire also includes 6 open ended questions. 

The first chart listed below is a summary of those students who had graduated in Early Childhood and ECE in 2003.  The second chart is a summary of students who graduated last year, 2003-2004.  Please note that last year the ECE students were evaluated both in Fall, 2003, and Spring 2004. The charts are listed by question number and the year the survey was conducted.

A Likert response of 4 is midpoint between Disagree and Agree. Any question falling below a 4 would be considered below average, and would not meet minimum requirements.  As the above chart indicates, the overall average for both majors was above 4 on all questions. However, an examination of the Early Childhood majors (EC) demonstrates a score on question #9 that falls below 4.  Additionally, the ECE majors rated this question lower (by 1 standard deviation) than the other questions.  Question #9 states, “My experiences in Child and Family Studies exposed me to professional organizations and publications in my field”.

The ECE majors also scored questions #6 and #14 one standard deviation below the mean, although they still are higher than the minimum score required (5.30 & 5.24, respectively).  Question #6 states, “My experiences in Child and Family Studies gave me an opportunity to meet professionals in my field”.  Question #14 states, “I feel I received appropriate support during my experiences in Child and Family Studies.      

The ECE majors rated item 12 two standard deviations above the mean.  Item 12 states “I would choose a major in Child and Family Studies if I could do it all over again.” This appears to be a very favorable comment on the department.

 Implications

Overall, the graduates who responded to this questionnaire appear to “agree” that their department experience prepared them for their career, taught them how to plan educational activities for children, and provided them with a very satisfactory experience. Furthermore, their experiences in the Melba Lehner Children’s School seem to be very beneficial.  It is very gratifying that the ECE students had a very positive response to the question about choosing a major in Child and Family Studies again.  This seems to reflect the overall satisfaction with the department and its program.

This questionnaire raises a slight concern about the students’ being exposed to professional organizations and publications in their fields. The faculty held discussions regarding introducing a requirement that students would belong to the National Association for the Education of Young Children (NAEYC).  Possibly, the department could pay for the initial membership fee.  Further discussions will be held regarding this issue.

For the ECE majors, the department discussed the feedback regarding question #14 (feeling support) at a departmental meeting.  From this discussion two ideas emerged.  First, the ECE major is a joint major with the department of Teacher Education.  These students may feel some confusion regarding their interaction with both departments. In trying to reduce the confusion, all students who want an ECE teacher license are assigned to the Child & Family Studies department; whereas, students used to have their choice between the two departments.

Another strategy that was implemented to reduce confusion was to hire a professional student advisor for the department.  This advisor provides in-depth student advisement and is the supervisor for the student organization.

4. Portfolio Evaluation

 The creation of a portfolio has taken on an increased importance and value for the Early Childhood Education students. An electronic portfolio is a Teacher Education and state of Utah requirement. The portfolio self evaluation is based on the NAEYC Standards for NCATE accreditation.  These standards are in five areas: Promoting Child Development and Learning; Building Family and Community Relationships; Observing, Documenting and Assessing; Teaching and Learning; and Becoming a Professional.

Students choose their best work and described how the selected artifacts documented their accomplishments in each standards area. The portfolios were first constructed in the Developmental Planning class (CHFAM 2620) and then modified during student teaching. At the beginning of student teaching a seminar session is devoted to a follow-up and review of the portfolio assignment. An evaluation discussion is held with the student teachers at the end of student teaching during which they explain modifications--what they added, what they deleted. They fill out a checklist summarizing which of their artifacts documented each standards area. Most students have ample documentation in all areas.  All of the students met the required levels of documentation for all five areas.

Because the ECE students take courses from the department of Teacher Education, that department has added more structure to their part of the portfolio process.  The ECE students are required to evaluate their artifacts relative to both the NAEYC and INTASC standards.

Teacher Education has created a rubric that is applied to the ECE portfolio evaluation at the end of the student’s public school student teaching.  These final portfolios are evaluated by the ECE faculty.

5. Pre-post teacher beliefs and practices survey

In Fall 2001, the Teacher Beliefs and Practices Survey , which is a modified version of the Teacher Questionnaire, was adopted to assess students’ understanding and implementation of developmentally appropriate practices (DAP) and developmentally inappropriate practices (DIP).  This assessment is a pre - post-test.  The students take the pre-test in a lower division course (CHFAM 2600) and then re-take the test at the completion of their student teaching experience in the Melba Lehner Children’s School. The purpose of this survey is to determine if the students’ attitudes and practices change as they progress through their course work.

Data was collected in Spring 2004 and sent to Weber State University’s department of Budget and Institutional Research for analysis.  These data have not been returned for inclusion in this report.

Mission Statement / Student Learning Outcomes / Curriculum Grid / Assessment Plan / Contact Person