Early Childhood Results of Assessment

Five outcome measures were implemented this year to determine the progress of students majoring in Early Childhood and Early Childhood Education. The five measures were a student teaching rating scale, a written capstone essay examination, an exit interview questionnaire, a self evaluated teaching portfolio, and pre-post teacher beliefs and practices survey.

The results from these assessments indicate that the program continues to provide an educational arena that encourages ethical instructors who plan open ended, developmentally appropriate activities that support creativity and the development of the whole child. Furthermore, the students’ overall competencies appear to be improving both as they progress through their course of study and because of the changes the department has made in the curriculum. The creation of a student portfolio system appears to be assisting students’ in identifying their areas of both strengths and weaknesses. The pre-post assessment on students’ attitudes towards developmentally appropriate practices suggests the department is able to encourage this belief as the student progresses through the Early Childhood major.

Graduates who responded to the Exit Interview Questionnaire appear to “agree” that their department experience prepared them for their career, taught them how to plan educational activities for children, and provided them with a very satisfactory experience. Furthermore, their experiences in the Melba Lehner Children’s School seem to be very beneficial.

Based upon this data, the department has implemented the following strategies to improve students’ comprehension and application of the course information:

The Student Teacher Evaluation rating scales has been used for four years. During Spring semester, 2003, the department revised this assessment to reflect NAEYC/NCATE’s preference for a three level rating scale. The professional lead teacher in the Melba Lehner Children’s School administers this assessment based on the student teacher’s performance. This is the fourth year that this assessment has been used.

This year, fifteen student teachers were assessed on the new form. This form used a 3 point criteria; Exceeds Criteria = 3, Meets Criteria = 2, Doesn’t Meet Criteria = 1. The average score of the students was 2.27, out of a possible 3. These scores indicate that the students are meeting the competency requirements established by the department. These scores were also examined by major. The Early Childhood Education majors averaged 2.33 and the Early Childhood majors averaged 2.20.

The revised rating form clustered the 21 items into 5 areas; guidance, planning, family, teamwork, and personal. The following chart indicates the averages:

AREA	TOT AVG	ECE AVG	EC AVG	CRITERIA PERFORMANCE
GUIDANCE	2.27	2.33	2.20	Pass
PLANNING	2.24	2.26	2.21	Pass
FAMILY	2.27	2.31	2.21	Pass
TEAMWORK	2.27	2.38	2.14	Pass
PERSONAL	2.43	2.62	2.21	Pass

A minimum score of 2.0 is required to meet the department’s competency level.

As the chart indicates both the ECE and EC majors meet the criteria. There is only a slight difference between the majors, with the ECE majors performing slightly better overall than the EC majors. The strength appears to be in the Personal Area. This area exams the students responsibility to the job and ethical behavior. An examination of all 21 questions demonstrates the students performed highest on:

Though the students met the overall criteria in the above five areas, the Early Childhood Education students did not meet one of the requirements under the Planning area, “addressing diversity issues” (question #12). Within this topic, the average score was 1.88, failing to meet the necessary 2.0 criteria.

The results from this assessment indicate that the program continues to provide an education arena that encourages ethical instructors who plan open ended, developmentally appropriate, activities that support creativity and the development of the whole child. Furthermore, the students’ overall competencies covered by this assessment appears to be improving. Additionally, the program should review its instruction in the realms of theory, anti-bias curriculum, and planning for non-traditional learning times.

The second measure was a capstone essay exam. This comprehensive essay examination includes 5 essay questions given at the end of the term to the students registered in the Early Childhood Senior Seminar (CHFAM 4990A), which is a capstone course for Early Childhood majors. The questions assess the students’ ability to describe, in written form, their understanding pertinent early childhood principles and practices. These include the goals, theories, and strategies of guidance; guidelines for constructing developmentally appropriate curriculum that includes an anti-bias strand relative to elements of diversity; using components of various developmental and learning theories for planning and evaluating instruction; describing the purpose of the NAEYC Code of Ethical Conduct and it’s major principles; and explaining the process for deciding when a child’s behavior required outside consultation and/or referral.

This exam has been given for 4 years. The following chart indicates the average percent, by year, that students received on each question of the comprehensive exam, and the overall average score by year. These scores only reflect students who are Early Childhood majors. The Early Childhood Education majors will not be required to take this exam until 2004.

The department established the minimum competency level at 80%. As can be seen by the chart, on the average, students met the competency requirements in both 2001 and 2002 (85.94% and 83.82%, respectively). As a group, they did not meet this requirement in either the year 2000 nor 2003.

Because of the students’ scores in 2000, in 2001 the Early Childhood Senior Seminar course (4990A) was changed to assist students in synthesizing the information that they accumulated during their studies. This course change may have been reflected in the increase in scores during the 2001 and 2002 years. However, it would not explain why students did poorer as a whole did poorer in 2003. The data does not reflect individual student differences on the exam.

An examination of the individual questions from the exam suggests that students over the four year period demonstrate meeting the competency on both questions 3 and 4 (80% and 90%, respectively), and come extremely close on question 1 (79%).

Question 3 asks the student to “state one idea from each [theorists] that you would consider in planning and another idea of each you would consider in evaluation of your instruction.” Question 4 requests the student to “describe the purpose of the NAEYC Code of Ethical Conduct and it’s major principle.” Question 1 requires students to state the goal of guidance in the early childhood classroom, describe the components of a trust relationship as they relate to guidance, and describe theoretical basis for three guidance strategies they might use.

The students appear to need more assistance with understanding the elements contained in question 5. This question asks the students to describe how they would decide when a child’s behavior requires outside consultation and/or referral.

Question #2 requests the students to explain the guidelines they would apply in order to construct developmentally appropriate curriculum. They need to include an anti-bias strand relative to elements of diversity such as culture, race, gender, and/or disabilities.

The box below indicates the range of students’ scores related to the year. As can be seen, the two years where the students’ average scores did not meet the required cut-off, the range was much greater. In 2000 the range was 36 - 79, and in 2003 the range was 48 - 97. The corresponding averages were 75.50 and 75.39, respectively. These ranges are much different than those occurring in either 2001 or 2002, suggesting that one or two students who did not meet the passing criteria effected the overall average for that year.

Based upon this data, the department has implemented the following strategies to improve students’ comprehension and application of the course information. As already stated, the department has redesigned the seminar course, CHFAM 4990A (which is the capstone course), to include both Early Childhood Education majors and to facilitate the synthesis of early childhood principles and practices. Additionally, because the students’ scores were low on question #5, the department will incorporate more instruction in two courses regarding assessing and referring children. These courses are CHFAM 3500, Children at Risk, and CHFAM 4710, Advance Planning and Guidance.

The results from question #2, regarding anti-bias curriculum, is a concern that the department is trying to address but currently does not have a resolution. As previously stated, a similar problem regarding anti-bias curriculum was demonstrated in the Student Teacher Evaluation.

In Spring 2003, a pilot questionnaire was sent to 81 recent graduates in the department of child and family studies. The recipients included graduates from all three of the majors within the department; Early Childhood (EC), Early Childhood Education (ECE), and Family Studies (FS). Thirty-one questionnaires were returned (38%). These included 10 Early Childhood, 8 Early Childhood Education, and 14 Family Studies.

The questionnaire consisted of 13 questions (items 2-12, & 14) that the recipient rated 1 - 7 on a Likert Scale; 1=strongly disagree, and 7 = strongly agree. Question 13, which asked about overall satisfaction of the respondent’s education in the department, was rated on a 10 point Likert Scale; 1=Extremely Dissatisfied and 10 = Extremely Satisfied. The questionnaire also included 6 open ended questions.

The chart listed below is a summary of the students’ responses. It is listed by question number (2-12, & 14) and major; Early Childhood (EC), and Early Childhood Education (ECE).

As the above chart indicates, most of the responses rated a score of almost 6, which is between agree and strongly agree. The average response for Early Childhood majors was 5.97, and the Early Childhood Education majors was 6.03.

Any question falling below a 4 would be considered below average. None of the scores fell below a 5 (“agree”). A comparison of the questions indicates both majors rated question 14 one standard deviation below the mean. It should be noted that this question still received an average score of 5.1 (EC) and 5.5 (ECE). This question states “I feel I received appropriate support during my experiences in Child and Family Studies.”

Additionally, the EC majors rated item 12 one standard deviation above the mean, and the ECE majors rated both item 5 and 7 one standard deviation above the mean. Item 12 states “I would choose a major in Child and Family Studies if I could do it all over again.” Item 5 reads, “My experiences in Child and family Studies allowed me to gain skills needed to be an educator of parents and/or children. Question 7 states, “My experiences in Child and Family Studies taught me how to plan educational experiences.”

A review of the open ended questions (items 15 - 20) suggests some very positive departmental traits. One trait that was frequently mentioned was the positive experience in the Melba Lehner Children’s School. For example, one student commented “I would say that all of my experiences in the Children’s School were very important and valuable.” Another student commented that the “hands on learning - working with the kids” was the “only way (and best) to really learn about and understand them.”

Overall, the graduates who responded to this questionnaire appear to “agree” that their department experience prepared them for their career, taught them how to plan educational activities for children, and provided them with a very satisfactory experience. Furthermore, their experiences in the Melba Lehner Children’s School seem to be very beneficial.

These questionnaires raise a slight concern about the students’ feeling “support during my experiences in Child and Family Studies.” In anticipation of this issue, in 2001 the department hired a professional student advisor to provide more in-depth student advisement.

The responses on question 14 may also stem from some ambiguity with the term “support”. As stated previously, this was a pilot questionnaire. Therefore, in future questionnaires the term “support” will be listed in different questions as faculty, student, or advising support. Hopefully, in future evaluations the rating for this item will increase.

The creation of a portfolio has taken on an increased importance and value for the Early Childhood Education students. An electronic portfolio is now a Teacher Education and state of Utah requirement. The portfolio self evaluation is based on the NAEYC Standards for NCATE accreditation. These standards are in five areas: Promoting Child Development and Learning; Building Family and Community Relationships; Observing, Documenting and Assessing; Teaching and Learning; and Becoming a Professional.

Students chose their best work and described how the selected artifacts documented their accomplishments in each standards area. The portfolios were first constructed in the Planning class and then modified during student teaching. At the beginning of student teaching a seminar session was devoted to a follow-up and review of the portfolio assignment. An evaluation discussion was held with the student teachers at the end of student teaching during which they explained modifications--what they added, what they deleted. They filled out a checklist summarizing which of their artifacts documented each standards area. Most students had ample documentation in all areas. They improved in documenting last year's weak area Building Family and Community Relationships. Individual student comments indicated that the checklist helped them identify areas of both strengths and weaknesses.

The Early Childhood Program committee decided to use the Teacher Beliefs and Practices Survey, which is a modified version of the Teacher Questionnaire, to assess students’ understanding and implementation of developmentally appropriate practices (DAP) and developmentally inappropriate practices (DIP). This survey was initiated in Fall 2001 as a pilot pre-test in CHFAM 2600 with 29 students. These students will take a post-test from 1 to 3 years later at the completion of their student teaching experience in the Melba Lehner Children’s School. The student teaching is a capstone experience. The purpose of using this survey is to compare the pretest and post-tests of the students to determine if their attitudes and practices change as they progress through their course work.

The Survey is composed of four parts: 1) Student Teacher Survey; consists of questions on the demographics of the students; educational status at the time of the pretest, and previous degrees, license and ethnic background, 2) beliefs about whose influence (parents, school system policy, principal, teacher, state regulations, and other teachers) the student believes will affect their planning and instruction, 3) beliefs about the importance of DAP and DIP practices and 4) instructional activities.

During the pilot session in Fall 2001, 29 students took the survey. However, only 4 students identified themselves which means we can only do a post-test match on these 4. These students will graduate anywhere from Spring 2004 to Fall 2005.

The Fall 2001 Pretest questions were analyzed according to DAP and DIP questions for each student and as a group. Group DAP mean is 4.31 and group DIP mean is 2. 13.

In Spring, 2002, 28 students took the pre-test survey. The Spring 2002 Pretest questions were analyzed according to DAP and DIP questions for each student and as a group. Group DAP Mean is 4.26 and Group DIP mean is 2.60.

As the chart below demonstrates, students in both groups tend to have a higher attitude towards developmentally appropriate practices (DAP) than developmentally inappropriate practices (DIP).

	DAP MEANS	DIP MEANS
PRETEST FALL 2001	4.31	2.13
PRETEST SPRING 2002	4.26	2.60

Two students from the Fall 2001 and 7 from the Spring 2002 groups completed their student teaching and took the post-test. Their pretest and post-test scores are listed in the chart below.

FALL 2001	PRE - DAP	PRE - DIP	POST - DAP	POST - DIP
Student 1	4.59	1.85	4.59	1.85
Student 2	4.38	1.46	4.72	1.23
Mean 2002	4.59	1.66	4.65	1.54

SPRING 2002	PRE - DAP	PRE - DIP	POST - DAP	POST - DIP
Student 1	4.00	2.54	4.38	1.77
Student 2	4.61	3.33	4.52	3.54
Student 3	4.83	4.00	4.45	2.15
Student 4	4.31	2.85	4.69	1.46
Student 5	4.48	2.08	4.90	1.38
Student 6	4.45	3.00	3.97	2.46
Student 7	4.57	3.31	4.93	1.85
Mean 2002	4.46	3.02	4.55	2.09

As demonstrated by the means, both groups (Fall 2001 and Spring 2002) increased their attitudes about developmentally appropriate practices (DAP) and decreased their attitudes about developmentally inappropriate practices (DIP). Overall, the increase on DAP went from a pretest average score of 4.47 to 4.60, and a decrease in DIP from 2.34 to 1.81.

Although three students actually showed a slight decrease in their post-DAP scores, this decrease is likely due to the small differences in the pretest/post-test scores.

Developmentally appropriate practices are a foundation of the departments instructional goals. The few students who participated in both the pretest and post-test survey demonstrate a slight progression towards having more positive responses to developmentally appropriate practices. This indicates the course work they are receiving either supports their beliefs that they held when they began the program and/or the course work has increased their positive beliefs towards developmentally appropriate practices.

CHF - Early Childhood / Early Childhood Education