Weber State University

CHF - Early Childhood / Early Childhood Education

Results of Assessment

2004-2005 (submitted 01/10/2006)

Three outcome measures were implemented during the academic year 2004-2005.  These measures are designed to help determine the progress of students majoring in Early Childhood (EC) and Early Childhood Education (ECE), and assess the department’s curriculum.  The three measures were an exit interview questionnaire, a student teaching rating scale, and an evaluation of student teacher’s work samples.

Synopsis of assessment information

The results from these assessments indicate that the program continues to provide an educational arena that encourages ethical instructors who plan open ended, developmentally appropriate activities that support creativity and the development of the whole child.  

The information gleaned from these assessment tools suggests the department’s curriculum is effective in teaching students how to implement instructional programs as they pertain to guidance, developmental planning, respecting and supporting families, cooperative and collaborative teamwork, and ethical and considerate teaching practices.  Additionally, this information indicates that the student teachers go beyond the minimal requirements and are consistently prepared for teaching, which is a strong complement to the EC and ECE majors.

Furthermore, all three assessments are based on the NAEYC Standards for NCATE accreditation.  Additionally, these assessments were conducted on students who had completed their student teaching.  The results from these assessments demonstrate students majoring in Early Childhood and Early Childhood Education, and have completed their early childhood student teaching, meet the NAEYC standards for Promoting Child Development and Learning; Building Family and Community Relationships; Observing, Documenting and Assessing; Teaching and Learning; and Becoming a Professional.

Implementation of strategies based upon pass outcome assessments

Changes that have been made in the department in response to past outcome assessments appear to be having positive results. For example:

1. Past exit surveys indicated that the ECE majors did not feel they received adequate advisement.  In order to better assist the ECE students, the department’s advisor enhanced the communication between the departments of Teacher Education and Child and Family Studies’ faculty, and the students.  As a result, the ECE students responded positively to the question “I feel I received adequate academic advisement assistance” on the exit survey.  The ECE majors had not responded positive to this question in past surveys.  In addition, both the ECE and EC majors replied positively on “I would choose a major in Child and Family Studies if I could do it all over again.”

2. Both the 2004 and the current 2005 outcome assessments have indicated the ECE students do not believe they received opportunities to meet professionals within their field, or was exposed to professional organizations and publications within their field.  The department, through its student organization, has already begun to implement strategies to address this.  For example, the student organization is attempting to develop a student affiliate to the Utah Association for the Education of Young Children.  Additionally, the department was able to provide funds from a private donor to help send students to professional conferences.

3. The 2004 outcome assessment indicated the department needed to increase the student’s familiarity with anti-bias curriculum.  Because of this, the department implemented more direct instruction and modeling during student teaching.  Also, more information regarding anti-bias curriculum has been added to courses.  As evidenced by the Student Teacher evaluation, this curriculum change has been effective.

Recommendations

Based upon the following data the department may want to consider the following:

1. Although the data from the Student Teacher evaluation resulted in “Did not meet criteria” on student’s ability to tie their plans directly to the goals (outcomes) of the Children’s School, assessments, and individual children’s needs and interests; the data from Teacher Work Sample demonstrates that the student teachers are very good at using assessment data to profile student learning and communicate information about children’s progress and achievement.  These two results appear to be conflicting.  On one hand it appears that the student does not tie their plans directly to assessments but, on the other hand, they are good at using assessment data to profile student learning.  This suggests that the students may need assistance on ways to articulate how their plans have been incorporated into their teaching, rather than focusing on methods to develop the plans.

2. The department should consider assessing the curriculum to determine if there needs to be additional information/activities regarding how to articulate theoretical and DAP reasons for guidance decisions.

Summary of the results from the three assessments

1.  Exit Interview Questionnaire

The Exit Interview Questionnaire is sent to graduates in our three majors; Early Childhood (EC), Family Studies (FS), and Early Childhood Education (ECE).  This questionnaire has been sent out since 2003.  Although most of the questions have remained the same, an additional question (#15) was added for 2005.  This question states “The faculty has been helpful and supportive.”

The data listed below (Table 1) is taken from students who graduated in Spring/Summer 2005.  Fifty-one surveys were sent and 15 students returned them, which is almost a 30% return rate.  The students who returned the survey included 3 Early Childhood, 9 Family Studies and 3 Early Childhood Education.  Although there were only 3 students in both EC and ECE groups, statistical analyses were still conducted that examined all three groups.

The Exit Interview Questionnaire consists of 15 questions.  The first question asks for the student’s major.  Questions 2-15 ask the students to rate their experiences on a 7-point Likert style scale (1=strongly disagree, 7=strongly agree).  Averages were complied on each question, overall by major, and the overall group of students.  The overall average was 6.0.  The EC, FS, and ECE averages were 6.55, 5.99, and 5.47; respectively.

A review of each question indicates question 6 fell 2 STD below the mean.  Question 6 asks, “My experiences in Child and Family Studies gave me an opportunity to meet professionals in my field.”  Two questions  (#2 & 14) fell 1 STD below the mean.  Question 2 states “My experiences in Child and Family Studies prepared me for my chosen career.”  Question 14 states “I feel I received adequate academic advisement assistance.”

An examination of these three questions (2, 6, and 14) was conducted by major.  The FS major appears to have had the most concern with questions 2 and 14.  The ECE major expressed the greatest concern with question 6.

Questions 3 and 5 were both one STD above the mean.  Question 3 states “My experiences in Child and Family Studies provided me with new knowledge.”  Question 5 states “My experiences in Child and Family Studies allowed me to gain skills needed to be an educator of parents and/or children.”  Further examination of these questions indicate that the FS major is most satisfied with question 3, and the EC major is most satisfied with question 5.

A review of the averages by majors was also performed.  EC majors rated both questions 4 & 5 as “7”.  This indicates they “strongly agree” “My experiences in Child and Family Studies allowed me to gain skills needed for my future career”, and “My experiences in Child and Family Studies allowed me to gain skills needed to be an educator of parents and/or children.”

Their lowest rating, 5.67, was on question 14.  This score still reflects they “agree” that “I feel I received adequate academic advisement assistance.”

As previously stated, FS majors rated question 2 the lowest, 5.11.  Although this is the lowest of the 14 questions, it still reflects they “agree” that “My experiences in Child and Family Studies prepared me for my chosen career.”

The highest question rated by FS graduates was on question 3 (6.56).  This indicates they “strongly agree” that “My experiences in Child and Family Studies provided me with new knowledge.”

ECE majors’ overall averages were over 1 STD below the mean.  They also rated both questions 6 and 9 a 4.33, which is over 2 STD below the mean (mean=6).  A score of 4.33 falls between “Disagree” and “Agree”.  As previously stated, questions 6 states “My experiences in Child and Family Studies gave me an opportunity to meet professionals in my field.”  Question 9 states “My experiences in Child and Family Studies exposed me to professional organizations and publications in my field.”

ECE majors rated question 12 the highest, 6.33.  This question states “I would choose a major in Child and Family Studies if I could do it all over again.”

Table 1  Exit Interview Spring 2005

Implications

In summary, overall it appears that Early Childhood majors “strongly agree” that they had positive educational experiences.  Family Studies majors fall between “agree” and “strongly agree”.  And ECE majors “agree” their experiences were positive.

A comparison between the EC and ECE 2004 and 2005 respondents (FS was not included in the 2004 data) indicate the average for the 2005 EC graduates was higher (5.93 vs. 6.55), and the 2005 ECE graduates were lower (5.70 vs. 5), 2004 and 2005 respectively.

In 2004, the ECE majors rated questions 6, 9, and 14 one STD below the mean.  In 2005, they continued to rate 6 and 9 below the mean, but not item 14.  Item 14 was rated an average of 6. Item 14 states “I feel I received adequate academic advisement assistance”. This indicates the department has improved its student advisement, which was a goal.

However, the ECE majors continue to “disagree” that “My experiences in Child and Family Studies gave me an opportunity to meet professionals in my field”, and “My experiences in Child and Family Studies exposed me to professional organizations and publications in my field”.

It is interesting that the EC and ECE majors share the same classes within our department, but had different opinions regarding their experiences.  The difference between the majors is that ECE students take courses in the department of Education.  It is not known if the differences in the responses are a function of this difference.

Finally, question 15, “The faculty has been helpful and supportive”, was rated an average 6.13.  The individual majors averages were EC 6.33, FS=6.22, and ECE=5.67.  This suggests that the graduates from all of the department’s majors “agree” the faculty is “supportive”.

2. Student Teaching Rating Scale

The Student Teacher Evaluation rating scales has been used for four years.  During spring semester, 2003, the department revised this assessment to reflect NAEYC/NCATE’s preference for a three level rating scale.  In 2004, in order to obtain more information related to NAEYC’s professional standards, the tool was revised again by increasing the total number of items to 24.  In the summer of 2004, an additional 12 items were added.  This brings the current form to a total of 36 items.

The professional lead teacher in the Melba Lehner Children’s School administers this assessment based on the student teacher’s performance. The questions continue to be clustered into 5 areas; Guidance, Planning, Family, Teamwork, and Personal. The form uses a 3 point criteria; Exceeds Criteria = 3, Meets Criteria = 2, Doesn’t Meet Criteria = 1. A minimum score of 2 is required for passing.

Eighteen student teachers were evaluated in 2005.  As indicated in Table 2, the overall average of the 18 students was 2.39, which is almost identical to the 2003-2004 student teacher ratings of 2.4. These scores indicate that the students continue to meet the competency requirements established by the department.  These scores were also examined by major, as shown in Table 2. The Early Childhood major obtained an average of 2.38.  The Early Childhood Education (ECE) major received an average of 2.39

An examination of the averages for each of the 36 individual questions demonstrates that the students received a “Pass” for all of them except item #12 (AVG=1.94).  Item #12 reflects the student teachers’ ability to tie their plans directly to the goals (outcomes) of the Children’s School, assessments, and individual children’s needs and interests.  An inspection of the averages for each major on this question reveals the EC students received a Pass (AVG=2.0), whereas the ECE majors “Doesn’t Meet Criteria” (AVG=1.89).  Furthermore, it was only 1 student, out of 18, that did not meet the criteria.  In other words, 17 out of the 18 students passed this area.

The next lowest average for both majors was on question #3 (AVG=2.0).  Question #3 asks the supervising teacher to rate the student teacher’s ability to articulate theoretical and DAP reasons for guidance decisions.  Additionally, ECE majors received an average of 2.0 on question #14.  This question asks the supervising teacher to rate if the student teacher’s plans are consistent with current early childhood theory and practice. 

The 2004 outcome assessment indicated the department needed to increase the student’s familiarity with anti-bias curriculum.  Because of this, the department implemented more direct instruction and modeling during student teaching.  Also, more information regarding anti-bias curriculum has been added to courses.  This curriculum change appears to have been successful in addressing this issue.  Question 18 addresses this area.  It asks if the student teacher “weaves anti-bias strands which emphasize culture, race, gender and/or handicapping conditions”.  The overall average was 2.28, which places it above the minimum requirement of 2.

The question that received the highest average score was #31.  This question falls under the Teamwork area.  The average score on this question (2.78) indicates that the student teachers go beyond minimal requirements and/or are consistently prepared ahead of time.

Table 2  Student Teaching Rating Scale Spring 2005

Implications:

A comparison of the results from Fall 2003 and Spring 2004 with Spring 2005 reveals the students’ scores were relatively similar in 4 of the 5 areas.  The Family area dropped slightly.  It still received a “Pass”.

A comparison of the overall averages between the EC and ECE majors for Spring 2005 does not reveal any clear differences that could not be accounted for by error.  The EC majors scored highest on Guidance and lowest on Family.  The ECE majors scored highest on both the Guidance and Personal areas, and lowest on the Family.

Within the individual questions, the faculty may want to discuss why the students’ scored low on both their  ability to tie their plans directly to the goals (outcomes) of the Children’s School, assessments, and individual children’s needs and interests; and articulating theoretical and DAP reasons for guidance decisions.  Additionally, ECE majors may need more information on developing plans that are consistent with current early childhood theory and practice.  Because the students do very well on virtually all of the areas, it may be that the problems are more in being able to articulate what they are doing, rather than the implementation of it.

In summary, the 18 student teachers that were evaluated in 2005 received an average of “Pass” in the 5 areas.  This data appears to support the department’s instructional programs as it pertains to guidance, developmental planning, respecting and supporting families, cooperative and collaborative teamwork, and ethical and considerate teaching practices.  Additionally, this information indicates that the student teachers go beyond minimal requirements and/or are consistently prepared ahead of time, which is a strong complement to the EC and ECE majors.

3.  Early Childhood/Early Childhood Education Teacher Work Sample (TWS)

Each student teacher in the Children’s School submits Work Samples (TWS) that correspond to the following six rubrics: contextual factors; objectives/intended learning outcomes; assessment plan; design for instruction & instructional decision-making; analysis of student (child) learning; and reflection & self-evaluation. The TWS particularly addresses the students’ ability to meet NAEYC’s Standard 4c, understanding content knowledge in early education, and 4d, building meaningful curriculum.

The Teacher Work Samples are based on the educational activities the students had implemented during their student teaching. The instructor for the Advanced Planning and Guidance course (CHFAM 4710) reviews the Work Samples and rates them on a 3-point scale; exceeds criteria (3), meets criteria (2), and doesn’t meet criteria (see example below).  The Advanced Planning and Guidance course (CHFAM 4710) is taken by the student teacher concurrently with student teaching.

The Candidates Work Samples need to contain items that relate to the rubrics, as listed below:

1. Contextual Factors: The teacher uses information about the learning-teaching context and student individual differences to set learning objectives and plan instruction and assessment.
2. Objectives/Intended Learning Outcomes:  The teacher sets a measurable objective based on the Children’s Schools goals.
3. Assessment Plan:  The teacher uses assessment tools aligned with the objective to assess student learning before, during, and after instruction.
4. Design for Instruction & Instructional Decision-Making:  The teacher designs instruction from assessments, taking into account students needs and interests.  The teacher uses ongoing analysis of student learning to make instructional decisions.
5. Analysis of Student Learning:  The teacher uses assessment data to profile student learning and communicate information about student progress and achievement.
6. Reflection and Self-Evaluation:  The teacher reflects on his or her instruction and student learning in order to improve teaching practice.

Table 3 lists the average scores and if the criteria were passed by the 19 students.  A score of 2 or higher reflects meeting or exceeding the criteria.

Table 3  Spring 2005 Teacher Work Samples

As indicated in Table 3, the average score of the 19 student teachers resulted in an overall “Pass” (avg=2.15).  Additionally, the overall scores on each of the three majors received a “Pass”.   The average scores for each of the 6 areas resulted in “Pass” for 5 of the areas, and a “Does not meet criteria” for one area, “Design for Instruction and Instructional Decision Making” (avg=1.97).

An examination of each area by each of the three majors reveals that only the EC major did not meet the criteria for the “Design for Instruction and Instructional Decision Making” (avg=1.89), but the ECE and Dual majors did meet the criteria (avg=2.06 & 2.00, respectively).

Implications

The data in Table 3 indicates that the student teachers met the necessary criteria and standards for virtually all of the areas.  This demonstrates they understand the content knowledge in early education and can build meaningful curriculum. 

The only area that falls below the minimum is for the Early Childhood majors in the area of “Design for Instruction & Instruction Decision Making”.  This area was also not met in the 2004 outcome assessment for TWS. 

The “Design for Instruction & Instruction Decision Making” attempts to assess the student’s ability to design instruction from assessments, taking into account individual student’s needs and interests.  Interestingly, the information gathered from the Student Teacher Evaluations (discussed above) indicates that students had difficulty demonstrating their ability to tie their plans directly to the goals (outcomes) of the Children’s School, assessments, and individual children’s needs and interests. Therefore, the department should examine the curriculum to determine if this area should be better addressed.

The highest area was in “Analysis of  Student Learning” (avg=2.36).  This area suggests that the student teachers are very good at using assessment data to profile student learning and communicate information about children’s progress and achievement.  The fact that the students are good at using assessment data to profile student learning, but score low on relating their plans directly to the goals, assessments and individual children’s needs suggests that the problem may involve articulating how they arrived at deciding on their curriculum and plans, rather than on implementation.

Mission Statement / Student Learning Outcomes / Curriculum Grid / Assessment Plan / Contact Person