Weber State University   

Psychology

Results of Assessment

2007-2008 (submitted 08/19/08)

Two assessments of psychology students were performed in 2007-2008. The first assessment was of WSU psychology students’ ACT scores. The purpose was to assess whether WSU psychology students have the cognitive abilities to handle a rigorous curriculum, particularly one which requires a scholarly capstone course. Although the evaluation was not of the impact of the curriculum on students, the analysis provides data necessary to tailor an appropriate curriculum to the abilities of our students.

The second assessment was a survey of students in psychology classes regarding their beliefs about the scientific status of the discipline. This assessment directly addresses a central goal of the department: To promote students learning to think like psychologists by adopting a scientific view of the discipline (see Knowledge under Department Goals). 

Study 1

The department recently discussed adding a capstone requirement to the curriculum in which students would be expected to complete a 20 page scholarly paper. The debate about the proposal centered, in part, on whether the demands of a capstone are within the cognitive capacities of WSU psychology students. To assess WSU psychology majors’ cognitive capacities we collected their ACT (American College Test) test scores. Such scores have been found to correlate with measures of general intelligence (Booth, 1983; Koenig, Frey, & Detterman, 2008; Lewis & Johnson, 1985). There is a broad consensus in educational psychology that general intelligence, academic skills, and reasoning ability are connected in important way (c.f., Perkins & Grotzer, 1997), suggesting that general intelligence may index students’ ability to benefit from a rigorous curriculum. Moreover, the ACT Corporation has identified benchmark ACT test scores which predict grades in various college classes (ACT, 2007), offering an even more specific assessment of students’ capacities.  

The Assessment Committee examined the ACT scores of psychology majors and compared them to the ACT scores of psychology majors at the University of Utah, the flagship institution in the state, which has a well recognized and rigorous curriculum. WSU students’ scores were also compared to national and state ACT test norms and to benchmark scores for university-level performance in Social Science, Mathematics, and Composition.   

ACT Mathematics, English, Reading, and Composite scores of psychology majors were collected from the Office of Institutional Research from the University of Utah and Weber State University. Each office provided average scores, standard deviations, and sample size of psychology majors, which are presented in Table 1. Table 1 also reports the ACT test performance of 2007 high school graduates nationally (http://www.act.org/news/data/07/pdf/National2007.pdf) and from Utah (http://www.act.org/news/data/07/pdf/states/Utah.pdf).

Table 1: ACT scores

Separate t-tests revealed that University of Utah psychology students had higher ACT Mathematics t(468) = 3.16, p < .01, English t(468) = 3.23, p < .01, Reading t(461) = 2.15, p < .05, and Composite t(466) = 3.31, p < .01 scores. The standard deviations were homogeneous and no school had a consistently higher standard deviation than the other. The findings suggest that although different in means, there good deal of overlap between the distribution of scores. 

Weber State psychology students’ Mathematics, English and Composite scores were no different from State and National averages. However, their Reading scores were higher than the State t(22,227) = 2.30, p < .05 and National t(1,300,219) = 2.90, p < .001 averages. University of Utah students ACT scores were higher than state and national averages (all ts > 2.07, p < .05).

The ACT Corporation benchmarks are empirically derived minimum scores on an ACT test that indicate a 50% chance of obtaining a B or higher or about a 75% chance of obtaining a C or higher in corresponding credit-bearing college courses. The relevant courses for the present analysis include English Composition (benchmarked by ACT English score of 18), Algebra (benchmarked by ACT Mathematics score of 22) and Social Science (benchmarked by ACT Reading score of 21). WSU psychology students have significantly higher scores than the ACT benchmark Reading score, t (227) = 11.00, p <.001 and English score, t (220.0) = 5.62, p<. 001, but significantly lower score than the benchmark Mathematics score t (227.0) = -4.80, p < .001. 

The findings suggest that although not performing at the same level as psychology majors at the University of Utah, Weber State psychology students; ACT scores are no different than national and state-wide averages. Weber State’s open enrollment policy likely explains why WSU students’ scores are no different than the state average. The benchmark data suggest that WSU psychology majors are above minimal standards in reading and composition skills, but below such standards in mathematics. Clearly a range of capstone classes should be offered which are all rigorous and demanding but do not all require advance mathematics-related skills, likely tapped by research capstones and some senior seminars.  

Study 2

To assess WSU psychology students’ beliefs about the scientific nature of the discipline, they were asked to complete the Psychology as a Science (PAS) questionnaire (Friedrich, 1996). The questionnaire was shown to have good measurement reliability, with internal consistency (Cohen’s Kappa) scores ranging from .7 to .8. Also, the scale was shown to have measurement validity. For example, Friedrich reported that Research Methods students at the end of the semester scored higher (M=5.55 on a 7-point Likert scale, reflecting a stronger belief in psychology as a science), than the same students at the beginning of the course (M=5.30) and different students from an introductory psychology course who were tested at the end of the semester (M=5.11).

The present study assessed students’ beliefs about the scientific status of psychology as a function of their Year in School (Freshman, Sophomore, Junior, and Senior) and Major Status (Potential or Actual Major, Potential or Actual Minor, and Neither Potential nor Actual Major or Minor). If the psychology curriculum impacts students’ beliefs about the scientific nature of the discipline, we would expect both Year in School and Major Status to predict PAS score. Specifically, we would expect higher PAS scores among upper- than lower-division students and among majors than non-majors.  

A total of 438 participants completed the PAS questionnaire (see Appendix) during the first 6 weeks of classes in the Spring 2008 semester. To insure the sampling of students from all Years in School (freshmen to seniors) and Major Statuses, the questionnaire was distributed to a range of classes. The distribution of participants by classes is presented in Table 2.

Table 2:  Distribution of Students by Classes

The distribution of participants by the two independent variables is presented in Table 3. Students’ Major Status was determined by their answers to two questions on the PAS questionnaire. Participants were asked whether they were a psychology major, minor, or neither a psychology major nor minor and (if they were neither) whether they were planning to become a psychology major, minor or neither a psychology major nor minor. Students who were psychology majors and those who were planning to become a psychology major were placed into the Potential or Actual Psychology Major group. Students who were psychology minors and those who were planning to become a psychology minor were placed into the Potential or Actual Psychology Minor group. Participants who were neither psychology majors nor minors and were not planning on becoming so were placed into the Neither Potential Nor Actual Psychology Major or Minor group. 

Table 3:  Distribution of Students by Major Status by Year in School.

Cohen’s Kappa for the 15 PAS items was .83, which indicates a very reliable test. Students’ PAS scores were positively correlated with the number of psychology courses taken, r (N = 414) = .21, p <.001. This finding suggests that the more psychology courses to which WSU students are exposed, the more they come to believe that psychology is scientific. 

A more systematic analysis of the impact of the curriculum on PAS scores was performed by subjecting the scores to a 4 (Year in School) by 3 (Major Status) ANCOVA, with sex and number of psychology classes as covariates.  Sex was treated as a covariate because there were fewer female (N=26) than male (N=51) Potential or Actual Minors, whereas the other categories revealed fairly equal distributions of males and females. The number of psychology classes was treated as a covariate because senior majors had substantially more courses than other groups.

Results revealed a main effect of Year in School, F (3, 396) = 3.80, p < .05, and Major Status, F (2, 396) = 18.74, p < .001. As can be seen in Figure 1, Bonferroni post-hoc tests revealed that Potential or Actual Majors had significantly higher PAS scores than the other two groups, and that Seniors had significantly higher PAS scores than Freshmen. Taken together, these findings suggest that those particularly interested in the discipline (majors) and those who are more advanced in their university career (seniors) have the strongest beliefs in the scientific nature of the discipline.

Figure 1: PAS scores by Year in School and Major Status

References

ACT (2008). The college readiness report. Downloaded, August 8, 2008 from http://www.act.org/news/data/08/index.html

Booth, R. (1983). An examination of college GPA, composite ACT scores, IQs, and gender in relation to loneliness of college students. Psychological Reports, 53, 347-352.

Friedrich, J. (1996). Assessing students’ perceptions of psychology as a science.  Validation of a self-report measure.  Teaching of Psychology, 23, 6-13.

Koenig, K. A., Frey, M. C., & Detterman, D. K. (2008). ACT and general cognitive ability.  Intelligence, 36, 153–160.

Lewis, M. L., & Johnson, J. J. (1985).  Comparison of WAIS and WAIS-R IQs from two equivalent college populations. Journal of Psychoeducational Assessment, 3, 55-60.

Perkins, D.N., & Grotzer, T.A. (1997). Teaching intelligence. American Psychologist, 52, 1125-
      1133.

Appendix 1:  PAS Questionnaire

Mission Statement / Student Learning Outcomes / Curriculum Grid / Assessment Plan / Contact Person