The 2004-2005 Assessment committee (Eric Amsel, Chair, Teri Kay, Leigh Shaw, and Joe Horvat) convened in early September, 2004 to set the agenda for the year.  Following up on last year's results, the goal of this year’s assessment was to more carefully index the actual impact of students' education in the psychology department on their thinking and reasoning about the discipline. Two studies were initiated on three aspects on different aspects of the psychology curriculum. Both studies were run by students with faculty supervision and both were presented at the WSU Undergraduate Research Symposium.  The first study examined the impact of majoring in psychology on students’ theoretical thinking about the discipline.  The second examined the impact of a major and minor in psychology on students' ethical and methodological reasoning. Two aspects of the curriculum which is the focus of the present research (theoretical, methodological reasoning) are central components the department’s general learning goal to have students “think like psychologists”.  The third aspect (ethical reasoning) is also part of our general learning goal but one separate from the other two.  At the present, there is no ethics course in the curriculum so the learning outcome is more of a wish than a real curricular goal.
 

 

---

 

Study 1:  Understanding Scientific Psychology
Eric Amsel and Brock Frost

Last year's departmental assessment (Psychology Assessment, 2004) demonstrated that WSU students enter into college classes with an intuitive theory about the nature of the discipline. This intuitive theory (called Folk Psychology) is an inherently unscientific account of behavior in terms of conscious mental states (D’Andrade, 1987). Folk Psychology is inconsistent with Scientific Psychology, which is an account of behavior based on the influence of genetic, biological, cognitive, and socio-cultural forces. The WSU Psychology curriculum in the psychology department is based on scientific psychology and its tenets is distinct from those of Folk Psychology, as presented in Table 1. 

Table 1:  Tenets of Folk and Scientific Psychology

The departmental assessment (Department Assessment, 2004) reported that Psychology students more strongly distinguish between the two theories than do students in other disciplines (Science and Arts & Humanities). The implication is that the two forms of explanation conceptually coexist.  That is, although students embrace the tenets of Scientific Psychology, that they do not reject those of Folk Psychology, despite seeing the two as antagonistic.

Following up on this previous work the present study was designed to explore how Psychology and other students think about how human beings are like animals (i.e., great apes) and machines (i.e., computers). Folk Psychology denies such similarities.  In contrast, in Scientific Psychology such commonalties are based more on relational analogies than literal similarities (Gentner & Woolf, 2000). For example, beyond any literal similarity, human beings, like computers, are thought to compute, store, and retrieve information. This is the foundation of Cognitive Psychology, which is a central theoretical orientation in psychology.  Similarly, humans, like animals, are thought to evolve species-specific behavior which may not be literally similar. This is the foundation for Comparative and Evolutionary Psychology, which are also theoretical orientations in the discipline.  So the present work examined whether compared to others, Psychology students would be more likely to judge commonalities between humans, animals, and machines based on relational analogies than literal similarities.

Method

A sample consisted of 30 upper-class (seniors or juniors) and 15 underclass (freshmen and sophomore) students who were majoring in Psychology, Science or Arts and Humanities  Participants’ judgments and justifications of the similarity of humans and animals and humans and machines were assessed in an interview format. The interview began with general questions regarding the similarity between humans and animals or machines. Other questions were also posed, but only the results from the general questions are reported. Participants were asked, “To what extent are human beings similar to animals (e.g., great apes)/machines (e.g., computers)?” Participants recorded their ratings on a 7-point scale, anchored by Not at all Similar (1), A Little Similar (2), Somewhat Alike (3) Moderately Similar (4), A Good Deal Alike (5), Very Alike (6), Identical (7).

Participants were probed about their ratings of entity similarity with follow-up questions such as, “Why do you say that they are _________?”, “What makes them ______?”, and “What exactly are you saying is alike about humans and _______?” The probes were designed to elicit justifications for participants’ similarity ratings. The justifications were coded as literal similarity (elements of one entity are also found in the other) or relational analogy (relations between elements in one entity are also found in another) based on Gentner and Woolf (2000) (see Table 2). An intermediate code was also established for cases with elements of each justification (partial analogy). Inter-rater reliability based on all the responses of 30% of the participants was 96%.

Table 2:  Justifications of Similarity Ratings

Results

Two analyses tested the prediction that Psychology students would be more likely than others to judge commonalities between entities based on relational analogies. The first analysis addressed similarity ratings and showed that Science students judged greater similarity than Humanities and Psychology students in the Animal condition but there was no Groups difference in the Machine condition (see Figure 1)

Figure 1:  Similarity Ratings by Groups and Entity

 
 

The second analysis addressed differences in response justifications and showed Psychology students had higher justification responses than Humanities and Science students in the Animal condition and equally high scores as Science students, compared to Humanities students, in the Machine Condition

Figure 2:  Justifications by Groups and Entity

Discussion

It was predicted that compared to others, Psychology students would be more likely to judge commonalities between humans, animals, and machines based on relational analogies than literal similarities.  The prediction was confirmed most strongly for Human/Animal comparisons. Although, compared to Science students, Psychology students judged less similarity between Animals and Humans, they justified their similarity judgments with relational analogies more so than others.

Such a pattern is consistent with the claim that Psychology majors continue to hold onto the tenets of Folk psychology, with its assumption of the uniqueness of human beings, despite adopting those of Scientific psychology with its relationally-based continuity between human beings, animals, and computers.

---

Study 2: Methodological and Ethical Reasoning Among Psychology Students
Eric Amsel, Teri Kay with Russell Riding and Calvin Tang

This study explores whether undergraduate psychology students learn methodological and ethical reasoning skills from their intensive educational training in psychology, broad educational training outside the department, or from both sources.  Methodological reasoning refers to planning, conducting, and interpreting research. Specific skills include generating hypotheses, managing variables by operationally defining, controlling and manipulating them, designing studies, and evaluating results.

Methodological reasoning may be acquired by students through their intensive educational training as psychology majors or minors as each class in the curriculum addresses methodological skills (Psychology Assessment, 2004).  However, such skills may also arise through broader educational training as many other courses outside of psychology also teach methodological reasoning skills.

 Ethical reasoning refers to the ability to recognize and react to cases of unfairness, subjugation, and injustice.  Specific ethical reasoning skills include appreciating multiple perspectives (i.e., victimizer and victim), empathizing with victims, and reacting appropriately. Like methodological reasoning, ethical reasoning skills may be acquired by students majoring or minoring in psychology or through broader educational training.

To test whether methodological and ethical reasoning skills are acquired by psychology students inside or outside the discipline, students in lower- and upper-division psychology courses completed questionnaires addressing these skills.

Method

The sample included 331 students (46% male, 54% female) in lower- and upper division psychology courses, who received extra credit for participating. Overall, most were Freshmen (49%) with the other statuses equally divided.  Of those expressing a major (68%), 30% were Psychology majors and of those expressing a minor (57%), 22% were Psychology minors. Of those not who had yet to select a major or a minor, a small percentage expressed a strong likelihood of choosing to major (15%) or minor (9%) in psychology. 

A subgroup of Psychology Students was composed of students who were Psychology majors and minors and those self-identified as likely to become so. There were 114 Psychology Students (36.5% male, 63.5% female) with 27 Freshmen, 14 Sophomores, 24 Juniors, and 39 Seniors. The larger number of Freshmen and Seniors than Sophomores and Juniors is in keeping with the distribution of students in the rest of the university. The students differed in expected ways as a function of student status (e.g., Age, Number of Psychology Courses Completed) and did were the same in expected ways as a function of status (Gender, GPA).

Each participant completed a 30 minute questionnaire in their classroom. The questionnaire contained three parts in a fixed order: Demographics, Ethics assessment, and Methodological Reasoning assessment.  The demographics questions included Age, Sex, GPA, Student Status, ACT or SAT scores (so few were completed that these variables were dropped), Major (or failing that, the certainty that the student would major in Psychology), Minor (or certainty of the student would minor in Psychology), and total number of psychology courses taken. Methodological reasoning was tested by the 36-item Test of Integrated Process Skills (TIPS) (Dillasaw et al., 1982). Internal consistency of the test (Cronbach's alpha) was previously determined to be 0.89.  Ethical reasoning was tested by a 10-item Ethical Reasoning test which was developed for this study.  The students made judgments of the ethics of various academic situations and their judgments of degree of “offensiveness”. Some of the scenarios depicted unethical behavior of student or faculty while others depicted perfectly accepted behavior. 

Results

The data were analyzed in a three-step process which reflects increasingly conservative tests of whether changes in students’ Methodological and Ethical reasoning are related directly to their training in psychology.

    Step 1:  The first step assessed whether there were changes over student status in task performance. The total frequency of correct responses on the TIPS was shown to be affected Student Status (Freshmen, Sophomore, Junior, and Senior).  Performance on the Ethics task was not significant, but Freshmen (M=7.0 out of 10) had significantly lower Ethics scores than did Seniors (M=7.54), t(309)=56.36, p<.001.  

    Step 2: The data from Step 1 suggest that TIPS and Ethical test scores change over student status.  However, because most of the Juniors and Seniors but few of the Freshmen and Sophomores were Psychology majors or minors (with only a few of the former interested in becoming so), the comparison may have been flawed. The next step was to correlate task TIPS and Ethics test scores directly to students’ number of psychology courses, independently of Age, Sex, and GPA. This analysis allowed for the assessment of whether increases in scores are directly related to the number of psychology courses taken, irrespective of other factors. TIPS score was significantly related to psychology courses taken (r= .26, p<001), but Ethics score was not (r=.04, ns) suggesting that only Methodological Reasoning was affected by training in Psychology.  

    Step 3:  The correlational data in Step 2 includes some Freshmen and Sophomore students who are very interested in becoming Psychology majors or minors and many others who are disinterested.  The third analysis examined only those students in the sample who are Psychology majors or minors or who are interested in becoming so. Only a third of the sample (N=114) could be categorized as a Psychology Student. Their TIPS performance was analyzed by student status, controlling for Age, Sex, and GPA.  Three aspects of their performance were analyzed:  Overall Total Score, the percentage of students who score in the top third overall (Figure 3), and the subscales of the TIPS, only three of which were significant (see Figure 4).  The data show that competence in methodological reasoning increases dramatically during junior year, perhaps by consolidating three specific methodological reasoning skills, which were strongly correlated:  Identifying independent, dependent and extraneous variables, articulating appropriate hypotheses, and designing effective tests of particular hypotheses.

Figure 3:  The Percentage of Psychology Students Scoring in the Top Third on the TIPS Test by Student Status.

 

 

 

 

 

 

 

 

Figure 4: Average Percent Correct of Psychology Students on Selective TIPS subtests, by Student Status.

 

 

 

 

 

 

 

 

 

Discussion

Undergraduate education is a complex marriage between learning intensively from a discipline in which one is a major and minor and learning broadly from General Education courses. This complicates identifying the source of changes in students’ ideas, abilities, and skills. The present study examined the source of change in two such skills: Ethical and Methodological reasoning.  

Ethical reasoning did show a moderate change, at least between Freshmen and Seniors in the entire sample. While the change was small, it reflects students’ recognition of the ethical complexity of college life. Further research exploring students’ ethical insights into academic life seems necessary as even seniors averaged only 7.5 correct out of 10 ethical dilemmas.  However, academic ethics was not specifically or uniquely learned in the discipline of psychology.  There was no relation between Ethics score and the number of psychology courses taken. While psychology class may provide training in the nature of academic ethics (cheating, deception, etc.), it is certainly not an exclusive source of such knowledge to students who are taking psychology classes.

In comparison, increases in Methodological Reasoning performance directly correlated with the number of Psychology courses taken.  Moreover, Methodological competence is acquired specifically in junior year, when students generally being to take a critical mass of courses in their discipline. The significant jump in above average performance on the TIPS tests during junior year is likely the result of students integrating together the multiple methodological skills being learned at the time across a variety of psychology courses.

General Discussion

Overall, the two studies demonstrate the significant impact that the psychology curriculum has on students’ discipline-related conceptual and methodological reasoning.  These findings suggest that psychology majors and minors are learning to  “think like a psychologist”, which is the central learning outcome of the department.  Moreover, the data provide further confirmation of the findings from the alumni survey (Psychology Department Assessment, 2004) which showed that most students who graduated from the department are satisfied or very satisfied with the academic training that they received in the department.

The department needs to address the weakness of ethics in the curriculum.  Although part of a number of courses, it is certainly not as regularly or as deeply presented in classes as is methodological or theoretical reasoning. 

References

D'Andrade, R. G. (1987) A folk model of the mind. In D. Holland and N. Quinn (Eds.) Cultural Models in Language and Though (pp. 112-148). Cambridge UK: Cambridge University Press.

Dillashaw, F. G., and Okey, J. (19802). Test of the integrated science process skills for secondary science students. Science Education, 64, 601-608.

Gentner, D. & Wolff, P. (2000). Metaphor and knowledge change. In E. Dietrich & A. Markman (Eds.), Cognitive dynamics: Conceptual change in humans and machines (pp. 295-342). Mahwah, NJ: Lawrence Erlbaum Associates.

Psychology Department Assessment (2004). http://departments.weber.edu/psychology/assessment/ASSESSPAGE.htm