Assessing the Impact of Study Abroad Joel D. Frederickson, Ph.D. Associate Dean of Institutional Assessment & Accreditation Professor & Chair, Psychology
Introduction Study abroad is considered by many to be a high impact educational practice. IES Abroad 50-year alumni survey found study abroad positively impacts Academic attainment (Mohajeri Norris & Gillespie 2008) Cultural development (Mohajeri Norris & Dwyer 2005) Career decisions (Peters 2004) Personal growth (Dwyer & Peters 2004)
Assessment of Three Cognitive Domains Dogmatism Anti-intellectualism Need for Cognition
What is Dogmatism? a relatively unchangeable, unjustified certainty (Altemeyer, 1996) Closed-mindedness An attempt to measure closed-mindedness regardless of political ideology.
Impact of Study Abroad on Dogmatism Intuitively, many of us would expect study abroad experiences to help students increase their worldview and reduce dogmatism. Living in another culture Bumping up against others who see the world differently IES Alumni survey 98% said their study abroad experience Helped me better understand my own cultural values and biases
Measuring Dogmatism Altemeyer s DOG scale: 20 items, 10 pro trait & 10 con trait Sample items The things I believe in are so completely true, I could never doubt them ; I am so sure I am right about the important things in life there is no evidence that could convince me otherwise ; and The people who disagree with me may well turn out to be right (reverse scored).
Reliability Altemeyer (2002) reports that the 190 interitem correlations are typically.30 and the alpha coefficients are around.90. Others have confirmed the strong alpha levels, with alphas from five samples ranging from.88 to.93 (Crowson 2009; Crowson, DeBacker, and Davis 2007).
Validity Changing Attitudes Toward Homosexuals Contending Bible is Free of Errors & Inconsistencies Among Christians Archeological Study
Research Question Do study abroad experiences result in lower dogmatism among students?
Year 1: Method We gathered data on dogmatism for two consecutive years. Dogmatism Items included on Assessment Day online institutional survey. Non-equivalent control group design Year in School: Served as a covariate Item that asked about study abroad experience: Studied Abroad Definite Plans to Study Abroad No plans to study Abroad
Year 1: Participants 760 students participated in the larger survey; 66% female Of these, 737 students completed both the dogmatism scale and the study abroad demographic item.
4.1 4 3.9 3.8 3.7 3.6 3.5 3.4 3.3 3.2 Sample Data for No Significant Impact of Study Abroad on Dogmatism Studied Abroad Plans to Study Abroad No Plans to Study Abroad
4.1 4 3.9 3.8 3.7 3.6 3.5 3.4 3.3 3.2 Sample Data for Significant Impact of Study Abroad on Dogmatism Studied Abroad Plans to Study Abroad No Plans to Study Abroad
Year 1: Results The covariate, Year in school, was significantly related to Dogmatism, F(1,733) = 8.45, p=.004 Controlling for year in school, students who had a study abroad experience had significantly lower dogmatism scores than students in the two comparison groups, F(2,733) = 7.41, p=.001.
4 Year 1: Raw & Adjusted* Mean Dogmatism Scores by Study Abroad Experience 3.9 3.8 3.7 3.792 3.874 3.6 3.5 3.4 3.3 3.551 Raw Adjusted 3.2 Studied Abroad Plans to Study Abroad No Plans to Study Abroad *Adjusted Means Control for Year in School
Year 2: Results The covariate, Year in school, was significantly related to Dogmatism, F(1,688) = 8.15, p=.004 Controlling for year in school, students who had a study abroad experience had significantly lower dogmatism scores than students in the two comparison groups, F(2,688) = 6.92, p=.001.
4 Year 2: Raw & Adjusted* Mean Dogmatism Scores by Study Abroad Experience 3.9 3.8 3.7 3.81 3.881 3.6 3.5 3.4 3.3 3.561 Raw Adjusted 3.2 Studied Abroad Plans to Study Abroad No Plans to Study Abroad *Adjusted Means Control for Year in School
Conclusion A study abroad experience does appear to have a significant impact on lowering dogmatism in college students. This is certainly one of the many intended benefits of a study abroad experience: broader worldview, greater tolerance, more openness to different ways of thinking and seeing the world.
Anti-intellectualism Another year the Anti-intellectualism Scale was included in the Assessment Day Institutional Survey. Given the IES Alumni survey, we would expect less anti-intellectualism from those who have had a study abroad experience. 80% said study abroad Enhanced interest in academic study
Anti-intellectualism Scale Hook (2004) created this scale to assess the anti-intellectual attitudes of college students, particularly as they relate to educational activities. Hook (2004) provides good evidence for the reliability and validity of this scale.
Anti-Intellectualism Sample Anti-intellectualism items: I would rather just pay money for a diploma than have to take so many useless courses. A big reason I m in college is that I value learning for its own sake. I see college as a necessary evil it s the price I have to pay to find a good job.
Results The covariate, Year in school, was NOT significantly related to Anti-intellectualism, F(1,930) = 2.07, p=.150 Controlling for year in school, students who had a study abroad experience had significantly lower anti-intellectualism scores than students in the two comparison groups, F(2,930) = 25.59, p<.001.
95 90 Raw & Adjusted* Anti-intellectualism Means by Study Abroad Experience 90.649 85 84.813 80 Raw 75 77.41 Adjusted 70 65 Studied Abroad Plan to Study Abroad No Plans to Study Abroad *Adjusted Means Control for Year in School
Conclusion Study abroad experiences do appear to decrease anti-intellectualism in students. This converges with the IES alumni survey results that indicate study abroad experiences increase student interest in academics.
Need for Cognition During another Assessment Day Survey we used the Need for Cognition Scale. We expected increases in need for cognition based on previous research that indicates that study abroad experiences are cognitively demanding (Rourke 2012).
Need for Cognition This refers to the tendency for an individual to engage in and enjoy effortful cognitive activities (Cacioppo & Petty 1982). An individual high in need for cognition tends to seek, acquire, and process complex information to make sense of the world. Individuals low in need for cognition rely on others, simple cognitive heuristics, or social comparison processes to understand the world. This scale has been widely used and the authors have provided strong evidence for the reliability and validity of this measure (Cacioppo, Petty & Kao 1984).
Need for Cognition Sample items include: I find satisfaction in deliberating hard and for long hours The notion of thinking abstractly is appealing to me Thinking is not my idea of fun (reversed scored)
Results The covariate, Year in school, was significantly related to Need for Cognition, F(1,1234) = 23.51, p<.001 Controlling for year in school, students who did not and plan not to study abroad had significantly lower need for cognition scores than students in the two study abroad groups, F(2,1234) = 7.72, p<.001.
4 Raw & Adjusted* Need for Cognition Means by Study Abroad Experience 3.5 3.421 3.452 3.305 3 Raw Adjusted 2.5 2 Studied Abroad Plan to Study Abroad No Plans to Study Abroad *Adjusted Means Control for Year in School
Conclusion Not enough evidence to say that study abroad experiences increase need for cognition.
Overall Conclusions Study abroad experiences appears to significantly reduce dogmatism and antiintellectualism in students. This is after controlling for year in school and compared to students who plan to study abroad but haven t done so yet. The impact on anti-intellectualism was particularly pronounced.
Further Study Using pre/post designs. Examining the impact of various types of programs Length of program Region Inclusion of homestay