Are data and statistics bulletproof for informing educational policy— objective and not subjective? This is an eternal debate between the positivist and the constructivist. For context, I am a constructivist —a reformed positivist who took 8 statistics courses during my doctoral program.
To enter the constructivist v. positivist fray, I introduce a new peer-reviewed paper that will soon be published in the prestigious American Journal of Education entitled A Story within a Story: Culturally Responsive Schooling and American Indian and Alaska Native Achievement in the National Indian Education Study. Click here for the paper.
Citation: López, F., Vasquez Heilig, J. & Schram, J. (2013). A Story within a Story: Culturally Responsive Schooling and American Indian and Alaska Native Achievement in the National Indian Education Study, American Journal of Education.
Abstract: The National Indian Education Study (NIES) is the only large-scale study focused on American Indian and Alaska Native (AIAN) students’ cultural experiences. Given that the study also includes achievement on the National Assessment of Educational Progress (NAEP), it has the potential to inform and guide policy directed specifically toward AIAN students. In consideration of the call to increase quantitative studies examining the role of Culturally Responsive Schooling (CRS) on AIAN achievement as well as the recent recommendations by the expert panel to the National Assessment Governing Board, which include an increase in the use of questions like those in NIES to inform policy, we examined the degree to which AIAN student experiences as reflected in NIES are associated with achievement on NAEP. We then examined NIES against a CRS framework, and found that NIES could inform policy to the detriment of AIAN students.
The NIES collects information about cultural experiences that address many of the CRS dimensions outlined by Demmert and Towner (2003): language, community participation, and the infusion of traditional culture into contemporary instruction are evident. Yet, the NIES does not collect information about perceptions of intellectual rigor, academic expectations, or quality of instruction held by either students or teachers, nor does it ask students about the degree to which they value their culture despite its importance in identity development. Moreover, NIES lacks the necessary information to discern the kind of training teachers have had to incorporate culture into instruction. Without these considerations, researchers who use the NIES to examine how culture is related to achievement may assert that the practice is associated with lower achievement, and make policy recommendations to limit AIAN culture—returning education for AIAN students to the 1950s policies that diminished the AIAN curriculum (Tippeconnic and Swisher 1992) or even used to justify retrogressive standards policies in U.S. history and other subjects that neglect the history of AIAN students.
In sum, what was interesting about this study is that we utilized Value Added Models (VAM) (Yes those same models that politicians and actual experts disagree on) We wrote that VAM models show a negative association between achievement and Culturally Relevant Schooling. We then proceeded to critique why these surprising results from VAM models were inadequate because of the absence the most relevant data/information (perceptions of intellectual rigor, academic expectations, or quality of instruction held by either students or teachers etc.) for Native American achievement is absent in the NIES.
People believe data and statistics to be objective. This conception is true and false. The math in the models is objective. However, the inclusion of different types of data in datasets as well as the decisions about which data will be included in models are subjective decisions. As we demonstrate in A Story within a Story: Culturally Responsive Schooling and American Indian and Alaska Native Achievement in the National Indian Education Study, the use data and statistics to form suppositions may influence educational policy is highly subjective and is not bulletproof or immune to criticism.
Please Facebook Like, Tweet, etc below and/or reblog to share this discussion with others.
Want to know about Cloaking Inequity’s freshly pressed conversations about educational policy? Click the “Follow blog by email” button in the upper left hand corner of this page.
Twitter: @ProfessorJVH
Click here for Vitae.
Co-Author Bios:
FRANCESCA A. LOPEZ is Associate Professor of Education at the University of Arizona. Her current research interests include examining the ways teacher-student dynamics inform the development of identity and achievement among Latino English Language Learners.
JACQUELINE SCHRAM, a band member of Sagkeeng First Nation, is a doctoral student in the Department of Educational Policy and Leadership at Marquette University. Her current research interest focuses on college access and student support services for Native American students.
Cloaking Inequity 