In 2018, I was invited to a mock trial entitled Public School System Charged with Fraud: Guilty or Not Guilty? about public education at Freedom Fest, a Libertarian conference that is held yearly in Las Vegas. Which basically says that if you have a conference in Vegas— I will say yes to just about any opportunity to speak. 🙂 I was asked by the Freedom Fest producers to add a little levity because the audience wouldn’t be very interested in hearing academics argue about effect sizes and other statistics. During this mock trial I brushed off questions about the so-called “gold standard” of research— Randomized Controlled Trials, or RCT.
The education reform trolling has been particularly intense the past few days on Twitter likely because of the release of Diane Ravitch’s new book Slaying Goliath. I hope to read it soon and post my review. Anyways, there is a video of me making the rounds on the internet from the Freedom Fest “trial” where I flippantly comment about RCT that is taken out of context because it does not describe the desired entertainment value of the event. I would like to mention we did hang the jury about the “fraud” of public education at a Libertarian event— which itself was entertaining. Anyways, I’d like to take this opportunity on my blog to be more precise about our thoughts about RCT and its role in school choice research.
Citation: Vasquez Heilig. J., Brewer, J. & Adamson, F. (2019). The politics of market-based school choice research: A comingling of ideology, methods and funding, In M. Berends, A. Primus and M. Springer (Eds.) Handbook of Research on School Choice, 2nd (pp. 335-350). New York, NY: Routledge.
I have taken this excerpt from our chapter in the Handbook on School Choice that I blogged about when it was published. You can the entire chapter here.
In the quest to determine if, and to what extent, a policy is effective, there remains the possibility that the types of questions, methods employed during research, and the funding of that research can be ideologically tainted. The first decade of the 21st century revved the quantitative and qualitative debate that has divided the social science community for decades. More precisely, No Child Left Behind (NCLB) reified a commitment towards quantitative ‘objective’ scientism while relegating qualitative and contextual understandings to a subpar practice. The National Academy of Sciences held that,
Federal and state agencies should prioritize the use of evidence-based programs and promote the rigorous evaluation of prevention and promotion programs in a variety of settings in order to increase the knowledge base of what works, for whom, and under what conditions. (Boat & Warner, 2009, p. 371).
That is, educational research and social inquiry are to be approached in systematic experiential trials (often with the so-called “gold standard” of Randomized Controlled Trials, or RCT) that create the foundational for universality and generalizability. There has long been a push to assert RCT or the “gold standard” in research as the pièce de résistance in educational research as the most ideal setting as it represents “random assignment” to the treatment or control group and eliminates selection bias (Mosteller & Boruch, 2002).
For example, in the quest to determine whether school choice models such as vouchers “work,” researchers (largely funded and supported by ideological organizations such as EdChoice and the University of Arkansas— a point we take up below) have increasingly proposed the use of RCTs to compare variance of outcomes among students receiving the school choice treatment and those remaining in public schools (Abdulkadiroglu, Pathak, & Walters, 2015; Barnard, Frangakis, Hill, & Rubin, 2003; Bitler, Domina, Penner, & Hoynes, 2015; Chingos & Peterson, 2015; Cowen, 2008; EdChoice, 2017; Greene, 2001; Greene, Peterson, & Du, 1998; Howell & Peterson, 2002/2006; Jin, Barnard, & Rubin, 2010; Krueger & Zhu, 2004; Mills & Wolf, 2016; Rouse, 1998; Wolf et al., 2013). Though, despite the glaring possibility of bias developing as pro-charter organizations like EdChoice (formerly the Friedman Foundation) and the University of Arkansas (heavily funded over the years by the Walton Family Foundation) promoting such research, there remains a considerable level of skepticism surrounding the unwavering power of RCTs in educational research (Lubienski & Brewer, 2016) and the elevation of quantitative over qualitative methods in general (Berliner, 2002).
In the age of hyper-accountability and assessment, policy makers have increasingly linked funding to the results of evaluations. Given the rampant existence of the “Protestant Work Ethic” dispositions outlined by Max Weber (1930) that has informed the myth of meritocracy, it has become commonsensical in our rhetoric and practice that one should be held accountable for the practices and monies to which they have been made responsible. In education this is manifested as students being held accountable for their grades, teachers for the production of good and better grades, administrators for the reduction in documented discipline problems, and school districts being expected to do more despite having less money. And if everyone is to be held accountable, everyone must be evaluated by test scores. In turn, these various outcomes serve as a guiding example— or exemplars— of “best practices,” or to the contrary, become negative examples. This myopic quest for certainty can be seen in school choice research that seek to ignore contextual factors in favor of generalizability across contexts.
Yet, while statistical inquiry can provide legitimate insight into some social phenomena, it stands little chance of drowning out the realities of contextual factors that are, to be sure, the most important factors in the development of lived realities and shared understandings.
The practical consequences of scientism in education are that it will institute a notion of the curriculum as “cookbook,” teaching and learning as “proven method” or “best practices,” research as “funded enterprise,” and educational inquiry as only “what works” (Baez & Boyles, 2009, pp. 51-52).
In sum, the quest for a-contextual certainty lies social inquiry that not only acknowledges context but also understands its powerful force in shaping outcomes. Given the political foundations and rationales of school choice, there remains an insatiable requirement to conduct experimental and non-experimental research that justifies school choice expansion. The expansion effort has required significant levels of funding to create a body of quantitative research and an appearance of a predominance of objective observation.
I also wanted to take this opportunity to also honor #24 and #8 Kobe Bryant. Sad, sad times for his family and fans. In 2009 I had my only opportunity to see Kobe play in person in Los Angeles, and he scored 40! on my Detroit Pistons. I’ll have to admit, I was a Kobe hater. He was just soooo good. Not as good as Jordan— which it pains me to say as a Pistons fan— but probably a clear #2 to the GOAT. Anyways, we can argue about that later.
Not only was Kobe a good basketball player, but he had some wisdom too.
RIP Kobe.
Please Facebook Like, Tweet, etc below and/or reblog to share this discussion with others.
For all of Cloaking Inequity’s posts on charters click here.
Check out and follow my YouTube channel here.
Twitter: @ProfessorJVH
Click here for Vitae.
Cloaking Inequity 
You must be logged in to post a comment.