I usually ignore Patrick Wolf and the other nonsense and commentary produced by the Department of Education Reform at the University of Arkansas, but I couldn’t this time. Here is the press release from my new analysis of their report on charter school versus public school “productivity.” From the National Education Policy Center (NEPC) Press release:
Bigger Bang, Fewer Bucks?: The Productivity of Public Charter Schools in Eight U.S. Cities, published by the University of Arkansas Department of Education Reform, contends that charter schools produce more achievement per dollar invested, as compared to public schools.
Professor Julian Vasquez Heilig of California State University Sacramento reviewed the report and identified a variety of methodological choices made by the authors that threaten the validity of the results.
The report is focused on city-level analyses in eight U.S. cities (Atlanta, Boston, Denver, Houston, Indianapolis, New York City, San Antonio, and Washington D.C.) and uses cost effectiveness and Return on Investment ratios. It concludes that charter schools deliver a weighted average of an additional 4.34 NAEP reading points and 4.73 NAEP math points per $1000 invested.
However, Professor Vasquez Heilig points out that the report fails to account for the non-comparability of the student populations in charter and comparison public schools. Four other problems also undercut the report’s claims. First, the report uses revenues rather than actual expenditures, despite well-established critiques of this approach. Second, the report’s lack of specificity (e.g., using state-level data in city-level analyses and completely excluding race and gender) plagues the accuracy and validity of its calculations.
Key Review Takeaway: Flawed evidence provides no valid guidance to educators or policymakers evaluating cost effectiveness or return on investment.
Third, the authors fail to reconcile their report with the extensive literature of contrary findings. Finally, even though the think tank’s earlier productivity report included a caveat saying that causal claims would not be appropriate, and even though the new report’s analyses also are insufficient to make causal claims, the new report omits that caution.
The evidence in this report is so flawed that it provides no valid guidance to educators or policymakers.
Suggested Citation: Heilig, J.V. (2018). NEPC Review: “Bigger Bang, Fewer Bucks?” (University of Arkansas Department of Education Reform, February 2018). Boulder, CO: National Education Policy Center. Retrieved [date] from http://nepc.colorado.edu/thinktank/review-roi.
I am honored to be a National Education Policy Center fellow. One of the most exciting and incredible things that they do is the Bunkum Awards! It’s the annual salute to the worsteducation think tank work in the United States. They write:
With the Oscar celebration next week, and the Emmys and Pulitzers on the way, the National Education Policy Center announces this year’s winner of its Bunkum Award. We invite you to enjoy our 10th annual tongue-in-cheek salute to the most egregiously shoddy think tank reports.
It’s not easy to laugh when data are manipulated and made to fit foregone conclusions or when the research literature is misrepresented or ignored and low-quality or dishonest “evidence” has real impact on policy and on children. As best they can tell, polar bears aren’t laughing at reports from the American Petroleum Institute.
Yet “humor is one of the best ingredients of survival,” according to Aung San Suu Kyi—whose travails have been far weightier than ours. So we will persevere in our commitment to having a bit of fun each year with the evidentiary farce-lympics.
The Think Twice Think Tank Review Project arose as a response to the often-outsized policy influence of glossy, well-publicized reports that have not been vetted by peer-review. These reports regularly wrap themselves in the veneer of research, but we are frequently little more than propaganda masquerading as social science.
This year’s awards announcement, available on the NEPC website, is hosted by Dr. David Berliner, the Regents’ Professor Emeritus and former dean of the College of Education at Arizona State University. Berliner is a member of the National Academy of Education and the International Academy of Education, a Fellow of the Center for Advanced Study in the Behavioral Sciences, a past president of the American Educational Research Association, and a widely recognized scholar of educational psychology and policy.
The 2015 Bunkum winner is…
My favorite Berliner quote during the award ceremony…
The authors of the report are anonymous, perhaps justifiably seeking anonymity.
For all of Cloaking Inequity’s posts on charter schools click here.
Please Facebook Like, Tweet, etc below and/or reblog to share this discussion with others.
Want to know about Cloaking Inequity’s freshly pressed conversations about educational policy? Click the “Follow blog by email” button on the home page.
How about some good news in your inbox today? The National Education Policy Center is seeking to recognize “Schools of Opportunity,” a recognition of top high schools across the nation. The application deadline is February 3 and is rapidly approaching. Here is their press release:
BOULDER, CO (January 19, 2016) – High schools from across the nation are now submitting applications to be recognized as part of the Schools of Opportunity project of the National Education Policy Center. The project recognizes public schools for what they do to give all students the chance to succeed, rather than turning to test scores to determine school quality. The application deadline is February 3, 2016.
The Schools of Opportunity project highlights schools that use research-based practices to close the opportunity gaps that result in unequal opportunities to learn, in school and beyond school.
For example, although schools cannot directly integrate neighborhoods by race and class, they can do their best to integrate classrooms by race and class. And although it is difficult for schools to increase learning resources in neighborhoods or homes, they can ensure that rich, engaging learning opportunities are provided to all students while they are in school.
The National Education Policy Center (NEPC), housed in the CU-Boulder School of Education, designed the Schools of Opportunity project as a way to highlight the nation’s best schools and practices. The project is led by NEPC director and CU-Boulder School of Education Professor Kevin Welner, and Carol Burris, director of the Network for Public Education, who was the 2013 New York State High School Principal of the Year.
Each state’s effort will also be assisted by a team of evaluators, including New York State Regent Betty Rosa and Vermont State Board of Education member William Mathis, a former finalist for National Superintendent of the Year. The Ford Foundation and the NEA Foundation have both provided funding assistance.
“This project is about rewarding schools for doing the right things, even if they do not enroll the nation’s top students,” said Welner. “It’s also about highlighting the work of schools that are energetically closing the opportunity gap by engaging in research-based practices designed to make sure that all students have rich opportunities to succeed.”
Burris, whose high school had consistently been given top ranks in popular lists of the nation’s top high schools, points out their limitations. “Current programs aimed at identifying the nation’s best high schools include many high-quality schools,” she said. “But the approach they use tends to reward schools that are affluent and/or those that enroll a selective group of students. It is time we recognize schools that do outstanding work with a wider range of students.”
The Schools of Opportunity project will recognize schools based on 11 specific principles identified by experts in the 2013 book, Closing the Opportunity Gap, published by Oxford University Press, which Welner edited along with Stanford University Professor Prudence Carter. The project will recognize schools that use these principles to help to close opportunity gaps in order to improve academic performance.
“The first step in changing the conversation on school quality requires us to acknowledge that achievement gaps are a predictable and inevitable consequence of opportunity-to-learn gaps, which arise in large part because of factors outside of the control of schools,” Burris said. “However, even as schools are affected by larger societal forces, schools and educators can make decisions that either widen or close opportunity gaps.”
The specific practices include effective student and faculty support systems, outreach to the community, health and psychological support, judicious and fair discipline policies, little or no tracking, and high-quality teacher induction and mentoring programs. All identified practices are listed on the Schools of Opportunity website at http://opportunitygap.org.
The project is grounded in two basic, interrelated truths. Opportunity gaps beyond the control of schools contribute to gaps in achievement. At the same time, excellent schools can help narrow achievement gaps by closing those opportunity gaps within the school’s control.
“It’s because of the first truth,” Welner explained, “that excellent schools cannot be identified by just looking at outcomes. An awful school can have pretty good outcomes if its students are lucky enough to have rich opportunities to learn outside of school. And an outstanding school won’t necessarily have excellent scores if its students are disadvantaged by severe life challenges outside of school.”
“When schools and communities focus resources and efforts on closing the opportunity gaps, they should be recognized, supported and applauded,” he said. “They should also serve as models for those who wish to engage in true school improvement.”
The Washington Post’sAnswer Sheet blog will announce schools that receive recognition in the spring. Top schools will receive acknowledgement at awards ceremonies and in other venues as well.
The Schools of Opportunity recognition process is designed to allow applicants to explain how and why their school should be recognized, and the project will provide any assistance needed to help applicants easily complete and submit their information.
Schools of Opportunity recognitions will be made at gold and silver levels, as well as a special recognition for top schools. Applications are welcomed until February 3rd, with all nomination information and forms available online at http://opportunitygap.org.
The National Education Policy Center (NEPC), housed at the University of Colorado Boulder School of Education, produces and disseminates high-quality, peer-reviewed research to inform education policy discussions. Visit us at: http://nepc.colorado.edu
Please Facebook Like, Tweet, etc below and/or reblog to share this discussion with others.
Want to know about Cloaking Inequity’s freshly pressed conversations about educational policy? Click the “Follow blog by email” button on the home page.
The world is experiencing the spread of privatizing GERMs (Global Education Reform Movements).
School “reformers” are pressing a variety of “school choice” and “market-based” policies that move the control of schools from democratic control to private control. Market-based approaches such as vouchers, charters, and parent trigger take the control of schools out of the hands of democratically elected governmental officials and local communities.
Not only are “choice” and “market-based” policies spreading across the United States, but the world is also facing privatization efforts. The United Nations recently convened a meeting to address the GERMs. I will begin with a quick discussion of primary domestic privatization efforts and then segue to the global pushback.
The two most prominent forms of private control pressed in educational policy in the United States are charters and vouchers. Choice proponents focus on the limitations of traditional public schools but rarely discuss the predominance of the peer-reviewed research literature that demonstrates limited or no effect of choice (i.e. vouchers and charters) on student success. Are there examples of student success in charters? Of course, as is there also in public schools. However, the most prominent CREDO study of charter schools across the nation showed that nationwide only 15% of charters perform better than traditional neighborhood public schools.
While it is true that you can find an occasional peer-reviewed study that shows small effects for vouchers, the predominance of the research literature in the United States and elsewhere in the world shows no statistical significance or limited effects. What is also notable about the rare voucher study that shows a positive effect is that they are typically churned out by researchers primarily funded by foundations that are ideologically privatization advocates (aka Kool-Aid drinkers). What charters and vouchers clearly do is hand the control of the capital in schools to religious organizations, politically appointed boards and/or corporations.
Parent Trigger is a policy where parents are allowed to vote to turn over a public school to private control. Parent Trigger is parent empowerment without the empowerment. Parental involvement without the involvement. Why? Students and parents rights are actually more limited. Once a trigger petition is voted on and goes forward, the rights that are often guaranteed to parents, students and teachers under the democratically defined education code are usually squelched. Under private control, parents and students no longer have guarantees on class size limits, disciplinary decisions, or qualified teachers among others.
Also, as real estate agents often say “Location!, Location!, Location!” Schools often sitting on very valuable parcels of land. It would be a coup for those shopping for real estate and facilities if just a few parents from any particular year are given the reins to easily transfer hundreds of millions of dollars in public assets. Furthermore, parent trigger legislation is typically flawed, as parents should have the option to cancel the contract and return to public management if they so choose. Parent Trigger shouldn’t be a one-way street. Families should also be able to make the choice to leave private control and return to public control if they are underwhelmed by the undemocratic, non-local, control of their school.
Another hot topic is the rise of virtual (online) charter schools. What high schooler wouldn’t want to sit at home and pretend to go to school and have the school pretend that they attend? The results of virtual schools has been dismal. For example, In the Public Interest’s reported entitled Virtual Public Education in California: A Study of Student Performance, Management Practices and Oversight Mechanisms at California Virtual Academies, a K12 Inc. Managed School System found,
The virtual education model advanced by K12 Inc. in California does not adequately serve many of its students. In every year since it began graduating students, except 2013, CAVA has had more dropouts than graduates. Its academic growth was negative for most of its history and it did not keep up with other demographically similar schools after 2005. Its Academic Performance Index scores consistently ranked poorly against other demographically similar schools and the state as a whole. Evidence of low quality educational materials, under- staffing of clerical employees and low teacher salaries all indicates that an additional investment of resources in the classroom is necessary for improvement.
The financial profiteering and lack of success of students enrolled in virtual charters by the National Education Policy Center was summed up by Kevin Welner, University of Colorado Boulder Professor, who stated that they “raises enormous red flags.”
Private control is problematic. What occurs is a few individuals— often elites— gain control of districts and schools rather than democratically controlled education policy. Disingenuously, there is a simultaneous promotion of an elegant, but false, narratives of Civil Rights and economic efficiencies as benefits of their top down approaches. However, what the anti-democratic policies really offer is the opportunity for the elites to profit handsomely from the education system by privately controlling the capital.
The United Nations recently took notice of the move to privatize education in the United States and elsewhere. Education International reported,
In what is being hailed as a landmark resolution, the United Nations Human Rights Council has urged States to regulate and monitor private education providers as well as recognise the threat of commercialised education.
The HRC is the leading global inter-governmental political body dealing with human rights. In the resolution adopted by consensus of its 47 members, the HRC has, for the first time, responded to the growing phenomenon of privatisation and commercialisation of education.
Camilla Croso, of the Global Campaign for Education, reacted: “the rapid, unregulated growth of private providers of education is already creating – and enabling – violations of the right to education, threatening to erase the last 50 years of progress in access to education. This resolution shows that States have realised that they must act now to regulate such providers – before it is too late.”
Sylvain Aubry, of the Global Initiative for Economic, Social and Cultural Rights elaborated: “Our research has consistently shownthat privatisation in education leads to socio-economic segregation and discrimination against the poorest children in schools, in violation of States’ obligations, as was recently recognisedin the case of Chile. The resolution adopted today, crucially highlights the obligation to provide educational opportunities for all without discrimination.”
I was listening to NPR the other day and they had a story about the low cost, standardized curriculum for-profit private schools that are proliferating in India and elsewhere. My first thought was that it was a good idea. But, as I thought more carefully, I realized that the point of these schools is to destabilize the public systems and profit simultaneously.
The Global Campaign for Education held a meeting in Geneva to bring together organizations and scholars to discuss privatization and commercialization. Frank Adamson attended the recent meeting. I first met Frank when we were working on our doctorates at Stanford (he plays a mean striker in intramural soccer). He is a senior policy and research analyst at SCOPE and co-editor of the forthcoming book “Global Education Reform: How Privatization and Public Investment Influence Education Outcomes” due out in January. He travelled to the recent meeting in Geneva.
Frank writes:
The meetings in Geneva revealed that the privatization and commercialization of public education is indeed a global phenomenon. The panel discussions ranged from low-fee private schools in Uganda and Ghana to public-private partnerships in Nepal and Philippines, vouchers in Chile, markets in Sweden, and corporate charters in the U.S. The pace and mechanisms of these approaches differ in each country context. However, the theme of profits displacing learning is consistent across contexts.
The profit in question comes from taxpayer money intended for public education. A variety of actors – from venture capitalists to the World Bank, technology firms, and local companies – have labeled public education as a “$4.625 trillion dollar space” and set their sights on taking home a slice of that pie. The Geneva meetings provided a more coordinated critique of this attempted takeover of education, a sector that most people support as publicly funded for the public interest, not private profit.”
The focus on profit and its corollaries – standardization, high-stakes testing, and cheaper labor (inexperienced teachers) – isn’t producing better education globally. Geneva revealed the failures and fallacies of the privatized approaches in different countries, illustrating that it’s now time to ensure that the public investment in education is channeled into the public interest. The roadmap is also clear, as the highest performing and most equitable countries in the world (Finland, Singapore, Canada, and Cuba, to name a few) use public investment strategies that produce the education outcomes that parents expect and would like to see for children worldwide.
GERMs are bad for the health of public education. Vaccinate yourself with the necessary knowledge to combat their spread. In conclusion, I’ll discuss privatization tomorrow here at Netroots. If you are here at Netroots or in Phoenix community come by and say hi.
Want to know about Cloaking Inequity’s freshly pressed conversations about educational policy? Click the “Follow blog by email” button in the upper left hand corner of this page.
The video below includes a panel discussion that includes Dr. Adamson and a UN Special Rapporteur. The panel provided a variety of international perspectives on education privatization.
I have been called a charter school “lover” and a charter school “hater” in the same week. Are both true? Neither? I think we as a society must be pragmatic on charters. I recently had a conversation with my grandmother, who was a lifelong educator at a middle school in Saginaw, Michigan. She explained to me that based on what she had read in the newspapers and seen on TV— that charters schools had some promise. I think the jury is still out. In this post I will introduce my podcast on the education view on charters, include a new review of charters schools and student achievement released by the National Education Policy Center and then conclude with my comments on the new Edweek blog .
First here is a link to my interview with the Educational View on holding charters accountability for equity. I based my conversation on the post Don’t Trust Charters More than a Sweaty Used Car Salesman (A Citizen Research Template) where I lay out a citizen research template for parents and other stakeholders to study and understand the equity context for any particular charter school or charter chain. Click the link in the tweet below for the five minute broadcast.
Charter schools on average produce results that are at least on par with and, in many cases, better than district-run public schools
Charter schools are producing higher achievement gains in math relative to their district-run counterparts in most grade levels, particularly in middle school, and gains in reading that are similar to district-run schools in reading
Charter school effect size has risen for both math and reading over time, though this trend is not statistically significant
A small and growing body of literature on the relationship between charter school attendance and outcomes apart from achievement finds further evidence of large positive impacts of charter schools on high school graduation, college enrollment, and behavioral issues
The National Education Policy Center commissioned a peer review of the report by Francesca Lopez at the University of Arizona. Click here for the full brief.
So what about that charter Kool-Aid? Her findings:
This report attempts to examine whether charter schools have a positive effect on student achievement. From a review of 52 studies that the authors considered superior (40 of them used in an earlier report),it finds charters are serving students well, particularly in math. This conclusion is overstated; the actual results are not positive in reading and are not significant in high school math; for elementary and middle school math, effect sizes are very small, ranging from 0.03 to 0.08 s.d. The report does a solid job describing the methodological limitations of the studies reviewed, then seemingly forgets those limits in the analysis. For example, the authors include lottery-based studies, considering them akin to random assignment, but lotteries only exist in charter schools that are much more popular than the comparison public schools from which students are drawn. This limits the study’s usefulness in broad comparisons of all charters versus public schools. The report also seeks to examine whether the effects of charter schools have changed over time. Despite finding no change, the authors inexplicably assert that there is a positive trend. Claims of positive effects when they are not statistically significant, exaggeration of the magnitude of effects, reliance on simple vote-counts from a selected sample of studies, and unwarranted extrapolation of the available evidence to assert the effectiveness of charter schools further render the report of little value for informing policy and practice.
In conclusion, I just recently started a discussion with Dr. Jack Schneider on EdWeek’s Beyond the Rhetoric opinion on a variety of topics (See Introducing Julian Vasquez Heilig). EdWeek has placed our conversation behind a paywall. Since EdWeek has not compensated me for contributing in any way, in my view, I still own my work. So below I will include only my own language. I don’t believe I have the right to include Jack’s questions/probes, so you will have to guess what Jack asked me between questions or subscribe to EdWeek. Sorry. 🙂
Julian Vasquez Heilig: Charters are quite diverse. Just the word can set off critics and supporters. The challenge, as I see it, is to decouple the persona of charters from the positive and negative realities of their diversity.
Schneider:
Heilig: The types of charters and the chartering process varies greatly from state to state. So I think we have to decide as a nation what types of charters we really desire. For example, compare for-profit charters with community-based charters.
The issue with for-profit charters is the same issue that we often face with corporate malfeasance—the profit motive can overwhelm honesty and integrity. We thereby sacrifice serving students regardless of their circumstance, an important aspect of our national social contract for public education. We must also consider the quasi-private policies found in many corporate charters that have implications for equity and excellence.
By comparison, a charter school like Travis Heights Elementary in Austin, Texas, where the local school district, parents, teachers union, and faith community came together to build a community-based charter school, is a welcome contrast to corporate, profit-driven charter schools.
Schneider:
Heilig: Charter governance differs across states. For example, in Michigan, 61% of charters are run on a for-profit basis. An important question for the nation is whether we think charters on a for-profit basis are a desirable approach. We also need to consider whether communities should be able to veto corporate charters and for-profit charters if they so choose.
I also think we need to reframe our conception of choice. Why can’t communities access desirable characteristics in their neighborhood traditional schools? Why have Louisiana and Michigan created achievement authorities that turn a blind ear to community input? Who is actually doing the choosing for our communities? Also, we must continue to grapple with the question of whether the freedom that charters have received is being abused. Equity and access for students of different kinds and workplace conditions for faculty and staff are important questions that we are continually grappling with in our ongoing research.
Schneider:
Heilig: I don’t think we should limit ourselves to the current state of charters. Instead, I think we need to consider what the evolution of charters will be. Yes it is true that in most states charters are not for-profit. But we already have plenty of data and research to understand the pitfalls of for-profit charters in Michigan, Arizona and elsewhere. The Detroit Free Press recently conducted several investigative reports of charters in Michigan that uncovered wasteful spending, lack of accountability, nepotism, and corruption. Further, overall charter performance was no better than traditional public school performance.
Heilig: I use “corporate” as a term to describe the large chains and franchises that are dominating and leading the charter movement—KIPP, BASIS, Rocketship, Great Hearts, etc. They are schools that, as you said, are typically controlled externally to the communities in which they are located.
There are, of course, community-based charters. I mentioned previously Travis Heights charter school, which was founded by community and stakeholder groups. The question is: do we need corporations to run our schools or should we allow local community-based organizations the authority and resources to design and run schools? The recent political battle between the IDEA corporate chain of charters and local community groups in Austin was an example of this tension.
Schneider:
Heilig: I would argue that what we do in an imperfect world is to hold charters to the same standards (equity, access, and achievement) that we hold traditional public schools. No special favors or double standards. Being that charters are purportedly a better choice, if they fail at this mission and underperform traditional public schools, close them, no excuses.
For all of Cloaking Inequity’s posts on charter schools click here.
Want to know about Cloaking Inequity’s freshly pressed conversations about educational policy? Click the “Follow blog by email” button in the upper left hand corner of this page.
Cloaking Inequity 
You must be logged in to post a comment.