Dr. Julian Vasquez Heilig African Americans, Dropout, Higher Education Access, Latina/os, Teacher Quality

JVH on KUT: “Education Experts Weigh Proposals’ Impact on Students”

Julian Vasquez Heilig gave an interview about “college” and “career” readiness to Veronica Zaragovia from KUT News. We briefly discussed the education bills passing through the Texas Legislature that are stirring debate on impact to students, especially minorities. See my earlier post Career and Tech: “Show me the money!” for thoughts on funding for career programs and the implications tracking students of color.

Listen 1:49 Story as it aired on KUT News 90.5 FM on 3/29/2013

The Texas Legislature is debating bills intended to help more students graduate from high school, by reducing the emphasis on standardized tests and increasing the emphasis on the kinds of education they need to be productive members of the workforce.

This week, the House passed House Bill 5, which would let high school students take a path to college or take a route intended to lead them more quickly to work. The bill also would drop the number of STAAR exams from 15 to 5.

Professor Julian Vasquez Heilig will be measuring the impact of this bill and others in tended to improve public education in Texas. He says one concern is that students who graduate from high school without core courses can’t get into college if they decide to later. And he’s concerned about the impact particularly on African American and Latino students, who are the majority in K through 12, but not in higher ed.

“We’ll know if this works if 20 years down the road we see African Americans and Latinos better represented in institutions of higher education across the state today,” he said. “We’ll know if we see trades people and other career ready African American and Latino students in business and corporations across the state.”

Heilig teaches at the University of Texas at Austin’s College of Education. He says even the college track students are suffering from a shortage of teachers in key areas like math and science, which he traces to budget cuts enacted in the last legislative session.

Deann Lee, state president of the Association of Texas Professional Educators, is more optimistic about what she called the “flexibility” that these bills can bring. Lee said she believes the number of end-of-course exams have caused dropout rates to jump in Texas, although others say research is inconclusive.

“The dropout rate has been a huge concern. We already have seen the effect of that in just a short amount of time that the present system is in place,” Lee said.

Next week, the Senate may take up the first of its education bills, SB 3. It  also guides students toward college or more directly to the workforce.

Dr. Julian Vasquez Heilig Accountability, African Americans, Announcements, High-Stakes Testing, Higher Education Access, Latina/os

Decade of Education Outcome Data: Texas vs. California vs. New York vs. Nation

Today we are pleased to release the report Is Texas leading its peers and the nation?: A Decadal Analysis of Educational Data. The Institute for Urban Policy Research and Analysis at the University of Texas at Austin commissioned a policy this report in the spring of 2012 to examine education data for the state of Texas, its peers and the nation. In this report, we gather data from a variety of sources to understand student outcomes in elementary, secondary and higher education by race and ethnicity for the past decade. We believe this is the first report to gather educational data of these types all in one report.

What did we find? (The Executive Summary)

The Texas Education Agency’s (TEA) Texas Assessment of Academic Skills (TAAS) and Texas Assessment of Knowledge and Skills (TAKS) exit exam data show that African American and Latina/o students apparently made dramatic achievement gains and narrowed the achievement gaps during the TAAS and TAKS eras. Notably, these gains appear to be reflected in the national NAEP test scores as Texas had the highest 4th grade and 8th grade NAEP scale scores relative to its peer states of New York and California for African Americans and Latina/os.

However, while the achievement gap closes, overall student performance lags.  This is not unique to Texas; each of the most populous states performed worse over the past decade relative to other states in the nation. Texas dropped 21 spots in 4th grade math, four spots in 4th grade reading, and eight spots in 8th grade reading. The only bright spot was that Texas improved its standing by the end of the decade in 8th grade math, moving from 22nd to 18th.

Education policy has not only focused on academic achievement, as measured by test scores, but has also focused on drop out and graduation rates. Despite about 15 years of high-stakes testing and accountability policy, dropout rates and graduation rates for African Americans in Texas do not appear to have improved; in fact, if data from independent empirical sources noted in the report are to be believed, the situation has worsened.

Another important question is whether Texas’ early grade performance on standardized tests relative to peer states also transferred to college entrance exams. Overall ACT composite scores were highest in New York (ranked 4th), followed California (15th) and then Texas (33rd). For African Americans and Latina/os, New York students had the highest ACT composite scores followed by California, then Texas. For the SAT, we found that California had the highest scores followed by Texas and then New York. Only California increased their SAT composite scores (2 points) over the last decade with other two states showing decreases New York (-17 points) and Texas (-4 points). In terms of national rank, California’s SAT rank held steady at 35th in the nation followed by Texas (44th) and New York (47th). In 2009, for African Americans and Latina/os, Texas had the highest SAT composite scores for Latinos amongst peer states, while California had the highest composite scores for African Americans. Therefore, in sum, Texas apparent success in the state-level data released to the public and NEAP scores in elementary and middle school is largely not reflected in composite SAT or ACT scores— except for Latina/os

These results beg the question: Why, considering in the past 20 years of high-stakes testing and accountability policy in the state, have these policies not produced more competitive NAEP scores relative to the nation and greater levels of college readiness in Texas relative to peer states and all other states? Considering that the ultimate goal of our schools is frequently framed as college and career readiness by the legislature, it appears that our current system is not meeting those goals, but state-mandated tests (TAAS, TAKS, STAAR— even NAEP) are instead providing us a false sense of security regarding achievement our schools.

In terms of higher education outcomes, Texas, in comparison to its peer states, finds itself generally surpassing California, but trails New York in virtually all outcomes. New York’s performance on higher education measures suggests that policies in that state are generally assisting African American and Latina/o students in closing the gaps. New York students generally surpassed California and Texas students on every measure, save one very significant exception: New York students allocated far greater amounts of their income toward higher education costs. While this finding is troubling to an extent, students’ aspirations towards higher education, as well as the amount of time they spend outside of class devoted towards studying is noteworthy. Higher education remediation measures are a category that none of the comparison states can truly boast about. In California, New York, and Texas, all three states reported that approximately half of all African American and Latina/o students had taken remedial courses. This speaks to the inequity of educational opportunities before students get to college.

In conclusion, while Texas typically performs in the middle between New York and California on the K-12 and higher education measures, the state is seeing more trends of decline and stasis than growth between 2000 and 2010 relative to all states in the nation. To promote college and career readiness, policymakers need more policy research to help them pinpoint problems and potential solutions.  This report is an example of the beginning of research that illuminates public issues in education but much more is necessary. The analyses in this report find indicate that Texas must stop depending solely on high-stakes testing as a measuring stick of the state’s educational progress and get serious about funding our schools (K-12 and higher education) relative to other states in the nation . California, in this instance, may serve as a bellwether demonstrating the catastrophic effects of extended divestment in education.

Cite as: Vasquez Heilig, J., Jez, S. & Reddick, R. (2012). Is Texas leading its peers and the nation?: A Decadal Analysis of Educational Data. The Institute for Urban Policy Research and Analysis. University of Texas at Austin.

Dr. Julian Vasquez Heilig Accountability, High-Stakes Testing

Bad Apples?: Accountability, AYP, and Cheating

I gave another interview on the cheating scandal in El Paso (1,2,3) yesterday. The question that often arises is this only a case of bad apples? The research literature has long shown serial dishonesty in Texas Education Agency’s (TEA) publicly reported dropout and graduation numbers (See for example IDRA; Losen, Orfield, & Balfanz, 2006; Vasquez Heilig & Darling-Hammond) suggesting that student leavers have been underreported since the inception of accountability in the 1990s. Remember the Ysleta scandal in 1999 for false reporting of data or Houston where Sharpstown and several other high schools reported a 0% dropout rate?

What is the scale of the manipulation of data? After 2005, when the state began to use the National Center for Education Statistics (NCES) dropout definition for leaver reporting, the statewide dropout count tripled for Latina/os and quadrupled for African Americans. Clearly, El Paso and Atlanta are recent, but not isolated examples. The real issue is that accountability incentivizes “pushing out” secondary students who score poorly on high-stakes exams such as the STAAR. I blame our policymakers for a poorly designed educational policy— accountability is not as good as advertised.

Check out Collateral Damage, a fantastic book on the cheating issue published by the Harvard Education Press and authored by David Berliner and Sharon Nichols.

p.s. In our BRE article, we also found that this is a very serious issue for charter schools. In 2003, a Performance-Based Monitoring System (PBMS) was developed by TEA to validate the data submitted to the state. Audits are triggered when TEA suspects serious falsification of dropout reporting. To study this issue, for the BRE paper we submitted a public information request to TEA seeking recent PBMS audits conducted by the agency on district dropout data. Recent audits showed that charters were more likely to report false student dropout data. A review of TEA audits from 2008-2009 and 2009-2010 revealed that 10 of the 22 PBMS audits were conducted on charter school districts.

Dr. Julian Vasquez Heilig Charter Schools, KIPP

“Work Hard, Be Nice?”: A Response to KIPP

In April of 2012, KIPP responded to a press release for Is choice a panacea? An analysis of black secondary student attrition from KIPP, other private charters and urban districts, a peer-reviewed paper published in the Berkeley Review of Education (BRE) about African American secondary student attrition from charter schools across the state of Texas. KIPP began their response by criticizing the research on four main points. I will address those points.

1. Vasquez Heilig relied on previous studies that claimed KIPP achieves results through high student attrition, while completely ignoring findings from the independent research group Mathematica that KIPP loses fewer black male students than neighboring district schools.

Notably, the suite of Mathematica reports and conference presentations focus on middle schools only. Our analyses focus on secondary students— middle and high schools in grades 7-12 (See p. 160 in the article). Also, our study was published in BRE, a peer-reviewed journal which required a blinded review process. Saying Mathematica is “independent” when KIPP is one of their paying “clients” is disingenuous.

2. The researchers used faulty methodology to draw false and inflammatory conclusions, directly contradicting Mathematica’s research—which found that KIPP produces significant and substantial academic gains for all students, including African-American students, that are not due to attrition.

I posit that it is inflammatory to accuse a study of faulty methodology without providing a valid basis for that claim… Actually, I don’t believe KIPP read the paper. Why? Because we never made claims relating to the relationship between attrition and achievement. We simply conducted an analysis of publicly available attrition data that is collected by Texas law in the PEIMS (The Lone Star State’s school data repository). KIPP Houston’s African American secondary student attrition problem was sitting in the PEIMS for a decade— it is still there for anyone to see regardless of KIPP’s spin.

3. Vasquez Heilig makes the inaccurate claim that KIPP receives $3,361 more in total revenue than the Houston Independent School District, and incorrectly infers that KIPP Houston spends more per pupil than the district. In reality, KIPP Houston, like all public charter schools in Texas, receive less per pupil funding than district schools and no public revenue for facilities. Excluding private funds raised to cover facilities costs, KIPP Houston spends less per student per year than HISD.

KIPP is incorrect. NEPC also thinks so here. Its hard to argue with publicly available data that they themselves are required to report by law. Per student revenue for KIPP Austin ($17,286) and KIPP Houston ($13,488) relative to Austin ISD ($10,667) and Houston ISD ($10,127) is readily available online each year from the State of Texas. However, considering the current school finance debacle in Texas, where approaching $6 billion was cut from education in the last legislature, in retrospect, I think KIPP should be applauded for spending more on education— as should other charters such as Making Waves.

4. The paper consists largely of repackaged findings from a study on KIPP released last year by Western Michigan University (WMU), whose conclusions were immediately discredited by researchers at the liberal Brookings Institute and other respected organizations.

The BRE paper uses different data sources (Texas PEIMS data versus WMU’s Common Core federal data). Our study focuses specifically on African American students in all charters across the state of Texas whereas the WMU study conducted a national analysis focusing solely on KIPP campuses. Again, I don’t think KIPP bothered to read the paper (but that never gets in the way of spin) because they stated,

In comparing KIPP’s student attrition with that of neighboring school districts, Vasquez Heilig and his  colleagues relied on faulty assumptions. At the time of the study, not all of the cities in question had KIPP high schools, so the report’s conclusions are based on comparing attrition in charter middle schools to district middle and high schools—an apples to oranges comparison.

This is not a valid critique. We noted in that paper that only KIPP Houston had a decades’ worth of 7-12 data (we show this on p. 168), so the decadal KIPP attrition analysis was restricted to Houston (See Table 9 on p. 171).

In conclusion: Does 100% of 60% really add up to 100%? (See p. 172) That’s Enron math.

If you have any other questions shoot me an email at jvh@austin.utexas.edu

Dr. Julian Vasquez Heilig Charter Schools, KIPP

Is choice a panacea? An analysis of black secondary student attrition from KIPP, other private charters and urban districts

Vasquez Heilig, J., Williams, A., McNeil, L & Lee, C. (2011). Is choice a panacea? An analysis of black secondary student attrition from KIPP, other private charters and urban districts. Berkeley Review of Education, 2(2), 153-178.

Public concern about pervasive inequalities in traditional public schools, combined with growing political, parental, and corporate support, has created the expectation that charter schools are the solution for educating minorities, particularly Black youth. There is a paucity of research on the educational attainment of Black youth in privately operated charters, particularly on the issue of attrition. This paper finds that on average peer urban districts in Texas show lower incidence of Black student dropouts and leavers relative to charters. The data also show that despite the claims that 88-90% of the children attending KIPP charters go on to college, their attrition rate for Black secondary students surpasses that of their peer urban districts. And this is in spite of KIPP spending 30–60% more per pupil than comparable urban districts. The analyses also show that the vast majority of privately operated charter districts in Texas serve very few Black students.

See KIPP’s response here. See Austin Chronicle article here. See my response to KIPP’s criticism here.