Uncovering Lies and Damn Lies in @ArneDuncan Graduation Rates

Has your state experienced a meteoric rise in its graduation rate since Arne Duncan’s new graduation standards went into effect? Do you think it too good to be true? Nobody likes to be lied to. The latest Arne Duncan sleight of hand is the reporting and trumpeting of graduation rates. The Washington Post related:

Calling it “a profound milestone,” Education Secretary Arne Duncan said Monday that the country has reached its highest graduation rate in history, with 80 percent of students receiving a diploma in 2012, the most recent year for which statistics are available.

Fusion.net (ABC News-Univision joint venture) reported:

Texas is one of the exceptional performers; the state boasts a graduation rate in the upper 80s.

The fact that Texas is in that list is remarkable, given some of the unique challenges it faces in educating young people. More than half of the state’s students receive free or reduced-cost lunches, and there is a sizeable population of non-native English speakers.

What does TEA say about the amazing success of the Lone Star State?

“You can’t address the problem until you define it,” she said. “You’ve got to know that John Smith is the kid at risk of dropping out, not just look at percentages.”

or REDEFINE your “data”… what do I mean by that? The Fusion.net piece concluded the article with the following:

Skewed numbers

But Rodriguez and several other people with knowledge of the state’s education system said that some of the uptick in graduation rates may actually come from the way Texas reports numbers to the government. In other words, the improvement might not really be as great as the state says it is.

Julian Vasquez Heilig, an associate professor at the University of Texas at Austin, has been vocally critical of the state’s graduation measurements.

“What they’re doing that’s amazing when it comes to graduation rate is lying to the government,” he said.

There are a couple of ways for the state to calculate graduation rate. One, called the Adjusted Cohort Graduation Rate (ACGR), lets states make adjustments for students who leave, through transferring out of state, homeschooling, passing away and some other scenarios.

The measurement is intended to let states get a more accurate count of where their students end up, but it also gives the state control over which “leaver” code they use and, depending on the code, they can let themselves off the hook for any follow up. If the state enters the homeschool code, for instance, the kid might actually not continue their education and really be a dropout, but they aren’t flagged as a dropout by the state.

“It allows Texas to tell the feds what the denominator is,” Vasquez Heilig said. “People are really good at hiding dropouts.”

The other measurement to determine the graduation rate is something called the Adjusted Freshman Graduation Rate (AFGR), which doesn’t let the state alter the denominator. In most states, the two measurements are relatively similar. In Texas, there’s enough of a discrepancy to elicit concern.

A 2013 report on graduation rates noted, “Both graduation rate measures (AFGR and ACGR) broadly agree on the rate and level of progress achieved by hispanic students. There is, however, a consistent five-to-six percentage point difference in overall graduation rates produced by the two different metrics and a ten-point divergence on the graduation rate for African Americans, which gives pause.”

“There are some data questions,” said Robert Balfanz, co-director of Johns Hopkins University’s Everyone Graduates Center, which helped author the reports. “They allow themselves some exceptions not everyone else allows themselves.”

He cited homeschooling and said the state also has thriving private, charter and online schools. The state “can get kids off their books” if they can show they’ve transferred to another degree-granting institution, he said, and then they no longer bear responsibility, even if the student drops out the next week.

Last year Cloaking Inequity looked at Texas’ meteroic rise in the graduation rates in Texas in the post  Texas Lies to Feds: Enrontize Federal Graduation Data. I will reblog the post here to demonstrate how Texas has gone from 29th in the nation to 4th in the nation in the last three years.

At the end of 2012, with much fanfare Texas trumpeted that its 86% Adjusted Cohort Graduation Rate (ACGR) was 4th in the nation. (For more on the ACGR go here). This would be a miraculous achievement for the Lone Star State. As we show in Is Texas leading its peers and the nation?: A Decadal Analysis of Educational Data, Texas’ Averaged Freshman Graduation Rate (AFGR) was 75.4% as recently as three years ago and ranked 29th in the nation. [i]

Many Texans have expressed that the 86% graduation rate is contrary to what they are seeing in Texas high schools and have characterized the results as “dubious.” The Houston Chronicle reported on recent independent analysis of the graduation rate in Texas,

Bill Hammond, president and CEO of the Texas Association of Business, calculated a completion rate of 65 percent by comparing ninth grade enrollment with the number of seniors graduating from the same cohort. Children at Risk takes the process a step further, tracking student movement through “leavers” – students not counted by the Texas Education Agency as dropouts but who leave the school for reasons beyond transferring to another public school. At 71.6 percent, Children at Risk’s calculated graduation rate strives to account for students who fall through the cracks. The data in both figures show the fact of the matter: The dropout problem remains prevalent.

So how has Texas shown miraculous ACGR graduation rate and increased it by 15-20% compared to independent analyses and the AFGR? Cloaking Inequity has been on the case since early December 2012 when I submitted a FOIA request to the Texas Education Agency (TEA). TEA recently responded. So here for your viewing pleasure, I have pasted data excerpts from TEA’s response below:

1. The numerator of the four-year ACGR for the class of 2011, i.e. the number of students in the class who graduated with a regular high school diploma in four years, is 274,562.
2. The denominator of the four-year ACGR for the class of 2011, i.e. the number of students in the class who graduated, continued in school in year 5, received a General Educational Development (GED) certificate, or dropped out, is 319,588.
3. The number of first-time Grade 9 students in 2007-08 was 356,183.
4. The number of students who transferred into the cohort (i.e. who entered Texas public schools in Grade 10 in 2008-09, Grade 11 in 2009-10, or Grade 12 in 2010-11) was 22,589.
5. The number of students who transferred out of the cohort (i.e. “other leavers”) was 53,538.

The first thing I noticed was that the first-time ninth graders plus transfers in and minus transfers out equals 325,234 and not 319,588. (The difference is accounted for by “data errors,” essentially TEA has excluded them from the denominator because they don’t know where they are or who)

Now lets focus on the 53,538 transfers out of the 2011 ACGR cohort denominator.

Other Leavers, by Leaver Reason, Texas Public Schools, Class of 2011 Grade 9 Cohort

A dash (-) indicates data are not reported to protect student anonymity.

One of the interesting things about the homeschool PEIMS code is that it has tripled over the last decade. When students are coded as leaving school for homeschooling they are not consider dropouts nor are they included in the denominator of the ACGR. The Houston Chronicle ran a story in 2010 and said that “home-schooling in Texas doesn’t add up” and that Texas is “disguising thousands of middle and high school dropouts in this hands-off category.” Excluding students using in the homeschool PEIMS code would also inflate the ACGR.

Even the state’s biggest proponents of home-schooling admit that the structure is vulnerable to fraud.

“That seems to me to be a loophole,” said Tim Lambert, president of the Texas Home School Coalition.

The problem is not among legitimate home-schoolers, but among public school officials trying to run off problem students, Lambert said.

“We call it dumping,” he explained. Some advocates complain that Spanish-speaking and special-needs student are especially vulnerable to being pushed out of public schools.

In fact, until 2011-12, according to the Texas Education Data Standards (TEDS) rules, Texas districts could just state that students “intended” to be homeschooled. Now they are required to obtain a signed form from the parents.

Do you want to code 9,942 students as leaving the country? Here is what you need as official documentation, again from the TEDS:

Acceptable documentation is also a copy of the withdrawal form signed and dated by the parent/guardian or qualified student and a campus or district administrator.

Only the student and administrator signs..hmmmmm

Texas high schools could also state until 2011-12 that students “intended” to enroll in a public or private school out of state. This code was used for 19,430 out-of-state and 7,116 in-state. Now the state of Texas is requiring that school either get a transcript request, verification from recieving district or a signed letter from the parent. But in 2007-2008, well, this was not required. Sure, Texas has mobility, but it was not required to be verified.

So what is the bottom line here? The AFGR did not allow Texas to define away dropout and inflate their graduation rate because it averaged enrollments in the 8th, 9th and 10th grades for the denominator. The AFGR of course had its weaknesses, especially in states with high mobility. For example, in Is Texas leading its peers and the nation?: A Decadal Analysis of Educational Data, Texas’ AFGR for Asians was above 100% because of the influx of Asian American students into the state. However, the ACGR also has weaknesses because Texas has found creative ways to reduce the denominator by requiring very limited documentation from schools. These loopholes were apparently closed in 2011 as the state sought to meet federal reporting guidelines for its PEIMS codes.

Myself, Linda McNeil, Angela Valenzuela, Linda Darling-Hammond, Gary Orfield, Walt Haney, IDRA and many others have highlighted over the years how Texas data is exceedingly spurious. The ACGR is just the latest egregious example. So I guess that means that we have to wait until the class 2015 to get a valid picture of Texas’ graduation rates.

In conclusion, I will leave you with a few quotes from Texas high school administrators included in my EEPA article.

Testing Coordinator:

I think each year we get a new set of regs, and we try and figure out how is the best way to use it to our advantage… I mean, the game changes…it’s…like any – like a game that has a set of instructions. And everybody gets the same set of instructions, and everybody follows the same set of instructions…  If you’re really savvy, and if you’re really into everything as a principal you may see a problem… you may give your campus an advantage that another campus doesn’t have.

It’s human nature to…look at your game plan and to look at the rules of the game… You know, and to say that using a loophole is not right or is a bad thing to do, I don’t necessarily agree with, because it could be a good thing. It depends on the loophole… schools, yes, are under pressure to look for creative ways to be successful, okay, that’s obvious.

Ratcliffe, the state spokeswoman, said the state does have “a high mobility rate,” but that “we think our count is very accurate…We believe our districts are reporting the information to us accurately.”