How RPS Cheated Its Way to the Bottom ...
After John Butcher and I pointed out the cancerous growth of the VGLA (Virginia Grade Level Alternative) testing and its use to cheat on the SOL testing, especially in Richmond, the Virginia Department of Education (VDOE) set out to eliminate the VGLA. The new math test in 2012 mostly eliminated the VGLA as to math. What follows is the first of a series of blogposts analyzing how RPS cheated its way to the bottom. First up: John Butcher's number-crunching and analysis, with plenty of charts that show how Richmond compares with various localities.
As expected, the tougher new test produced a statewide drop in the scores in 2012. This year, the scores (pass rates) bounced back statewide. Richmond is another story.
The division scores minus the State average give a
measure of each division's performance in relative terms.
As you can see, Richmond, Petersburg, and Charles City did not share in the general improvement in 2013; they continued to decline vs. the state average. Note, however, that lowly Petersburg beat Richmond in both 2012 and 2013. Richmond was fourth from the lowest pass rate in the state in 2013.
English Disaster
As with the math test a year earlier, the new
reading test dropped scores statewide in 2013.
Versus the state, Richmond again
grossly underperformed.
FACT:
Richmond had the lowest pass rate in the state.
The scores dropped statewide in 2011 (and more in Richmond) because HB304 made it tougher to use the VGLA to cheat. In 2012, the statewide reading scores again dropped slightly. The Richmond reading score dropped even more. The new reading test in 2013 produced a huge drop in the Richmond scores, almost certainly because it deprived RPS of the VGLA cheat.
New Details
The new front end to the SOL database makes it much easier to extract data.
Those data cast a clear light on the effect of Richmond's VGLA cheating. We start with the math scores by year. Here are the Richmond and state scores for students with and without disabilities; the second graph shows the Richmond scores minus the state scores, both with and without disabilities.
Notice that Richmond has performed well below the state average but its students with disabilities performed well above the state average for students with disabilities until 2012.
Further notice the drop in scores for students with disabilities in 2011, perhaps related to the State Superintendent's belated "concern" about abuse of the VGLA.
To the point here, Richmond's kids without disabilities underperformed the state average while the kids with disabilities outperformed the statewide average for students with disabilities. Until 2012, that is, when the new test stopped Richmond's abuse of the VGLA and took the Richmond scores for students with disabilities below the state scores.
The new reading test in 2013, with no VGLA (except for LEP students), confirms this picture. Again we see Richmond's students before 2013 generally underperforming the state average but its students with disabilities outperforming the state average for students with disabilities. But the 2013 plunge in Richmond's scores, led by the students with disabilities, gives away the game.
"Plunge" understates the reality: Richmond's students with disabilities went from 3 points above the state average for students with disabilities to 15.2 points below from 2012 to 2013. Looking at these scores in context with the math test leaves only one explanation: the Richmond scores for students with disabilities were inflated until deflated by the new, VGLA-free math and reading tests.
The drops in scores for students without disabilities, in 2013 for the reading test and in 2012-13 for the math test, are consistent with the poor preparation, mentioned above, for the new tests.
Whether you credit that analysis or not, explain this: In 2013, Richmond had the lowest pass rate in the state in reading and the fourth lowest math. Here are the bottom ten on each test:
| Division | Reading |
| Richmond City | 50.0% |
| Petersburg City | 50.6% |
| Franklin City | 52.1% |
| Northampton County | 56.9% |
| Martinsville City | 59.2% |
| Norfolk City | 60.5% |
| Hopewell City | 60.6% |
| Prince Edward County | 60.6% |
| Sussex County | 61.7% |
| Charles City County | 62.2% |
| Division | Math |
| Franklin City | 43.8% |
| Buena Vista City | 45.2% |
| Charles City County | 46.0% |
| Richmond City | 47.0% |
| Cumberland County | 48.7% |
| Petersburg City | 48.9% |
| Lancaster County | 51.1% |
| Mecklenburg County | 52.4% |
| Covington City | 53.7% |
| Lynchburg City | 53.9% |
I don't think the (ex)Superintendent should merely have been fired; I think she should be sued for deliberately harming these kids.
Don't Blame the Kids
We hear that the Richmond student population is particularly difficult because the kids are [pick your excuse]. The only excuse that I might credit is socioeconomic: Poorer kids don't perform as well in school as their better-off peers. We'll investigate that below; in the meantime, nothing about the Richmond student population explains the recent plunges in the reading and math scores (while a lousy Superintendent explains the situation fully).
A few years ago, the Gates Foundation sponsored a huge data-gathering effort by S&P. Their Web site showed, with some clarity, that school performance correlates negatively with socioeconomic status: Poorer kids don't do as well.
The conventional proxy for socioeconomic status of a school population is the number of kids who qualify for free or reduced price lunches. Indeed, the F/R percentage is the criterion for Title I money from the feds. VDOE has settled on a more general measure, the percentage of students who areeconomically disadvantaged. That term includes any student who "1) is eligible for Free/Reduced Meals, or 2) receives TANF, or 3) is eligible forMedicaid, or 4) [is] identified as either Migrant or experiencing Homelessness."
The enrollment data are available here and the SOL scores are here for both the general population and the economically disadvantaged.
So let's look at the Virginia division pass rates as a function of the percentage of students whom VDOE identifies as "economically disadvantaged." First, the reading test:
The data give a decent least squares fit (R2 = 0.64), suggesting that the ED percentage indeed correlates with the scores. On this graph, Richmond is the gold square (recall that Richmond had the lowest reading score in the Commonwealth this year). The red diamonds are, from the left, Hampton, Newport News, Norfolk, and Petersburg. Charles City is the green diamond.
Here is the same graph for the math test:
The color codes are the same and the correlation is not as good.
Plainly, Richmond is grossly underperforming both the fitted line and a number of other divisions that have similar socioeconomic situations. Hampton and Newport News, both old, urban jurisdictions, are particularly instructive. In short: Socioeconomics does not provide an excuse, much less a reason, for Richmond's lousy performance.
Delving further: The database also gives data for students who are or are not economically disadvantaged. Plotting the pass rates of the economically disadvantaged students v. the rates for the students who are not, by division, gives the following for the math tests:
The R2 of 0.48 indicates a pretty good fit, suggesting that variations in the average pass rate of the kids who are not economically disadvantaged in a division explains about half of the variations of the rate for those who are economically disadvantaged.
The red diamonds are, from the top, Hampton, Newport News, and Norfolk. Richmond is the gold square. Charles City is the green diamond. No surprises there. The largest outliers from the fitted line are Middlesex County, the purple diamond, and Lexington, the yellow diamond.
The graph for the reading test provides an interesting contrast:
The color codes are the same as for the math test, with the outliers again being Middlesex and Lexington. The R2 suggests a much poorer correlation for this test than for the math test.
Of interest here is the relatively poor performance of Richmond's economically disadvantaged students. Those same students performed near the fitted line on the math test. In light of the very low scores on both tests, even after considering the large percentage of economically disadvantaged students in Richmond (see above), the implication is that Richmond is doing a poor job teaching both subjects but an even worse job teaching English to its economically disadvantaged students. Given Richmond's large proportion of economically disadvantaged students, that is consistent with Richmond's position as fourth worst division on the math test and worst of all on the reading test.
I used to think that, despite the obvious rigging of the Accreditation scores, the SOL scores were fairly honest. Then I learned how the Maggie Walkerscores go to the high schools the kids would attend if they did not attend Maggie Walker (more likely, "might attend"; the adjusters ignore the reality that many of these kids would attend private schools if they did not get into Walker). In contrast, the schools -- esp. the middle schools -- can ship their troublemakers off the CCP (Capitol City Program, aka Richmond Alternatative) and have both the troublemakers and their scores land at CCP. So the Richmond schools keep the scores of good students who do not attend those schools and get rid of the scores of lousy students. Nice!
This year they put up the SOL scores on July 1. I had the division and Richmond math scores downloaded when they took the database back down. When I filed a Freedom of Information Act request for the data, they denied the request. Their story, in an affidavit they filed after I sued them, is:
The alternative assessments given to significantly cognitively impaired special education students, known as the . . . VAAP tests, are always among the last results to be reported, not becoming available to VDOE until the end of July.
VDOE then gives the schoools the opportunity to access the VAAP data thro9ugh a secure web-based portal and reassign the scores appropriately per federal law [i.e., to meet the 1% passing cap]. These reassigned scores are then included in the VDOE data set. If the testing records are released prior to all the data being reported to VDOE, given the much smaller number of children represented in this last batch of test scores, one could discern how the Commonwealth's special education children scored on their tests based upon the impact on overall pass rates.
Let's put aside, for the moment, the absurd notion that the addition of a number of VAAP passing scores that cannot exceed 1% of the number taking the SOL would change the average enough to reveal the identities of the students taking the VAAP. Let's look at the actual change in the Richmond math scores from July 1 to the general release on August 20:
| School | Subject Area | July | August | Change |
| John Marshall High | Mathematics | 27.14% | 70.31% | 43.17% |
| Richmond Community High | Mathematics | 55.64% | 91.27% | 35.63% |
| George Wythe High | Mathematics | 32.28% | 53.18% | 20.90% |
| George W. Carver Elementary | Mathematics | 50.00% | 70.78% | 20.78% |
| Franklin Military Academy | Mathematics | 47.02% | 63.76% | 16.74% |
| Westover Hills Elementary | Mathematics | 45.19% | 60.71% | 15.52% |
| Huguenot High | Mathematics | 27.04% | 42.50% | 15.46% |
| Armstrong High | Mathematics | 20.38% | 35.23% | 14.85% |
| Martin Luther King Jr. Middle | Mathematics | 13.32% | 25.23% | 11.91% |
| Amelia Street Special Education | Mathematics | 44.74% | 55.17% | 10.43% |
| Thomas Jefferson High | Mathematics | 40.38% | 48.59% | 8.21% |
| Lucille M. Brown Middle | Mathematics | 49.65% | 56.85% | 7.20% |
| Overby-Sheppard Elementary | Mathematics | 57.14% | 63.49% | 6.35% |
| Henderson Middle | Mathematics | 23.25% | 29.05% | 5.80% |
| Oak Grove/Bellemeade Elementary | Mathematics | 46.43% | 51.68% | 5.25% |
| Albert Hill Middle | Mathematics | 55.23% | 57.52% | 2.29% |
| Binford Middle | Mathematics | 33.63% | 35.74% | 2.11% |
| Bellevue Elementary | Mathematics | 73.08% | 74.74% | 1.66% |
| Fred D. Thompson Middle | Mathematics | 39.92% | 41.01% | 1.09% |
| Richmond Alternative | Mathematics | 25.54% | 25.93% | 0.39% |
| Ginter Park Elementary | Mathematics | 36.44% | 36.52% | 0.08% |
| Open High | Mathematics | 82.14% | 82.18% | 0.04% |
| Miles Jones Elementary | Mathematics | 46.79% | 45.58% | -1.21% |
| J.B. Fisher Elementary | Mathematics | 56.86% | 55.20% | -1.66% |
| William Fox Elementary | Mathematics | 83.65% | 81.78% | -1.87% |
| Blackwell Elementary | Mathematics | 47.32% | 45.37% | -1.95% |
| J.L. Francis Elementary | Mathematics | 49.28% | 47.29% | -1.99% |
| Mary Munford Elementary | Mathematics | 80.50% | 76.64% | -3.86% |
| Southampton Elementary | Mathematics | 67.61% | 63.64% | -3.97% |
| Fairfield Court Elementary | Mathematics | 73.04% | 69.03% | -4.01% |
| E.S.H. Greene Elementary | Mathematics | 66.67% | 62.50% | -4.17% |
| Elkhardt Middle | Mathematics | 36.62% | 32.20% | -4.42% |
| Summer Hill/Ruffin Road Elementary | Mathematics | 57.40% | 52.17% | -5.23% |
| Swansboro Elementary | Mathematics | 62.31% | 55.47% | -6.84% |
| Patrick Henry School Of Science And Arts | Mathematics | 48.21% | 40.30% | -7.91% |
| Linwood Holton Elementary | Mathematics | 70.13% | 60.57% | -9.56% |
| Elizabeth D. Redd Elementary | Mathematics | 54.65% | 45.03% | -9.62% |
| Chimborazo Elementary | Mathematics | 50.93% | 40.85% | -10.08% |
| G.H. Reid Elementary | Mathematics | 51.45% | 40.52% | -10.93% |
| Broad Rock Elementary | Mathematics | 89.73% | 76.47% | -13.26% |
| Thomas C. Boushall Middle | Mathematics | 51.50% | 34.62% | -16.88% |
| George Mason Elementary | Mathematics | 60.78% | 43.43% | -17.35% |
| J.E.B. Stuart Elementary | Mathematics | 68.79% | 49.63% | -19.16% |
| Woodville Elementary | Mathematics | 55.21% | 35.84% | -19.37% |
| John B. Cary Elementary | Mathematics | 69.05% | 46.74% | -22.31% |
| Clark Springs Elementary | Mathematics | 88.03% | 60.56% | -27.47% |
Whoops! The math score at John Marshall went up 43 points from July 1 to August 20. There is no way on this planet that the small number of passing scores from kids with "significant cognitive disabilities," which is capped by law at 1% of the class size, could raise the average by forty-three points.
Looking at the division scores, the differences are smaller but, on the scale of the SOL's, huge. Here are the divisions whose scored improved by more than one per-cent from July 1 to Aug. 20:
| Division | July | August | Change |
| Covington City | 47.25% | 53.74% | 6.49% |
| Richmond City | 41.76% | 47.00% | 5.24% |
| Wythe County | 55.64% | 60.40% | 4.76% |
| Tazewell County | 63.49% | 66.99% | 3.50% |
| Suffolk City | 61.23% | 64.24% | 3.01% |
| Norton City | 62.01% | 64.97% | 2.96% |
| Lynchburg City | 51.08% | 53.89% | 2.81% |
| Washington County | 68.76% | 71.51% | 2.75% |
| Virginia Beach City | 70.37% | 72.03% | 1.66% |
| Roanoke City | 66.17% | 67.50% | 1.33% |
| Madison County | 60.30% | 61.56% | 1.26% |
| Waynesboro City | 64.65% | 65.80% | 1.15% |
| Hampton City | 65.95% | 67.06% | 1.11% |
| King George County | 69.82% | 70.90% | 1.08% |
| Charles City County | 44.95% | 46.00% | 1.05% |
As you see, Richmond improved its score by over five points over that sixty day period.
It will take some more digging to figure out what is going on here. What is already clear, however, is that there are HUGE variations of the scores from July to August and VDOE wants to keep them (and, doubtless, the reasons for them) secret.
Stay tuned.
As a former RPS employee, I can attest that RPS has been poorly led for MANY years. The 'ol boy network' was certainly alive and thriving for too many years. If you belonged to a certain sorority or fraternity, whether qualified or not...someone was hired for a leadership position.
ReplyDeleteIt was scary. Central office personnel bullying/intimidating their workers...and if you were a teacher or educator who dared to believe differently, and even spoke up and attempted to change the way things were being done...well...consider yourself a target!
The teachers were constantly blamed...not Central Office 'Specialists' who could not handle a classroom to teach a lesson and who were constantly shutting down different instructional strategies...yet oddly enough, these people were never in danger of losing THEIR positions...nope. They simply blamed the classroom teachers.
RPS has gotten what it deserves...unfortunately, it is the students, the VERY ones RPS is supposed to be serving, that are the ones continuously suffering from self-centered, egotistical and myopic leaders.
The kids used to check out the administrators with the beautifully manicured nails and fine clothing driving around in their Escalades....they knew, and know, what corruption looks like.
ReplyDeleteAs a current rps employee, you are exactly right. The CO specialists are all sorority and frat brothers and sisters. You can't get in if you're not in the sorority. You could be the best person for the job and it doesn't matter. You could care the most but you're not in if you're not DST or AKA or KA.
ReplyDeleteThey say it's not true but I'll believe it when they hire a white, non frat teacher with passion for what she/he does.
Go in any building and ASK "what has the CO Instructional Specialist done to help?" Nothing. They just come complain. I'd like to see the Specialist from CO or Vickie teach a lesson at a priority school on the southside.. If you're so damn good and can do it better. .. prove it.