VDOE admits that the accreditation scores are "adjusted." In fact, the scores are so tortured as to be meaningless.
Note: These data embrace a spectrum of tests but VDOE boils the scores down, generally, to a single number for each subject at each school. See 8VAC20-131-300.C.1.d. Thus, the elementary English score is produced by averaging scores of the reading tests in grades 3 to 5. So please be aware that the terms "test" and "score" below probably refer to results of several tests, generally at several grade levels. The score distributions tell part of the story. The math distribution is the less complicated so we'll start there.
The passing score is 70. Recall that the State both designs the tests and sets the passing score so the location of the line hardly comes as a surprise.
For English, there are two passing scores: 75 for the elementary schools, 70 for all the rest. So we have two distributions.
Interestingly, the elementary English test is harder and has a higher passing threshold. In any event, it's hard to flunk on any of these tests.
Richmond was up to the challenge, though. Statewide, only 6.5% of the schools failed to be fully accredited. 30.4% of the Richmond schools failed.
How Big the Boost?
In contrast to the past, the VDOE Web site gives a fairly honest description of the accreditation calculations:
What it doesn't tell us is the size of these "adjustments."
At my request, the always helpful Chuck Pyle of VDOE sent along a spreadsheet showing the one-year and three-year adjusted scores of the Richmond public schools (send me an email if you'd like a copy). With those data in hand, it's straightforward (albeit tedious) to pull the actual scores (with the high school scores already boosted by stealing the Maggie Walker scores) and compute the boosts from the accreditation adjustments. I'll stick to the English and math tests but the data are there for anyone who is interested in the other tests.
Let's start with the high schools. The boosted scores in English averaged 2.6 points higher than the Maggie Walker enhanced scores in English; 2.5 points higher in math. The three year boost (obtained by using the three year average of the adjusted scores) was 1.5 points in English but 22.4 points in math. You read that right: The average 3-year boost available to the math scores was 22.4 points. Said otherwise, the math accreditation scores could be only distantly related to the actual 2012 SOL scores. Here are those numbers in a table:
Here "Boost1" is the accreditation score minus the score on the Report Card. "Boost3" is the three year average of (boosted) accreditation scores, again minus the actual 2012 score.
The table by school shows the details. There are two entries for each test: The "Current" accreditation score and the "3-yr" average accreditation score. The "Pass Rate" column (named by VDOE, not by me) holds the current or 3-yr accreditation score. Note: This terminology implies that they are reporting the actual pass rate. But this is NOT the pass rate reported on the online School Report Card; it's the "adjusted" pass rate, i.e., the (boosted) accreditation score. That is, they are trying to sell the fiction that they are reporting an actual pass rate, not a manipulated number. This is the same kind of rank misrepresentation that in 2004 converted a 73.7 math score into a "perfect score" and embarrassed the Governor.
The last three columns are mine. "Report Card" contains the actual score from the online School Report Card. "Boost1" and "Boost3" again are the boosted 1-year and 3-year accreditation scores ("Pass Rates") minus the Report Card score, i.e., the scoring boosts available from the "adjustments."
Here is the table for the Richmond high schools.
Notice the Huguenot scores. That school flunked the math test with a score of 37. The VDOE spreadsheet reports an accreditation score ("Pass Rate") of 42, a 5 point boost. That still didn't make the required 70. The three year average, however, was 75, a 34 point boost over the actual 37 score. So Huguenot is fully accredited in math, despite a miserable test score this year.
Here we see the two schools that failed in math: Armstrong and Franklin. To the point of "adjustments," we also see that 3-year boosts ranging from 20 to 34 points saved Wythe, Huguenot, Marshall, and TJ. So, in fact the passing score was not 70; it was as low as 37.
Got that? Huguenot's dismal 37 gets boosted to a passing 71, which looks to be about the same as Community's actual, unboosted 72. Thus this profoundly dishonest process makes a grossly failing math program look as if it's within one point of a school that earned a passing score. Perhaps more to the point, the manipulated score at Huguenot tells the school its math program is "fully accredited," implying that a math program that has failed miserably does not need to be improved.
The middle schools enjoyed smaller one-year average boosts but a 24.3 point 3-year math boost:
The middle school table is slightly more complicated than the high school table. For example, AP Hill scored 57 on the math test and three points of "adjustments" left it at 60, ten points shy of accreditation. The three year average, however, raised the actual 57 point score to 82, comfortably above the required 70. Binford, in contrast, got a five point boost from the English adjustments, which raised it to a passing 71. Binford did not need the 13 point boost that was available from the 3-year average so VDOE reported Binford at 71.
Get that: Binford's 66 is reported at 71; Hill's 57 is reported as 82! Can you spell "ludicrous"?
I don't know what to make of the negative boosts at Alternative and Boushall.
The average boosts for the elementary schools were yet smaller.
But the effects were dramatic.
In all, five elementary schools (Greene, Fairfield, Stuart, Oak Grove, and Summer Hill) were saved by the one-year boosts while eighteen(!!) (Blackwell, Chimborazo, Redd, Reid, Mason, Carver, Ginter Park, Fisher, Francis, Cary, Jones, Oak Grove, Overby-Sheppard, Southampton, Summer Hill, Swansboro, Westover Hills, and Woodville) were spared by the 3-year boosts. Oak Grove and Summer Hill are twofers, both accredited by way of one-year boosts (4 and 5 points) in English and 3-year boosts (26 and 21 points) in math.
The one-year and 3-year dichotomy again produced strange results. Thus Holton scored 70 on the math test and was boosted to 71 (passing by one point). Just down the street and around the corner, Ginter Park got a 36 on the math test but a 34 point 3-year boost transformed that to a passing score of 70. Looking at the accreditation numbers, one might think that Holton and Ginter Park both did OK; in fact, Holton passed a tough test while Ginter Park failed miserably.
In terms of a graph, the English scores are seen to be boosted in most cases:
Oak Grove and Summer Hill (the rightmost red points) missed the 75 required of elementary schools but got "adjusted" into accreditation. As we saw above, King and Binford (the leftmost red pair) also got "adjusted" above 70 and into accreditation.
The graph for the math test is more dramatic:
The 25 schools left of the red line and above the green line bombed the test but were accredited in math. That's twenty-five of forty-six schools that scored below 70 but were accredited in math.
After the other subjects and the graduation requirement, nineteen Richmond schools were fully accredited despite bombing the math test and two (Oak Grove and Summer Hill) were fully accredited in spite of flunking scores on both the English and math tests. Ginter Park leads the Parade of Horribles with a 36 math score "adjusted" to 70.
The "Adjustments"
8VAC20-131-300, in the Accreditation portion of the Standards of Quality regulation, provides, in relevant part:
A search of the Virginia Administrative Code for "pass rate" produces the following result:
The first two are regulations of the Board of Education. 8VAC20-131-300, which launched this inquiry, is quoted in relevant part above. 8VAC20-630-40 provides
Hmm. That's not much help.
8VAC20-131-5 defines "Eligible Student":
The two exceptions there relate to students with disabilities, whose participation in the SOL testing is prescribed by the applicable IEP or 504 plan, and LEP students enrolled in Virginia schools for fewer than 11 semesters.
The regulation at 8VAC20-131-280 creates the three-year average score option and allows the accreditation to be based (in unspecified part) on the "number of students who successfully complete a remediation recovery plan."
The regulation does not otherwise define the term "pass rate." Standing alone, the regulation would seem to use the term in its usual sense of number passing divided by number taking. Yet the VDOE Web page (quoted above) explicitly provides for "adjustments" to the pass rate that go considerable beyond the exceptions set out in the regulation. The VDOE procedures for calculating the accreditation "pass rate" exclude the following test records from the calculation:
Under "Accreditation Benchmarks," the procedures further provide:
Can you see VDOE's thumb on the scale here?
All these "adjustments" go in one direction: Up. All these adjustments are totally unnecessary: VDOE writes the test and can make it as easy or hard as they want. The adjustments only contribute opacity to the process . . . and, as we see above, create opportunities for accreditation scores that are divorced from a school's actual performance.
Your tax dollars at work.
|
A forum to discuss Richmond, VA Public Schools and the politics of the City of Richmond
Thursday, November 15, 2012
Hard to Flunk? SOLs Scores & More ...
A Must-read: My friend, John R. Butcher, a/k/a "The Cranky Taxpayer" explains how RPS and VDOE conjure those SOL scores.
Subscribe to:
Post Comments (Atom)
No comments:
Post a Comment
Remember: I will review all comments before posting and if you wish your information to remain confidential, please know that I will honor your request.