John Butcher Does it Again!

(While I have been busy with a FOIA FIGHT at Richmond City Hall to discover what really happened when former City of Richmond CAO Byron Marshall abruptly quit, John Butcher and Brian Davison have been waging their own FOIA FIGHT with the Virginia Department of Education to pry loose data that shows one way that Richmond public schools (and other localities across Virginia) have been boosting their scores. Check it out!)

IF AT FIRST YOU DON’T SUCCEED, TRY AGAIN AND AGAIN AND . . .

There’s a Whole Lot of Retesting Going On.
On Tuesday of this week, Brian Davison posted the SGP databases he had obtained from his Freedom of Information Act suit against the State Department of Data Suppression (SDDS), aka VDOE.
I’ve been looking at the 2013-14 reading database, 2013spgReading.txt, which is small enough to fit into Excel (the 2014 math database is too large).  In particular, I’ve been looking at the scaled SOL scores and the retest flag.
First some background: SDDS allows a student to retake a failed SOL test during the same administration (pdf at 39) (an “expedited retest”) if the student
o  Failed the test by a narrow margin (between 375-399), or
o  Failed the test by any margin and has “extenuating circumstances,”* or
o  Did not sit for the regularly scheduled test for “legitimate reasons”*
*The division superintendent is responsible for making the determination of what constitutes “extenuating circumstances” and “legitimate reasons.” 
More generally (pdf at viii),
A student is considered to be a retester if the student: 
  • has previously taken and passed a course but failed the associated EOC SOL test and is retaking the EOC SOL test to verify the standard credit and requires the verified credit in order to fulfill specific diploma requirements. Note: Students who have fulfilled their specific diploma verified credit requirements should not be re-tested. 
  • is retaking the SOL test under the expedited retake policy having taken it once during the Spring administration. 
  • is in high school and pursuing a Modified Standard Diploma, had previously taken a Grade 8 Mathematics and/or Grade 8 Reading SOL test. The student is retaking the Grade 8 Mathematics/Reading test to satisfy the Numeracy and/or Literacy requirements for the Modified Standard Diploma.  
  • is in grade 9, has previously taken a Grade 8 Mathematics SOL test, and is participating in a recovery program for Grade 8 Mathematics but is not enrolled in an EOC Mathematics course. This student may be coded retest and recovery.
  • is in grade 9, has previously taken a Grade 8 Reading SOL test, and is participating in a recovery program for Grade 8 Reading but is not enrolled in an EOC Reading course. This student may be coded retest and recovery. 
Looking at the database, we see a student at Marshall, no. 2P66913255795, who shows twelve(!) test results, all of them retests:
image
Note: The database does not tell us the order in which the student obtained these results so I have sorted them in increasing order.
Another Note: This database has a problem with duplicates so these data could represent as few as six test results.
All of the data for this student are retests so these apparently are not expedited retests.  You can read the bureacratese quoted above to try to see where it says a student gets twelve, or even six, retries.  Also note that the retries produced a 39 point score range, perhaps all of it a score increase.
Another Marshall student, no. 5B66913230306, also shows twelve results:
image
It looks like the database has triplicated this student’s results.  The student was within 25 points of the 400 passing cutoff, so he/she got to try again, and again, and again.  Hard to see where it says the kid gets three (or maybe nine) tries.  But it’s easy to see the 28 point score increase.
Then we have the Huguenot student, 5F66913295273, who got five (or as many as ten) retries after scoring a dismal 355.  
image
Perhaps there were “extenuating circumstances.”  In any case, five (or ten) tries were not sufficient to get this one a “pass.”  Perhaps more to the point, this not-very-good student managed to raise the reading score by 44 points through multiple retries.  That suggests, again, that multiple retries are in the interest of the schools.
To summarize the Richmond retries, I told Excel to delete all the Richmond data where the student ID, retest flag, and SOL score were the same.  That took out 842 records (no telling whether some of those were cases where two or more retests produced the same scores) and led to this summary:
image
Looks like Marshall gave more retests than not.  And Armstrong came close.  Huguenot, Alternative, and TJ all were above one quarter retests.  
As we have seen, retries can improve scores significantly.  Do you think we should believe any of the results from those schools?
And do you think that SDDS will try to figure out whether some of these schools are cheating?  If so, would you like to bet me on it?