2021 SOL Results, First Cut

by John R. Butcher

cranky

2020 was the first spring since 1998 without SOL tests in Virginia. Then came 2021, when participation in the testing was voluntary.

The VDOE press release says, “[2020-21] was not a normal school year for students and teachers, in Virginia or elsewhere, so making comparisons with prior years would be inappropriate.” The first line of the very next paragraph of the press release then quotes the Superintendent making a comparison: “Virginia’s 2020-2021 SOL test scores tell us what we already knew—students need to be in the classroom without disruption to learn effectively.”

Let’s look at some data and see whether they offer any principled implications.

But first: As we have seen, economically disadvantaged students (“ED”) underperform their more affluent peers (“Not ED”) by around twenty points, depending on the test. This renders the school and division and state averages meaningless because of the varying percentages of ED students. Fortunately, the VDOE database offers data for both groups. Hence the more complicated analyses below.

To start, let’s look at the numbers of students tested by year for reading in Richmond and statewide.

As we might expect, the ED tested counts dropped in 2021 even more precipitously than the Not ED. In Richmond, the Not ED decrease was about double the state average, the ED, nearly three-fold.

The pass rates (of those fewer students) fell, relative to 2019, but not as far as we might have feared.

The Richmond ED score drop was nearly twice the state average; the Not ED, a bit short of 1.5 times.

Note, however, that even a 6.7 point drop in the pass rate is huge.

We can only speculate about the effects of the various factors that might lie beneath these numbers. Those factors might include:

• Quality of the online instruction,
• The students’ capability to learn online,
• Students’ efforts in the presence of an online teacher,
• Parental desire to see results for their children,
• Parental opposition to SOL testing,
• Parental concern for COVID exposure during the testing, and
• Relaxed graduation requirements.

The ED/Not ED difference increased, both in Richmond and Virginia.

The math counts show decreases similar to the reading.

Those who took the math tests, both ED and Not, did not do well.

And, again, the Not ED/ED gap of those tested in ‘21 increased.

The H&SS test counts were so low that the pass rates must be close relatives of meaningless.

Hmmm. Even so, the Richmond Not Ed pass rate edged above the state average.

The science counts were close to the math numbers.

The writing data painted yet another messy picture.

Note the increase in the Richmond ED rate. Looks like that 13.3% were a special group.

There is one clear inference available from these data: The test results may well have been helpful to individual students but making the SOL testing voluntary made the collected results meaningless. Those collected results do not even give any measure of “what we already knew,” although they are not inconsistent with the notion that the online instruction did not work well. We’ll have to wait for the 2022 results to get a clearer idea of the COVID effects.

That’s enough for one post. It will take some time to mine the information in this year’s data; if you’re interested, please stay tuned.