Equity, Equality and Quality ...

In Schools, Teacher Quality Matters Most

By  Dan Goldhaber
Reposted from Education Next 

Fifty years after the release of  “Equality of Educational Opportunity”—widely known as the Coleman Report—much of what James Coleman and his colleagues reported holds up well to scrutiny. It is, in fact, remarkable to read through the 700-plus pages and see how little has changed about what the empirical evidence says matters. 
The report’s conclusions about the importance of teacher quality, in particular, have stood the test of time, which is noteworthy, given that today’s studies of the impacts of teachers use more-sophisticated statistical methods and employ far better data. Moreover, many of the Coleman findings foretold debates over school and teacher policy that continue to rage today. 

What the Report Didn’t Say 

The Coleman Report focused on differences in schooling resources available to white and minority students and on the degree of racial segregation in America’s public schools. It was also the first major, large-scale study to try to document the influence of schooling resources on student achievement, and how the influence of schooling resources compares to the influence of student background and socioeconomic status. 

This comparison resulted in the oft-cited finding that “schools don’t matter.” Interestingly, that quote does not appear in the Coleman Report, yet it is widely interpreted as a central conclusion. The actual text is far more nuanced, suggesting that schools are remarkably similar in the way they relate to the achievement of their pupils when the socioeconomic background of students is taken into account.… When these factors are statistically controlled…it appears that differences between schools account for only a small fraction of differences in pupil achievement.

The phrases “small fraction” and “between schools” are important. The finding that differences between schools only explain a small fraction of the variation in student achievement does not suggest that policymakers wishing to improve the lives of students are necessarily hamstrung. That differences in resources do not explain a large share of the differences in test scores between white and minority students (the report focused on African American students) does not necessarily mean those resources do not affect student achievement. 

Not only do we now know more definitively that the quality of schools and teachers do matter, but also, importantly, these are resources over which policymakers have direct control (at least more so than socioeconomic status). And the fact that the Coleman findings are based on differences between schools means that it ignores important differences in resources—teacher quality in particular—that we know today exist within schools.

What Did Coleman Say about Schooling and Teacher Quality?

Beyond the headline finding about the impact of schooling overall, the report contains a fair amount of nuance on which school characteristics do (and, importantly, which do not) predict student achievement. The primary analytical technique used involved assessing the proportion of the variation in student achievement explained by different factors. Across grades and different student subgroups, the Coleman study found that most of the variation in student achievement is within rather than between schools, but a larger share of the variation is found between schools in earlier grades and among more disadvantaged subgroups. 

Regarding teacher quality specifically, one of the key conclusions is that the quality of teachers shows a stronger relationship [than school facilities and curricula] to pupil achievement. Furthermore, it is progressively greater at higher grades, indicating a cumulative impact of the qualities of teachers in a school on the pupil’s achievements. Again, teacher quality seems more important to minority achievement than to that of the majority.

The finding that “teacher quality is one of the few school characteristics that significantly affects student performance” is quite consistent with more-recent research. Also in line with current studies is the report’s finding that “for any groups whether minority or not, the effect of good teachers is greatest upon the children who suffer most educational disadvantage in their background, and that a given investment in upgrading teacher quality will have most effect on achievement in underprivileged areas.” Recent studies, for instance, find that higher funding levels, smaller classes, and more-qualified teachers all have larger effects on disadvantaged students than on other students. 

What characteristics of teachers are predictive of student achievement? 

The report includes various caveats about the findings, including that “many characteristics of teachers were not measured in this survey; therefore, the results are not at all conclusive regarding the specific characteristics of teachers that are most important.” But of the characteristics and attitudinal factors that were measured, “those that bear the highest relationship to pupil achievement are first, the teacher’s score on the verbal skills test, and then his educational background—both his own level of education and that of his parents.” 

Also measured were teaching experience (in years), professional journals read, and teachers’ perceptions of the ability and effort levels of their students.
The finding that teachers’ verbal skills appear to be predictive of student achievement is consistent with later reviews of the factors predicting student achievement and with evidence from the last decade showing that teachers’ licensure test scores are also predictive of achievement. There is far less evidence from research today that teachers’ educational background (having a master’s degree in particular) matters for students. 

One possibility is that teacher degree level was more predictive of teacher quality in the 1960s than it is today. School systems today are not very discriminating when it comes to crediting teachers with a master’s degree (with a substantial pay bump). Most reward the degree regardless of the focus of the master’s work—it is often unrelated to the teacher’s classroom assignment—and pay no attention to the quality of the institution granting the degree. Moreover, a far lower proportion of the teacher workforce had an advanced degree in the 1960s; obtaining such a degree may have been more likely to reflect the quality of those teachers who pursued this credential.

Students assigned to high-value-added teachers are more likely to graduate from high school, go to college, be employed, and earn higher wages.

One finding from the Coleman Report that is rarely mentioned relates to the structure of the teacher labor market. The data collection for the Coleman Report included several questions about where teachers in a school grew up and went to high school and college. As is the case today, “In the Nation, there is considerable evidence that [minority students] are more likely to be taught by teachers who are locality-based, in the sense that they are products of the area in which they teach and that they secured their public school training nearby.” This finding reflects what is now popularly known as the “draw of home” in the teacher labor market. Like much in the world of education, this aspect of the teacher labor market appears not to be very different today than 50 years ago.

The Coleman et al. study has been subject to a number of critiques, including, for example, that the cross-sectional nature of the data used did not support causal claims about schooling effects, and that the percentage of variance explained by different subgroups of variables are sensitive to the order in which these are entered into statistical models. It is worth noting that the report itself addresses many of the issues brought up by critics.

For instance, it reports the findings on the proportion of explained variation associated with entering explanatory variables in different order and notes the possibility thatschool effects were not evident because no measurement of educational growth was carried out. Had it been, then some schools might have shown much greater growth rates of students than would others and these rates might have been highly correlated with school characteristics.

My interpretation of the Coleman Report findings is consistent with the reanalysis and reinterpretation by scholars in the early 1970s: in short, the findings hold up remarkably well. 

What Have We Learned since Coleman about Teacher Quality? 

New empirical work, using better data (e.g., that enable researchers to estimate the relative impact of factors affecting student achievement growth from year to year) and more-sophisticated statistical techniques has, in broad terms, reinforced the Coleman Report conclusion that teacher quality is the most important schooling variable. 

Some of the acknowledged limitations of the data used in the Coleman study—the need to focus on the relationship between teacher variables averaged to the school level and student achievement, in particular—have been addressed by more-recent research. Specifically, the Coleman study was unable to explore the extent to which teacher quality varies within schools or estimate how much of the impact of individual teachers might be related to teacher attributes not associated with those school-level variables.

Researchers today have the benefit of longitudinal data sets that link individual teachers and students over time. This allows for the use of statistical models to estimate the total contribution—that attributable to both observable and unobserved teacher attributes—of teachers toward student test-score gains (often referred to as “value added”). Although these models are controversial, the weight of the evidence suggests that they produce valid estimates of teachers’ contributions to student learning.

The importance of being able to estimate the value added of teachers for both policy and research cannot be overstated. Admittedly, many observable teacher characteristics—
gender, age, an advanced degree, or even state certification of competence—are not ordinarily found to be associated with effectiveness in the classroom. Yet qualities less easily (or commonly) quantified appear to matter a great deal, as the differences between individual teachers have been found to have profound effects. Not surprisingly, teachers who are successful with students in one year tend to be successful in other years; hence, measures of a teacher’s performance in the past tend to be a good predictor of how well future students assigned to that teacher will achieve.  

And recent studies that consider within-school differences in teacher effectiveness show just how important teachers are (see Figure 1). For instance, the median finding across 10 studies of teacher effectiveness estimates that a teacher who is one standard deviation above the average in terms of quality produces additional learning gains for students of 0.12 standard deviations in reading and 0.14 standard deviations in math. These within-school differences likely understate the overall import of teacher effectiveness because, as recent evidence suggests, there are also differences in teacher quality across schools. Despite this, the impact of having an effective teacher (one at the 85th percentile) in a particular school versus having an average teacher (one at the 50th percentile) is several times larger than the differences we typically observe between a novice and third-year teacher.

For instance, a 2015 study by The New Teacher Project found that districts spend an average of $18,000 per year per teacher on professional development, but most professional development programs fail to yield changes in teacher effectiveness that are detectable in student test scores.

We also now know that it is difficult to enhance teachers’ performance by a substantial margin. Most studies find that teachers improve with additional experience only early on in their careers. Gains in average teacher quality after five years are seldom detected, however.

This is despite the fact that school districts invest considerable resources in professional development in an effort to improve teacher performance. 

Finally, although the lion’s share of teacher-quality research since the Coleman Report has focused on the connections between teacher quality and student test scores, new evidence is shining a light on the extent to which teachers affect other long-term non-test student outcomes as well. Important work by Stanford University researcher Raj Chetty and his colleagues finds that value-added measures of teacher quality predict students’ outcomes long into the future. Students assigned to high-value-added teachers are more likely to graduate from high school, go to college, be employed, and earn higher wages (see Figure 2).

This has profound implications: Chetty and colleagues estimate that replacing a teacher whose value added is in the bottom 5 percent of the distribution with an average teacher would increase the present discounted value of students’ lifetime income by more than $250,000 for a typical class (of 
28 students).

Coleman and Policy Debates Today 

Have the last 50 years of education research led us to fundamentally different conclusions about the impact of teachers on the educational achievement of students? There is a bit more nuance to the answer than “not really,” but “not really” comes awfully close to hitting the mark. If anything, the half century of research on student achievement has strengthened arguments for a policy focus on teacher quality. More-sophisticated research has been conducted over the last two decades, since states began collecting longitudinal data that connect teachers and students. This work shows both how different teachers are from one another, in ways not readily captured by their qualifications, and how important these differences are for student achievement and long-term outcomes.

Those who buy the notion that the Coleman Report basically got it right might ask why we have not made more progress in improving the quality of the teacher workforce (or schools more generally). Certainly, one part of the problem is that, 50 years later, we are still debating the extent to which education policy ought to focus on teacher quality, and on the performance of individual teachers in particular. The research showing the important variation in teacher quality within schools and its connection not only to test scores but also to other important outcomes ought to strengthen arguments for teacher-oriented policy interventions. But it is precisely the focus on teacher evaluation—and whether it is connected to student test scores—that is at the center of the most hotly contested education policy debates. 

Recent revisions to the most prominent federal law dealing with school quality—the Elementary and Secondary Education Act—mark a sharp rollback of the federal role in teacher evaluation and accountability. It is not clear whether states and localities will consequently focus less attention on teacher quality, but if this is the outcome, policymakers will have failed to internalize the important lesson of both the Coleman Report and subsequent research: the main way that schools affect student outcomes is through the quality of their teachers. 

Dan Goldhaber is director of the National Center for Analysis of Longitudinal Data in Education Research at American Institutes for Research and director of the Center for Education Data and Research at the University of Washington.