Are there flaws in Value-added Models (VAM)?
Dan Goldhaber and Duncan
Chaplin, 2012 find fault in a recent study by Jesse Rothstein in 2010 that finds
flaws in standard value-added models that result in implausible effects for
teachers and past student achievement. Rothstein developed the Rothstein
falsification test to examine possible biases in VAM. Rothstein’s results indicate
bias in VAMs estimates of teacher contributions. The falsification test is
specifically designed to ascertain if students are randomly assigned in VAMs.
If true these findings could throw doubt on the growing body of research now
using VAMs studies in school reform and the national education policy debates.
The Goldhaber and Chaplin research theoretically and
through simulations refute the findings of Rothstein. They find incidents in which the Rothstein
falsification test rejects VAMs even when students are randomly assigned and
when there is no bias in estimated teacher effects. In this paper they criticize
the Rothstein method and reject his hypothesis. They agree that the paper raised
important concerns about the ability of VAMs to produce unbiased estimates of
teacher effectiveness and this issue needs further study, but the Rothstein
test does not provide useful guidance regarding VAMs.