Are there flaws in Value-added Models (VAM)?

Dan Goldhaber and Duncan Chaplin, 2012 find fault in a recent study by Jesse Rothstein in 2010 that finds flaws in standard value-added models that result in implausible effects for teachers and past student achievement. Rothstein developed the Rothstein falsification test to examine possible biases in VAM. Rothstein’s results indicate bias in VAMs estimates of teacher contributions. The falsification test is specifically designed to ascertain if students are randomly assigned in VAMs. If true these findings could throw doubt on the growing body of research now using VAMs studies in school reform and the national education policy debates.

The Goldhaber and Chaplin research theoretically and through simulations refute the findings of Rothstein.  They find incidents in which the Rothstein falsification test rejects VAMs even when students are randomly assigned and when there is no bias in estimated teacher effects. In this paper they criticize the Rothstein method and reject his hypothesis. They agree that the paper raised important concerns about the ability of VAMs to produce unbiased estimates of teacher effectiveness and this issue needs further study, but the Rothstein test does not provide useful guidance regarding VAMs.

http://www.caldercenter.org/upload/Assessing-the-Rothstein-Test_wp71.pdf