Tuesday, October 27, 2009

Bad data interpretations make bad policy

A bad interpretation of an LSU study makes for a poor conclusion in this article by Times-Picayune reporter Sarah Carr.

Most of the article is about U.S. Secretary of Education Arne Duncan's unabashed praise for Louisiana's "value added" analysis of student achievement. The model, according to the article, "links the success of educators in the classroom to their teacher preparation program."

There are arguments to be made for and against the value-added model, but that's grist for another mill (although it's a shame that the article uncritically accepts Duncan's premise). Where this article really fails is in its assertion that "novice Louisiana teachers trained through an alternative program called The New Teacher Project on average outperformed experienced teachers in helping their students progress in math, reading and language arts."

If true, that bolsters arguments that teachers don't really need the specialized training provided in schools of education, and that it is not educationally sound to reward teachers for earning advanced degrees.

But the data reported in the article don't really support that conclusion.

The comment is based on a study that LSU did on graduates of The New Teacher Project, which puts prospective teachers on a fast track to certification. Most of Louisiana's Teach for America participants have been certified through a TNTP-like program called the Louisiana Practitioner Teacher Program.

But the problem, as this AFT critique of the study shows, is that the TNTP teachers surveyed in the study were not novices at all, but were in fact veterans of at least two years in the classroom:

This is a critical detail. This means that the TNTP teachers who were labeled
new teachers” were usually third- and fourth-year teachers. They should not
have been considered “new” because they already had been working in the
classroom for roughly two years while they were completing their 18-month
certification program. For this reason, comparisons to other new teachers in
Louisiana are not valid.

The point is that TNTP teachers are not necessarily more effective because of their training, but because of their classroom experience. It's simply wrong to draw conclusions about the relative worth of experience or degrees from this LSU study.

No comments: