Here is an excerpt from Charles Wheelan discussing the difficulty in assessing school outcomes:
We don’t really even know which schools are good schools.
Think of it this way: If a golf pro gives Tiger Woods a lesson, and a different golf pro gives me a lesson, can we conclude that Tiger‘s teacher is better than mine because Tiger beats me by three shots in a match after our lessons?
That’s usually how newspapers and real estate agents pick the “best schools.” The true measure of quality — with golf pros or elementary schools — is how much value they add. In education, that turns out to be extremely difficult to calculate.
You may be thinking that we could probably fix this with a simple pre- and post-test — let’s measure what kids know when they walk in the school door, and then measure how much they know when they leave, one or four or eight years later. The best schools will be the ones with the students who show the most improvement.
Not exactly. Gifted students don’t merely begin at a higher level, they also learn at a faster rate. So, to stick with our athletic example, suppose that neither Tiger Woods nor Michael Moore has ever played tennis before. If we give them each a tennis coach, can we evaluate the quality of those coaches based on the subsequent outcome of a tennis match between Tiger Woods and Michael Moore? (Pause for a moment and consider what a great pay-per-view event that would be.)
So schools with high test scores may or may not be doing a great job; perhaps their students are capable of much more. And conversely, some schools with middling or poor test scores may be doing a terrific job educating students who would otherwise be failing abjectly.