February 3, 2012
More on the revelation that Claremont McKenna College had submitted falsified data about students’ SAT scores to U.S. News and World Report and other college ranking services. In an interview with a campus newsmagazine, Pamela Gann, CMCC’s President, blamed “inadequate controls” for the misrepresentation, the New York Times Blog, The Choice, reports:
The misreporting occurred because “a sole person had too much authority over reporting,” Ms. Gann said. People with knowledge of the matter identified that person as Richard C. Vos, a vice president and dean of admissions at the college who resigned last weekend. “If you look at how flat the reported data were, it did not raise suspicions,” Ms. Gann said. “Unlike corporate controls over financial matters, we didn’t have internal checks and balances in place.”
I think that’s likely right: that admissions officers who had an interest in the data’s looking good were also responsible for reporting the data to others. With adequate controls, the data would have been entered by admissions staff but reported by the office of institutional research. (I still have this question: were SAT scores of individual students falsified as they were entered into the college’s MIS system, or were just the aggregate scores derived from all those individual records doctored? If the latter, I wonder why institutional research didn’t catch the error earlier, because they could have checked the aggregate scores. If the former, then we have a more serious issue.)
Lost sight of here is the question of why we might care about the mean or median SAT scores of a college. Is a college better because its students SAT scores are higher? Any scheme of rankings that includes SAT scores is, in essence, saying that’s so. But does that make sense? What makes a college better, I think, is whether students are likely to learn there: a ‘better college’ is one where students make greater gains in learning while at the college (in comparison with other colleges), not whether the students were more gifted students when they started. (I’m assuming that ‘more gifted’ students are those with SAT scores, but let’s note that’s subject to considerable argument.)
That more gifted students want to attend a particular college may be a favorable circumstance for a college, but does it make it ‘a better college?’ Not in my view: I believe a better college is one that induces more learning on the part of the students it enrolls. That’s a value-added approach.
In The Politics, Aristotle asks how flutes should be distributed to flute players: should the best players receive the best flutes? He says yes, but then he is no believer in human equality; he believes more talented people are more deserving. If our goal is to provide a quality education for everyone, then Aristotle’s answer is less convincing. And Aristotle already supposes he knows which are the best flutes as well as who are the best flute players. Leaving aside the question of whether SAT scores tell us who are the best students, how do we tell which colleges are the best? By seeing which colleges students with high SAT scores want to attend? Or by seeing which colleges add the most value (learning) to their students?