When I was applying to Ph.D. programs in 1996, I consulted the National Research Council’s rating of anthropology departments. The data were relatively fresh, the rankings were clearly numbered, and I had the confidence that they had been compiled by a neutral observer.
After a long wait, the new ratings have been released. Unlike the previous version, these rankings are unlikely to settle any debates. They rely on departmental data from 2005, they employ mind-boggling complex methodologies, and end up with wildly disparate conclusions.
As pointed out in Inside Higher Ed, by reporting confidence levels and not absolute rankings, the results end up conveying very little. The Communications Department at UT Austin could be the #1 program in the country or the 69th. Nonetheless, Boston University already trumpeted the performance of its departments on its website.
The problem is, BU reduced a very complex system to a single number. The NRC refused to endorse one system of ranking over another, so BU can be accused of selectively highlighting the most favorable outcomes. For instance, Geography, one of BU’s top-rated programs in terms of faculty research, sends only 35% of its graduates onto academic jobs and has one of the lowest percentages of female faculty in its field. As exhaustive as the NRC data are, without a clear understanding of what constitutes academic excellence, the rankings question will go unanswered.