Is one web ratings black box better than another black box? Why should we believe Alexa is better?

Alexa, the Amazon.com subsidiarywhich provides website traffic and rankings information, has just announced it has revamped its rankings system. Since I have the Alexa toolbar installed in my browser, which shows the ranking of whichever website you’re currently looking at, I actually noticed this morning that the rankings were changing. Techcrunch reports that the change has downrated many technology blogs. It has actually increased the ranking for this blog by about one third.

However the announcement gives absolutely no information on how the ranking is calculated or the sources used. There are bold, completely unsubstantiated claims:

– MORE SOURCES: Alexa rankings are now based on more sources of data to give a better indicator of website popularity

– BETTER RANKINGS: The rankings are an even better indication of website popularity due to new and improved algorithms

Alexa has been much criticized for the inaccuracy of its rankings system, which until now has relied entirely on picking up the web browsing activities of those who have installed its toolbar. This is a skewed population, and there are various ways to game the system.

While I don’t necessarily expect Alexa to reveal its exact algorithms, I think people would be far more likely to have confidence in its measures if it gave at least some indication of what the new sources are, or what they look like, or what changes to the algorithm were made. In fact we know less now about how the Alexa rankings are compiled than we did before. Alexa think they’re better. I guess I think they’re better, since I’m ranked higher. But why should anyone else believe they’re better? Tell us please, oh Alexa, just something that supports your claims that Alexa is improved…