The IE team published an in-depth post over the weekend, raising a few concerns about StatCounter’s methodology (or lack thereof) for reporting browser market share. Their points were interesting to consider, but one of them stood out to me:
You’ll notice some pretty big differences in the weighting of StatCounter versus Net Applications. First and foremost, the most populous country in the world, China, doesn’t make the top 20 for StatCounter, when in fact it represents the world’s largest internet population.
To further explore this problem, we re-ran the StatCounter numbers and weighted their publicly reported individual country browser share numbers by the CIA internet population data. This calculation would then represent a true country or geo-weighted view of worldwide browser data based on the actual world’s internet population.
It’s true that we should be wary of methodology issues that can creep into data extracted from analytics services that weren’t designed with aggregate statistics in mind. StatCounter’s data is often accepted at face value, without any detailed scrutiny. However, I believe this geo-weighting approach they’ve explored may be as flawed as the raw, unadjusted data itself.