Advocacy

  Myths
  Press

Dojo (HowTo)

  General
  Hack
  Hardware
  Interface
  Software

Reference

  Standards
  People
  Forensics

Markets

  Web

Museum

  CodeNames
  Easter Eggs
  History
  Innovation
  Sightings

News

  Opinion

Other

  Martial Arts
  ITIL
  Thought


HOW ARE STATISTICS BIASED?
Here are a few ways that the statistics get bent.

By:David K. Every
©Copyright 1999


Not only is the press biased, but so too are some companies that do tracking of statistics (especially in the computer market).

When you compound their errors on top of the presses spinning those errors, things get way out of hand - and the truth is no longer recognizable.

SPA

The SPA (Software Publishers Association) tracks Software sales. But they do a really poor job of it. They generally ask members (stores and software companies) to submit results based on what they estimate their sales are like for the previous quarter (or year) and then take those predictions and estimate total sales. Anyone who knows anything about studies should see a few holes -

  • User polls are almost never accurate. Each company makes mistakes on their reporting and this can make the ending results much worse.
     
  • Often the results are predicted based on sales into the distribution channel. But that is not sales to consumers and does not factor in returns. Some titles have return rates as high as 50%.
     
  • SPA takes into account only the largest companies. Microsoft alone get 60 cents on every software dollar in the PC marketplace - yet well under half that number in the Mac market. That means that the platform that has lots of smaller software companies (Mac Market) - is automatically short-changed and tracked worse.
      
  • On sales projections SPA does not account for hybrid sales (titles that have both Mac and PC in the same box). Usually all these sales get registered as PC sales - which is a double hit. Macs are not getting their volume, and PC titles are getting credit for Mac sales. This favors the PC.
     
  • The SPA makes no effort to accurately track a markets new sales vs. upgrade sales. But the difference to the consumer is dramatic. Users are often not pleased to be forced to upgrade their software. So often Windows sales are not happy consumers rushing out to buy the latest and greatest - but consumers forced to buy new titles because their 16 bit titles no longer work right under Win95.
     

Then there is creative accounting -

On September 11, 1995, http://www.spa.org/research/95q2noam.htm, the SPA announced that for the first half of 1995, Macintosh software sales in North America were $473.2 million, down 4.1% from the same period in 1994.

On September 25, 1996, http://www.spa.org/research/releases/96Q2nort.htm, the SPA announced that for the first half of 1996, Macintosh software sales in North America were $603.9 million, down 12.5% from the same period in 1995.

So going from $473 million in '95 to $604 million in '96 is down 12.5%? I do not know how they do math, but I get about 28% growth!

The SPA explained this (only after they received lots of complaints). It turns out that they were revising the previous years numbers (based on unreported sales, etc.) all during the year - and comparing the previous years "revised" numbers to next years reported numbers. But if one were to assume that this years projections were off by as much as the last years - then sales were still way up. Also at no time did SPA have a press release announcing the real numbers for the previous year and stating that they had miss estimated those numbers (which were also up over the previous year - despite the report to the contrary).

It seems that the SPA gets quoted far more often when they have doom and gloom reports about Apple - and so they cater to that, and make little attempt to be objective or accurate - especially when inaccurate gets more attention.

Conclusion

I will add to this section as time goes on. But for now it is quite obvious that spin effects things. It is not only the press that is spinning things either. Companies make nice with MS and Intel because they see a cut in it for themselves - of some sort or another. If the truth is likely to upset their readers or advertisers, then the truth becomes unimportant, or creatively interpreted. Well what else is new? We just have to smarter than they are, are keep our critical skepticism in place.


Created: 02/18/97
Updated: 11/09/02


Top of page

Top of Section

Home