Lies, and why people get annoyed with MS.
Apple sponsors independent companies to do studies on -
"Which computer is easier (or less expensive to use) the Mac
Apple has always won -- by a land-slide. These companies are
often research firms and Apple does little more than request
the study be done. Since Apple is better by far than
Windows, Apple is quite fair and objective about what to
test. The information on most of these studies are
available, and most (except for the most flaming of
Microsoft Moonies) agree that the studies were fair.
So Microsoft decided to use marketing instead of science
to achieve the same results. They sponsored the most
offensive and extreme example of survey-stacking that I have
ever seen. MS pulls this type of stacking all the time, but
at least this time they got challenged by Apple and IBM. MS
was (for once) called to the carpet for their marking lies
-- but the media pretty much brushed over it, choosing
instead to report on the study and not the controversy. This
is a perfect example of why MS is disliked by many. If they
were trying to make their product better then there is no
problem -- but they are not above deceit when they fail to
achieve the desired results. Bill Gates says things like
"Truth is a matter of Perspective" and writes an article
about that for the New York Times. Not only does that fluff
get printed, but the New York Times just happens to have
many business dealings with him, and just happens to be a
big Apple-Bashing Paper. So MS is actually believing their
own hype, and no longer recognizes that there is anything
called truth -- its all subjective to them, and if they can
distort truth, they feel they have done their job.
MS chose to not only sponsor the study, but then tried to
hold their methods and the study itself in complete secrecy,
and only show the press release. The tactic worked pretty
well, and I still can't find the study or any documentation
of it (I've gone through IDC's and Microsofts sites), but
Apple/IBM were able to get the results, and made a small
stink over the studies.
Below are some of the highlights on how to taint a study
to achieve your ends.
- Microsoft hand-picked the testees (not testes)
instead of letting IDC do so. Who did they choose from?
MS chose the participants from MS office users.
MS-Applications are generally considered the best
available for PC's, and often considered the worst
applications available for the Mac, and are not even
available for OS/2. Imagine how tainted the pool of users
is to begin with.
- Microsoft hand-picked the test criteria. They chose
tests in which Windows excelled, and in which the Mac did
things a little differently. There can't be much more
blatant study stacking than that.
Or MS chose things Mac users wouldn't normally have to
do. Example -- install a print driver for the Mac. Macs
have most of their print drivers preinstalled, so not
only is this unneeded on the Mac, but it is confusing
because Mac users don't usually have to do this.
In keeping with that theme MS also only tested the
sub-set of OS
functionality that they did do well, so it was not an
exhaustive test -- only a test of what they did best.
- Microsoft hand-picked the testing terms. They would
choose particular naming conventions that perfectly
matched Windows, but was likely different from the Mac or
OS/2. Example -- MS would say find a file "between" a
date -- the Mac uses the term "around" a date. The
studies are timed and any hesitation or error shows up in
MS's favor. In Apples studies and others, specifically
neutral terms are chose so as NOT to taint the
- Microsoft factored out of the test procedures things
that would show their OS's flaws. There was no evaluation
for troubleshooting a problem (which is much easier on a
Mac), no allowance for learning the system (Mac is easier
to learn/use), and nothing that could be considered
Only Microsoft could taint a study so blatantly and call
that "unbiased". It was a revolting example of MS trying to
lie to people -- and to make matters worse, the press
basically ignored it. One small blurb by Infoworld because
IDG (he parent company of both Infoworld and IDC who did the
study) wanted deniability of bias. That was it. If Apple,
IBM or anyone else had spewed that garbage I believe the
study would have been dissected and the company lambasted --
but most magazines and Newspapers don't want to challenge
one of their most affluent advertisers.
MICROSOFTS PRESS RELEASE
Windows 95 Users Outperform Mac & OS/2 -
REDMOND, WASHINGTON, U.S.A., 1995 NOV 22 (NB) --
International Data Corp. (IDC) has released results of a
productivity test which compares tasks performed on the
Windows 95, Macintosh, and OS/2 operating systems. Overall
test results claim to show Windows 95 users performing 19%
faster than Mac users, and 50% faster than OS/2 users.
Additionally, the results show 76% of Windows 95 users
completed eight or more of the tests successfully, while
only 58% of Mac users achieved the same success level,
followed by 31% of OS/2 users.
Working with Andre Associates of Oakland, California, IDC
says it developed 10 families of tasks which included:
managing and printing local and networked files; managing
documents and applications; checking system resources;
creating an alias or shortcut; customizing the desktop; and
attaching an external CD-ROM drive.
The tests were administered to a focus group of 54
Windows 95 Preview Program participants, 55 Mac users, and
52 OS/2 users. Unlike many studies which compare performance
on applications, IDC says this study focused on tasks which
identified operating system performance. IDC says "minimal
interaction" with applications was controlled through
identical applications in each environment.
IDC also found the Windows 95 group was able to finish
the assembled tasks much faster than the other two groups.
Eighty-five percent of Windows 95 users performed the test
in less than one hour, while 47% of Mac users completed the
test in the first hour.
Responding to the results, Jeff Price, a product manager
at Microsoft, told Newsbytes, "A lot of people will be
surprised with the results of this test. Apple's OS
(operating system) has long been positioned as the leader in
ease- of-use and best performance, but now there is hard
evidence that the mantle has changed to Microsoft. These
results show Windows 95 is now the productivity benchmark
for operating systems."
IDC is an independent consulting and testing organization
which has been commissioned to survey and test products and
services of many large computer Hardware
and software companies, including Apple Computer.
Newsbytes learned this test began with a large pool of
users with general computing skills. To evenly weight the
sampling groups, IDC used three matched users, one for each
OS. This means, for example, if a user in one OS test group
had two years experience using a word processor and a
spreadsheet on a local area Network,
matched users for the other two operating systems were used
to balance the study.
"This study shows the tens of thousands of focus groups
we tested to develop Windows 95 has paid-off as planned,"
continued Price. "Our major goal was to make Windows 95 easy
to use and these results are the hard evidence which shows
we accomplished our goal," he said.