Vista Security by the Numbers
|
Last week, Jeff Jones, Microsoft's security strategy director, released a rosy report about Windows Vista's security progress. Counting Jones' way, Vista has a pretty good 90-day track record compared with other operating systems. But counting another way, the vulnerability number is much higher. |
By Jones' reckoning, Microsoft disclosed five security vulnerabilities during Vista's first 90 days of availability. He then shows how much better Vista looks compared with other operating systems, including Windows XP, during a comparable 90-day period.
Jones makes the classic mistake of using the number of disclosed vulnerabilities as a measure of Windows Vista security. Microsoft has taken this approach before, in what I described as "the counting problem."
Three years ago, Microsoft Chairman Bill Gates sent an executive e-mail touting Windows Server 2003's improved security based on a counting of security alertsnine "critical" or "important" compared with 40 for Windows 2000 Server, during both products' first 320 days of release. In early 2004, Microsoft had a mechanism by which anyone could search and count up the security notices; the utility vanished sometime after I wrote about the counting problem. By my count, including all Windows Server security alerts, Version 2003 had 20 vulnerabilities, compared with 28 for Windows 2000 Server during the products' first 11 months of release.
Microsoft CEO Steve Ballmer also bandied around the number of alerts as a measure of improved security. So, I went back and compared Windows NT 4 Server with Windows Server 2003. In the six months before April 2004, when Ballmer was touting alerts as a measure of improved security, Microsoft issued fewer security vulnerability notices for Windows NT 4 Server than Version 2003. By Ballmer's alerts measure, NT 4 Server would be more secure than Windows Server 2003. Realistically, would it really be?
Apologies to Jeff Jones, but he has fallen into a similar counting trap. His report casually mentions the Department of Homeland Security National Vulnerability Database. It's a great utility, which I highly recommend. How does Vista measure there? Four vulnerabilities during the 90-day period Jones cites, but another 11all in the past three weekssince.
The Vista information doesn't reflect components Microsoft considers to be part of the operating system. Alerts mentioning Internet Explorer 7 number about two dozen during Jones' 90-day period. Some relate to third-party applications using ActiveX controls that either crash or exploit the browser.
The March increase in security vulnerabilities is the more telling number. Windows Vista had limited availability until Jan. 30. The real-world pounding on Vista started two months later than Jones' date. The 90 days that matters is under way, and so far, the National Vulnerability Database count is 14. Internet Explorer 7 would add another nine, during the incomplete 90 days since Jan. 30; I didn't count general IE flaws that might affect Version 7, so the number may be understated.
The point: Alerts are not a measure of security progress. I don't doubt that Vista security is improved over its predecessors. Number of security notices isn't the way to make the case.


Comments (16)
Just for you Joe I will repeat the article from bink.nu
Stop bad mouthing Windows Vista on what you "Think" rather than facts that are generally available which you don't take any notice of because you are biased against Microsoft.
"Microsoft security strategy director Jeff Jones this week published a report comparing the security vulnerability profile for various high-profile operating systems in the 90 days of their existence. Surprisingly, Vista came out in front: Vista had 5 vulnerabilities in its first 90 days, one of them fixed, and one pending with a High severity rating. By comparison, XP had a total of 17 vulnerabilities in its first 90 days, 8 of which were rated High, when it shipped in 2001. The surprises, however, come when you compare the non-Microsoft competition. Mac OS X 10.4, a darling of the press, actually suffered from 20 vulnerabilities in its first 90 days, 8 of which were rated High. Worse, OS X 10.4 still suffered from 17 publicly disclosed but unpatched vulnerabilities at the end of those 90 days. "The data doesn't support [Apple's] marketing," Jones writes. Linux fared even worse: Ubuntu 6.06 suffered from a whopping 71 vulnerabilities in its first 90 days, 27 of those rated High. And there were at least 29 unpatched vulnerabilities in that OS after the 90 day period ended."
These are fact Joe and NOT your opinion that is warped against Windows Vista.
Posted by Neil | March 26, 2007 7:39 PM
Neil I always read your posts with such interest. You always have a way of undoing all your previous points. But on this one you out did yourself.
Ever since the Vista launch to general public you've been all over the board saying how Vista is soooooo great and how marvelous it is. How Linux and Mac are so bad and how few apps actually exist for these systems. Then the whole hardware compatibility thing and the games bla bla bla.
And now this! With this you throw all your hard work down the drain. Why in the world did you have to get so concerned about a security article by Joe?
If all you said previously was true then this article would be irrelevant. The whole study would be irrelevant. Who in the world would change or even consider an alternative as Linux or Mac on security alone? Would I change to a super secure OS that has no apps and on which I can do no work, games, fun, multimedia etc.? Obviously not!
The simple truth is this. The fact that you and other Microsoft shills and supporters are banging their drums around this issue means the other products are truly functional alternatives.
Microsoft is out to show people how secure its system is compared to Linux and Mac. But who cares? After all they didn't care before. Back when Microsoft products were not as secure as they are today all they said was get a proper antivirus and firewall.
I say this today. Get yourself a Mac with proper antivirus and firewall and forget about Windows. Get a Linux box with proper antivirus and firewall and forget about Windows. Enjoy Beryl on Linux with your old 128Mb video card and don't worry about over bloated Vista Aero.
Oh and don't worry about these security warnings and reports. After all we didn't worry about them when Windows was the only game in town. We just got our antiviruses and firewalls. If it worked back then it is bound to work today on other systems too.
And above all don't worry about people like Niel who like to spin the perfect Vista view. Sadly enough people like him have come to use the same sad lame excuses Linux supporters had a few years back. You know, driver support oh that's the hardware manufacturer fault. Applications, oh that's just UAC and the greater security settings in Vista. And now these 90 day reports? Give me a break!
Posted by Gerardo Tasistro | March 26, 2007 8:37 PM
Gerado
So you are a mac user are you, well in actual fact I don't think that you needed to say you were.
Your saying how good a mac is would speak for itself.
I am sorry that you do not like the actual statistics on your OS, but facts are facts aren't they, and no one can urgue with them. Not even you.
I bet that you are one of those who just loves those "mac vs. pc" ads which are "grossly" biased against the windows pc. The UAC part of those ads is very inaccurate indeed.
They are like a firewall doing the same thing, well we both know it doesn't.
The UAC is only a "type" of firewall and does not do what the ad says at all.
Oh and one last thing, if macs were so good why are they now using "Intel" ... a pc cpu ???
Not to mention the OS 10.4.9 upgrade bebarkle !
Posted by Neil | March 26, 2007 9:04 PM
Well Neil for starters it isn't "My OS". The fact that you want to see it that way shows a rather erroneous perception of user reality.
Am I an OS? Am I a brand? Far from that. We are not the OS or machine that we use. Only die hards and fanatics like yourself would portrait an OS as "own".
I have Windows, Linux and Mac boxes. I'm not a particular supporter of any one. Although I use mostly my Mac and Linux I do see the benefits of the Windows box. I also see that there is no need to limit yourself to one particular OS and that Windows does not fit all.
I am troubled though by people like you. Who a few years back disregarded the same arguments you now use against Linux and Mac. As an end user and a paying customer I like to pay for what gives me a benefit and/or money directly.
I bought my Mac because it has a BSD (Unix) based OS. Which as a developer allows me to run all the systems I run on my production Linux boxes. It is also the 12" PowerBook G4 (not the Intel one). I realize it is under powered compared to the newer Intel ones. But it gets the job done. Will I buy a new Intel one? Probably when I wear this one out or if a 12" Intel one comes out soon ( I really like the fact that this one fits nice inside my Canon camera pack and on an airplane's tray with peanuts and a drink by the side).
Could I have bought a cheaper Toshiba or Dell? Sure, but I didn't because a) I had the money b) it does a better job at what I needed it for. I honestly don't see the need to buy something that costs less, but is less functional. For example a laptop with Windows.
My desktop PC runs mostly Linux. Although I have about 6 drives with 3 versions of Linux (Mandriva, Kubunto and Gentoo) and then Windows (XP Home, Pro and 2003 Server). So believe me I know a bit too much about these systems to be a fanatic about any particular one.
So please spare me the "your OS" line. I've had my fair share of problems with all systems and believe me the ones that have cost me more money and time to fix have been those on Microsoft platforms. So when someone like you comes to preach a shallow founded line about security all I can do is laugh.
All systems will have their security problems, but Microsoft products have presented me with other more important issues. Lack of support for previous products, upgrades that break previous file formats, web site issues with explorer, etc. Things that take you real time and money to fix. Not just 50 bucks for an antivirus and firewall.
At the end of the day all that matters is what costs me less to get the most out of. Once Microsoft had, with all its nice tools, and edge on this. I used Microsoft products, but as of a few years to date MS has been loosing the edge on this. It's no cybernetic jihad, no electronic vendetta. Just simple economics. I work better on a Mac than on Windows and I'm sure a lot of people out there would if they took a dive into it. Some like you will not. But that is ok. One OS does not fit all. Particularly in an era of web based applications that make the choice of OS less relevant.
Posted by Gerardo Tasistro | March 26, 2007 10:00 PM
Boy can you talk a lot !!
I would like to again attract your attention to what "actually" matters around here mate ... the facts !
Please read them as I first posted, it doesn't matter to me what OS you use, just read the facts.
Posted by Neil | March 26, 2007 11:13 PM
Lets translate Neiltalk to english.
"Boy can you talk a lot !!
I would like to again attract your attention to what "actually" matters around here mate ... the facts !
Please read them as I first posted, it doesn't matter to me what OS you use, just read the facts."
-----
"Man you totally beat my points and I don't know what to say. Nobody answers at Redmond this late at night.
So, uhhh I'll dodge and point in another direction. I'll say something about the facts. Even though you didn't actually question them. Heck you even agree with them. But maybe the readers will have forgotten about that like the forgot about the track record of XP after the first 90 days.
Oh and I'll say something respectful about his OS preferences. Even if I hate to the core the fact that he proves other OSes are viable alternatives."
Posted by Gerardo Tasistro | March 26, 2007 11:26 PM
The facts are:
"Microsoft security strategy director Jeff Jones this week published a report comparing the security vulnerability profile for various high-profile operating systems in the 90 days of their existence. Surprisingly, Vista came out in front: Vista had 5 vulnerabilities in its first 90 days, one of them fixed, and one pending with a High severity rating. By comparison, XP had a total of 17 vulnerabilities in its first 90 days, 8 of which were rated High, when it shipped in 2001. The surprises, however, come when you compare the non-Microsoft competition. Mac OS X 10.4, a darling of the press, actually suffered from 20 vulnerabilities in its first 90 days, 8 of which were rated High. Worse, OS X 10.4 still suffered from 17 publicly disclosed but unpatched vulnerabilities at the end of those 90 days. "The data doesn't support [Apple's] marketing," Jones writes. Linux fared even worse: Ubuntu 6.06 suffered from a whopping 71 vulnerabilities in its first 90 days, 27 of those rated High. And there were at least 29 unpatched vulnerabilities in that OS after the 90 day period ended."
Posted by Neil | March 27, 2007 12:30 AM
Further
I do not work for microsoft mate !
These facts are on bink.nu
If you do not believe me at least look for yourself.
For those of you think that these are not factual figures, they can be easily verified by checking around.
They are correct.
Posted by Neil | March 27, 2007 12:34 AM
Yes Neil. I'm quite aware of the facts. I don't question them.
I do raise the point that your bashing of the issue sets Linux and OS X as viable platforms. Otherwise why worry about security? A platform that has no functionality, use or applications would be dropped from the list before we even got evaluating to the security aspects of it. Aspects which can be easily solved by third parties and proper system lock down.
Posted by Gerardo Tasistro | March 27, 2007 12:37 AM
Neil, I was ready to post these facts when i first read this article. You got there first. How about for the "microft watch" to post some favorible reports for a change.... Like the latest Symantec Report on Internet security that lists windows as the most secure OS.
Posted by evan | March 27, 2007 6:45 AM
@ Neil: it appears as though you're one of these people who defend the black art of turning lies and damn lies into statstics, even when apples and oranges are compared.
Posted by Roy Schestowitz | March 27, 2007 7:08 AM
Wow - sorry, I couldn't get past the first couple of sentences before I knew what all these comments would be about.
There is simply no getting around it. Some idiot out there (actually NOT employed by Microsoft) will try to champion their operating system as opposed to Linux, Mac, etc.
Figure it out, folks: Microsoft (for the time being) is not only the BIGGEST in terms of distribution, but it is also the wealthiest, and (unfortunately for Microsoft) it is absolutely the most arrogant.
In case you didn't know it, the majority of banks and on-line sales and on-line transactions involving one's personal data was also accomplished by users running Microsoft (if you discount the fact that 60% of the internet was still Linux/Sun).
Microsoft will probably ALWAYS be the number one target of naughty people wanting to exploit your personal information for illegal purposes.
That translates into the same CONTINUOUS revelation of security flaws with Microsoft's latest OS (regardless of WHICH OS it is), the same continuous revelation of security flaws in IE (I remind you that you CANNOT get a version of Windows WITHOUT IE). IE is quite literally PART of the OS - shame on Microsoft for trying to exclude IE in their OWN FAVORABLE report about how well Vista is doing compared to all other OS's, without including the newly revealed flaws in IE and in Defender (again - these are BUNDLED with the OS - ya can't get Vista without them).
We all watched Bill Gates flail his hands around in near exasperation when a reporter asked him what there was about Vista that REALLY made it the cat's pajamas.
I find it incredibly insulting - this recent headline I read on the net: "Microsoft Gives Windows Vista an A-Plus for Security".
Wow - Microsoft is very generous here, don't you think? They are giving THEMSELVES an A-Plus. Hey, I am a database developer - I would LOVE to be in a position to work for such a large and successful company. I don't see that happening any time soon, however. And in the meantime, My Mac and my Linux web server and my Mandrake and my Fedora boxes just keep on chugging along.
I couldn't even read through the blogs - it is the same every time: An article calls Microsoft's credibility into question, and some guy out there (with no knowledge or experience with any other OS except Microsoft, with no willingness whatsoever to give a nod to what Mac has done or what the Open Source sector has accomplished) has a problem with it. So doubt my contribution here will be as dry reading for him as his contribution was to me.
Posted by Paul | March 27, 2007 7:46 AM
Evan
If you have had a look at most of the articles written by Joe Wilcox here, you will see that he is NOT interested in those sort of facts.
I am... he's not !
Mostly negativaty around here as far as MS is concerned !
Roy
It turns out that you don't fully disclose things do you ? "Roys loves SUSE" on your weblog hey ??
Why is it that according to Joe Wilcox we MUST disclose whether or not we have a financial interest or work for MS, but you do not disclose the fact that you (by your own admission on your blog) hate MS.
To me it is not fair that this site howls down anyone who dares to stick up for MS.
You are of course entitled to your views but this site is supposed to be an unbiased look into MS and really you have your OWN blog to express your side of it don't you ??
That is the trouble with this site since Joe took over the reigns from Mary Jo Foley, when she was here no mac users bothered to comment, and no Linux users either.
Now its virtually overrun with them, and the MS are being howled down by them!
If Linux is so great why bother with this site, if MAC is so great why bother with this site??
A lot of you (thankfully not all) are very insecure about your OS (MAC and Linux) and that is why you are putting Windows down isn't it !!
You don't like to see figures say that YOUR OS has a very small following so you take it out on Windows users.
These MAC ads are a prime example, and now Linux is starting up as well.
You have yours and we'll have ours.
I suggest rather than tell us how bad microsoft is, that you go and preach to the already converted. Oh ! sorry that why your here, there is no converted, so your taking it out on us instead.
Posted by Neil | March 29, 2007 9:17 PM
Nice, I don't have to sign up to post...
Anyways, Neil you're coming off like an absolute idiot, just so you know(at least in that last post).
As for your "facts are facts" post: are you serious? Facts are whatever the presenter of those facts wants them to be.
Let me give you an example(this is all hypothetical):
Microsoft PR rep Bill cites a survey where an independent company surveyed 1000 people and found that on average people rated their happiness with Microsoft vista as a "9" on a scale of 1-10.
Now in our story, that is a fact. There is no denying it... except it's not true. While indeed it is true that the average rated it as a "9", if we take a look at the raw data in this survey, we might see a very different story. We might come to see that the actual happiness average was 4.6.
"But how can this be?" you say.
Well I was going to leave it at that, but since you asked...
There are 3 different averages available to anyone who wants to put out a statistic. There is the Mean, Median and Mode.
The Mean: This is the most common function used when someone is speaking of averages; it is the adding of all of the data(in this case, numbers from 1-10) and dividing it by the amount of data(in this case, 1000).
The Median: This is the central value in a list sorted into increasing order. For an odd numbered list it is the middle number, and for an even numbered list it is 50% between the two middle numbers.
The Mode: The mode is the most frequently occurring number in a list. There can be more than one Mode in a list.
Now let's look at the raw data from the survey. The first column will be the 1-10 scale, and the second column will be how many people rated their happiness with that number.
10 0
9 300
8 0
7 0
6 0
5 0
4 200
3 200
2 200
1 100
As you can plainly see there is no way that someone could interpret this information favorably just averaging them up(remember, average is vernacular for the Mean), so they used a different average. They used the Mode method of averaging to interpret the data. In doing so they are able to report, without lying, that the average person polled rated their happiness with Microsoft Vista as a "9". However if they had used the Mean to interpret the data they would have arrived at the answer "4.6".
Now there were 2 things in this story that should have warned you at the very beginning that the data was going to be skewed. I've talked about one of them: the use of the term "average". The other one is the term "PR rep"; if it's coming from the company and it's favorable towards the company, it's probably a lie. It's either a lie through a manipulation of the data, by omission, through misstated purpose or sometimes it's just flat-out untrue.
The only FACT I saw about Jeff Jones' report was just that: it was Jeff Jones' report. Everything else was just interpreted data, to be taken at face value(as you are so wanton to do) or to be examined for it's honesty and truthfulness.
Note: I use XP, and I really like it(however I've never used anything else in my adult life) so this isn't me crapping all over Microsoft. It's just a statement about the realities of the business world and the gullibility of most people.
Different Averages: http://www.shodor.org/interactivate/lessons/IntroStatistics/
Do your own research on security flaws instead of being a fanboy:
http://www.google.com/search?sourceid=navclient-ff&ie=UTF-8&rlz=1B2GGGL_enUS176&q=windows+vista+security+flaws
Posted by Aaron | April 30, 2007 5:14 AM
@Aaron:
You forgot that anyone can answer a survey saying they are happy when their mood is angry or sad or...
@Evan:
You DEFINITELY don't work in the IT field, or you would know for a fact that Symantec's anti-virus programs just don't work. (W-A-Y too much dependancy on IE & Windows DLLs.)
@Neil:
Whoever is paying you to inject these comments is making you look a fool sir. Just because one article writes that the sky is brown dows not make it so.
Posted by Dan | August 26, 2007 7:09 PM
neil i am sorry but you do not know one thing i am a beta tester for microsoft i tested vista and it sucks you may like it but i do not it has nsa software in it and there is holes in it for the start more than xp it is not safe becuase of the holes it calls home 2 time a week to microsoft and the gov. it reports all you do and it does not let you do what you can do with it that is how it was made to stop from running anything but microsoft so you can love vista but i will not
Posted by jt | September 19, 2008 8:46 AM