By Daniel Eran Dilger

Published: 12:00 AM EST (09:00 PM PST)

A preview version of Microsoft's Internet Explorer 9 browser engine passed more elements of a web standards test than a random selection of other browsers in the Worldwide Web Consortium's "vastly incomplete" HTML5 test suite, creating a news story that W3C members decried as "incorrect information" based on a "rather buggy test results page."

A variety of sources have reported the published results of the W3C test suite without noting that the browser Microsoft submitted for testing on October 28 was its "hot off the press" Internet Explorer Platform Preview 6, introduced to developers at the company's Professional Developers Conference just days earlier. Microsoft's "platform preview" was compared against Apple's currently shipping Safari 5.0.2 (publicly released two months ago), as well a beta version of Google Chrome 7, Mozilla's Firefox 4 beta 6, and an alpha build of Opera 11.

The results, published and amplified by Microsoft in a blog posting, suggest exceptional HTML5 web standards compliance in the company's upcoming Internet Explorer 9 and depict a significant lead over other browsers in web standards performance.

"Bogus," "buggy test results" used in "creating PR events"

The W3C's internal HTML5 test results were copied to a public web site labeled "Official HTML5 Test Suite Conformance Results," and were subsequently picked up by the tech media to falsely suggest Internet Explorer 9 "outperforms other browsers in HTML5 compliance."

The result was then discussed in emails between W3C members, including Anne van Kesteren of Opera, Maciej Stachowiak of Apple, Jonathan Griffin of Mozilla, Mikkel Toudal Kristiansen of Google, and Kris Krueger of Microsoft.

Opera's van Kesteren wrote, "This test suite is vastly incomplete. Publishing unverified results of a vastly incomplete test suite without a big fat warning is extremely silly. Why was this done?"

Stachowiak responded, "It's also strange that the results include alpha/beta/preview versions of most browsers, but the stable version of Safari [rather than the latest nightly build]. Wouldn't be a big deal other than the fact that this rather buggy test results page was labeled as 'Official' and then picked up in the press as authoritative. We should probably be cautious about the chance of creating PR events based on incorrect information."

Google employee Ian Hickson, the author and maintainer of the Acid2 and Acid3 tests and the HTML 5 specification itself, added, "I agree with Anne that it's rather pointless to be publishing results for this test suite. Realistically speaking the test suite isn't even 0.1% complete yet."

Another member of the email thread, identified only as "miss.verstaendnis@nurfuerspam.de," responded, "Apologies to all. That was just a transmission of a test result to the public list. I did not expect that this result is published immediately." The individual's email address literally means "Miss Understanding" in German, with the placeholder domain name "only for spam."

Microsoft's big push to catch up in HTML5, reach iOS users

Microsoft's currently shipping version of Internet Explorer 8 fails nearly every HTML5, SVG, CSS3, DOM, and JavaScript test across the board. The dramatic reversal of Microsoft's browser performance in the W3C's HTML5 test suite appears to align with the company's recently stated intent to switch gears in pushing HTML5 web standards over its own Silverlight proprietary web-replacement plugin, in large part an effort to reach Apple's installed base of iOS users who can't install Silverlight.

Microsoft has also channeled resources in its money-hemorrhaging Online Services Division to focus Bing web search and maps features on using HTML5 embellishments rather than using Silverlight, an effort the company profiled on stage at Apple's WWDC event this summer. At the time, Apple's Steve Jobs announced that both iOS devices and Safari 5 would gain access to Microsoft's Bing service as a web search option.

The company has also changed its mind about supporting Canvas, an element of HTML5 used to create dynamic, scriptable rendering of 2D shapes and bitmap images within web pages. Canvas, originally created by Apple and contributed royalty free to the HTML5 specification, enables HTML5 developers to create animations and games without needing a plugin helper like Flash or Silverlight.

This spring, Microsoft was promising HTML5 support in IE9 but was still reserving the right to exclude support for Canvas, likely in an effort to protect a market for Silverlight. The company is now including support, albeit in a limited fashion.

Microsoft aces its own tests

However, there's another reason why Microsoft has vaulted from dead last to first place in HTML5 benchmarks: the company has essentially written the W3C's test suite used to generate the scores.

Microsoft has openly contributed thousands of test cases to the W3C pertaining to elements of the HTML5 technology portfolio (which collectively includes CSS, JavaScript and a variety of other components in addition to HTML itself).

Additionally, the test also completely ignores HTML5 features critical to real world web applications, such support for drag and drop, Web Workers, the File API, local storage, and CSS3 transforms and other animations. Further, Microsoft only reports IE's performance against its own tests.

This allows Microsoft's engineers to focus on scoring well on the tests they wrote, while not necessarily competing with other browsers to actually support web standards properly. Internet Explorer certainly does not lead in the performance rankings of independent web standards tests that were not invented by Microsoft, such as Acid3, which focuses on DOM and JavaScript compliance.

Internet Explorer 9 still isn't standards compliant

For example, despite publishing scores that suggest Internet Explorer 9 will pass all of the W3C tests it submitted related to SVG (scalable vector graphics) on its website, Microsoft's data "doesn't actually show SVG compliance as such," noted "Haavard," a blogger who works at Opera.

Haavard cited CodeDread, a site detailing real world SVG support in different browsers. "As you can see, IE9 still does poorly compared to other browsers. So while Microsoft's own page would give you the impression that IE9 has excellent SVG support, that is not the reality," Haavard wrote.

CodeDread's tests, based on the official SVG Test Suite, assigned shipping versions of Safari, Chrome and Opera "A" scores, while Firefox got a "C" and Internet Explorer 8 earned an "F," failing every test. Microsoft's IE9 Previews have only inched up from "F" to "D" over the past year, despite passing with flying colors the limited set of SVG tests that Microsoft invented for the W3C.