To make Internet Explorer as the world’s fastest browser, Microsoft spent last over 5 years to have designed and built an immerse performance lab that is one of the world’s most sophisticated web performance measurement systems.
The IE Performance Lab is a private network completely sealed from both the public Internet and the Microsoft intranet network, and contains over 140 machines. The lab contains the key pieces of the real Internet, including web servers, DNS servers, routers, and network emulators, which simulate different customer connectivity scenarios. It’s like a mini real world Internet.
The infrastructure can be broken into 3 main categories:
- Network and Servers
- Test clients
- Analysis and reporting
and the common tools used in the lab is the Windows Performance Tools (WPT)
For analysis, the lab uses 11 server class machines, each of which has 16 cores and 16GB of RAM. During analysis, each trace file is inspected and thousands of metrics are extracted and inserted into a SQL server. Over the course of 24 hours these analysis machines will inspect over 15,000 traces that will be used for trend analysis.
To measure, there are 4 different types of scenarios measured in the lab on daily basis, Loading content, Interactive web apps, IE “the application”, and Synthetic benchmarks. To reflect the test to the real world, the lab uses the real web pages that represent real world patterns and exercise different browser subsystems.
In order to determine which sites to use for testing, we regularly crawl millions of sites and compile a list of site attributes and coding patterns. We use 68 different data points to determine commonalities across sites – things like the depth and width of the resulting DOM, CSS layout patterns, common frameworks used, international features, and more. From the results we chose sites that best represent the common patterns and diversity of the broader Web.
There are also 5 main metrics to form a multi-dimensional performance, display time, elapsed time, CPU time, resource utilization, and power consumption.
Knowing through all these details, it surely helped Microsoft build the fastest browser over its main competitors. But unfortunately, I didn’t see anything mentioned about usability test. While a secure, fast browser matters a poorer UI design can surely ruin all of these hard work. And unfortunately, UI design and test can’t be done through the Lab like this one Microsoft has built. It’s a human job.[via Building Windows 8]