Lately Microsoft put a lot effort in testing and this combined with the whole process & dogfooding create better products. Even better they do not hide it but are generous and share details with us. Recently a new test center was announced.
The Microsoft Tester Center showcases the test discipline as an integral part of the application lifecycle, describes test roles and responsibilities, and promotes the test investments required to deliver high-quality software.
Kirill Osenkov - SDET @ C# team shares some facts about his job:
- We develop all of our test code using most recent builds of Visual Studio "10" and C# 4.0.
- We do almost no manual testing - almost all of our tests are automated, which means they can run on a lab machine unattended.
- We run our tests on a variety of operating systems, languages and VS SKUs (e.g. Visual C# Express 2008 on Japanese Vista or VSTS on Windows Server 2008 with Visual Studio 2005 already installed side-by-side).
- The Visual Studio Managed Languages Team has about 30000 functional C# and VB tests which run overnight for about 12 hours on 5 to 10 lab machines (paralleled).
- C# tests are written in C#, VB tests are written in VB, F# tests are written in F#.
- We have a lot of automated UI tests - where mouse moves automatically and keyboard "types" on its own - its fun to sit and watch a test creating a C# project, opening files, typing, using refactorings, navigating, etc.
- We prefer testing on the component level, bypassing the UI.
- Our developers write unit-tests, while we concentrate on the functional testing and end-to-end scenarios.
- We reach more than 70% code coverage with our tests - this is a required minimum for our team.
- We use TFS (Team Foundation Server) for source control and work item tracking.
- Many of our tests use a new data-driven test framework where our test scripts are written in XML and are interpreted by a script-like engine.
- Most of Visual Studio tests are out-of-process - our tests attach to a Visual Studio process and control it using Remoting.
- We have a "bug-bash" day about once a month where the entire team just uses Visual Studio and logs bugs.
- We also have AppWeeks, where we form groups and build applications using our product.