Delivering Delightful Performance for More Than One Billion Users Worldwide

Delivering Delightful Performance for More Than One Billion Users Worldwide

Over the past year, we have made major strides in improving fundamentals across Windows.  From enhancing Windows’ speed and reliability to optimizing the performance of its flagship applications to improving Window’s sustainability, we have been working diligently to deliver faster, more reliable, and more efficient computing experiences that our users love.

Among the highlights from the last year:

• We improved frequently used shell interactions like Taskbar, Notifications, and Quick Settings by as much as 15%!
• Getting users into productivity faster by improving the time it takes to get to the desktop by 10% and reducing the impact of startup apps by over 50%.
• Making PCs more sustainable with our new Energy Recommendations and Carbon-Aware Windows Update. We see a 6% average reduction in energy consumption for the users who use the recommended energy settings
• Windows 11 is the most reliable version of Windows ever, with recent updates continuing to improve reliability.
• Microsoft Edge starts up faster than ever before and saves memory with features like Sleeping Tabs, which saves up to 83% of memory for background tabs.
• The new Teams app for Windows is 2x faster while using half the resources.

Today, we’re going to give you a behind-the-scenes peek into how performance improvements are delivered!  Windows operates at a massive scale – with over one billion users and a rich PC ecosystem, and there are innumerable combinations of usage patterns, key apps, and hardware.  Windows is foundational to many products too, each with varying needs – it powers your PC, it is the host OS for Azure, and it is core to Xbox and HoloLens.  With Windows 11, we are also releasing more frequently, increasing the need to protect against regressions and shortening our cycle for driving improvements.  Thankfully, we have built infrastructure and tools that help us find, understand, and fix performance problems at the massive scale Windows operates at.

Delivering Performance at Scale

If you go back 10 years ago, life as a performance analyst in Windows was challenging!  We would develop an operating system for months before it was released to external users as a beta release, and then we responded to bugs that were filed by our users and partners.  When you received a bug indicating users had performance issues in some scenarios, we did not always have the information necessary to root cause the bug.  Reproducing the issue was often a challenge.  Once you have found and understood the issue, how do you know how common it is?  As you can imagine, this process was slow and cumbersome.  Fast forward to today, when it’s hard to even remember those days.

Today we have a richer, more comprehensive story to guarantee great performance.  It starts during the design phase, where teams define strong performance goals to ensure our products meet the needs of our users.  Significant research has gone into users’ expectations, and we rely on that research to guide our goal-setting for user interactions.  We call these “interaction classes”, and they help us set targets and ensure we have the right UX affordances for experiences, whether they’re a nearly instant interaction like opening the Start Menu or longer like deleting files.

During development, we protect performance with daily performance gates that measure key Windows performance indicators ranging from Windows UX experiences, application experiences, resume from standby/sleep, memory and storage, and more.  Every day, we run over one million machine hours of validation.  This helps protect our daily build quality and ensures that teams can be productive.  Once we complete selfhost and evaluate our performance measures on our own devices, we flight builds to our vibrant Windows Insider community who provides us rich feedback on quality issues they observe across a breadth of device configurations and use cases.

Diagnostic data by the numbers: (1) Over 2.5 million cabs per day; (2) over 12 million performance traces per year; (3) over 70.4 billion scenario performance data points per year.

The scale of feedback we receive, both through feedback and perf diagnostic data, is massive.  Over the past few years we have invested in capturing performance and other Windows quality measures via diagnostic data in addition to backend services to analyze the information we get and to identify issues.  For important scenarios, we can now see the distribution of the real-world performance observed by our users across our broad ecosystem of devices.  Performance is a spectrum, not one number, and what you experience on your system may be quite different from another user with different hardware, running different apps, and using their system differently.  Another key innovation for us is the ability to analyze traces at scale and find issues that are on the critical path of a scenario’s responsiveness. This is a rich tool that allows us to tease apart what is causing performance issues in traces – ranging from scenario-specific code, IO delays, all the way down to kernel level scheduling optimizations and even driver quality issues.

Deploying These Techniques to Ensure Delightful Experiences

Delivering a great Instant On experience (Resume from Modern Standby with Windows Hello)

With Windows 11, you can get back to productivity more quickly than ever with Windows Hello and Instant On.  Most laptops today use Modern Standby and Windows Hello to offer a password-less logon experience.  Our goal is to get you back to productivity in seconds, and our latest generation of Surface PCs, like the Surface Laptop 5 and Surface Pro 9 are 10% faster than the prior generations.  We were able to achieve this with performance improvements in silicon, camera drivers, display drivers, and the OS over the last few releases.

A responsive, fluid, and fast Windows experience requires great baseline performance and the ability to deliver that consistently.  The negative performance feedback we receive for Windows Hello is infrequent, but when it occurs, it is often due to longer times to unlock your device.  Here is a look at how Windows engineers improve the baseline performance and consistency for scenarios like Instant On.

• Measurement: All performance analysis starts with adding code to emit diagnostic data events that can be used for measurement or identifying system resource usage. We rely heavily on ETW to measure daily results to ensure we prevent regressions prior to selfhost.
• Optimize the baseline: We analyze ETW traces through automated trace analysis techniques using DataLayer and in some cases manually using Windows Performance Analyzer (WPA). In WPA, engineers deploy a range of analysis techniques from resource usage (like CPU Analysis) to critical path analysis to identify issues in software and hardware that can be optimized.
• Optimize the distribution at scale:
  • Using the optics that engineering teams add to their features, we observe light weight performance data (e.g. overall time for Windows Hello face authentication) across a diversity of Windows hardware. From this data, we can see the gap between the 90th/95th/99th percentiles and more common cases at the 10th and 50th percentiles.
  • We then collect verbose performance diagnostics at scale from a smaller sample of Windows users across a diversity of hardware and Windows OS configurations.
  • We then leverage our automatic trace analysis techniques using resource usage and critical path analysis, to build a stack chart of performance issues across the distribution and their relative impact. This scale analysis helps us deeply understand the performance bottlenecks across a broad range of devices and helps us prioritize the most impactful opportunities.  This chart shows an example of this analysis for Windows Hello face authentication.
  • Finally, we double-check a set of the top performance traces in these buckets from the scale approach to confirm the insights from scale analysis and build a solution with engineering teams.

• Listen to customer feedback: Feedback from customers through channels like Feedback Hub help us identify areas where we need new optics or investigation outside of our current flow. For Instant On, our team monitors a few Feedback Hub categories and contexts like “Power and Battery >> Sleep” and “Security and Privacy >> Windows Hello Face.” When customers “Add more details” in Feedback Hub by reproducing and hitting “Record my issue,” we have the rich data necessary to diagnose problems effectively.

Across recent Windows 11 updates, we have optimized:

• More security with better performance for Enhanced Secure Sign-On (ESS). ESS provides increased security by building memory enclaves that can be hidden from the rest of the system.  We improved performance by up to 10% in cases where the system is under memory pressure by more efficiently handling the secure enclaves’ large slabs of memory.  This also enables us to ship on a larger range of RAM sizes.
• Consistent Windows Hello performance with app orchestration. When you resume your machine and kick off Windows Hello, apps also start resuming in the background and can consume system resources, slowing down your Hello performance.  We delay the resume of these apps until after Windows Hello camera startup and face authentication is complete.

And when customers update to the May 2023 release, they will have an even faster Instant On experience with fewer Remote Procedure Calls (RPCs) on the lock screen that can save up to 8% in cases where many apps are resuming.

What Am I Getting with the May Update?

We continue to invest in ensuring core Windows scenarios have great fundamentals.  Over the course of the last year, we also asked each feature team-building experiences in Windows to take a close look at their features to find additional opportunities to optimize the OS experiences.  We were surprised and energized by the breadth of optimizations made across the entire product, and we are excited to bring these improvements to all our users as part of the February and May updates.

We started this effort by conducting trainings across the entire organization on best practices, techniques, and educating the organization on the latest tools and techniques for performance analysis.  The energy and momentum that this created was invigorating as every team was empowered to improve any part of the operating system they chose.  This effort resulted in over 300 different performance improvements across Windows!  Here is a subset of some of the improvements available for you on all your Windows 11 devices.

Faster, more responsive post-boot experience

Today, when you boot a PC, there are a number of things that happen behind the scenes.  If you have a lot of apps registered to start at boot, it can be a stampede of apps all clamoring for resources to launch.  While investigating a boot perf issue, we experimented with moderating app startups.

What we found was we can both ensure the apps the user is trying to interact with are responsive and prioritized, and also that in many cases the background apps finish launching faster due to reduced contention.  Rather than unleash all the start up apps at once, we moderate their launches.  This helps avoid significant contention at launch and ensures the foreground app the user is interacting with stays performant and responsive.

These improvements most benefit devices with fewer resources, and we measured a 25-50% improvement in application launch times in the time period after logon in our experiment with multiple startup apps.  The image below helps visualize this, where we moderate startup apps to ensure that user-initiated operations can run with less contention.  In this example, you can see the user’s app completes its launch faster, and the background startup apps also complete in a similar timeframe due to less contention for resources.

File Explorer improvements

We made several improvements in File Explorer, including improving launch time of File explorer for users who have files stored in the cloud by avoiding I/O to enumerate cloud files.  We also looked at deleting files and made improvements to bulk file delete operations by over 40% as shown in our results below!

While it may be rare for users to delete large numbers of files on their device all at once, the example shows the breadth of the improvements we set out to bring to Windows 11 users.

Reduced game stutter with high report rate mice

We know gamers hate stutter, and we’re fixing issues throughout Windows so that an untimely frame-freeze won’t take gamers out of their immersive experiences.  Gamers love to push their systems to the limit in search of the best experiences in-game.  Many also use sensitive, high DPI, high report rate mice to shave milliseconds off their response times and increase precision.  At the same time, the world of gaming is more connected than ever with gamers relying on an increasing number of background apps while playing, like voice chat, streaming, apps for configuring your keyboard, mouse, or graphics card, and more.  The Windows input stack was being pushed to its limits with high report rate mice and their input being delivered to not just the game, but also multiple background processes.  In turn, that caused a significant amount of time processing input rather than providing as many cycles as possible for rendering the game experience.

We set out to reduce the amount of processing time it took to handle input requests by throttling and coalescing background raw mouse listeners and capping their message rate.  Prior to these changes, we observed on a Surface Laptop Studio with a 1000 Hz mouse, a test bed of background listeners, and popular games that there was significant stutter.  After the improvements, on the same setup, we now deliver a smooth, uninterrupted gaming experience and preserve the low latency, high precision input experience in games while being efficient with input for background listeners!

Something for everyone

While the examples above highlight some of the improvements in the February and May updates, there are many other improvements available today, including:

• Taskbar improvements to ensure Taskbar is readily available after logging in
• Task View performance improvements when devices are under heavy load
• Improved performance of Windows Notification Service
• Improved launch time of Quick Settings and Notification Center
• Reduced threads, dlls, and graphics device count in Explorer
• Reduced CPU usage of explorer
• Reduced memory usage of Notification Center
• Touch keyboard performance improvements
• Resource usage reductions in multi-user scenarios
• Reduced voice access and Narrator latency in larger documents
• Improved voice typing activation time
• Improved speed of theme changes
• Reduced size on disk by leveraging features-on-demand via the Store

For developers, there is a lot coming in the May release too.  Some highlights that intersect fundamentals:

• New enhanced kernel minidump format for better debugging
• Live kernel dumps can now be collected directly via Task Manager
• New richer, more actionable third-party reliability data views in Partner Center
• Windows Performance Analyzer (WPA) is now supported and available on Arm devices

Applications That Run on Windows Have Improved

There are great recent wins across key apps on Windows as well that we should celebrate.  We partner closely with these teams, share approaches to performance and fundamentals, and collaborate on improvements.

Microsoft Edge performance improvements

Browsing the web is one of the most common activities users perform.  Edge continually invests in not just improving the browsing experience for users, but in improving performance and minimizing resource usage.  Sleeping Tabs was introduced in Edge over a year ago, and the team has continued to improve the effectiveness of it to reduce memory and power usage of tabs.  Most users have up to 7 tabs open at any point in time, and while only 1 tab is often visible, those other tabs can have significant impacts on both memory and power.

With Sleeping Tabs, when you switch away from a tab we can reduce the memory and CPU usage of the background tabs, and just as importantly, we are able to resume faster than if we fully discarded the background tab.  Our diagnostic data shows that at the 75^th percentile, a tab can consume as much as 400MB of private working set.  With up to 6 tabs in the background, this adds up to nearly 2.4GB of memory being consumed.  With Sleeping Tabs, we can reduce the private working set of a tab down to 75MB!  A savings of nearly 2GB of memory (83% reduction), providing memory to other applications and services.  In addition, we have seen an over 15% improvement in the startup time of the Edge browser!

When you are mobile, getting the most out of your battery is important so you can keep working until you can plug in again.  Efficiency mode in Microsoft Edge helps you get the most out of every charge.  Through efficient reduction of CPU usage, Microsoft Edge can extend battery life on average of 25 minutes when efficiency mode is active.

The New Microsoft Teams App for Windows

Microsoft Teams has exploded in popularity over the past few years, and the team intently focused on scale and rapid growth.  Together, we spent quite a bit of time understanding the common use cases and scenarios.  A big focus for the team was moving to a new architecture and backend services that drove a 2x performance improvement, while consuming half the resources.  Bold fundamentals goals were set that required the team to step back and rearchitect significant parts of the software that enables all of us to connect and collaborate across the globe!  For more details on the improvements made with the new Teams client refer to the Teams blog.

Thank You and What’s Next

We continue to invest in driving performance, efficiency, and reliability with every update, and we are excited to have you experience these improvements firsthand.  Please continue to share feedback, and a very special thanks to our Windows Insider community for helping us continue to improve Windows for the over one billion users worldwide.  We have more on the way this coming year, and we are looking forward to ensuring your Windows experience gets better with every update.
Source: Delivering Delightful Performance for More Than One Billion Users Worldwide