Appears to say: Microsoft Edge now provides auto-generated image labels

Many people who are blind or low vision experience the web primarily through a screen reader: an assistive technology that reads the content of each page aloud. Screen readers depend on having image labels (alternative text or “alt text”) provided that allows them to describe visual content – like images and charts – so the user can understand the full content of the page. Alt text is critical to making the web accessible, yet it’s often overlooked. Our data suggests that more than half of the images processed by screen readers are missing alt text.

To help fill that gap, Microsoft Edge will now provide auto-generated alt text for images that do not include it. Auto-generated alt text helps users of assistive technology such as screen readers discover the meaning or intent of an images on the web.

Using automatic image descriptions in Microsoft Edge

Modern image-recognition technology can help make things easier. When a screen reader finds an image without a label, that image can be automatically processed by machine learning (ML) algorithms to describe the image in words and capture any text it contains. The algorithms are not perfect, and the quality of the descriptions will vary, but for users of screen readers, having some description for an image is often better than no context at all.

After the user has granted permission, Microsoft Edge will send unlabeled images to Azure Cognitive Services’ Computer Vision API for processing. Most common web image formats are supported (JPEG, PNG, GIF, WEBP, etc.). The Vision API can analyze images and create descriptive summaries in 5 languages, and recognize text inside of images in over 120 languages. Images sent to this service are governed by our Privacy promise.

There are some exceptions. The following image types will not be sent to the auto-image caption service, nor provided to the screen reader:

• Images that are marked as “decorative” by the web site author. Decorative images don’t contribute to the content or meaning of the web site.
• Images smaller than 50 x 50 pixels (icon size and smaller)
• Excessively large images
• Images categorized by the Vision API as pornographic in nature, gory, or sexually suggestive.

Additionally, the feature can be controlled through an enterprise policy setting. Under the policy name AccessibilityImageLabelsEnabled there is an option to disable the feature.

We are excited to offer this built-in service today to all Microsoft Edge customers on Windows, Mac and Linux. The feature is not currently available for Microsoft Edge on Android or iOS.

It’s important to reiterate that alt text provided by a site’s author will always be preferable to automatically-generated alt text. Only the site author fully understands the context and intent of the image and its creative meaning and can supply the most relevant description (or mark that it’s decorative if it has no explanatory relevance). Automatically generated image alt text should not be viewed as a reason for authors to avoid properly labeled images.

How can I try this?

To try this out, go to edge://settings/accessibility and look for the new setting “Get image descriptions from Microsoft for screen readers.” When this setting is enabled, a prompt will appear with a summary of the feature, a link to additional privacy information, and ask for permission to continue.

For those who may have tinkered with Edge command line options or advanced accessibility configurations, also confirm that “Screen reader support” is enabled on the Accessibility Internals page at edge://accessibility.

Then use Narrator, or any popular screen reader to browse the web.

The screen reader will preface its description of images automatically labeled by this feature with “Appears to be…” and will preface any text that was detected with “Appears to say…”.

Note: When the screen reader encounters an unlabeled image, it may note that image descriptions are available from Microsoft via the context menu. A new context menu option “Get image descriptions from Microsoft” is available at any time to get all the unlabeled image descriptions “Just once” for the current page, or to enable automatic image descriptions “Always” for any page on which they are found. Choosing “Always” is the same as turning on the feature via edge://settings/accessibility.

I’m a web site author. How do I write better “alt text”?

Web sites are only as accessible as their authors make them. Image descriptions provided by web site authors will generally provide a better experience for screen reader users.

The W3C’s Web Accessibility Initiative (WAI) provides more information about creating good image alt text for site authors wishing to adhere to the Web Content Accessibility Guidelines (WCAG) specification. Tutorials provide patterns for alt text across a variety of functions that images may serve in the context of a web site. Microsoft also offers some recommended guidance.

Find more help in the DevTools

Providing great alt text for blind and low vision users is only one small part of improving the overall web experience for users with disabilities. The Microsoft Edge Developer Tools help remind authors to provide alt text and provide many other tools to help diagnose, debug, and improve accessibility on web sites.

Growth opportunities

This feature is still new, and we know that we are not done. We’ve already found some ways to make this feature even better, such as when images do have a label, but that label is not very helpful (e.g., a label of “image” or “picture” or similar synonym that is redundant with the obvious). Continuous image recognition and algorithm improvements will also refine the quality of the service.

We’re excited for our screen reader users to benefit from this new service and grateful for the opportunity to work to improve the web experience for all users. We look forward to hearing what you think.

How do I give feedback?

We’d love to hear from you! Submit feedback via the Microsoft Edge Feedback tool (Settings menu, Help and Feedback, Send Feedback), and let us know what we can do to improve this feature.

Microsoft is committed to meeting or exceeding WCAG and other accessibility compliance standards in our products and web sites. If you notice an issue we might have missed, we would love to hear about it and get it fixed.
Source: Appears to say: Microsoft Edge now provides auto-generated image labels