Assistive technology has always been a major focus for Microsoft and their latest effort will make the Microsoft Edge browser easier to use for those with poor vision.
Microsoft has announced that the Microsoft Edge will now provide auto-generated alt text for images that do not include it. Auto-generated alt text helps users of assistive technology such as screen readers discover the meaning or intent of an images on the web, but more than half of the images processed by screen readers are missing alt text.
Now with this latest innovation, after a user has granted permission, Microsoft Edge will send unlabeled images to Azure Cognitive Services’ Computer Vision API for processing. Most common web image formats are supported (JPEG, PNG, GIF, WEBP, etc.). The Vision API can analyze images and create descriptive summaries in 5 languages, and recognize text inside of images in over 120 languages. Images sent to this service are governed by Microsoft’s Privacy promise.
The following images will not be interpreted:
- Images that are marked as “decorative” by the web site author. Decorative images don’t contribute to the content or meaning of the web site.
- Images smaller than 50 x 50 pixels (icon size and smaller)
- Excessively large images
- Images categorized by the Vision API as pornographic in nature, gory, or sexually suggestive.
- Additionally, the feature can be controlled through an enterprise policy setting. Under the policy name AccessibilityImageLabelsEnabled there is an option to disable the feature.
The feature is available from today to all Microsoft Edge customers on Windows, Mac and Linux. The feature is not currently available for Microsoft Edge on Android or iOS.
While Microsoft is offering a Machine Learning-based solution for missing image alt-text, Microsoft notes that the ideal solution would be for the website author to label the images appropriately on upload, since only the site author fully understands the context and intent of the image and its creative meaning and can supply the most relevant description (or mark that it’s decorative if it has no explanatory relevance).
To try this out, go to edge://settings/accessibility and look for the new setting “Get image descriptions from Microsoft for screen readers.” When this setting is enabled, a prompt will appear with a summary of the feature, a link to additional privacy information, and ask for permission to continue.
When the screen reader encounters an unlabeled image, it may note that image descriptions are available from Microsoft via the context menu. A new context menu option “Get image descriptions from Microsoft” is available at any time to get all the unlabeled image descriptions “Just once” for the current page, or to enable automatic image descriptions “Always” for any page on which they are found. Choosing “Always” is the same as turning on the feature via edge://settings/accessibility.
The screen reader will preface its description of images automatically labeled by this feature with “Appears to be…” and will preface any text that was detected with “Appears to say…”.
See the feature demoed below:
