Any modern mobile application built for public use needs to support a large and diverse userbase. Mostly we think of designing apps for people of various ages, backgrounds, using a variety of hardware supporting a range of technical features. But that also includes people with various impairments which make usage of small displays cumbersome. According to some statistics these are not miniscule populations. For example, the number of visually impaired internet users in the US alone is larger than the whole userbase of Canada. Many of these people turn to accessibility features of their devices for help. Both Android and iOS provide a number of such capabilities – including font size and contrast manipulation, screen magnification, animation overrides, and others.
One of the popular accessibility features is screen reading, which allows users to navigate through visual elements by a number of gestures, and provides audible descriptions of application widgets. I urge everybody who wonders how these work by enabling TalkBack or VoiceOver on their mobiles and step into the shoes of many millions of people that rely on these tools every day. You’ll discover that your app has a secret facet that you haven’t even thought about.
You might be wondering if your Flutter app is going to make use of screen reader capabilities. The answer is (obviously): yes. There are several methods that one can use to help users in enhancing their experience. But before I describe how to do that, you need to check if your app doesn’t already behave in the way you want. Many of the widgets provided by Flutter have accessibility features implemented, and only if you are not satisfied with the default results should you try manually wrangling your app’s accessibility.
Flutter’s Semantics widget
The Flutter SDK doesn’t provide a fine-grained widget for controlling just accessibility. Instead, it contains a class called Semantics, with its description:
A widget that annotates the widget tree with a description of the meaning of the widgets.
As you can see, the widget doesn’t really care about the specific technology or feature, but rather describes a general information about the widgets it wraps. That said, most of the features are tailored for the two screen readers mentioned in the previous section.
When you check its constructor, you can get a sense of how many potential attributes it can take (and features it can support). Most of them end up in the SemanticsProperties class. I won’t describe them all here – but let’s take a look at several common ones.
Label, value & hint
One of the common use cases for the Semantics widget is to add a description to your widgets. In Semantics, we have (among others) three useful String parameters for that:
label, which is a brief description of the widget,
hint, which is an explanation of the action that will occur when the widget is interacted with,
value, which provides a textual description of the widget’s state.
Naturally, you might not need all of them, but in some cases all of them fit. Let’s see one case where they do:
For the following text (which in fact is also a button, since we specify
onTap handler), a user of screen reader will – assuming that the
_counter variable stores zero, hear Counter button, zero, press to increase.
onCut, onCopy & onPaste
A useful bunch of properties that can help when interacting with text fields:
onPaste are callbacks that get executed when user executed matching actions on text fields. You can use them to enhance user experience by providing a visual or audible feedback when user manipulates text.
The above code doesn’t do much, especially from the end user’s perspective, but you should get the idea.
Combining a11y features
While Semantics allows us to provide many options for adding accessibility features, we also need tools to suppress, or modify the already existing ones. Luckily, Flutter provides several mechanisms for that.
Let’s revisit the first example. I lied a bit when I wrote that it’s going to be read as:
Counter button, zero, press to increase.
In fact, you’re going to hear:
Counter button, zero, zero, press to increase.
Where did the second zero come from? As I mentioned, many Flutter widgets come with semantics already embedded in them, and so does Text. By default Flutter will attempt to combine semantic information from multiple widgets into one. To prevent this from happening, you can use the ExcludeSemantics class, which makes Flutter discard all the semantic information from the widget subtree starting with ExcludeSemantics’ child. See this example below on how to get rid of the extra zero.
There are other cases where you might want to use this widget. Say, you don’t like the built-in features of widgets provided by Flutter – while useful they might not fit all cases, and sometimes be very verbose.
While ExcludeSemantics works in many cases, you can imagine how cumbersome would it be, to add it whenever you open a popup in order to hide the background widget’s features. In such cases, it’s better to use BlockSemantics, which is a more convenient way to exclude semantics of sibling widgets. It’s smart enough to only do that for widgets that were painted before the BlockSemantics child, which means that you don’t need to manually determine which ones should, and which shouldn’t be ignored.
Let’s take a look at an example:
If you have a screen reader enabled, all three containers will be selectable. What would happen if we wrap each of the Semantics widgets in BlockSemantics?
- Wrapping the “red” widget will not change a thing, as it’s painted first.
- Wrapping the “green” widget will prevent the “red” widget from being selectable. However, the “blue” one, as it’s painted after the “green” widget, is still going to be selectable.
- Wrapping the “blue” widget will make disable semantics on both other widgets. Notice, that this also includes the “red” widget, with which the “blue” one doesn’t overlap. This is because what matters in the painting order, not the area taken by the rendered widgets.
The last of the useful Semantics widgets is MergeSemantics. Imagine that you have a container which appears as a single widget, but actually is composed of multiple widgets. You might want to avoid having multiple semantically-annotated elements, but not by dropping some information, but rather combining them into one.
Let’s take a look at our previous example. If we wrapped the whole Stack in MergeSemantics, it would appear to the screen reader as a single item with combined description Red, Green, Blue.
Under the hood
If you’re curious, you might wonder how these widgets actually work. Flutter, apart from a tree of Widgets, keeps a separate structure called the semantics tree, which gets updated after the widgets are built, laid out, and painted.
The Semantics tree is composed from SemanticsNode objects, which are generated from RenderObject’s properties and SemanticsConfiguration (which, in turn, closely resembles SemanticProperties mentioned above). Not every RenderObject has its own SemanticsNode, rather they are maintained at certain points in the tree, called the semantics boundaries. RenderObjects that are not themselves a semantic boundary, have their configuration (if any) merged into the nearest parent boundary, or discarded (for example, if you use ExcludeSemantics).
Whew, that’s a lot of info. One thing we haven’t covered yet is: what to do when something we can’t get something right? The Flutter SDK has a simple tool to let us examine the semantics tree as we progress through the app. To enable it, you need to add the showSemanticsDebugger: true argument to the app’s top WidgetsApp class (or its subclasses). Below you can see how this looks like for the default Flutter app.
We went through some of the ways we can control accessibility features in Flutter using the *Semantics widgets. The topic of accessibility is, however, much larger than what fits into a single blog post. Sadly, Google doesn’t provide a concise and complete guide on that topic, that you can use. Instead, I recommend that you also take a look at these articles to get some more knowledge on the topic:
And remember – if you can’t figure something out, join the community and ask away :)
I’d like to thank Michał Pierzchała for his input on this article.