When people don’t understand how an AI-driven service comes up with results, they may feel less able to discern whether or not to trust it. We therefore need to design mechanisms to explain to people when, how and why they are affected by AI systems, to help support greater comprehension of AI.

These mechanisms should drive practical understanding and agency for everybody, not just for the narrow few. That means the explanations must be provided in a way that caters for non-experts, and gives people the information they need to make a reasonably informed judgement.

The goal is not product explanations, but people’s understanding.

Through our interactive workshops and research, we’ve found that people draw on their wider knowledge and experience to develop their understanding of an AI in an active manner. Their understanding can change depending on how a product behaves and surfaces results, as people situate explanations within the context of their experience. This means that we must adopt a more interactive concept of explainability, moving beyond the idea that product makers provide explanations and product users merely receive them. Product makers can take a more collaborative approach to explainability through the use of interactive touchpoints, user controls and implicit explainability mechanisms.

Comprehensive transparency and highly detailed explanations aren’t always useful, desired, or even possible. The key to creating positive user experiences is knowing how much to reveal and when.

Product makers need to ensure people are provided with the information most relevant to their respective needs and contexts. They also need to consider how explainability is balanced between different audiences, such as general product users and expert stakeholders, and acknowledge the trade-offs that come with this. When a product serves different user groups, the information and control appropriate to each can vary significantly.

It also means accounting for the fact that different people engage with information in different ways, by designing multiple opportunities for a user to develop understanding in a way that is meaningful to them.

Controls take many forms; they include mechanisms that allow people to influence the way an AI-powered product generates results, to modify their declared interests and preferences, or to override or remove AI-inferred data inputs. They also include opportunities for people to provide feedback on recommendations and to challenge AI outcomes and explanations if they disagree with them.

Comprehensive control is not necessarily possible or desirable, but appropriate controls can enhance people’s understanding of AI systems. There is more work to be done to identify and develop user controls that can improve transparency and explainability of AI systems for different audiences.

Control is about empowering people over their interactions with AI systems. The relative complexity of AI means explaining how and why a system uses information, such as someone’s personal data, is not always straightforward.

And giving people a lot of information about how their data might be used is not the same as meaningful control – it outsources responsibility to the user, instead of the product or system maker.

Meaningful agency means control over inputs, and the ability to feedback into AI processes. It may also include a choice of outputs of AI systems.

Technical documentation, such as model or system cards, can provide insights into why and for what purpose an AI model or system was developed, how it was trained, how it produces results and how it will evolve over time.

By producing this documentation product makers are given an opportunity to reflect on their processes which promotes accountability within an organisation, and allowing external experts to verify it can provide assurances to non-experts that the necessary oversight mechanisms are in place, and working effectively.

With adequate information, and clear routes to redress, users are empowered to make an informed decision about the AI systems they use