Amazon Rekognition is one of the most widely-used, popular, and successful AI programs on the web. Companies use Rekognition to identify individuals, logos, and other details in images across the Internet. So why did one study find that Rekognition misclassified 31% of dark-skinned women in a sample?
Rekognition is not alone, and the failure of its AI is not solely a UX problem. But cases of software products with racially insensitive patterns, workflows that don’t take into account gender, or technology used to harm particular groups all point to a widespread user experience issue.
What can we as UX designers do to lessen racial and gender bias in the products and services we help design? Here are four ways to get started.
Be aware of your biases. We often think of bias as a negative trait that reflects poorly on our ethics and morality. But biases don’t have to be learned and taught. In fact, many of our biases occur because of the shortcuts we take in our thinking. For example, confirmation bias takes place when we prioritize facts that support our opinion rather than challenge it. In other cases, many people are unaware of how their implicit biases can shape their viewpoint on race, gender, and sexuality without consciously thinking.
Think about your “default user.” When examining a popular WordPress plugin, content strategist Tekla Szymanski made a surprising discovery: the default image for a user profile was a businessman in suit and tie. This is just one example of how the idea of “default users” can alienate all other users who don’t fit the “standard” profile. In an era when it is easier than ever to allow users to customize their experience, UX designers should consider whether the default is good enough.
Consider the hidden consequences of “easy” features. AirBnB uses the principles of “frictionless design” to make it simple for hosts to approve guests by viewing their names and profile photos. But research by a civil rights attorney found that this feature left guests vulnerable to hosts’ unconscious bias, or the preconceived ideas about race and gender that they may not have fully considered. This type of “easy” user feature, while intended to make the user experience smoother, may inadvertently replicate unsettling discriminatory patterns.
Test among diverse users. A focus on diversity in user testing can provide design teams with many benefits, from a more realistic portrait of user needs to a deeper perspective on user flows. Critically, user testing and research among diverse populations can also highlight product flaws and blind spots that designers might overlook. As UX designer Nhung Nguyen writes, “All of this matters and enriches the final solution you’ll come up with.”
Making products and services more equitable and accessible isn’t entirely on the shoulders of user experience professionals. But as the people in the room who are uniquely responsible for successful user outcomes, it is often up to UX designers to ensure that teams address their biases head on for the best possible outcome for all.