Triumph kid for all: meme accessibility for the blind

By Miriam Fauzia
BU News Service

BOSTON — Social media has revolutionized the way humanity connects on a global scale, but in the blind and visually impaired community, barriers to social media inclusion persist.

“Social media accessibility has gotten better but continues to be a huge issue,” said Sassy Outwater-Wright, director for the Massachusetts Association for the Blind and Visually Impaired. 

“A huge amount of information is communication through infographics, movies, GIFs and they’re all image-based,” she said. 

While there are assistive technologies available to help navigate, such as screen readers or Braille displays, Outwater-Wright, who is visually impaired, feels they still fall short of fully communicating visual content.

But one group at Carnegie Mellon University’s Human-Computer Interaction Institute (HCII) is seeking to remedy this issue by creating a new technology that would make memes – images taken from popular culture and overlaid with text to express an idea, behavior or style – more user-friendly for the visually impaired. 

Currently, social media platforms use alternative text, also known as alt text, to translate visual information into text. When an image cannot be displayed, the alt text appears instead. For the blind and visually impaired, alt text is what is primarily read by screen readers when accessing websites or smartphone apps. Memes, unfortunately, do not contain any alt text.  

Cole Gleason, a Ph.D. student involved in the study, stated the method involves an optical character recognition software which automatically extracts the alt, which is then viewed by a human who elaborates with more details, such as color, shape, size or what the object actually is. The software then feeds the new information back in, and is able to reproduce it whenever it comes across the same image again.  

“Basically the idea here is that we think a visual meme codes some theme that’s common to all times you see that same meme,” said Gleason, who is also visually impaired. 

He also mentioned the possibility of including sound to provide more sensory information and better understanding of the joke. 

Gleason used the Success Kid meme, the image of a determined looking baby clenching a fist, as an example. 

“You [would] read out the top text and then we [would] have the sound effect going ‘Yes!’ and then read out the bottom text,” he said.  

Once a standard template is created, the software can immediately match it to any iteration of the same image employed by the meme. Blind and visually-impaired internet users would be able to utilize the software via a downloadable web browser extension.  

The critical obstacle that stands in the way of integrating improved accessibility tools are the social media platforms themselves.

Outwater-Wright stated accessibility prioritization is not something emphasized or considered by company executives early on. 

“The senior [executives] really need to have a leadership role and say accessibility needs to be part of our development process from day one, not launch day when we launch a product and then hear from a blind person that they can’t access it,” Outwater-Wright said. 

Gleason also agreed that when senior executives demonstrate concern for accessibility, the sentiment trickles down to every individual in the company. He cited Microsoft’s CEO Satya Nadella who, in July 2017, launched Microsoft’s Seeing AI, a free iPhone app which tells the blind what is around them. Nadella’s eldest son Zain was born with severe cerebral palsy and has been the CEO’s motivation for improved accessibility and inclusivity.

Another aspect to making memes accessible is providing the blind and visually impaired tools for creating memes themselves. 

“I think there is something to be looked at, like blind people as producers and figure out what tools we could give them to create things as well,” Gleason said.  

Gleason spoke of an authoring interface which would allow users to browse the internet for images and convert the image details into pre-existing alt text meme templates. Users would also have an option to add sound effects chosen from an online sound library.  

Going forward, Gleason wants to apply his research to the newest trending visual content app, TikTok. 

“[TikTok] is a shared audio that people are maybe performing different visuals to the same audience,” he said.

When asked if the new technology could keep pace with the rapid generation of content on social media, Gleason expressed no concerns.

“Every time someone invents a new medium, you’re going to have to explain what the template should be. The benefit is that it’s a much smaller problem because it’s very easy to make a new instance of a meme that already exists. New memes that come along are [a] little bit few and far between.”