Our New Technology Lets You Swipe Away Communication Barriers

27th February, 2019

People face communication barriers in all kinds of environments—while travelling, at business meetings with people of different nationalities and when interacting with people who have hearing impairments—and we at Mitsubishi Electric are no exception. Communication difficulties actually inspired a new technology that leverages tablets, called the "User Interface for Voice-activated Drawing."

The Desire to Communicate Inspired True Innovation

Our mission is to enhance the quality of life for all, and Mitsubishi Electric’s support for the United Nations’ Sustainable Development Goals (SDGs) underscores this commitment. The goal to reduce inequalities particularly resonated with Masato Hirai, a developer at the Mitsubishi Electric Industrial Design Center in Kamakura, when he met with an intern who was hard-of-hearing. The challenges he faced in interacting with her gave him an idea for an app. "This project started from my desire to talk to her more, even though I didn’t know how to use sign language," Hirai said.

With his colleagues at the "Design X" project—a team of designers that works with different business units to solve society’s problems through design—Hirai developed a completely new user interface that combines voice recognition, swiping on a tablet and drawing pictures. To use the User Interface for Voice-activated Drawing, all a person has to do is speak into the tablet’s microphone and swipe their finger across the screen—then voila!—the spoken words sprout out on the screen as text from the person’s fingertip.

It’s as Easy as "Speak and Swipe"!

While we have yet to put the app on the market, its intuitive approach promises to make conversations with people who have hearing impairments smoother, richer and more profound. With conventional sign language and written communication, those with hearing difficulties struggle to follow explanations that involve pointing to documents and diagrams. That’s because it’s difficult to watch a speaker’s lips and hands at the same time. With the app, however, spoken words from the speaker’s fingertip can be displayed next to text and illustrations, all on the same screen. On top of that, the app does not require any special skills, and saves a lot of time since text doesn’t need to be written or typed.

Bringing People Closer Together through Technology

As leisure and business travel around the world increases, the app could also help break down language barriers thanks to software that translates text and enables illustrations. For example, a hotel concierge could give accurate directions to a guest by drawing a map and combining it with translated text that is inserted into the display by fingertip.

From start to finish, the development of this next-generation communication tool was guided by Masato Hirai’s desire to better connect people on an emotional level and break down communication barriers. "What we saw as the most important things when making the app were an intuitive and enjoyable ease of operation, and warm connections among people," he explained. In a world that’s becoming increasingly connected and diverse, this is an inclusive technology that we can all look forward to.

Learn more

Related Contents

Our Stories Top