At one university in Israel, scientists are creating portals between the senses and even reprogramming our senses.

Iddo Wald shows me two abstract shapes, one angular and the other rounded.

“One of these shapes is called Bouba and the other is Kiki. Which is which?”

“That’s Bouba,” I declare, pointing to the rounded shape.

“That’s what 90 percent of people say,” Wald tells me with a smile.

“It has nothing to do with culture or language. It’s related to how we move our mouths to pronounce those sounds,” he says.

“The sharp sound of ‘Kiki’ and the way our mouths move to pronounce it is associated with the sharp angles of the shape you see.”

The “Bouba/Kiki Effect” was first reported by psychologist Wolfgang Köhler in 1929. The Baruch Ivcher Institute for Brain, Cognition & Technology at Reichman University (formerly IDC) in Herzliya is taking that basic sensory research to places ol’ Wolfie couldn’t have imagined.

“There are a lot of connections between the senses. In our lab, we look at those connections and how we can create portals between the senses and even reprogram our senses.”

Seeing with the ears

Last July, I reported how a 50-year-old man, blind from birth, learned to recognize objects using Amedi’s EyeMusic invention.

EyeMusic converts visual images into “soundscapes” that activate dormant neural circuits in a blind person’s vision-processing occipital cortex. This “sensory substitution” concept is one of Amedi’s specialties.

“With a lot of practice listening to pictures you can train people to see through their ears,” explains Wald.

“A blind person with no previous concept of colors can actually reach a stage where they can pick a green apple from a bowl of red apples. It means the part of the brain used for sight needs to be redefined, and it also shows that our assumptions about brain plasticity after a certain age were incorrect.”

Amedi discovered that the brain is much more malleable than previously believed.

“What you learn later in life is not less important that what you learn early in life. You can teach the brain at any age even if you are missing input from when you were a kid. If you were never exposed to visual stimuli, it can emerge even at age 50 or 60,” Amedi tells me.

“All the technologies here are inspired by this discovery.”

Speech to touch

EyeMusic training is intense and long, but the lab also is developing sensory substitution techniques requiring little training.

Amedi and postdoctoral research fellow Katarzyna Ciesla pioneered a sensory substitution device that doubles speech comprehension by delivering speech simultaneously through audition and as vibrations on fingertips.

This has obvious benefit for people with hearing impairment. But everyone needs help understanding speech, especially now that facemasks prevent us from seeing the speaker’s mouth.

“All the senses are processed simultaneously. If there is a conflict between vision and audition, you will hear what you see and not what you really hear,” Amedi says.

People with normal hearing lose about 10 decibels – or half of speech content — if there’s a lot of background noise or they can’t see the speaker’s lips. After an hour of training with the speech-to-touch device, 16 out of 17 test subjects gained 10 decibels and understood speech twice as well.

Future wearable implementations of this speech-to-touch device could be useful in situations such as talking on the phone or learning a foreign language.