Tapping Research on Songbirds, Argentine Scientist Aims to Give Voice to Speech-Impaired Humans

BUENOS AIRES, ARGENTINA – In a laboratory where books, cables and birdcages coexist, scientist Bernardo Gabriel Mindlin points to the motherboard that allows him to synthesize birdsongs and that promises to give voice to people with speech impairments. 

“Through these cables here arrive the signals, a product of the muscular movements of the bird’s vocal apparatus,” says Mindlin, 51, a University of Buenos Aires professor who holds a doctorate in physics. “And this small chip transforms them into synthesized song.”

A small speaker then emits the song.

Mindlin aims to use the same framework to synthesize the human voice to create a device to enable speech-impaired people to talk. He hopes the device will be ready for commercial development in three or four years.

In talking about his research, Mindlin mixes the didacticism of a physics professor with the enthusiasm of someone who has dedicated 10 years to a groundbreaking project. That decade passed quickly because he constantly noted progress, he says.

“The discovery that the birds confuse this [synthesized] song with their own was one of the most gratifying moments of my career,” he says.

To his right, three diamond birds hop in their cage as they sing a cheerful melody. Mindlin pauses and observes them affectionately.

“Those are the ones that remained from the study,” he says. “We took care of them, and now we keep them as pets.”

Mindlin and his team published a study on their research last year. Nature, a weekly international science journal, hailed the study as one of the best scientific reports published in 2013.

Nine other investigators, most of them students working on theses grounded in Mindlin’s research, collaborate with him. That custom has allowed for the rotation of investigators participating in the research over the past decade.

Mindlin and his team conduct their research at a laboratory in the School of Exact and Natural Sciences at the University of Buenos Aires in Argentina’s capital. The research, including the researchers’ salaries, is subsidized by the U.S. National Institutes of Health, the University of Buenos Aires, and two other Argentine government institutions – the National Scientific and Technical Research Council and the National Scientific and Technological Advancement Agency.

Mindlin began his research by studying how human vocal systems function. He then focused on birds because, like humans, they learn to speak by imitating mentors, but their vocal systems are much simpler. Researchers muted the laboratory birds with tracheotomies, then restored their voices by reversing the procedures.

“Now that we have the model on birds perfected, we return to humans,” he says.

The device that Mindlin aims to produce through his research would return speech to people who lost their voices because of facial paralysis, tracheotomies or other physical problems, he says enthusiastically. Using brain signals, it would permit a person to emit a synthesized human voice.

The devices now used by people with speech impairments form phrases from preprogrammed words, so the user cannot control vocal tone and speed, Mindlin says.

“There is no other method until now that is adaptable to the will of the user,” he says. “What I want to make is a device that allows the user to modulate – to speak much slower, much faster, or to give certain emphasis, if that is what they desire – something that interprets what you want to do muscularly and generates it at your will, producing exactly the sound that you wanted to produce.”

As far as Mindlin knows, no similar research is being conducted anywhere in the world.

Mindlin’s interest in speech production stems from another of his passions: music. He began studying physics at his family’s urging, and he was eager to finish his degree so he could focus on playing the trumpet, he says. But he discovered that physics gave him another way to work with sound.

“All of the subjects that interested me always had to do with sound – birdsongs, timbral properties, music, voices,” he says. “And how much better that this can provide a benefit to other people!”

Martina Inés Brocca, 14, is one potential beneficiary. A genetic lesion in the basal ganglia of Martina’s brain affects her motor system. She has difficulty modulating and, as a consequence, speaking.

Because brain signals activate the device Mindlin is developing, it would empower her to communicate more quickly, fluidly and independently. Martina smiles and, with great effort, voices her opinion on the project.

“It is good,” she says. “I want to be able to speak more clearly.”

Martina’s whole body tenses as she modulates each word. It is exhausting, she says.

“I have to concentrate a lot,” she says. “And when I concentrate, I drool a lot, and I get very hot.”

Martina’s mother, Victoria Herms, sits at her side, facilitating the communication.

Herms says that her daughter does not have a mental illness; the lesion affects only her motor systems. She is sad that there is currently no device that can help her daughter to communicate fluently.

“She is very intelligent,” Herms says. “She has a large vocabulary and a lot of things to say. It is a shame that there is no technological assistance. Almost no one takes a prolonged enough pause to listen to what she wants to say.”

Paula Andrea Lema, a public school teacher specializing in mental and social disabilities, says Mindlin’s research could also improve the lives of her young and adult students who have a variety of speech impairments.

“There is one young woman who has a permanent tracheotomy that they did on her because of a health problem that prevents her from speaking,” Lema says. “Maybe she could talk using that device.”

The key to helping people with disabilities is to shift the focus from their limitations to their potential by coordinating all their capabilities, Lema says. That is why she appreciates how Mindlin’s team coordinates physics and biology.

The research combines biological knowledge of vocalization with physics equations that convert brain signals into sounds.

At this stage, Mindlin’s team is simultaneously investigating two areas. One analyzes the distances between the lips, jaws and tongue that a speaker creates in modulating the voice. The other analyzes which groups of neurons activate portions of the vocal apparatus.

To analyze the vocal apparatus, researchers attach magnets and sensors to a volunteer’s lips, jaw and tongue. As the volunteer speaks, the sensors send the mouth’s positions through Wi-Fi or Bluetooth to a computer chip with a speaker that transforms the data into sounds that the movements would generate. The team bases this framework on the results of its bird research.

The researchers are developing a prototype device for Spanish speakers, Mindlin says. They will then modify the device to produce sounds needed to speak other languages. But the process always begins the same way – determining which muscular position or movement corresponds with each sound.

To advance the second avenue of research, the investigators measure the signals the brain sends to the speech organs. This research also draws on the findings from the bird experiments.

Specific portions of the brain give orders to produce each facial movement, Mindlin says. To measure these signals, he and his team examine epileptics who have had subcranial electrodes implanted for the sake of their diagnoses. As these patients read certain texts to produce specific sounds, the researchers observe which nuclei are activated.

The device Mindlin has conceived will consist of a sensor attached to a speech-impaired person’s brain that will transmit cerebral signals to a chip that synthesizes the corresponding sounds via a speaker.

The work with epileptics is not ideal because the electrodes are implanted in places affected by epilepsy, not spots directly pertinent to the team’s research, Mindlin says. This makes identifying the nuclei activated during vocalization a slow, complicated process.

“In any case, we are going to continue working on the two methods since they can help people with different problems,” Mindlin says.

Research engineer Franco Martín Pessana, who has pursued the application of electronics to medicine in his research and holds positions at the National Technological University and Favaloro University in Buenos Aires, says the work of Mindlin and his colleagues fascinates him.

“It is a study that is unique in the world,” Pessana says, “because they not only made a mathematic model working with knowledge of physics and biology, but they also gave an extremely important initial kick in synthesizing voice.”

The recognition bestowed by the journal Nature is a major honor, he says.

“That Nature has chosen them as one of the best studies of 2013 is a fact to highlight because it is the most important scientific journal at the global level,” Pessana says. “To publish there is very difficult; they are extremely demanding. What this group has achieved is very commendable.”

Mindlin says he and his team were obsessed with the study because they believe in their research’s potential.

“We know that the device can radically change the quality of life of many people,” he says.

GPJ translated this article from Spanish.