For people with hearing loss it can very difficult to understand and separate voices in noisy environments. This problem may soon be history thanks to a new groundbreaking algorithm that is designed to recognise and separate voices efficiently in unknown sound environments.

The vision on intelligent hearing aids

“People with normal hearing are usually able to understand each other without effort when communicating in noisy environments. However, for people with hearing loss, it is very challenging to understand and separate voices in noisy environments, and a hearing aid may really help. But there’s still some way to go when it comes to general sound processing in hearing aids,” explains Morten Kolbæk, PhD student at Aalborg University: “When the scenario is known in advance, as in certain clinical test setups, existing algorithms can already beat human performance when it comes to recognising and distinguishing speakers. However, in normal listening situations without any prior knowledge, the human auditory brain remains the best “machine”.”

But this is exactly what Kolbæk has worked on changing with his new algorithm. “Because of its ability to function in unknown environments with unknown voices, the applicability of this algorithm is so much stronger than what we have seen with previous technology. It’s an important step forward when it comes to solving challenging listening situations in everyday life’,” says one of Kolbæk’s two supervisors, Jesper Jensen, Senior Researcher at Oticon and Professor at the Centre for Acoustic Signal Processing Research (CASPR) at AAU.

Noise reductions and speech separation

Specifically, Kolbæk’s PhD project has dealt with two different but well-known listening scenarios. The first track sets out to solve the challenges of one-to-one conversations in noisy spaces such as car cabins. Hearing aid users face such challenges on a regular basis. “To solve them, we have developed algorithms that can amplify the sound of the speaker while reducing noise significantly without any prior knowledge about the listening situation. Current hearing aids are pre-programmed for a number of different situations, but in real life, the environment is constantly changing and requires a hearing aid that is able to read the specific situation instantly,” explains Kolbæk.

The second track of the project revolves around speech separation. This scenario involves several speakers, and the hearing aid user may be interested in hearing some or all of them. The solution is an algorithm that can separate voices while reducing noise. This track can be considered an extension of the first track, but now with two or more voices. “You can say that Morten figured out that by tweaking a few things here and there, the algorithm works with several unknown speakers in noisy environments. Both of Morten’s research tracks are significant and have attracted a great deal of attention,” says Jensen.

Deep neural networks

The method used in creating the algorithms is called “deep learning”, which falls under the machine learning category. More specifically, Kolbæk has worked with deep neural networks, a type of algorithm that you train by feeding it examples of the signals it will encounter in the real world.

“If, for instance, we talk about speech-in-noise, you provide the algorithm with an example of a voice in a noisy environment and one of the voice without any noise. In this way, the algorithm learns how to process the noisy signal in order to achieve a clear voice signal. You feed the network with thousands of examples, and during this process, it will learn how to process a given voice in a realistic environment,” Jensen explains.

“The power of deep learning comes from its hierarchical structure that is capable of transforming noisy or mixed voice signals into clean or separated voices through layer-by-layer processing. The widespread use of deep learning today is due to three major factors: ever-increasing computation power, increasing amount of big data for training algorithms and novel methods for training deep neural networks,” says Professor Zheng-Hua Tan, who is also affiliated with CASPR and supervisor of the project.

A computer behind the ear

One thing is to develop the algorithm, another is to make it work in an actual hearing aid. Currently, Kolbæk’s algorithm for speech separation only works on a larger scale. When it comes to hearing aids, the challenge is always to make the technology work on a small computer behind the ear. And right now, Kolbæk’s algorithm requires too much space for this. Even if Kolbæk algorithm can separate several unknown voices from each other, it isn’t able to choose which voice to present to the hearing aid user. So there are some practical issues that we need to solve before we can introduce it in a hearing aid solution. However, the most important thing is that these issues now seem solvable.

Source: Aalborg University