Newswise — Intelligent hearing aids that adapt to the individual needs of the user: for four years, researchers at the Collaborative Research Center (CRC) Hearing Acoustics have been working in this direction. Now the German Research Foundation (DFG) has extended funding for the project. Led by Prof. Dr. Volker Hohmann, hearing researcher at the University of Oldenburg, the CRC will receive up to 8.1 million euros for a second phase of funding from 2022 to 2026. With the official title “Auditory Acoustics: Principles Perceptuals, Algorithms and Applications” (HAPPAA), CRC is focused on developing hearing aids and hearing assistance systems that use artificial intelligence (AI) to automatically adapt to different environments, making these devices more adaptable to the specific needs of individual users. Besides the University of Oldenburg, the Jade University of Applied Sciences, the Fraunhofer Institute for Digital Media Technology IDMT, the Hörzentrum Oldenburg gGmbH, the RWTH University of Aachen and the Technical University of Munich – all leading institutions in the field of auditory research – are involved in this major project which is expected to last twelve years in total.
“In our aging society, it is becoming more and more urgent to develop hearing aids and other communication aids that work effectively in difficult acoustic environments and really help people in everyday life. Oldenburg Hearing Research is doing an excellent job and is highly recognized both nationally and internationally, and the renewed funding commitment from the German Research Foundation underscores this impressively,” said university president Professor Ralph Bruder.
When humans interact with their acoustic environment
The Acoustic Collaborative Research Center for Hearing brings together various disciplines, including acoustics, psychoacoustics, audiology, engineering sciences and physical modelling. During the first funding period, the focus was on the interactions between people with hearing loss and their acoustic environment. “In real life, the auditory situation constantly changes because people react to voices and sounds. For example, they turn their head towards the sound source or move their gaze in that direction. We call this the ‘acoustic communication loop ‘” says Hohman. This dynamic loop had received little attention in auditory acoustics in the past, he notes.
In recent years, the team has succeeded in integrating the hearing aid into this acoustic communication loop. “We have developed a first prototype of the so-called ‘immersive hearing aid’ which continuously assesses the acoustic situation and identifies which sound source a test person is directing their attention to at any given time,” says Hohmann. The device determines the direction of the test person’s gaze and head movements, then adjusts the signal processing to ensure that the targeted sound source can be heard optimally by the test person. The current prototype can be used in field experiments as well as in the laboratory.
Among other factors, new perceptual models developed by the research team for use in different auditory situations paved the way for this success. “These models predict how a test person will perceive a sound signal in a given situation – whether or not they will be able to follow a conversation in a noisy environment, for example,” says Hohmann. Simulating hearing with and without hearing loss in different hearing situations involving background noise and reverberation is essential for the development and evaluation of innovative signal processing methods in hearing aids, he points out.
Test algorithms directly in the ear
Another important result of the first funding period is the “earphone”, a special, particularly high-quality earphone for research purposes. Inserted into the ear and featuring several built-in microphones and small speakers, the device can amplify sound in exactly the same way as a hearing aid. Researchers can use it to test new signal processing algorithms directly in the ear, for example. The special thing here is that the earpiece is acoustically transparent – which means that hearing with this device corresponds to normal hearing with an open ear. “Thanks to the interdisciplinary collaboration within the CRC, we were able to combine acoustics and signal processing methods and thus achieved considerable progress,” says Hohmann.
The team also developed an interactive audio-visual virtual reality lab facility to conduct auditory experiments with test subjects under controlled conditions. Thanks to this technology, real situations can be simulated more realistically than before. To this end, the team created several complex audio-visual scenarios that test takers can “immerse” themselves in, including a virtual restaurant, subway station, and lounge. These scenarios, along with the associated data, have been made freely available to research laboratories around the world so that they can conduct their own auditory experiments.
Active noise control
During the second funding period, which will now begin, the CRC team plans to refine and merge its perception models, algorithms and applications. One of the goals is to develop algorithms for the receiver and the immersive hearing aid that can actively control the noise depending on the acoustic scenario. To do this, the researchers use state-of-the-art AI methods that they themselves have developed. The long-term goal is for each hearing aid to constantly learn and improve its ability to predict which setting is optimal for the respective user in a specific situation. People who are hard of hearing must be able to enter the necessary comments themselves via their smartphone. “However, we still have a lot of work to do before we achieve this goal,” notes Hohmann.
The team is also working on establishing international standards for complex acoustic scenarios in hearing and audiology research to facilitate and improve exchanges between different laboratories. In addition, the CRC aims to develop new auditory-acoustic tests in virtual environments that allow researchers to better identify differences in individual perception. This should make it possible to design diagnostic and rehabilitation measures for hearing aids that are perfectly tailored to individual needs.
The Collaborative Research Center complements the research conducted by the Hearing4all Center of Excellence, which is also led by researchers from the University of Oldenburg. In addition, it actively supports PhD projects of early career scientists with its own integrated research training group.