​eLECTRO-hAPTICS rESEARCH

Helping cochlear implant users hear better with haptics
Electro-haptics Research
  • The project
  • RealSpeech auditory training
  • Virtual acoustics
  • Get in touch

A new approach to implant listening


Hearing loss is a global challenge, particularly in countries with ageing populations. Unaddressed hearing loss poses an annual global cost of $750 billion and recent work has shown a strong link between hearing loss and dementia. Over a billion young people are at risk of hearing loss because of the way they listen to music.

Hundreds of thousands of people depend on implanted electronic devices to hear. These devices, known as auditory or cochlear implants, aren't perfect. Implant users often have difficulty knowing where sounds originate, are commonly unable to enjoy music, and find it difficult to understand speech when there is background noise - like in a busy workplace, crowded restaurant or chaotic classroom. We have a new approach to solve this problem by transferring crucial sound information through the skin.
​
We are developing a low-cost haptic device that could revolutionise the treatment of hearing loss.

Implants in noise


Cochlear implant users struggle to understand speech in noisy environments.

People with auditory implants hear the world in a very different way to people with healthy hearing. In an implant user, the sound that is usually transmitted to the brain by thousands of extraordinarily sensitive cells in the ear is instead transmitted by just 22 micro-electrodes. This means that the information transmitted to the brain is severely limited.

We've made a quick demo that simulates how hard it can be for auditory implant users to understand speech in complex sound environments. It's available to use for free as a teaching or demonstrating tool (YouTube or Download). 
Note: these are just simulations based on models of how cochlear implants work with the brain. Cochlear implant users experience their device in different ways depending on a range of factors.

Implants and Music


Note: these are just simulations based on models of how cochlear implants work with the brain. Cochlear implant users experience their device in different ways depending on a range of factors.

​Music sounds very different to implant users. They can struggle to distinguish different pitches, and they have poor access to the quality, or "timbre", of sounds.

This short demo that simulates what it is like to hear music as a auditory implant user. Like the demo above, it's available to use for free as a teaching or demonstrating tool (YouTube or Download).

​Hearing through your skin


"The brain takes information from the senses to build a model of the world. When information is missing from one sense, we can use another sense to add it back in." - Dr Mark Fletcher, principal investigator

Our research is all about finding ways to send crucial sound information through the skin as small vibrations. We've shown that this can work in the lab–now we're taking it to the real world.

We are developing a wrist-worn device that we have shown can improve implant users’ speech understanding and could transform their lives.

Our new haptic device


3D ​render of our mosaicOne_C device. We've been working hard to improve our device and get it ready to be used outside of the lab. Further iterations of the mosaicOne will be coming very soon!
  • Discreet: We're developing a slim, discreet wrist worn device that won't get in the way of daily activities.
  • Precise: we're focused on making sure our vibrations are reproduced perfectly in sync with the sound, without any distortion of the signal.
  • Comfortable: a priority in our design and manufacture process has been making the mosaicOne lightweight and comfortable to wear for long periods.
  • Quiet: our algorithms and motors have been carefully developed so our device is near-silent.
Picture
​This short video shows people reacting when listening to music and feeling it through the first prototype of our mosaicOne_A haptic device.

Training at home


Picture
Learning to use new information is an important part of the electro-haptic effect. We're building a remote training system to see how people learn to use electo-haptic stimulation over time.

Our research uses RealSpeech, an auditory training app developed by Dr Mark Fletcher and Dr Ian Wiggins from the electro-haptics team. RealSpeech has a large library of talkers and background environments, over 20 hours of synchronised high-definition video and audio, and is set up for remote data collection.

Virtual acoustics 


Working with the University of Southampton Virtual Acoustics and Audio Engineering Research Group, we're developing a system for creating carefully controlled realistic sound environments for use in the lab, in the clinic, and at home. Currently, a large, acoustically-treated space and an expensive system with several loudspeakers in a ring around the listener is required to create such sound environments. We hope our system can opening up the possibility of more realistic testing much more widely for clinics and research labs and allow for much more sophisticated at-home training and testing.
Picture

Peer-reviewed publications


Our first paper of 2021, published in Nature Scientific Reports
  • Fletcher, M.D., Zgheib, J., & Perry, S.W. (2021) Sensitivity to haptic sound-localisation cues. Nature Scientific Reports (pdf)
Picture
Our 2020 review paper on the challenges of developing a new haptic device, published in Expert Review of Medical Devices
  • Fletcher, M.D. (2020) Using haptic stimulation to enhance auditory perception in hearing-impaired listeners. Expert Review of Medical Devices (pdf)
Picture
Our forth paper of 2020, published in Nature Scientific Reports
  • Fletcher, M.D. & Zgheib, J. (2020) Haptic sound‑localisation for use in cochlear implant and hearing‑aid users. Nature Scientific Reports (pdf)
Picture
Our third paper of 2020, published in Nature Scientific Reports
  • Fletcher, M.D., Song, H., & Perry, S.W. (2020) Electro-haptic stimulation enhances speech recognition in spatially separated noise for cochlear implant users. Nature Scientific Reports (pdf)
Picture
Our second paper of 2020, published in Nature Scientific Reports
  • Fletcher, M.D., Thini, N., & Perry, S.W. (2020) Enhanced pitch discrimination for cochlear implant users with a new haptic neuroprosthetic. Nature Scientific Reports (pdf)
Picture
Our first paper of 2020, published in Nature Scientific Reports
  • Fletcher, M.D., Cunningham, R.O., & Mills, S.R. (2020) Electro-haptic enhancement ​of spatial hearing in cochlear implant users. Nature Scientific Reports (pdf)
Picture
Our 2019 paper in Nature Scientific Reports
  • Fletcher, M.D., Hadeedi, A., Goehring, T., & Mills, S.R. (2019) Electro-haptic enhancement of speech-in-noise performance in cochlear implant users. Nature Scientific Reports (pdf)
Picture
Our first EHS paper, published in Trends in Hearing 
  • Fletcher, M.D., Mills, S.R., & Goehring, T. (2018) Vibro-tactile enhancement of speech intelligibility in multitalker noise for simulated cochlear implant listening. Trends in Hearing (pdf)
  • ​In the top 3% of the 12.5 million research outputs scored by Altmetric
  • 2nd most discussed article ever published in Trends in Hearing
Picture

Presentations


Mark talks about improving sound localisation in cochlear implant users with haptics at the 2020 British Cochlear Implant Group meeting
  • Thank you to the University of Nottingham for hosting a wonderful event​
Picture
Our 2020 visit to Oticon Medical and the Danish Technical University to formally kick-off the Electro-Haptics Research Project
  • Thank you to Soren Riis, Kathleen Faulkner Scalzo, and Jeremy Marozeau for being wonderful hosts and to everyone else on the EHS project team for a great meeting
Picture
Mark's 2020 electro-haptics guest lecture for the "Current Developments in Bioengineering" 3rd year BEng module at  Nottingham Trent University School of Engineering and Technology
  • Thank you to Fred Vanheusden and everyone at NTU for hosting
Picture
Our 2019 talk at the National Cochlear Implant Users’ Association AGM
  • Thank you to the National Cochlear Implant Users' Association for hosting Carl's talk on the project
Picture
Electro-haptics visits University College London
  • Thank you to everyone at UCL Speech, Hearing and Phonetic Sciences (SHaPS) for hosting Mark's talk on Electrohaptics research​
Picture
Mark's 2019 talk at Nottingham Trent University School of Engineering and Technology
  • Thank you to everyone at Nottingham Trent for hosting our talk on Electro-Haptics
Picture
Our 2018 presentation at the British Society of Audiology
  • Fletcher, M., Hadeedi, A., Goehring, T., & Mills, S. (2018). Using tactile stimulation to improve speech-in-noise performance in cochlear implant users. British Society of Audiology Basic Auditory Science 2018
Picture
Our new device at the 2018 British Cochlear Implant Group meeting
  • The mosaicOne prototype shakes things up at BCIG
  • Our new poster from the meeting (pdf)

Media and outreach


The EHS team support the Robosapiens permanent exhibition by the Science Museum of Minnesota
  • Mark and Sam help curate the part of the exhibition focusing on cochlear implants (set to open in 2022)
  • Sam develops an advanced real-time cochlear implant simulator app for the exhibition
Picture
The EHS project featured in the British Society of Audiology magazine 'Audacity'
  • Piece authored by Professor Carl Verschuur (page 44) highlights work on the EHS project as well as other important work going on within the University of Southampton Auditory Implant Service
Picture
Our latest work featured in The Conversation
  • Vibration on the skin helps hearing-impaired people locate sounds, The Conversation, February 2020​​
Picture
Mark talks about neuroscience and the Electro-Haptics Project to secondary-school students for the Smallpiece Trust
  • Thank you to everyone for coming and to the Smallpiece Trust and University of Southampton for organising this wonderful event.
Picture
Our work featured in The Conversation
  • Playing sound through the skin improves hearing in noisy places, The Conversation, September 2018​​
  • Here’s what music sounds like through an auditory implant, The Conversation, February 2019
Research workshop at the University of Cambridge
  • Electro-Haptics Workshop at the MRC Cognition and Brain Sciences Unit at the University of Cambridge. Working with researchers from the Cambridge Hearing network and a visiting group from the Laboratoire de Mécanique et d’Acoustique, CNRS, Marseille.
Picture
EHS featured in Hoorzaken
  • "Bracelet makes cochlear implant wearers hear better" (translated from Dutch)
  • Thank you to Rene van der Wilk for sharing our work.
Picture
​Some of the professional bodies that have shared our work:​
  • National Acoustic Laboratories (NAL)
  • Acoustical Society of America (ASA)
  • British Psychological Society (BPS)
  • British Society of Audiology (BSA)
Picture

Funding


£500k grant to support Electro-Haptics Research
  • The Electro-Haptics Research project is now supported by grants awarded by the Oticon Foundation and the Danish Innovation Fund
  • The new multi-centre project is led by Dr Mark Fletcher (University of Southampton), with a team from the University of Southampton, University of Cambridge, University of Iceland, University of Nottingham, Danish Technical University and Oticon Medical.
  • Researchers will be working on ways to improve CI listening using a new haptic device.
£155k grant to support Ahmed Bin Afif, our new PhD student
  • The Electro-Haptics Research project has a new Post-Graduate Researcher, Ahmed Bin Afif​.
  • Funding from King Saud University to support his research into improving CI hearing with haptic stimulation.
Picture
£80k grant to support our Virtual Acoustics project 
  • The Electro-Haptics Research project has a post-doctoral researcher, Eric Hamdan, thanks to funding awarded by the Oticon Foundation.
  • The team now also includes Associate Professor Filippo Fazi who, leads the University of Southampton Virtual Acoustics and Audio Engineering Research Group, and Professor Nicci Campbell from the University of Southampton Auditory Implant Service, who's research focuses on the development of more realistic listening tests.
Picture
£10k to fund an internship for 6 months on the electro-haptics project 
  • We welcome Jana Zgheib who is joining us for 6 months, thanks to £8k of funding from Oticon Medical and £2k of funding from the University of Southampton Auditory Implant Service.
  • Jana will be looking at enhancement of spatial hearing using EHS.
Picture
£1k equipment fund to seed a collaboration between the University of Southampton and the University of Iowa awarded by the Iowa Neuroscience Institute
  • Thank you to the Iowa Neuroscience Institute for supporting this project through their Barrier Buster Grant.
  • We are excited to begin our collaboration with Dr Joel Berger and Dr Zsuzsanna Kocsis, using measurements from cortical electrodes in human subjects to explore the underlying mechanisms of haptic enhancement of listening.
Picture

The team


Picture

Dr Mark Fletcher

Principal Investigator
​University of Southampton
Picture

Ama Hadeedi

EHS Speech Enhancement
​University of Southampton​
Picture

Sean R. Mills

Tactile Neuroscience
​University of Southampton
Picture

Prof. Carl Verschuur

Director, University of Southampton Auditory Implant Service
Picture

Robyn Cunningham

EHS Spatial Hearing
​University of Southampton
Picture

Ahmed Bin Afif

EHS Speech Enhancement
​University of Southampton
Picture

Dr Tobias Goehring

Implant Signal Processing
​University of Cambridge
Picture

Nour Thini

​EHS Music Enhancement
​University of Southampton
Picture

Dr Devyanne Bele

Clinical Scientist
University of Southampton Auditory Implant Service
Picture

Dr Ian Wiggins

CI Neuroimaging
​University of Nottingham
Picture

Marianna Vatti

DSP Engineering
Oticon Medical
Picture

Sam B.P. Perry

Real-Time DSP
​University of Southampton
Picture

Dr Søren Riis

Sr. Director of Technology Development​
Oticon Medical
Picture

Haoheng Song

EHS Spatial Hearing
​University of Southampton
Picture

Helen Neale

Design & Manufacturing
Picture

Jana Zgheib

Haptic Sound-Localisation
​University of Southampton
Picture

Prof. Rúnar Unnþórsson​

Mechanical Engineering
​University of Iceland
​
Picture

Dr Mark Steadman 

Electronics & Mechanical Engineering
​Imperial College London
​
Picture

Dr Ben Lineton

Lecturer in Audiology
​University of Southampton
Picture

Dr Jeremy Marozeau

Music in CI users
Technical University of Denmark
​
Picture
Dr Joel Berger
Neurophysiology
University of Iowa
Picture
Dr Zsuzanna Kocsis
Neurophysiology
University of Iowa

Project partners


Picture
Picture
Picture
Picture
Picture
Picture
Picture
Picture
Picture
Picture
Powered by Create your own unique website with customizable templates.
  • The project
  • RealSpeech auditory training
  • Virtual acoustics
  • Get in touch