Electro-Haptics Research

The EHS Research Group

The EHS team is involved in a range of research projects, exploring whether vibration can be used to enhance speech, music perception and spatial hearing for hearing-impaired listeners (including hearing aid and cochlear implant users). This includes both work exploring the limits of the human tactile and auditory systems, work with clinical populations, and the development of new haptic and auditory assistive-technologies.

The team is also working on research and consultancy projects exploring whether haptic devices can be used to enhance balance perception and gunshot localisation, as well as a project focused on realistic sound reproduction using virtual acoustics (e.g., cross-talk cancellation) and enhancing gaming and virtual reality.

Research projects:
Hearing with Haptics
Balancing with Haptics
Virtual acoustics

Consultancy and collaboration:
Human testing for hearing, vibration, and balance (including with clinical populations)
Vibration and acoustic system calibration and testing
AI for noise reduction, system optimization, and big data
App development for hearing and acoustics

For more information about working with the EHS team, please contact Dr Mark Fletcher at: [email protected]
You can also follow us on Twitter at: https://twitter.com/ElectroHaptics

Peer-reviewed publications

2024

Our third paper of 2024, presenting our work maximizing the speech information we present through haptics, published in Nature Scientific Reports

Fletcher, M.D., Akis, E., Verschuur, C.A., Perry, S.W., (2024) Improved tactile speech perception and noise robustness using audio-to-tactile sensory substitution with amplitude envelope expansion. Nature Scientific Reports (pdf)

Our second paper of 2024, presenting our neural-network-based noise-reduction approach for haptic hearing aids, published in Nature Scientific Reports

Fletcher, M.D., Perry, S.W., Thoidis, I., Verschuur, C.A. & Goehring, T. (2024) Improved tactile speech robustness to background noise
with a dual‑path recurrent neural network noise‑reduction method. Nature Scientific Reports (pdf)

Our first paper of 2024, presenting our new approach for converting audio to haptics, published in Nature Scientific Reports

Fletcher, M.D., Akis, E., Verschuur, C.A., & Perry, S.W. (2024) Improved tactile speech perception using audio‑to‑tactile sensory substitution with formant frequency focusing. Nature Scientific Reports (pdf)

2023

Our 2023 paper, published in Nature Scientific Reports

Fletcher, M.D., Verschuur, C.A., & Perry, S.W. (2023) Improving speech perception for hearing-impaired listeners using audio-to-tactile sensory substitution with multiple frequency channels. Nature Scientific Reports (pdf)

Ahmed Bin Afif successfully defended PhD thesis

Congratulations to Ahmed

Ahmed_Thesis.pdf
File Size:	4390 kb
File Type:	pdf

Download File

2022

Sam Perry's successfully defended PhD thesis

Congratulations to Sam for becoming the first completed PhD student on the EHS project

Our new paper on virtual acoustics for clinical audiology

Hamdan, E.C. & Fletcher, M.D. (2022) A Compact Two-Loudspeaker Virtual Sound Reproduction System for Clinical Testing of Spatial Hearing With Hearing-Assistive Devices. Frontiers in Neuroscience

2021

Mark's review paper on the use of haptics to enhance music perception in hearing impaired listeners, published in Frontiers in Neuroscience

Fletcher, M.D. (2021) Can Haptic Stimulation Enhance Music Perception in Hearing-Impaired Listeners? Frontiers in Neuroscience

Our review paper on the state of the emerging field of electro-haptics in audiology, published in Frontiers in Neuroscience

Fletcher, M.D. & Verschuur, C.A. (2021) Electro-Haptic Stimulation: A New Approach for Improving Cochlear-Implant Listening. Frontiers in Neuroscience

Our first paper published in Sensors

Fletcher, M.D., Zgheib, J., & Perry, S.W. (2021) Sensitivity to haptic sound-localization cues at different body locations. Sensors (pdf)

Our first paper of 2021, published in Nature Scientific Reports

Fletcher, M.D., Zgheib, J., & Perry, S.W. (2021) Sensitivity to haptic sound-localisation cues. Nature Scientific Reports (pdf)

2020

Our 2020 review paper on the challenges of developing a new haptic device, published in Expert Review of Medical Devices

Fletcher, M.D. (2020) Using haptic stimulation to enhance auditory perception in hearing-impaired listeners. Expert Review of Medical Devices (pdf)

Our forth paper of 2020, published in Nature Scientific Reports

Fletcher, M.D. & Zgheib, J. (2020) Haptic sound‑localisation for use in cochlear implant and hearing‑aid users. Nature Scientific Reports (pdf)

Our third paper of 2020, published in Nature Scientific Reports

Fletcher, M.D., Song, H., & Perry, S.W. (2020) Electro-haptic stimulation enhances speech recognition in spatially separated noise for cochlear implant users. Nature Scientific Reports (pdf)

Our second paper of 2020, published in Nature Scientific Reports

Fletcher, M.D., Thini, N., & Perry, S.W. (2020) Enhanced pitch discrimination for cochlear implant users with a new haptic neuroprosthetic. Nature Scientific Reports (pdf)

Our first paper of 2020, published in Nature Scientific Reports

Fletcher, M.D., Cunningham, R.O., & Mills, S.R. (2020) Electro-haptic enhancement of spatial hearing in cochlear implant users. Nature Scientific Reports (pdf)

2019

Our 2019 paper in Nature Scientific Reports

Fletcher, M.D., Hadeedi, A., Goehring, T., & Mills, S.R. (2019) Electro-haptic enhancement of speech-in-noise performance in cochlear implant users. Nature Scientific Reports (pdf)

2018

Our first EHS paper, published in Trends in Hearing

Fletcher, M.D., Mills, S.R., & Goehring, T. (2018) Vibro-tactile enhancement of speech intelligibility in multitalker noise for simulated cochlear implant listening. Trends in Hearing (pdf)
In the top 3% of the 12.5 million research outputs scored by Altmetric
2nd most discussed article ever published in Trends in Hearing

Presentations

2024

Mark giving a talk and presenting a demo of the mosaic 2 haptic device on SoundByte Studio's Endeavour 2 boat (London, UK)

Thank you to the James Mather and Bronwyn Elizabeth for hosting a great event

Mark presenting and demoing our mosaic 2 haptic device, and chairing a session on hearing-assistive technology at the Auditory Science Meeting in Cambridge (UK)

Thank you to the Tobi Goehring and the organizing team for hosting an excellent conference

Mark's invited talk and demo of our mosaic 2 haptic device at Med-El, Innsbruck (Austria)

Thank you to the Med-El team for arranging a wonderful trip

Mark's invited talk at the Sussex Hearing Lab Meeting, Brighton (UK), describing our work developing the mosaic 2 haptic aid for hearing

Thank you very much to Mood Bhutta and Alan Sanderson for the invitation and for being excellent hosts

Mark's haptic hearing aids lecture

Mark gives a lecture to Audiology students at the Southampton University on using hearing through vibration on the skin.

Mark and Sam present a poster and a demo of our latest electro-haptic devices to the ARO in Los Angeles (USA)

The work presented has been published in the paper: Fletcher, M.D., Akis, E., Verschuur, C.A., & Perry, S.W. (2023) Improved tactile speech perception using audio‑to‑tactile sensory substitution with formant frequency focusing. Nature Scientific Reports (pdf)

2023

Sam's talk at the Basic Auditory Sciences Meeting, London (UK)

Sam gave a talk in the Multi-Modal Hearing session titled: Developing a haptic hearing-assistive device to improve speech perception in hearing-impaired listeners
In addition to Sam's talk, Mark chaired a session on Hearing-Assistive Technologies and Modelling

2022

Mark's invited talk at the Mathematics and Statistics department of the University of Exeter (UK)

Mark spoke about the latest developments on the EHS project
Thank you to James Rankin and Kyle Wedgwood for being wonderful hosts

Sam's invited talk at the UK Hearing and Audiology Sciences Meeting, Southampton (UK)

Sam spoke about our recent work developing a new haptic device
The session on "Hearing-Assistive Technologies" was chaired by Mark Fletcher

Mark's invited talk at the National Cochlear Implant Users Association, London (UK), giving an overview of our project on enhancing hearing in CI users using haptics

Thank you very much to Paul Tomlinson, Ray Glover, Nigel Williams and the other organizers of the meeting for the invitation
Thank you also to everyone who attended the meeting for giving us such a warm and enthusiastic welcome

Mark's invited talk at the Sussex Hearing Lab Meeting, Brighton (UK), describing our work on enhancing speech perception and spatial hearing in hearing impaired listening using haptics

Thank you very much to Mood Bhutta for the invitation and for being an excellent host

Mark's guest lecture on the Fundamentals of Auditory Implants module on the University of Southampton Audiology MSc course

Thank you to Nicci Campbell for leading an excellent module

Alberte's poster on using haptics to improve music perception in CI users, presented at the ACI Alliance CI2022 DC conference

2021

Mark's invited talk at the Music & CI Symposium, Cambridge University (UK), on enhancing music perception in CI users using haptics

Thank you very much to the organizing committee and to Oticon for the invitation
This talk discussed work in collaboration with Ama Hadeedi, Robyn Cunningham, Jana Zgheib, Nour Tahini, Vinson Song, Tobias Goehring, Sean Mills, and Sam Perry

Mark & Sam's invited talk at the Center for Music in the Brain at Aarhus University (Denmark), on enhancing cochlear implant listening using haptics

Thank you very much to Bjorn Peterson and everyone at Aarhus for being such welcoming hosts
This talk discussed work in collaboration with Ama Hadeedi, Robyn Cunningham, Jana Zgheib, Nour Tahini, Vinson Song, Tobias Goehring, and Sean Mills

Sam's poster at the Music & CI Symposium, Cambridge University (UK), on enhancing pitch discrimination with haptics

The described work in collaboration with Mark Fletcher and Nour Tahini

CambridgeCIMusicPoster
File Size:	462 kb
File Type:	pdf

Download File

Mark's invited talk at the Conference on Implantable Auditory Prostheses

Thank you very much to the CIAP organising committee for the invitation
This talk discussed work in collaboration with Ama Hadeedi, Robyn Cunningham, Jana Zgheib, and Sam Perry
Click to see a recording of this talk (Q&A forum questions and answers are shown in the video description)

Mark's invited talk at the Oticon Shaping the Future study day, London

Thank you very much to Oticon and Andrew Causon for the invitation

Mark's electro-haptics guest lecture at the Nottingham Biomedical Research Centre

Thank you to Joe Sollini and everyone at the NBRC for hosting

2020

Mark talks about improving sound localisation in cochlear implant users with haptics at the British Cochlear Implant Group meeting

Thank you to the University of Nottingham for hosting a wonderful event

Our visit to Oticon Medical and the Danish Technical University to formally kick-off the Electro-Haptics Research Project

Thank you to Soren Riis, Kathleen Faulkner Scalzo, and Jeremy Marozeau for being wonderful hosts and to everyone else on the EHS project team for a great meeting

Mark's electro-haptics guest lecture for the "Current Developments in Bioengineering" 3rd year BEng module at Nottingham Trent University School of Engineering and Technology

Thank you to Fred Vanheusden and everyone at NTU for hosting

2019

Our talk at the National Cochlear Implant Users’ Association AGM

Thank you to the National Cochlear Implant Users' Association for hosting Carl's talk on the project

Electro-haptics visits University College London

Thank you to everyone at UCL Speech, Hearing and Phonetic Sciences (SHaPS) for hosting Mark's talk on Electrohaptics research

Mark's talk at Nottingham Trent University School of Engineering and Technology

Thank you to everyone at Nottingham Trent for hosting our talk on Electro-Haptics

2018

Our presentation at the British Society of Audiology

Fletcher, M., Hadeedi, A., Goehring, T., & Mills, S. (2018). Using tactile stimulation to improve speech-in-noise performance in cochlear implant users. British Society of Audiology Basic Auditory Science 2018

Our new device at the British Cochlear Implant Group meeting

The mosaicOne prototype shakes things up at BCIG
Our new poster from the meeting (pdf)

Media and outreach

Mark interviewed as an expert on haptic technology for supporting hearing in the Secret Science documentary series for ABC (2024)

Thank you to Adam Finney and Bruce Permezel at ABC

Mark interviewed by the New York Times for an article on emerging haptic technology (2023)

Thank you to Sarah Bahr and Katie Van Syckle at the New York Times

Mark interviewed by the ABC (Australian Broadcasting Corporation) on Sunday Extra (2023)

Thank you to Julian Morrow

EHS team provide materials and advice to Good Morning Britain on ITV for a piece on cochlear implants (2023)

Thank you to Ellie Stephenson

Mark was the expert for the ABC news article "Haptic technology creates new ways to experience music for people who are Deaf or hard of hearing" by Sam Nichols and Sam Carmody (2023)

Mark has contributed an article to A World of Sound, a free science eBook for children (2023)

Thank you to Chris Sumner and Caryl Hart

Mark interviewed by Newstalk for the Moncrieff show (2023)

Thank you to John Byrne Newstalk

EHS featured in the newsletter of the Nation Cochlear Implant Users Association (2022)

Thank you to the association for your interest in our work, and particularly to Alison Heath, who edits the NCIUA newsletter

"Listen with your wrists" is our new article published in Frontiers for Young Minds (2021)

Based on work with Robyn Cunningham, Sean Mills, Jana Zgheib, Ama Hadeedi, Sam Perry, and Tobi Goehring

Live webinar for school children on our research (2021)

Based on upcoming article in Frontiers for Young Minds "Listening with your Wrists"
Thank you to Dr Hannah Semeraro for leading the event

Interview with Mark in The Hearing Journal (2021)

The article "Leaps in Haptic Technology Improve CI outcomes", focuses on our recent review paper in Frontiers in Neuroscience and the origins of the EHS project

The EHS team support the Robosapiens permanent exhibition by the Science Museum of Minnesota

Mark and Sam help curate the part of the exhibition focusing on cochlear implants (set to open in 2022)
Sam develops an advanced real-time cochlear implant simulator app for the exhibition

The EHS project featured in the British Society of Audiology magazine Audacity

Piece authored by Professor Carl Verschuur (page 44) highlights work on the EHS project as well as other important work going on within the University of Southampton Auditory Implant Service

Our latest work featured in The Conversation

Vibration on the skin helps hearing-impaired people locate sounds, The Conversation, February 2020

Mark talks about neuroscience and the Electro-Haptics Project to secondary-school students for the Smallpiece Trust

Thank you to everyone for coming and to the Smallpiece Trust and University of Southampton for organising this wonderful event.

Our work featured in The Conversation

Research workshop at the University of Cambridge

Electro-Haptics Workshop at the MRC Cognition and Brain Sciences Unit at the University of Cambridge. Working with researchers from the Cambridge Hearing network and a visiting group from the Laboratoire de Mécanique et d’Acoustique, CNRS, Marseille.

EHS featured in Hoorzaken

"Bracelet makes cochlear implant wearers hear better" (translated from Dutch)
Thank you to Rene van der Wilk for sharing our work.

Some of the professional bodies that have shared our work:

Funding

£60k funding from the Del Monte Institute for Neuroscience (University of Rochester) to prime a new project exploring the neural correlates of haptic enhancement of hearing (2024-present)

This project is a new collaboration with Ed Lalor (University of Rochester) and Matt Dye (Rochester Institute of Technology)

Funding from the University of Southampton Auditory Implant Service for Esma Akis to work as a research assistant on the EHS project for 6 months (2023-present)

Esma will work with us to evaluate new algorithms for converting audio to haptic stimulation

£20k funding from the Web Science Institute for a project titled: "A web-based software training tool to improve music enjoyment for cochlear implant users" (2023-present)

The project will be lead by Alan Sanderson with support from Mark Fletcher and Sam Perry

£568k funding award from the EPSRC for a new collaboration, with James Rankin and Farzaneh Darki at the University of Exeter, modelling the perception of haptics for hearing (2022-present)

We will develop mathematical models of human perception of haptic stimuli used to enhance hearing

£8k equipment support fund from Oticon Medical (2022)

Thank you to Soren Riis from providing these funds

£6k app development fund from Audika (2022)

Thank you to Alexander Larsen for organizing this

£6k award from the Signal Processing, Audio and Hearing Research Group at the Institute of Sound and Vibration Research to explore the use of haptics to aid people with balance disorders (2022)

Thank you to Jordan Cheer and Stefan Bleeck for approving the use of these funds

£450k award by the William Demant Foundation to explore electro-haptic enhancement of music perception (2022-present)

The project will focus on haptic enhancement of music perception and how audio and haptic stimulation are combined in the brain
This project is a collaboration between Mark Fletcher and Sam Perry (University of Southampton Auditory Implant Service and Institute of Sound and Vibration Research) and Peter Vuust and Bjørn Petersen (Centre for Music in the Brain and Royal Academy of Music, Aarhus, Demark)

£40k invested into the Electro-haptics project by the University of Southampton Auditory Implant Service and Faculty of Engineering and Physical Sciences (2019-2022)

Our particular thanks to Carl Verschuur for his support
This funding was critical for us to build world-class laboratory facilities for this project

EPSRC Doctoral Prize awarded to Sam Perry, allowing him to join EHS as a Research Fellow (2021-2022)

Sam will be working on the EHS project for 1 year and 3 months on the doctoral prize
Additional funding has been provided by the University of Southampton Auditory Implant Service (extending Sam's time by 3 months) and Oticon Medical (£5k for equipment and travel)

£10k equipment fund to seed a collaboration between the University of Southampton and the University of Iowa awarded by the Iowa Neuroscience Institute (2020)

Thank you to the Iowa Neuroscience Institute for supporting this project through their Barrier Buster Grant.
We are excited to begin our collaboration with Dr Joel Berger and Dr Zsuzsanna Kocsis, using measurements from cortical electrodes in human subjects to explore the underlying mechanisms of haptic enhancement of listening.

£500k grant to support Electro-Haptics Research (2019-2024)

The Electro-Haptics Research project is now supported by joint grants awarded by the Oticon Foundation and the Danish Innovation Fund
The new multi-centre project is led by Dr Mark Fletcher (University of Southampton), with a team from the University of Southampton, University of Cambridge, University of Iceland, University of Nottingham, Danish Technical University and Oticon Medical.
Researchers will be working on ways to improve CI listening using a new haptic device.

£155k grant awarded to support Ahmed Bin Afif, our new PhD student (2019-2024)

The Electro-Haptics Research project has a new Post-Graduate Researcher, Ahmed Bin Afif.
Funding from King Saud University to support his research into improving CI hearing with haptic stimulation.

£82k grant to support our Virtual Acoustics project (2020-2023)

The Electro-Haptics Research project has a post-doctoral researcher, Eric Hamdan, thanks to funding awarded by the Oticon Foundation.
The team now also includes Associate Professor Filippo Fazi who, leads the University of Southampton Virtual Acoustics and Audio Engineering Research Group, and Professor Nicci Campbell from the University of Southampton Auditory Implant Service, who's research focuses on the development of more realistic listening tests.

£10k to fund an internship for 6 months on the electro-haptics project (2019)

We welcome Jana Zgheib who is joining us for 6 months, thanks to £8k of funding from Oticon Medical and £2k of funding from the University of Southampton Auditory Implant Service.
Jana will be looking at enhancement of spatial hearing using EHS.