Have you ever considered how much of our day to day lives rely on audio?
From entertainment, to healthcare, to communication – and increasingly so in remote working environments – we turn to sound technology to connect us. But imagine trying to conduct a Zoom call, access TeleHealth services, or tap into an immersive metaverse experience without the ability to hear. Or hear well.
That reality may not be so far off. For starters, we’re all aging (sorry to be the bearer of bad news – although perhaps you’re all too aware of ageism in the tech world, anyway). And in fact the World Health Organization predicted that by 2020, nearly 2.5 billion people will have some degree of hearing loss.
This is an indication of the millions of people who already struggle to access services and platforms due to their hearing challenges.
We asked the Girls in Tech community to get creative.
To put their tech skills to good use and make a meaningful impact in people’s lives.
Partnering with Hewlett Packard Enterprise, we ran a virtual hackathon that challenged participants to think outside of the box and design creative tech solutions for the hard of hearing community.
Participants focused on creating innovative solutions for areas like:
- Communication: Technology that transcribes or adds subtitles to virtual collaboration tools like Zoom or phone calls.
- Healthcare: Solutions to enable hard-of-hearing people to access in-person or TeleHealth services without sign language translation.
- Wellbeing and fitness: Apps that link hearing aid devices to health technology like heart rate monitoring, step counting, workouts or meditation practices.
- Emergency access: Ways to connect with emergency services in a nonverbal way.
- Culture and entertainment: Ways to make audio-based apps (Spotify, Clubhouse, Netflix) accessible to the hard-of-hearing.
And in classic GIT community style – the results were stunning. Here’s a wrap-up of our winning entries.
1st Place: Sonder
There were several driving forces that inspired our first place winner Lucy Low to create a mobile app that improves learning experiences for deaf children:
- A hard of hearing mentor – “I was motivated by attending Hack for RARE, organized by the Children’s Tumor Foundation and MIT Hacking Medicine for rare diseases. One of my mentors there… was deaf and that is what motivated me to build Sonder,” she said.
- The United Nations Sustainable Development Goals – particularly the wider goal to make education accessible for all.
- The realization that hearing loss affects 308,648 children between the ages of 5 and 17 in the US – not just “grandma and grandpa”.
Lucy also shared the inspiration for the app’s name, Sonder, which means:
The profound feeling of realizing that everyone, including strangers passed in the street, has a life as complex as one’s own, which they are constantly living despite one’s personal lack of awareness of it.
She explained, “This is important because the ‘lived experience’ for a deaf student is extremely different. Developing with empathy – thinking of people other than yourself – is important.”
About the app
Sonder is an online learning management system for hearing-impaired children and their teachers. It’s designed to make communication easier, so children can learn like the rest of their peers.
Features include:
- Tech that transcribes or adds subtitles to virtual collaboration tools like Zoom or phone calls, and can connect with emergency services in a nonverbal way for safety purposes.
- A portable translator for deaf or hard-of-hearing students, allowing users to transcribe or live caption voice-to-text for classrooms, business meetings, doctor’s appointments, shopping, events, and more.
- A video interpreter: Reading American Sign Language with Computer Vision Google Tensorflow Object Detection Model CNNs trained on object recognition for visual recognition tasks.
“This is an important market because children are the future and deaf children and their education matter – we can’t leave anyone out,” Lucy explained.
Take a look at how Sonder works, the app features, and plans for the future, on the Devpost page.
2nd Place: CallBuddy
The team behind the CallBuddy app wanted to bridge the gap between the hearing and hard of hearing community by allowing users to easily make phone calls, without having to go through a relay operator.
They shared, “Our teammate, who is HoH (hard of hearing), faces… severe anxiety every time they have to call the pharmacy to refill their medication or call the local restaurant to make a reservation.”
And while you may not be aware of the challenges people face in making phone calls through a relay service, the team’s research really highlighted how imperfect it is. Key challenges include:
- Lack of widespread adoption
- Slow and laborious processes resulting in delayed relays
- Scammers abusing free relay services
- Access challenges
- Being hung up on.
CallBuddy is designed to simplify and streamline the process of making a phone call as a deaf or hard of hearing person.
About the app
CallBuddy enables deaf and hard-of-hearing people to access services by making and receiving calls from anyone, without having a relay operator present or using any additional hardware.
Users can make calls from their mobile phone or on the Web. After they sign into their account, which is connected to a Twilio number, they can choose who to call by selecting from a category first.
Categories include:
- Medical
- Restaurants
- Emergency
- General calls
You can check out the details of the CallBuddy app on its Devpost page, or give the real thing a try.
3rd Place: PeaceTree Wellness
The team behind our third place submission wanted to transform how the deaf and hard of hearing community interact with meditation apps – from which they are largely excluded. (Despite the mental health and wellbeing app market being extremely lucrative.)
The app they’ve designed uses visual, sensory, and light cues (rather than audio) to guide hard of hearing users to engage in meditative practices, allowing them to truly relax.
“We had never thought that deaf people would have a hard time concentrating (concentration fatigue) and straining to listen to a guided meditation,” the team shared. “Now we understand why there is a need for apps like ours, and hope that there are more accommodations made for people in the d/Deaf community.”
About the app
The PeaceTree Meditation App uses American Sign Language (ASL) to guide individuals through meditation, in addition to providing visual, sensory, and light cues for users to know when to breathe, listen, and relax, practicing mindfulness.
Transcripts are provided, in addition to a set of hands that signs in ASL and guides individuals through meditation. There are a few ways users can interact with the PeaceTree App:
- ASL guided meditation – features an interpreter signing and saying the instructions while a transcription is also provided on the screen.
- Sensory cues with instructions on the screen – the user can close their eyes and let the dimming of the light, and the brightness of the light, guide their breathing.
- Visual cues – such as a picture expanding and deflating to focus on breathing.
The PeaceTree Wellness team found themselves truly challenged by this hackathon, particularly given its focus on eliminating the need for a primary sense: sound. They said, “Figuring out how to communicate without sounds was a learning curve for us, as it was something that we hadn’t considered ourselves, and was really fascinating to learn about how we could meet the needs of the community.”
Take a look at the details of the app and how it works on the Devpost page.
We’d like to congratulate all teams that submitted an entry into the HPE Hackathon. The quality of submissions shows us just how influential our community will be in developing real-world solutions and technologies in future.
Keep an eye on our Events Page to learn about future hackathons with Girls in Tech.
