Whispy - AI for Accessibility

Created by team shrimple on July 30, 2023

Whispy is an accessibility tool built for voice chat accessibility. Using multiple models running concurrently, we can completely substitute a user in a voice chat. Users of Whispy can stick to using their preferred input method, whether that be Speech to text, or Text to speech, and other users in the voice chat continue to use the platform as is. This seamless integration into the Discord platform for our Demo allows users to have complete, real-time, and thorough conversations via Text or Voice, regardless of their preference. We leverage ElevenLabs streaming API and an audio queue to return any written text to the users of the voice call with a custom TTS voice. Text users can choose from all default voices, and their preferences are stored in the bot files. Our solution allows for text to be streamed back into the voice call rapidly, ensuring fluid conversation. Additionally, OpenAI's Whisper large model is analyzing and transcribing audio from any number of users in a voice call, separated out by speaker, and returning their speech as text into the same channel as the ElevenLabs user is typing in. This essentially replicates the Voice Call audio into a text conversation. For international users, both ElevenLabs and Whisper models can handle other languages, mostly limited to the Whisper supported languages. Our demo showcases Spanish as a secondary.

Category tags:

Utility and Tools

Github Presentation Demo

Explore more applications

sdffasdfas ds df asdf sd d d

sdffasdfas ds df asdf sd d dsdffasdfas ds df asdf sd d dsdffasdfas ds df asdf sd d dsdffasdfas ds df asdf sd d dsdffasdfas ds df asdf sd d dsdffasdfas ds df asdf sd d dsdffasdfas ds df asdf sd d dsdffasdfas ds df asdf sd d dsdffasdfas ds df asdf sd d dsdffasdfas ds df asdf sd d dsdffasdfas ds df asdf sd d dsdffasdfas ds df asdf sd d dsdffasdfas ds df asdf sd d dsdffasdfas ds df asdf sd d dsdffasdfas ds df asdf sd d dsdffasdfas ds df asdf sd d dsdffasdfas ds df asdf sd d dsdffasdfas ds df asdf sd d dsdffasdfas ds df asdf sd d dsdffasdfas ds df asdf sd d dsdffasdfas ds df asdf sd d dsdffasdfas ds df asdf sd d d

sdfasdfasdf

BabyAGI

Google Vertex AI Hacka

Google Vertex AI Hackathon Google Vertex AI Hackathon Google Vertex AI Hackathon Google Vertex AI Hackathon Google Vertex AI Hackathon Google Vertex AI Hackathon Google Vertex AI Hackathon Google Vertex AI Hackathon Google Vertex AI Hackathon Google Vertex AI Hackathon Google Vertex AI Hackathon Google Vertex AI Hackathon Google Vertex AI Hackathon Google Vertex AI Hackathon Google Vertex AI Hackathon Google Vertex AI Hackathon Google Vertex AI Hackathon Google Vertex AI Hackathon Google Vertex AI Hackathon Google Vertex AI Hackathon Google Vertex AI Hackathon Google Vertex AI Hackathon Google Vertex AI Hackathon Google Vertex AI Hackathon

Google Vertex AI Hackathon

BabyAGI

fsadfasdf asdf

asd fasd fasdf asd fasd fasd fasd fasd fasdf asd fasd fasd fasd fasd fasdf asd fasd fasd fasd fasd fasdf asd fasd fasd fasd fasd fasdf asd fasd fasd fasd fasd fasdf asd fasd fasd fasd fasd fasdf asd fasd fasd fasd fasd fasdf asd fasd fasd fasd fasd fasdf asd fasd fasd fasd fasd fasdf asd fasd fasd fasd fasd fasdf asd fasd fasd fasd fasd fasdf asd fasd fasd fasd fasd fasdf asd fasd fasd fasd fasd fasdf asd fasd fasd fasd fasd fasdf asd fasd fasd fasd fasd fasdf asd fasd fasd fasd fasd fasdf asd fasd fasd fasd fasd fasdf asd fasd fasd fasd fasd fasdf asd fasd fasd fasd fasd fasdf asd fasd fasd fasd fasd fasdf asd fasd fasd fasd fasd fasdf asd fasd fasd fasd fasd fasdf asd fasd fasd fasd fasd fasdf asd fasd fasd fasd fasd fasdf asd fasd fasd fasd fasd fasdf asd fasd fasd fasd fasd fasdf asd fasd fasd fasd fasd fasdf asd fasd fasd fasd fasd fasdf asd fasd fasd fasd fasd fasdf asd fasd fasd fasd fasd fasdf asd fasd fasd fasd fasd fasdf asd fasd fasd fasd fasd fasdf asd fasd fasd fasd fasd fasdf asd fasd fasd fasd fasd fasdf asd fasd fasd fasd fasd fasdf asd fasd fasd f

test team

BabyAGI

xcvvbsdffgedf

DSFASDF ASDF sd fasd asdfasdf asdf asdf asdf asd fasd fasdfdfdsafasdf asdfasdf asdfasd asdfadsf ddasd asd asd ad as DSFASDF ASDF sd fasd asdfasdf asdf asdf asdf asd fasd fasdfdfdsafasdf asdfasdf asdfasd asdfadsf ddasd asd asd ad asDSFASDF ASDF sd fasd asdfasdf asdf asdf asdf asd fasd fasdfdfdsafasdf asdfasdf asdfasd asdfadsf ddasd asd asd ad asDSFASDF ASDF sd fasd asdfasdf asdf asdf asdf asd fasd fasdfdfdsafasdf asdfasdf asdfasd asdfadsf ddasd asd asd ad asDSFASDF ASDF sd fasd asdfasdf asdf asdf asdf asd fasd fasdfdfdsafasdf asdfasdf asdfasd asdfadsf ddasd asd asd ad asDSFASDF ASDF sd fasd asdfasdf asdf asdf asdf asd fasd fasdfdfdsafasdf asdfasdf asdfasd asdfadsf ddasd asd asd ad as

wdGFASDFFGA

OpenAI

Lolllll

gfdgdfgfdgdf gfd gdfg dfg dfg dfg dfgfd g dfg fdg dfg df

testingoo musi

GPT-3.5

Toven Nevot
Technical Community Lead

"good work"

Walaa Nasr Elghitany

Data scientist and doctor

"Wonderful, as within Discord, the voice transcription is major issue, which with this solution can be resolved easily. I tried it myself and it was very smooth. Moreover, the idea is very impactful and looking forward what the future holds for this project. Well Done! "