HACK@BROWN 2019 WINNER
What is Babel Fish?
Babel Fish is a video chat app. Whenever someone speaks, that speech is translated in real-time for everyone else in the chat. This lets people who don't speak a common language to easily converse over the internet. Check out the video above for a demo or try it out yourself at https://blogwithme.net!
As the world gets more interconnected, speaking different languages can be a major barrier to communication. At institutions like the UN, it is not uncommon to see people walking around with earphones in, listening to live translations of each other.
Now think about the refugee crises in Syria or Venezuela. So many people lack good access to medical and legal assistance, because the doctors and lawyers around the world who want to help out often don't speak the languages of the people who need help. If we remove the language barrier from communication, we can give people the power to build community and ultimately bring the world closer together.
How it works?
We used the Twilio Programmable Video API to set up our video chat. We use the Google Speech-to-Text API to transcribe speech. We wrote our own backend server to forward the transcribed text between clients. We then use the Google Translate API to translate language for each different user.
Challenges we ran into
Video is such a difficult medium to work with, so simply getting video chat up and running was tough. In addition, because this project has so many moving pieces, getting every component from transcription, to video, to translation to all work together was very difficult.
Accomplishments that we're proud of
We are really proud of building a working real-time-translation video chat app, because last night we didn't know if that was even possible.
We can see so many potential applications for this technology. We hope that this kind of technology will be used to break down barriers, and build community around the world.
What's next for us?
After this hackathon, Facebook invited our team to compete in the Facebook Global Hackthon Finals for the second year in a row!
As for Babel Fish, we want to build a more scalable app with more features. We also want to add a text-to-speech feature to allow visually impaired users to use Babel Fish, by allowing them to listen to live translations.