The Illinois Institute of Technology has been busy building on the WebRTC API to create a unique and extremely beneficial solution. The school has created the BaBL project, a real-time closed-captioning application for WebRTC. The multi-user video conferencing application also provides speech recognition and instant translation into a variety of languages as well as transcription storage using HTML5 APIs.
Luis Villasenor Munoz, a graduate student at the university, developed the not-for-profit project and demonstrated its features at the recent WebRTC Expo in Atlanta. He walked the audience through the various features of the application and how they are using services from Google and Microsoft to add exciting functionality.
“The main goal was to develop a WebRTC application that was able to provide real-time close captioning in order to help hearing impaired people,” said Munoz. BaBL was built using the WebRTC API of course, with all signaling achieved through websocket and everything coded in Javascript. It works by User A setting up a meeting with another user, as Munoz demonstrated with a remote friend.
Participants are then prompted to choose a language, and once they begin speaking, closed captioning appears almost immediately. On the back end, User A’s request for captioning is sent through a signaling server to User B. User B’s voice snippets are then sent along to Google’s speech recognition engine, which sends back the transcribed text. This is then sent back to User A using WebRTC.
In addition to the close captioning, an entire conversation may be stored, and all or parts of it referred back to at any time. Conversations may also be translated into different languages on the fly, using Microsoft Translator Server. In a reverse configuration, text may be translated into a different language and then read back through an automated voice through a text-to-speech conversion.
The solution was developed using free resources and technology and features and capabilities are still being refined and added.
Edited by
Maurice Nagle
