Translation Results Requested Through Conyac Made Public

Chinese Researchers Team Up With Microsoft to Teach Kinect How to Understand Sign Language (VIDEO)

The folks at Microsoft Research Asia have collaborated with researchers from the Institute of Computing Technology at the Chinese Academy of Sciences (CAS) to work on sign-language recognition. What they’re doing – as shown in the video below – is teaching Microsoft’s Kinect, a motion detection system usually used for gaming, to understand sign-language from around the world. Though it’s all very much a work-in-progress, the research could lead to a new way for hearing-impaired people to communicate online – particularly with those who don’t understand signing. Using the Kinect also means that any resulting solution will be cheap to purchase.

CAS professor Chen Xilin explains:

From our point of view, the most significant contribution is that the project demonstrates the possibility of sign-language recognition with readily available, low-cost 3-D and 2-D sensors. As Mashable notes, the experimental project has two settings. The translation mode prompts the Kinect to convert sign language into text or speech whether they’re single words and entire sentences. There’s also the communication mode that uses an avatar to convert text or speech input into sign for a hearing-impaired person to view.

The program manager at Beijing-based Microsoft Research Asia, Wu Guobin, notes, “While it is still a research project, we ultimately hope this work can provide a daily interaction tool to bridge the gap between the hearing and the deaf and hard of hearing in the near future.”

Here’s the demo video: