Mátételki, Péter and Pataki, Máté and Turbucz, Sándor and Kovács, László (2014) An assistive interpreter tool using glove-based hand gesture recognition. In: International Humanitarian Technology Conference, 2014, 1 June 2014, Montreal, Canada. (Unpublished)

Slideshow Peter_Matetelki_An_assistive_interpreter_tool_using_glove_based_gesture_recognition_v07.pptx Download (4MB)

Abstract

An assistive tool (InterpreterGlove) for hearing- and speech impaired people is created, enabling them to easily communicate with the non-disabled using hand gestures and sign language. An integrated hardware and software solution is built to improve their standard of living, consisting of sensor network based motion-capture gloves, a low-level signal processing unit and a mobile application for high-level natural language processing. This paper introduces the overall system architecture and describes our automatic sign language interpreter software solution that processes the gesture descriptor stream of the motion-capture gloves, produces understandable text and reads it out as audible speech. The main logic of our automatic sign language interpreter consists of two algorithms: sign descriptor stream segmentation and text auto-correction. The software architecture of this time-sensitive complex application and the semantics of the developed hand gesture descriptor are described. We also present how the beta tester’s feedback from the deaf community influenced our work and achievements.

Item Type:	Conference or Workshop Item (Speech)
Subjects:	Q Science > QA Mathematics and Computer Science > QA75 Electronic computers. Computer science / számítástechnika, számítógéptudomány
Divisions:	Department of Distributed Systems
Depositing User:	Péter Mátételki
Date Deposited:	12 Dec 2014 10:39
Last Modified:	12 Dec 2014 10:39
URI:	https://eprints.sztaki.hu/id/eprint/8050

Update Item