Cognitive study of an assistive multimodal user interface for hands-free human–computer interaction

The process of research and development of a multimodal user interfaces aimed at hands-free interaction with a personal computer by means of the natural speech input and pointing gestures/movements by user's head is described. The proposed interface uses a low-cost audio-visual equipment for information input and provides an universal access to computer systems both for regular human-operators for contactless (without using hands) personal computer control, and for handicapped users having difficulties with hand motion or even without arms. The methods and results of quantitative evaluation of speech and performance and contactless human–computer interaction are described and comparison with contact-based means of information input is made.