Abstract
A multimodal system accepting touchpad gesture input and speech input has been developed to support browsing online newspapers for the visually impaired. The issue addressed in this work is how to combine the two input modalities for nonvisual text-information browsing. An exploratory study with fourteen sighted subjects suggests that: (1) A dependence between users’ choice of input modality and the type of input operations undertaken exists – speech is preferred for abstract input and gesture for location-based navigations. (2) Continuous repetitive operations are more likely to be performed using the touchpad. (3) Among several factors including changes in operation types, input failures, and interruptions in task performance, the first factor is the major cause of users’ input mode mixes. (4) Prevalent user strategies for error corrections do not involve input modality switches but repetitions of failed operations and tryouts of different operations in the same input modality. (5) Training orders have limited effect on users’ preference on input modalities in later usage. (6) Skipping is common in a familiar information organization. In speech, the main skipping strategy is speaking the desired item directly. On the touchpad the main strategies include approximating the location of a desired item and sequential skips. A confirmative experiment will be conducted with visually impaired users and will lead to suggestions for designing non-visual multimodal input for the addressed application domain.
Recommended Citation
Chen, Xiaoyu, "Designing Non-Visual Multimodal Dialogues to Support Information Access for the Visually Impaired" (2005). AMCIS 2005 Proceedings. 98.
https://aisel.aisnet.org/amcis2005/98