A Managing Agent for Sharing Multiple Modalities

Boi's Computer Science Engineeing Degree Thesis

Abstract:

Multimodal user interfaces promise more natural man-machine communication, as well as improved accessibility for disabled people. When extending multimodal user interfaces beyond single application scenarios, providing a multimodal interface in a desktop, applications have to share access to input and output devices. This requires an agent that arbitrates between applications and low-level drivers. This thesis proposes a design for communication between driver, manager and application, and discusses a prototype implementation and preliminary testing results.

The printed version consists of five separate documents:

The colored cover page: frontpage.ps.bz2 (227KB) | frontpage.pdf (88KB)
The main document: thesis.ps.bz2 (2.2MB, expands to 17.4MB Postscript!) | thesis.pdf (1.2MB)
Appendix A: abstract of the student poster presentation at HCI International99, called Multimodal User Interface Management: hci_99.ps.bz2 (279KB) | hci_99.pdf (409KB)
Appendix B: the abstract of an article submitted to Specom99, called Speech Based, Multimodal Information Systems: specom.ps.bz2 (28KB) | specom.pdf (53KB)
Appendix C: internal document, Considerations for writing applications: apps.ps.bz2 (27KB) | apps.pdf (77KB)
Appendix D: internal document, Modality Manager User Manual: manual.ps.bz2 (36KB) | manual.pdf (150KB)

The larger documents have hi-res bitmapped pictures, which is very space-inefficient in PostScript, unfortunately.

Reference documentation on the Modality Management Library, which was written in JavaDoc during the development of the library, is not part of the thesis but is available here in printable format. It's called the MIM Library JavaDoc reference: docs.ps.bz2 (83KB) | docs.pdf (327KB)