ITR: Adaptive Spoken Dialog with Human and Computer Partners

This project takes an innovative and multidisciplinary approach to characterizing how spoken language production and interpretation are coordinated in dialog. It examines how people adapt to both human and computer conversational partners, through variation in pronunciation (e.g, dialect), rhythm, word choice, sentence structure, and perspective. Adaptations include those that make processing easier for both partners, such as converging on the same wording; they also include adjustments made by one partner "for" the other. Findings from human dialog will be applied to systems that use speech recognition and generation, with the goals of (1) getting users to adapt utterances to forms the system can process more robustly, and (2) whenever feasible, adapting the system's vocabulary, dialect, and perspective to the user's' needs. The project brings together methods and theoretical perspectives from computer science, linguistics and psychology to advance theories and improve applications. Methods include laboratory experiments, corpus studies, and simulation studies, integrated with prototyping and evaluation of spoken dialog systems. Three applications are planned: a picture matching game, a PDA-based calendar system, and a telephone-based course evaluation system for an undergraduate community. The project will enhance training of young scientists in computer science, linguistics and psychology, and will include underrepresented groups in basic research and user-interface engineering. The broader impact will be a scientific foundation for developing flexible and robust spoken dialog systems that serve the needs of diverse users. Specific application opportunities include mobile computing, educational assessment, machine speech recognition of non-standard pronunciation, and tutoring systems for the hearing impaired or second-language learners.