Conversational Speech Recognition for Creating Intelligent Agents on Wearables


By Benjamin A. Wong and Thad E. Starner

Future Computing Environments Group,
College of Computing and GVU Center
Georgia Institute of Technology, Atlanta, GA, USA 30332-0280

Published: August 2001


Paper-based planners and calendar programs run on personal digital assistants are common methods to remind users of tasks and appointments. Both require a certain amount of time to record an entry. Is it worthwhile (or even possible) to try to create an appointment scheduling system which takes significantly less time and effort? We argue that it is and describe one such system currently being built, its potential benefits, and some of the issues it must overcome. This system, called the Calendar Guardian Agent (CGA), runs on a wearable computer to assist in scheduling appointments based on captured speech from a user’s everyday conversations.


There are a variety of methods to remind people of tasks and appointments. These methods include paper-based planners, such as the Franklin Day Planner, and calendar programs run on personal digital assistants (PDAs), such as the Palm Pilot. Often these scheduling systems are used during a conversation in which a person is orally scheduling a meeting: “I’ll meet you at 3pm tomorrow.” However, using these current scheduling systems can incur a significant overhead. There are other methods of scheduling which do not require a large overhead in time, for example, using a human assistant or just trying to memorize an appointment. However those methods have considerable trade-offs in cost efficiency and reliability, respectively. Wearable computing’s capability for continuous interaction permits new techniques to facilitate everyday life and social interaction. One of our latest projects, the Calendar Guardian Agent (CGA), is a system to schedule appointments using an agent on a wearable computer which attends to the user’s everyday conversations. Although we use techniques similar to a human assistant and memorization to reduce the overhead of scheduling, we expect the costs in terms of monetary expense and unreliability to be reasonable. Note that while portions of the CGA have been completed, the system described in this paper is still under development.

About The Author(s)

Thad Starner is the director of the Contextual Computing Group and is also a Technical Lead/Manager on Google’s Project Glass. In general, our academic research creates computational interfaces and agents for use in everyday mobile environments. We combine wearable and ubiquitous computing technologies with techniques from the fields of artificial intelligence (AI), pattern recognition, and human computer interaction (HCI).

Leave a Reply