This content was uploaded by our users and we assume good faith they have the permission to share this book. If you own the copyright to this book and it is wrongfully on our website, we offer a simple DMCA procedure to remove your content from our site. Start by pressing the button below!
Editorial Intelligent Help Systems for UNIX: Computational Models and Systems
This is the first in a series of three special issues focussed on intelligent help systems for UNIX,1 each with its own emphasis: (1) computational models and systems, (2) planning and knowledge representation and (3) natural language dialogue. In this first issue focussing on computational models and systems there are five papers, one addressing empirical foundations, another virtues and problems, with the final three describing comprehensive implemented systems. Jerrams-Smith presents some empirical studies conducted on UNIX users which resulted in a taxonomy of error types so that users’ errors could be classified. In turn, this information is encapsulated as production rules within a knowledge base which forms the basis for an intelligent interface to UNIX. The prototype makes inferences about users’ mental models and uses these to select appropriate tutorial advice. A comparison was conducted between users of the prototype intelligent interface and the usual UNIX interface. Users of the prototype made fewer errors, had fewer misconceptions and took less time to complete given tasks. Virvou et al. look at the virtues and problems of an active help system for UNIX. A passive consultant accepts questions from the user, but is otherwise ignorant of the user’s goals, while an active consultant continually monitors users’ actions, attempting to discover their goals from these actions. An empirical study encompassing a cross-section of UNIX users at an academic site showed the need for an active, rather than passive, help system. Examples of system interaction supporting this view are given, and a discussion of the help needed in these examples is provided. It is proposed that to provide such help requires the construction and maintenance of a model of each user. The next three papers focus on comprehensive systems which have been built to perform as intelligent help systems for UNIX. In the first, Wilensky et al. describe UC (UNIX Consultant), a natural-language interface that allows naive users to learn about the UNIX operating system. The development of UC was undertaken because the task was thought to be a useful domain
2
EDITORIAL
for AI research, as well as a useful application of AI work in planning, reasoning, natural language processing, and knowledge representation. UC contains a number of components for language understanding (ALANA), inference (concretion mechanism), goal and plan analysis and generation (PAGAN, UCEgo), domain planning (KIP), content expression (UCExpress), and language generation (UCGen), user’s knowledge state with respect to UNIX (KNOME), and enabling addition of knowledge of both English and UNIX facts to UC’s knowledge base (UCTeacher). All components of UC make use of a knowledge representation system called KODIAK. Mc Kevitt describes OSCON (Operating System CONsultant), which is also a natural-language operating system consultant capable of answering queries on over 40 commands from both the UNIX and MS-DOS2 operating systems. OSCON differs from UC in that it is specialised more on the domain of operating systems, whereas UC has the broader goal of studying knowledge representation, planning and natural language processing through operating systems. UC employs a much more general knowledge representation and planning mechanism with a more cognitive flavour. Mc Kevitt also points to more recent developments where empirical studies with subjects using UNIX through Wizard-of-Oz techniques give frequencies of types of questions asked and enable the incorporation of user modelling into OSCON. Matthews et al. present USCSH (University of South Carolina SHell), an active intelligent help system for UNIX. USCSH can operate in two modes, the active mode and the intelligent mode. In the active mode, USCSH monitors users’ interactions and at appropriate times makes suggestions on how the user might better use the system. This sense of active is similar to that used by Virvou et al. above. In the intelligent mode, the system accepts natural language questions and responds to them, taking into consideration the context and user’s ability. A trend that has been showing up in commercial applications of AI: translating a working prototype into a language that can be more easily embedded into a running application, was employed in the development of this system. A prototype in Lisp was rewritten in C so that it could be called directly from the C shell. To sum up, there are a number of ways of looking at a domain of application such as UNIX, depending on whether one is interested in modelling the domain itself or the cognitive models applied to the domain. Also, empirical studies are useful for obtaining knowledge about how subjects interact with domains such as UNIX, and then this knowledge can be incorporated within systems. Finally, “active” rather than “passive” help systems are more appropriate for situations in which the system attempts to model the plans of the user over time. The latter problems of planning and knowledge representation
EDITORIAL
3
will be the subject of the next special issue on intelligent help systems for UNIX. The articles in this issue are derived from papers originally presented at a workshop entitled Knowledge representation in the UNIX help domain, organised by Peter Norvig, Wolfgang Wahlster and Robert Wilensky at the University of California, Berkeley, USA, in December, 1987. The workshop was funded, in part, by the International Computer Science Institute and all participants were invited. The area of intelligent help systems for UNIX provides such a fruitful example domain of application for many AI techniques that we have decided to publish this work, which is still timely, more widely and particularly now where we have many spoken dialogue systems applied to such fixed domains (see Bernsen et al. 1998, BusinessWeek 1998). Each article here has been reviewed by the editors and has been subsequently revised; furthermore, all authors have been asked to include a section on recent developments on their work. Related work which may be of interest to the reader can be found in Kobsa and Wahlster (1988) which focusses on user modelling and in Maybury and Wahlster (1998) which is a comprehensive publication on intelligent user interfaces. Members of the Kluwer editorial and publishing staff are to be thanked for their help in producing this issue. We regret that John Jones passed away in 1994 in a climbing accident in Peru. A special thanks to David Chin for preparing the paper on UC herein. The Editors, April 2000, Stephen J. Hegner Umeå University Sweden Paul Mc Kevitt The Queen’s University of Belfast Northern Ireland Peter Norvig NASA Ames Research Center California, USA and Robert Wilensky University of California at Berkeley USA
4
EDITORIAL
Notes 1 UNIX is a trademark of X/Open, Inc. 2 MS-DOS is a trademark of Microsoft Corporation.
References Bernsen, N-O., Dybkjær, H. & Dybkjær, L. (1998). Designing Interactive Speech Systems: from First Ideas to User Testing. Springer-Verlag: New York, NY. BusinessWeek (1998). Speech technology is the next big thing in computing, Special Report. BusinessWeek, Feb. 23. New York: McGraw-Hill. Kobsa, A. & Wahlster, W. (1988). User Models in Dialog Systems. Springer-Verlag: Berlin. Maybury, M. & Wahlster, W. (eds.) (1998). Readings in Intelligent User Interfaces. Morgan Kaufmann Publishers: Los Altos, CA.
Artificial Intelligence Review 14: v–ix, 2000. Stephen J. Hegner et al. (eds.), Intelligent help systems for UNIX.
v
About the Authors
Gautam Biswas is an Associate Professor of Computer Science, Computer Engineering, and Management of Technology at Vanderbilt University. He received a B.Tech. degree in Electrical Engineering from the Indian Institute of Technology, Bombay, India, in 1977, and M.S. and Ph.D. degrees in Computer Science from Michigan State University, East Lansing, in 1979 and 1983, respectively. Dr. Biswas conducts research in Artificial Intelligence, Knowledge Based Systems, and Cognitive Modeling, and their applications to the development, analysis, and evaluation of Intelligent Systems. He has primary interests in hybrid modeling and analysis, and their applications to analysis, control, diagnosis, and design of complex embedded systems. He has applied hybrid modeling and combined qualitative-quantitative reasoning techniques to design fault monitoring and isolation systems for fast breeder reactors and automobile engine systems. Another area of interest is the design and implementation of Intelligent Learning Environments. Working as part of a Cognition and Technology group at the Learning Technology Center in Vanderbilt University, he has developed multi-media systems and simulation-based intelligent learning environments for middle school math and science education. In ONR-funded projects he has developed methods for assessing expertise in troubleshooting of Alternating Current circuits and sys-
tems. Dr. Biswas has published in a number of journals and contributed book chapters. He is an associate editor of the International Journal of Approximate Reasoning and the Journal of Applied Intelligence, and has served on the Program Committee of a number of conferences. He was co-chair of the 1996 Principles of Diagnosis Workshop, and a senior program committee member for AAAI 97 and 98. He is a Senior member of the IEEE Computer Society, ACM, AAAI, and the Sigma Xi Research Society. David N. Chin is an Associate Professor of Information and Computer Sciences at the University of Hawaii at Manoa. He received his B.S. in Physics and B.S. in Computer Science/Engineering from M.I.T. in 1980 and his Ph.D. in Computer Science from the University of California at Berkeley in 1987. While at Berkeley, he lead the design and implementation of UC, the UNIX Consultant, a natural language consultation program for the UNIX operating system. His doctoral dissertation concerned treating UC as an intelligent agent which has its own goals, plans, and model of the user. By detecting the right goals in appropriate situations, UC can take the initiative in its dialog with the user in order to volunteer information, correct user misconceptions, and reject unethical requests. UC’s user model, KNOME (KNOwledge Model of Expertise), ana-
vi
ABOUT THE AUTHORS
lyzed user queries and inferred the user’s level of UNIX expertise based on what kinds of facts the user does or does not know. Dr. Chin has served on the editorial board of the international journal User Modeling and User-Adapted Interaction since its inception and has twice chaired the International Conference on User Modeling. Professor Chin’s current interests include cognitive science, software engineering, geographic information systems, and artificial intelligence, particularly intelligent agents, natural language processing, user modeling, and intelligent interfaces. Stephen J. Hegner received the Ph.D. degree in Computer and Information Science from the University of Massachusetts at Amherst in 1977. Currently, he is a senior lecturer in the Department of Computing Science at Umeå University, Sweden. His areas of professional interest focus on algorithmic aspects of knowledge representation and inference in database and knowledgebase systems. Jenny Jerrams-Smith is a Principal Lecturer in Information Systems at the University of Portsmouth, UK, and director of the Advanced Interface research centre. She was formerly a Principal Researcher in Artificial Intelligence at Philips Research Laboratories, UK. She completed her PhD in Computer Science in 1986 at the University of Birmingham and her Master’s degree in Computer Science in 1982. She has a Post Graduate Certificate in Education (1968) and a Bachelor’s degree in Zoology (1967), both from the University of London. Her primary research interest is in intelligent/adaptive user interfaces, in which she has many refereed publications. She
has recently completed funded research projects on an adaptive interface for a telecare system and on adaptive navigational guidance for the WWW. John Jones was born in 1956 in England and died in 1994. He achieved a BSc First Class degree in Mathematics from the University of Nottingham in 1980. He completed his Ph.D. in Mathematical Logic at the Department of Mathematics, University of Nottingham. He was a Research Fellow in Artificial Intelligence at the University of Edinburgh from October 1983 until September 1986. He joined the Department of Computer Science at the University of Hull as a Lecturer in October 1986 and remained there until he died. James Martin is an Associate Professor in the Department of Computer Science and the Institute of Cognitive Science at the University of Colorado at Boulder. He received his Ph.D. in Computer Science from the University of California at Berkeley in 1988. He earned his B.S. degree in Computer Science from Columbia University in New York in 1981. His current research interests are in Natural Language Processing, Information Retrieval and Machine Learning. Manton Matthews is Associate Professor and Director of Graduate Studies in the Department of Computer Science at the University of South Carolina. He received his Ph.D. in Mathematics (graph theory) while simultaneously obtaining a MS in Computer Science in 1980. He has served as program chairman for the 1990 International Conference on Industrial and Engineering Applications of AI and Expert Systems in 1990 and then again as
ABOUT THE AUTHORS
co-chairman in 1997. He has served on the editorial board of the Journal of Applied Intelligence. His research interests include intelligent user interfaces, natural language processing (NLP), user-modelling, graph theory, and forensic computing. James Mayfield is a Senior Computer Scientist at the Johns Hopkins University Applied Physics Laboratory. Prior to joining Johns Hopkins, he was an associate professor of computer science at the University of Maryland, Baltimore County. Dr. Mayfield received an A.B. in 1979 from Harvard College, and a Ph.D. in 1989 from the University of California at Berkeley. Since the time of the reported plan recognition work, he has carried out research in hypertext, information extraction, distributed inference, and agent communication languages. His current projects include work in information retrieval, computer forensics, and simulation. Paul Mc Kevitt is 36 and from Dún Na nGall (Donegal), Ireland. He is Associate Professor (Senior Lecturer) in the School of Computer Science at The Queen’s University of Belfast, Northern Ireland. Previously, he has been Visiting Professor of Intelligent MultiMedia Computing in the Institute of Electronic Systems at Aalborg University, Denmark and a British EPSRC (Engineering and Physical Sciences Research Council) Advanced Fellow in the Department of Computer Science at the University of Sheffield, England. The Fellowship, commenced in 1994, and released him from his Associate Professorship (tenured Lecturership) for 5 years to conduct full-time research on the integration of natural language, speech and vision pro-
vii
cessing. He completed a Master’s degree in Education (M.Ed.) at the University of Sheffield in 1999. He completed his Ph.D. in Computer Science at the University of Exeter, England in 1991. His Master’s degree in Computer Science (M.S.) was obtained from New Mexico State University, New Mexico, USA in 1988 and his Bachelor’s degree in Computer Science (B.Sc., Hons.) from University College Dublin (UCD), Ireland in 1985. His primary research interests are in Natural Language Processing (NLP) including the processing of pragmatics, beliefs and intentions in dialogue. He is also interested in Philosophy, MultiMedia and the general area of Artificial Intelligence. Mark Millington is 39 and from Chester, England. He is a consultant specialising in finance and telecommunications applications, working in the Chicago area. He completed his Ph.D. in Computer Science in 1985 at the University of Edinburgh, Scotland, and his Bachelor’s degree in Computer Science and Mathematics in 1980 at the University of Essex, England. Harish Neelakandan, 34, completed his Master’s degree in Computer Science from the University of South Carolina in 1987 and worked at Cognitive Systems in New Haven, Connecticut, from 1988 to 1992. While at Cognitive, which was founded by Roger Schank, then at Yale, to commercialize his group’s Natural Language research, Harish worked on the development of a Natural Language system for processing financial funds transfer telex messages. This system was deployed at some of the major money center banks in the USA and Europe. Subsequently, he worked on a Finan-
viii
ABOUT THE AUTHORS
cial Portfolio Evaluation system for use by Portfolio Managers and then on a Case-Based Reasoning system to detect suspicious trading activity at a major North American stock exchange. He left Cognitive in 1992 to get a graduate degree in Management at MIT’s Sloan School of Management in Boston, and upon graduation from Sloan in 1994, joined Merrill Lynch in New York where he is currently a Foreign Exchange Options trader. Peter Norvig received a B.S. in Applied Mathematics from Brown University and a Ph.D. in Computer Science from the University of California at Berkeley. He has been a Professor at the University of Southern California and a Research Faculty Member at Berkeley. He has over forty publications in various areas of Computer Science, concentrating on Artificial Intelligence, Natural Language Processing and Software Engineering. He has also served as a Senior or Chief Scientist at Sun Microsystems Labs, Harlequin Inc. and Junglee Corp., and is currently Chief of the Computational Sciences Division at NASA Ames Research Center. Walter Pharr is an Assistant Professor of Computer Science at the College of Charleston, in Charleston, South Carolina, USA. He is currently completing his Doctoral degree at the University of South Carolina. His primary research interests are natural language processing, formal semantics for natural and artificial languages, and functional programming. Maria Virvou is 34, from Athens, Greece. She has been a lecturer in the Department of Computer Science at
the University of Piraeus, Greece since 1994. She completed her Ph.D. in Artificial Intelligence and Computer Science at the University of Sussex, England in 1992. She received her M.Sc. degree in Computer Science from University College London in 1987. Her first degree in Mathematics was obtained from the University of Athens, Greece in 1986. Her current research interests are in user modelling, Artificial Intelligence in Education, distance learning and adaptive user interfaces. Robert Wilensky received his B.A. and his Ph.D. from Yale University. In 1978, he joined the faculty of the University of California at Berkeley, where he is now Professor in the Division of Computer Science, and in the School of Information Management and Systems. He has served as Chair of the Computer Science Division, the director of BAIR, the Berkeley Artificial Intelligence Research Project, and the director of Berkeley Cognitive Science Program. Professor Wilensky has published numerous articles and books in the area of artificial intelligence, planning, knowledge representation, natural language processing, and digital information systems. He is currently Principal Investigator of UC Berkeley’s Digital Library Project. Dekai Wu is an Associate Professor of Computer Science at the Hong Kong University of Science and Technology (HKUST). He received his PhD in Computer Science from the University of California at Berkeley, and was a postdoctoral fellow at the University of Toronto (Ontario, Canada) prior to joining HKUST in 1992. He received his BS in Computer Engineering from the University of California at San Diego
ABOUT THE AUTHORS
(Revelle College) in 1984. He was also a visiting researcher at the Technische Universität München (Munich, Germany) during 1986–87. He currently serves on the Editorial Boards of Computational Linguistics, Machine Translation, Computational Linguistics and Chinese Language Processing, and Communications of COLIPS and on the Executive Committee of the Association for Computational Linguistics (ALC). His research interests include statistical
ix
and corpus-based natural language processing, machine translation, language modeling, speech recognition, Chinese computing, machine learning, language acquisition, dialog systems, and Internet information processing. Since 1992 he has led the SILC Machine Translation Learning Project, and helped found the Human Language Technology Center (HLTC) at HKUST. He is also active as a Principal Investigator in the Cyberspace Center of HKUST.
An Intelligent Human-Computer Interface for Provision of On-Line Help JENNIFER JERRAMS-SMITH Department of Information Systems, University of Portsmouth, University House, Winston Churchill Avenue, Portsmouth, Hants, PO1 2UP, UK (E-mail: [email protected])
Abstract. Some user interfaces, such as that of Unix,1 are difficult for novices to use, and this paper suggests a possible solution to such problems. The results of a study of Unix users enabled the development of a taxonomy of error types so that users’ errors can be classified. This information is encapsulated as production rules within a knowledge base and forms the basis for the design and development of an intelligent interface to Unix. The prototype makes inferences about users’ mental models and uses these to select appropriate tutorial advice. Performance of users of the prototype intelligent interface was compared with that of users of the usual Unix interface. The prototype users were found to make fewer errors, exhibit fewer misconceptions and take less time to complete a standard set of tasks. Keywords: user models, intelligent interface, knowledge based system, adaptivity, Unix
1. General Interface Problems and Possible Solutions Many reports indicate that computer applications are difficult to use and that users have insufficient expertise to benefit from their full range of functions (Hayes et al. 1981; Lang et al. 1981; Nickerson 1981; Hayes and Szekely 1983). More problematic, there is evidence that the application’s response to an error may be confusing (Mishra et al. 1984) or even positively unhelpful and misleading, thus causing additional errors (Hanson et al. 1984). In particular, the Unix operating system is seen as being difficult for novices to learn and use, as indicated by the interest in the provision of improvements such as COUSIN-Unix (Hayes and Szekely 1983), and the “natural language” interfaces of the Unix Computer Consultant (Douglass and Hegner 1982), the Unix Consultant (Wilensky et al. 1984), and the Unix File Orienter (Mishra et al. 1984). Many of the current suggestions for remedying users’ problems with interfaces focus on studying users (Ramsay and Attwood 1979; Hollnagel 1983; Norman 1983). Other guidelines recently suggested for interface development indicate that the interface should facilitate error handling but allow
6
JENNIFER JERRAMS-SMITH
freedom of expression, and should be easy to learn but appealing to experienced users (Shneiderman 1979); should provide feedback and a sense of presence (DuBoulay et al. 1981; Ehrenreich 1981); should provide not just answers, but answers which help users to achieve their goals (James 1981); should protect users from their own errors (James 1981) both to prevent serious consequences and because errors made in the first few attempts at a new activity tend to become ingrained and should enable commands to be chosen by recognition rather than recall (Bailey 1982). Associated with recent work on difficulties with interaction, there are numerous recommendations that an effective method for enabling users to make the best use of a system is by including a tutorial element within the system (James 1981; Tagg 1981; Relles and Price 1981; Bailey 1982; Scapin 1981), instead of (or in addition to) the provision of a stand-alone tutor such as the Unix tutor (Irgon and Martin 1984). Further, exploration-based learning is believed to be educationally preferable to that of instruction-based learning (Carroll 1984; Kamouri et al. 1986). The author provides a solution to the problems users have with Unix and other applications. Unix was chosen as an example application because it is an operating system whose use is becoming more widespread, but which causes serious problems for novice users because of the enigmatic nature of its responses to their commands. The solution involves a study of Unix users, leading to the design and development of an additional intelligent interface to Unix, which includes a knowledge based system (KBS) and which also incorporates the current recommendations for interface design and development within the interface, as outlined above. This interface includes a tutorial component and is able to make inferences about users’ mental models so that inaccuracies can be remediated as soon as they are detected, which relates the work to that in the field of intelligent interfaces (Innocent 1982) and also to tutoring systems such as that by (Self 1974; Clancey 1986; Brown and Burton 1978; Burton and Brown 1979; Sleeman and Hendley 1979; Sleeman and Smith 1981; Finin 1983; and Johnson and Soloway 1984). In particular, the solution makes use of previous work on the acquisition of mental models by novices: Bayman and Mayer (1983) found that novices’ mental models (of BASIC) are often inaccurate, while Sleeman (1983) proposed the concept of malrules (the wrong rules which children use when they learn algebra). All user commands pass through the intelligent interface before they reach the application. This transparent interface monitors the user’s interaction with the application, but users are unaware of it except when the interface has inferred that tutorial help is required or that the user’s action could have harmful consequences. In this event, the interface immediately provides
PROVISION OF ON-LINE HELP
7
tutorial information at the moment when it is most needed and therefore of most value to the user. This enables “minimalist training”, as advocated by Carroll (1984), so that users can work in experimental mode while at the same time it traps their errors and helps them to develop a correct mental model of the application. For such a new type of interface there were few guidelines available for its development. Since possession of a correct mental model of the application cannot be assumed for novice users (Ramsay and Attwood 1979), it was decided to extend Norman’s (1983) suggestion that some rules for the design of the interface for a specific application might be derived from analysis of human error. The author therefore investigated user errors in order to discover if they indicate what help users need, and where the user’s mental model is inaccurate, and therefore what remedial tutoring is required. A method was devised which enabled the development of a taxonomy of error types so that errors could be classified. Subsequent investigation indicated the misconceptions (inaccuracies in the mental model) which were associated with some of the errors, and hence the remediation requirements. An investigation of the differences between errors made by experts and novices was also made. Fewer of these mistakes were expected to be caused by inaccuracies within the expert user’s mental model, since the accuracy of the mental model depends on the level of the user’s expertise (Carey 1983). The resultant information was encapsulated as production rules in the knowledge base within the intelligent interface.
2. An Empirical Study of Unix Users 2.1. Method of study A study was made of 55 novice users of Unix. These were first year undergraduates in software engineering, who were familiar with other operating systems. Observations were made over a four week period by logging all input to the Unix shell and simultaneously collecting verbal protocols from the subjects. 2.2. Data Figure 1 shows a sequence of commands given by one of the subjects. It also has the error categories into which each of the incorrect commands was subsequently classified. Figure 2 provides typical examples of verbal protocols provided by the subjects.
8
JENNIFER JERRAMS-SMITH Command
Error category (see Table 1)
elp help passwd robins help passwd passwd logot logout passwd help passwd hcat/manual/summary/passwdi passwd YYYYYYY login login passpasswd passwd help help passwd passwd (v) passwd (u) help passwd passwd aaaaaa aaaaaaa hoqrr uoou hcat/manual/summary/passwdi login cat/manual/summary/passwd help passwd hcat/manual/summary/passwd why not help goto help passwd help passwd help login /etc/passwd hcar/etc/passwd
Figure 1. Typical log of user commands their errors and categories.
2.3. Error analysis and classification Error analysis was carried out by an expert user of Unix who was also experienced in teaching novices how to use Unix, so that a taxonomy could be developed and errors classified in terms of this taxonomy. Errors were considered to be not only those actions which would produce a system error message, but also included commands which, although valid,
PROVISION OF ON-LINE HELP
9
June 14, 1983. 15.31. Just dabbling to see if I can do anything. It would appear that @ doesn’t work and removing underlines. Don’t know why that is. Just put in my password. Think I’ll leave it and come back after I’ve looked at the literature. June 15th, 1983. 13.23. Objective today to try and understand what this thing does. Since yesterday I’ve failed to do anything. Actually understand some of the text handling . . . files and things. Couldn’t manage to write to terminal G. Don’t know why that was. Well, so far we have achieved nothing again today. So I’m leaving off for today. June 16th, 1983. 13.32. Well, I’ve now got half an hour on this machine. We’ll call it junk.txt (using the ned screen editor). We appear to have – what about ˆF? If we can find the right spot in our Unix papers we can actually do something. Totally stuck on the literature. Well, we can honestly say it’s hopeless finding anything in this manual. (Consults friends. One explains how to use the em line editor). Well, I think I’ll start again. End of session three (note of despair or disgust in his voice). Figure 2. Typical verbal protocols.
make inefficient use of the system or do not produce the intended result. Errors which were caused by misconceptions were differentiated from errors caused by mistyping. 2.4. The meaning of errors – the differentiation between mistypes and misunderstanding Examples of command sequences which indicate inaccuracies in the user’s mental model were derived from the error analysis. Mistyping categories were also derived from the error analysis. The following gives additional guidelines to those provided by Damerau (1964) for the formulation of the rules indicating that mistyping has occurred. These were derived by analysis of the logs generated by the subjects’ interaction with Unix. It is very important to detect mistyping and to differentiate it
10
JENNIFER JERRAMS-SMITH
Table 1. Errors and percentages for the first 2 weeks and the second 2 weeks Error
Category
1st
%1st
2nd
%2nd
Total
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19
Unable to give command Failed try for help Wrong mode or directory Using previous knowledge Mistype Prevented Misunderstanding Inefficient use Misread documents Unknown Misconception Obscenities Guesses Loss of attention Forgot own file name Easier option available Known to be wrong Forgotten Documentation errors
40 17 7 21 20 7 5 6 5 0 0 0 0 0 0 0 0 0 7
29 12 5 15 14 5 3 4 3 0 0 0 0 0 0 0 0 0 5
14 39 18 32 77 10 5 55 11 57 45 2 2 26 3 8 9 3 0
3 9 4 7 18 2 1 13 2 13 10 0.5 0.5 6 0.5 1 2 0.5 0
54 46 25 53 97 17 10 61 16 57 45 2 2 26 3 8 9 3 7
from misunderstanding so that experienced users in particular are not given unnecessary advice (Number of occurrences are shown in brackets). 1. Omission of one character – this includes the omission of space between arguments in the case of an experienced user. (12) 2. Substitution of another character – next one to the right or left on the QWERTY keyboard. (8) 3. Repetition of one character. (16) 4. Repetition of two characters. (2) 5. Inversion of two characters. (4) 6. Extra characters at the start – this could be caused by incomplete deletion of incorrect characters. (17) 7. Repetition of a previous character instead of the correct one – example nror for nroff. (4)
PROVISION OF ON-LINE HELP
Command
Interpretation
em SC em SC
does not know the !hshell commandi facility when in em
SC f > f2 lf f2
does not understand > and pipes
rm f1 rm f2 rm f3
does not know that all files could be supported on one command line
cat f|lf
cat is redundant
create f em f
create is redundant since em automatically creates
SC f1 SC f2 SC f3
use of same command could indicate lack of knowledge of pattern match such as ∗
11
Figure 3. Typical command sequences indicating inaccuracy in the user’s mental model. (SC = any Shell Command, em = Editor, lf = List File, create = Create File).
3. Development of an Intelligent Transparent Interface for Unix Since this was an unusual application, the production of a prototype is an important part of the process of development and gave useful indications for the final design. The prototype, and the final version, were written in FRANZ LISP on a VAX 11/730. During prototype development the code remained uncompiled and underwent continual modification. The prototype contained an intelligent spell-checker in addition to the components for recognition and remediation of other types of errors. 3.1. Rule based inferencing A knowledge-based component was developed for inclusion within the prototype interface, and was based on an expert system suggested by Winston and Horn (1981). The knowledge base consisted of a set of production rules,
12
JENNIFER JERRAMS-SMITH
and the control mechanism allowed goal-directed search or pattern-directed search. The knowledge base consists of modules of knowledge in the form of production rule sets grouped according to function and accessed when required for solving particular parts of a problem. A production rule expresses knowledge in the form of a conditional part followed by some action. A simple example might be: IF (command has failed) (intended command has been deduced) (deduced command = passwd) (user is at novice level) THEN (explain how to set the password) The inference engine of the KBS uses facts deposited in the database to produce new facts or actions. It matches the conditionals of the rules with the facts in the database, and causes the required action when a match occurs. Thus deduction is carried out by forward chaining only. Forward chaining has previously been shown to be useful for modelling cognitive processes and for solving problems which require very broad but shallow knowledge. An explanatory facility was developed, similar to that of MYCIN (Shortliffe 1976), which enabled the user to ask for justification: how a fact was derived or why a fact was required. The output to the user was in pseudo natural language, which was aided by coding in LISP so that chosen parts of a rule were directly displayed on the screen. Pseudo-probabilities (confidence factors) were provided which were similar to those of MYCIN. It became clear from the experience of using the experimental KBS that it was more difficult than had been anticipated to decide on the values of confidence factors, and they were therefore omitted from later versions. As indicated above, it was also realised that the backward chaining mechanism (and therefore the “why” explanations) would also be un-necessary for this application and it was decide to omit them too. The “how” explanations allow users to follow the reasoning strategy of the KBS and was used during the testing and modification stage of development to ensure that the interface correctly deduced the user’s intentions. 3.2. User models The user model consists of two components: the generalised or static model and the dynamic or specific model. The static (generalised) model is derived from error and protocol analysis (held within the rules of the knowledge base), from published information on the use of Unix, from command usage frequency information and from likely previous knowledge of other Oper-
PROVISION OF ON-LINE HELP
13
ating Systems, such as Multics, CP/M or Tops-20 (held as a list of possible substitutes for each command). It is well known that people will attempt to apply their previous knowledge when attempting a new task, and numerous examples are visible in the user logs. However, as the logs indicate, this can sometimes hinder the successful completion of the new task. The Dynamic User Model includes the user’s model of the system, the user’s behaviour patterns, the user’s expertise level and the trace of the user’s behaviour for the current session. The record of previous activity is held on property lists: attributes for each input line include original command, deduced command, intended command, original arguments, and deduced arguments, intended arguments. Sequences which may indicate the occurrence of inefficient use are maintained, and the frequency of usage of commands and files is also recorded. In future versions, as part of the planning operation, the Dynamic User Model should also maintain possible sequences which might indicate a command which is likely to be used next, or a file which is currently being operated upon (example: edit file1 cat file1 lf file1). 3.3. Pre-testing As soon as the prototype was complete and had been tested to eliminate all apparent errors in the code, pre-testing was carried out in order to design the test-evaluation stage of development. The pre-test subjects answered questions which helped to validate the rules of the knowledge base. The interface with the completed interpretation module was pre-tested on a small group of expert users of Unix (one research associate, two research students). They were not constrained to use any specific operations, but asked to use it in whatever way they liked. The explanations facility of the KBS was used as a development tool which enables the subject to ask how each deduction was made and work back through a chain of reasoning. The rules were shown to the user in a natural language version; a property list holds the English language description of each function. The subjects indicated whether they believed each deduction to be true or false. Pre-testing revealed that additional rules and hypotheses were required. For example, if “cc fred” were given as input, “fred” is a valid file, but not a valid argument for “cc”. However “fred” would be a valid argument for “bc” or “wc” and this is discovered by the interface. The final hypothesis is that the input cannot be interpreted because the relevant rules state that it is only likely to be “cc” if no other command has matching arguments. Thus a rule is required which will choose between the two possibilities or offer both. Currently the interface attempts to choose the most likely and rules out all others.
14
JENNIFER JERRAMS-SMITH
The final pre-test was carried out using a standard sequence of operations which were completed by a single test subject (a computer expert but new to Unix). The tester followed a standard sequence of frequently used operations such as display a file on-screen, make a hard copy of a file, find which files are present in the current directory, delete a file, make a copy of a file, use the editor, use the text-formatter. By the end of this stage the LISP coding had been compiled in order to improve speed. 3.4. Action of interface for the final pre-test The following is an ordered list of the actions carried out by the intelligent interface for the final pre-test. 1. Production rules and Unix data are read in. 2. Menu, file-list and prompt simulator are output. 3. A command line from the user is read in. 4. A scanner divides this string of characters into tokens (example: divides “fred&” into “fred” and “&”). 5. Each token is checked for the types it might fit (examples: username, string, filename, file in current directory, valid command, digits, switch). 6. Each token is checked to discover if it could be a mistyped command or a mistyped file in the current directory. 7. The arguments are matched against each of the possible commands (examples: ps = ps, pr, ls; cap = cmp, cat, cal; ll = ln, l, ls; cp = cc, cp, cmp, cd). 8. The production rules are used to determine the user’s intention. This results in a list of hypotheses about the user’s intentions. 9. The explanations of the KBS are shown to the subject to discover the validity of the rules. Subjects indicate the truth/falsity of rules for their situation. 10. The command is passed to Unix to show its response. 11. The subject is asked for comment on the Unix response. 12. A record is kept within the interface of user commands, and of the user’s response to the rules and to Unix. 3.5. Modification of the rule base after pre-testing New rules were easily added to the rule set. Such rules were formulated after pre-testing on the basis of information provided by the user in response to the deductions offered by the interface. Rules were also modified and added after consultation with users during the full test. Ideally, the new rules would be added by the interface, but there was insufficient time to automate this activity.
PROVISION OF ON-LINE HELP
15
3.6. Evaluation of the interface Although usability studies were not in common use when this research was conducted, the transparent interface was subsequently evaluated by tests with users in order to discover how effective it is for Unix and also to discover which aspects might be generally applicable to other systems which are difficult for novices to learn and use. For this between-subjects study; 13 joint honours undergraduates in Mathematics and Computer Science were divided randomly into an experimental group (7 subjects) and a control group (6 subjects). Each subject in the experimental group was provided with a version of the intelligent interface, which was removed as soon as the study was complete. Testing involved completing a standard sequence of operations to be attempted in a given order and involving frequently-used Unix commands, an improved version of that used for the pre-test since, for instance, the pre-test indicated that the operation to find on-line help should be successfully completed before any other command is attempted. Logs were recorded for subjects and in addition a record was made of all the analyses and interpretations which the intelligent interface carried out for each input line plus the responses made by the subjects to the questions provided by the tutoring module. The following summarises the results. Further details are in Jerrams-Smith (1989). 1. Fewer mistakes were made by the experimental group. 2. The control group made many more repeats of identical errors. 3. Most members of the experimental group completed the sequence; most of the control group did not. 4. Fewer commands were given by members of the experimental group. 5. More misconceptions were found in the control group than in the experimental group. Protocol analysis was carried out in which the subjects attempted to describe their intentions and activities while they carried out the standard sequence, and this was recorded on tape. However, in most cases there was a greater emphasis on debriefing because many users found it extremely difficult to talk about what they were doing while also trying to learn new and complex tasks. This proved to be a fruitful exercise because it indicated important aspects of the intelligent interface which were not otherwise obvious. These included: 1. The directory listing provided as part of the intelligent interface provided feedback: it was used to verify that commands had acted as expected.
16
JENNIFER JERRAMS-SMITH
2. There was evidence of a wrong mental model in use, which was not detectable from the log: one subject thought that files had to be created before using the ‘mv’ or ‘cp’ commands. 3. There was evidence that the interface corrected misconceptions about Unix as shown in the following extract from the protocol transcript: “Edited ‘test3’ using vi editor and now using the spell command to check for spelling mistakes in ‘test3’. (pause) 5 spelling mistakes recorded and these are going to be put into a file ‘test.sp’ using piping function. (pause) Pipe can’t be used since I’m trying to put it into a file so I’ll use the greater-than sign. (pause) Fine. That works.” 4. It was possible to gain some idea of the subjects’ impression of the interface in action, as indicated in the following: questioner: “Did you actually get anything out of the tutorial comments that you got back? Can you make a comparison between the INFO information and the tutorial comments?” subject 1: “Yes. The tutorial – it aims at the problem more closely. The INFO is general information, but the tutorial has found out where you’ve got a problem and actually aims for that – that’s pretty useful.” The results indicate that members of the control group gave many more commands and showed more misconceptions than the members of the experimental group. The mode of activity may be different for learners where help is not easily available. They may adopt a more experimental approach and hence use many more commands to effect the same results. There is an indication that misconceptions are corrected by the intelligent interface as soon as they appear and so are not repeated, as tends to occur in the control group. 4. Recent Developments Research on adaptive (intelligent) interfaces, which respond to meet the needs of the individual user, is still in progress (Jerrams-Smith 1989), and a good case has recently been made for the provision of various kinds of adaptive system (Benyon and Murray 1993; Elsom-Cook 1993). The essential component of such adaptive systems is a user model (also referred to within tutoring systems as a student model) which stores the system’s current beliefs about the attributes of a specific user of the system. In addition, recent research has identified a number of potential problems endemic to hypermedia which could ultimately restrict its usefulness but which might be solved by the provision of adaptivity. For instance,
PROVISION OF ON-LINE HELP
17
Waterworth (1992) suggests that there remain significant unresolved usability issues, while Whalley (1993) discusses a potential problem associated with hypertext when used within an educational context: the fragmented nature of a hypertext document means that the most natural way to study hypertext is by browsing or exploring the various paths of the document. In some instances, however, the browsing activity may be an inappropriate method of learning, for example, when an author is trying to develop a series of ideas within a particular context or framework. Further problems which need to be addressed were identified by Conklin (1987), who describes two main difficulties associated with reading hypertext documents (which apply equally to hypermedia): disorientation and cognitive overhead. Disorientation or ‘getting lost in space’ can occur if the user is uncertain as to his/her location on the hypertext network. Arguably this problem could also exist in traditional linear text documents, however, in this case the reader is limited to searching either earlier or later in the text. Because hypertext offers more dimensions in which the user can move, the likelihood of a user becoming lost is increased, especially in a large network. The problem of cognitive overhead occurs when the user is presented with a large number of choices about which links to follow. These directional decisions are absent in a traditional linear text document (or in a film or TV programme) when the author has already made the choices for ordering of material. With hypertext (and also with hypermedia), the moment a link is encountered, the reader must decide whether or not to follow the link. If many links are encountered the reader may become distracted, leading to what Conklin (1987) refers to as information ‘myopia’. Adaptive systems may provide a solution to such problems and the current interest in intelligent and adaptive multimedia/hypermedia systems is indicated by recent work which involves an investigation into the development of intelligent multimedia systems, such as Jerrams-Smith (1991), Bocker et al. (1990), Hendley et al. (1993), Edwards et al. (1995) and Perez et al. (1995). The author is currently engaged in a research programme developing a number of adaptive hypermedia systems in which a flexible and detailed user model is constructed and applied. These systems include: • HyperLearner (an adaptive multimedia authoring/tutoring system; see 4.1 below for details) • Telecare Companion (a prototype adaptive hypermedia system which is part of the British Telecom ‘Telecare’ programme; see 4.2 below for details) • Adaptive Hypermedia Memory Remediator (for use with persons with various memory disorders)
18
JENNIFER JERRAMS-SMITH
• Adaptive Navigation Guidance for the World Wide Web; see 4.3 below for details. These systems provide vehicles with which to identify the variables responsible for individual user differences and to adapt the system to accommodate user requirements in terms of attributes such as level of domain knowledge, personality, preferences, information processing styles, goals and tasks, roles within an organisation. Work currently in progress indicates that the following variables are important in enhancing the usability of the system. A variable which appears to be fundamental to adaptivity is Field Dependence-Independence (FD-I) (Witkin et al. 1972). FD-I refers to a fundamental individual difference in information processing. The field dependent user is easily influenced by information in his/her environment and tends to incorporate environmental information non-discriminately. In contrast, the field independent user tends to be influenced by internally generated cues and is more discriminating in the use of environmental information. Adaptive systems could take advantage of user differences in cognitive style to present information in a form consistent with the user’s field dependency. Such design features could have value for tutoring applications because fielddependent students’ cognitive style appears to be less suited to hypermedia instructional systems than does that of field-independent students because they are less able to create a framework with which to connect the information (Witkin et al. 1977). A further variable which may be of importance to adaptivity is that of Locus of control (Rotter 1966). Two types of user can be distinguished: internals and externals. Those with an internal locus of control regard outcomes as the result of their own efforts, whereas those with an external locus of control regard outcomes as the result of factors beyond their influence. While a system which removes control from the user may be compatible with an external user, its usability is reduced for internal locus users. Indeed, when internals are required to use restrictive systems they often ignore instructions and some ultimately cease to use the system. The adaptive systems currently in development will identify the user’s locus of control using a brief modified version of the I-E scale (Rotter 1966) and subsequently modify the interactive mode to accommodate the user’s locus of control. A variety of further user differences are being investigated as the basis of adaptivity. A more sophisticated form of adaptivity is also to be investigated: the identification of variables of particular relevance to the user and variables which are irrelevant to the user. In this way the system can focus on the factors which are important in the system’s usability and ignore those which are not.
PROVISION OF ON-LINE HELP
19
4.1. The HyperLearner project Combining hypermedia technology with adaptive tutoring provides a tutoring system which can adapt the sequencing of material and can also adapt the presentation of material to suit the potentially diverse needs and abilities of individual users. In addition, adaptive hypermedia tutoring systems are well suited to an exploratory approach to learning, and therefore encourage the active learning which has long been advocated (Piaget 1973). The author has recently developed HyperLearner, a prototype hypermedia authoring system which has been used to help tutors to build tutorials of course material. The first phase of the project investigated the issues involved in the development of a framework/authoring system for adaptive hypermedia tutoring systems. The long-term aim is the delivery of such a system, incorporating the results of the ongoing investigations. Current trends indicate that in the future, working, and learning, are likely to become increasingly homebased and the provision of systems which enable effective distance learning is therefore a topic of considerable importance. The HyperLearner prototypes learn about the student user and therefore adapt the interaction to suit the individual. The aim of the HyperLearner project is primarily to help the tutor, and thus to help the student to learn about a specific domain, but the prototypes have also been used to help students to learn about some of the fundamental issues connected with intelligent hypermedia tutoring systems (Jerrams-Smith 1995). 4.2. The Telecare Companion The Telecare Companion prototype (John et al. 1998) developed for British Telecom supports the provision of Telecare (telecommunications for disabled and elderly people) by applying adaptive features identified during the HyperLearner project. The Telecare system is designed to form a community network that will provide primary care to the elderly, housebound, disabled or otherwise disadvantaged living in the community. The Telecare Companion investigates the provision of adaptive support for users to access information stored in hypermedia networks, including the Internet, and communicate using a video-phone and e-mail. 4.3. Adaptive Navigation Guidance for the World Wide Web The Computer Aided Internet Navigation project (CAIN) (Lamas et al. 1996) addresses the problem of disorientation, cognitive overhead and information overload. It provides weak hypertext linearisation so that users follow an individual and ordered sequence of selected web pages. It combines techniques of user modelling with data modelling in order to select and order pages.
20
JENNIFER JERRAMS-SMITH
In conclusion, the systems currently under development are providing important insights into the design and production of adaptive hypermedia systems. While a fully adaptive hypermedia system has yet to be developed, the present research programme has already made important inroads into the design and construction of such a system. It should be stressed that the present research is ongoing and the author anticipates that more sophisticated systems will emerge. Note 1 Unix is a trademark of X/Open, Inc.
References Bailey, R. W. (1982). Human Performance Engineering: A Guide for System Designers. Bayman, P. & Mayer, R. E. (1983). A Diagnosis of Beginning Programmers’ Misconceptions of BASIC Programming Statements. Communications of the ACM 26(9): 677–679. Benyon, D. & Murray, D. (1993). Applying User Modelling to Human-Computer Interaction Design. Artificial Intelligence Review 7: 199–225. Bocker, H-D., Hohl, H. & Schwab, T. (1990). HYPADAPTER – Individualizing Hypertext. In Diaper, D. (ed.) Human-Computer Interaction – INTERACT ’90, 931–936. B.V. (North Holland). Brown, J. S. & Burton, P. R. (1978). A Paradigmatic Example of an Artificially Intelligent Instruction System. International Journal of Man-Machine Studies 10: 323–339. Burton, R. R. & Brown, S. (1979). An Investigation of Computer Coaching for Informal Learning Activities. International Journal of Man-Machine Studies 11: 5–24. Carey, T. (1983). User Differences in Interface Design. IEEE Computer 15: 125–129. Carroll, J. M. (1984). Minimalist Training. Datamation 30(18): 125 et seq. Clancey, W. J. (1986). Qualitative Student Models, 86–15. Stanford Knowledge Systems Laboratory: CA. Conklin, J. (1987). Hypertext: An Introduction and Survey. Computer 20(9): 17–41. Damerau, F. J. (1964). A Technique for Computer Detection and Correction of Spelling Errors. Communications of the Association for Computing Machinery 7(3): 254–258. Douglass, R. J. & Hegner, S. J. (1982). An Expert Consultant for the Unix Operating System: Bridging the Gap Between the User and Command Language Semantics. Proceedings of the 4th Conference of Canadian Society for Computational Studies of Intelligence, 119– 127. DuBoulay, B., O’Shea, T. & Monk, T. (1981). The Black Box Inside the Glass Box: Presenting Computing Concepts to Novices. International Journal of Man-Machine Studies 14(3): 237–249. Edwards, M., Powell, H. & Palmer-Brown, D. (1995). A Hypermedia-based Tutoring and Knowledge Engineering System. In Proceedings of Educational Multimedia and Hypermedia 1995. Carlottesville, VA: Association for the Advancement of Computing in Education (AACE). Ehrenreich, S. L. (1981). Query Languages: Design Recommendations Derived from Human Factors Literature. Human Factors 23: 241–247.
PROVISION OF ON-LINE HELP
21
Elsom-Cook, M. (1993). Student Modelling in Intelligent Tutoring Systems. Artificial Intelligence Review 7: 227–240. Finin, T. W. (1983). Providing Help and Advice in Task Oriented Systems. In Proceedings of The Eighth International Joint Conference on Artificial Intelligence, 176–178. Karlsruhe, FRG. Hanson, S. J., Kraut, R. E. & Farber, J. M. (1984). Interface Design and Multivariate Analysis of Unix Command Use. Association for Computing Machinery Transactions on Office Information Systems 2: 42–57. Hayes, P. J., Ball, J. E. & Ready, R. (1981). Breaking the Man-Machine Communication Barrier. Institute of Electrical and Electronic Engineers Computer 14: 19–30. Hayes, P. J. & Szekely, P. A. (1983) Graceful Interaction Through the COUSIN Interface. International Journal of Man-Machine Studies 19(3): 285–305. Hendley, R. J., Whittington, C. D. & Juraschek, N. (1993). Hypermedia Generation from Domain Representation. Computer Education 20(1): 127–132. Hollnagel, E. (1983) What We Do Not Know About Man-Machine Systems. International Journal of Man-Machine Studies 18(2): 135–143. Innocent, P. R. (1982). Towards Self-Adaptive Interface Systems. International Journal of Man-Machine Studies 16(3): 287–299. Irgon, A. E. & Martin, J. C. (1984). CLYDE: A Unix tutor. In Proceedings of the US–Japan Conference on Human Computer Interaction. Hawaii. James, E. B. (1981). The User Interface: How We May Compute. In Coombs, M. J. & Alty, J. L. (eds.) Computing Skills and the User Interface. Jerrams-Smith, J. (1989). An attempt to Incorporate Expertise About Users into an Intelligent Interface for Unix. International Journal of Man-Machine Studies 31: 269–292. Jerrams-Smith, J. (1991). Report on the PIE Project (Personalised Interaction for End-Users). Confidential Report, Philips Research Laboratories (UK). Jerrams-Smith, J. (1995). Combining Multimedia, Hypermedia and Artificial Intelligence to Support four Aspects of Learning. In Proceedings of Educational Multimedia and Hypermedia. Charlottesville, VA, USA: Association for the Advancement of Computing in Education. John, D., Jerrams-Smith, J., Heathcote, D. & Boucouvalas, A. (1998). The Telecare Companion – an Adaptive Interface for Telemedicine. Proceedings of the 1st International Symposium on Communication Systems. Sheffield Hallam University. Johnson, W. L. & Soloway, E. (1984). Intention-Based Diagnosis of Programming Errors. In Proceedings of The National Conference on Artificial Intelligence, 162–168. Austin, TX: AAAI Press. Kamouri, A. L., Kamouri, J. & Smith, K. H. (1986). Training by Exploration: Facilitating Procedural Knowledge Through Analogical Reasoning. International Journal of Man-Machine Studies 24: 171–192. Lang, T., Lang, K. & Auld, R. (1981). A Longitudinal-Study of Computer-User Behavior in a Batch Environment. International Journal of Man-Machine Studies 14(3): 251–268. Lamas, D. R., Jerrams-Smith, J. & Gouveia, F. R. (1996). Computer Aided Information Navigation: Project Description. Proceedings of Webnet ’96. Mishra, P., Trojan, B., Burke, R. & Douglass, S. A. (1984). A Quasi-Natural Language Interface for Unix. In Salvendy, G. (ed.) Human Computer Interaction. Nickerson, R. S. (1981). Why Interactive Computer Systems Are Sometimes Not Used by People Who Might Benefit from Them. International Journal of Man-Machine Studies 15: 469–483.
22
JENNIFER JERRAMS-SMITH
Norman, D. A. (1983). Design Rules Based on Analyses of Human Error. Communications of the Association for Computing Machinery 26: 254–258. Perez, T. A., Lopisteguy, P., Gutierrez, J. & Usandizaga, I. (1995). HyperTutor: From Hypermedia to Intelligent Adaptive Hypermedia. In Proceedings of Educational Multimedia and Hypermedia. Charlottesville, VA: Association for the Advancement of Computing in Education (AACE). Piaget, J. (1973). Memory and Intelligence. Rutledge and Kegan Paul: Cambridge, MA. Ramsay, H. R. & Attwood, M. E. (1979). SAI-79-111-DEN (NTIS:ADA 075 679), Human Factors in Computer Systems: A Review of the Literature. Science Applications, Inc.: Englewood. Relles, N. & Price, L. A. (1981). A User Interface for Online Assistance. In Proceedings of The Fifth International Conference on Software Engineering. San Diego, California: IEEE Computer Society. Rotter, J. B. (1966). Generalised Expectancies for Internal versus External Control of Reinforcement. Psychological Monographs 80(609). Scapin, D. L. (1981). Computer Commands in Restricted Natural Language: Some Aspects of Memory and Experience. Human Factors 23(3): 365–375. Self, J. A. (1974). Student Models in Computer Aided Instruction. International Journal of Man-Machine Studies 6: 261–176. Shneiderman, B. (1979). Human Factors Experiments in Designing Interactive Systems. Institute of Electrical and Electronics Engineers Computer 12: 9–19. Shortliffe, E. H. (1976). Computer-Based Medical Consultations: MYCIN. Elsevier: New York, NY. Sleeman, D. H. (1983). Intelligent Tutoring Systems and Student Modelling. Presented at AISB Conference on AI and Education. Exeter, UK. Sleeman, D. H. & Hendley, R. J. (1979). ACE: A System Which Analyses Complex Explanations. International Journal of Man-Machine Studies 11. Sleeman D. H. & Smith, M. J. (1981). Modelling Students’ Problem Solving. Artificial Intelligence 16: 171–188. Tagg, S. K. (1981). The User Interface of the Data-Analysis Package: Some Lines of Development. International Journal of Man-Machine Studies 14(3): 297–316. Waterworth, J. A. (1992). Multimedia Interaction with Computers. Ellis Horwood. Whalley, P. (1993). An Alternative Rhetoric for Hypertext. In McKnight, C., Dillon, A. & Richardson, J. (eds.) Hypertext: A Psychological Perspective. Ellis Horwood. Wilensky, R., Arens, Y. & Chin, D. (1984). Talking to Unix in English: an Overview of UC. Communications of the ACM 27(6): 574–593. Winston, P. H. & Horn, B. K. P. (1981). LISP. Addison-Wesley: Reading, MA. Witkin, H. A., Lewis, H. B., Hertzman, M., Machover, K., Meissner, P. B. & Wapner, S. (1972). Personality Through Perception: An Experimental and Clinical Study. Greenwood Press. Witkin, H. A., Moore, C. A., Goodenough, D. R. & Cox, P. W. (1977). Field-Dependent and Field-Independent Cognitive Styles and Their Educational Implications. Review of Educational Research 47: 1–64.
Virtues and Problems of an Active Help System for UNIX MARIA VIRVOU1 , JOHN JONES2 and MARK MILLINGTON3 1 Department of Computer Science, University of Piraeus, 80 Karaoli & Dimitriou St., Piraeus 18534, Greece. E-mail: [email protected]; 2 Department of Computer Science, Hull HU6 7RX, UK; 3 Interactive Business Systems, Inc., 2625 Butterfield Road, Oak Brook,
Abstract. An empirical study undertaken on a cross-section of UNIX users at an academic site reveals a role for an active form of help system, rather than the more usual passive kind. Sample scripts supporting this view are presented and the kind of aid required for these examples is discussed. It is then proposed that to provide such aid requires the construction and maintenance of an individual model of each user. Keywords: advice generation, intelligent help systems, user’s errors, user interfaces, user modelling, UNIX commands
1. Introduction The aim of this paper is to motivate, through examples and discussion, work on active aid systems for the UNIX file store manipulation domain. Most current help systems are passive, requiring a user to explicitly ask for help on a particular topic. Examples of this approach include the on-line UNIX manual and intelligent help systems such as UC (Wilensky et al. 1986). In contrast (in common with Breuker 1988; Jerrams-Smith 1985; Kemke 1986; Matthews et al. 2000) we will propose a role for an (automatic) active aid system to act as an “over the shoulder” adviser. Such a system is very difficult to construct and we discuss a number of the problems and issues that arise if this route is taken. The remainder of this section introduces a number of challenging examples used to illustrate points made in later sections. Section 2 discusses some advantages of an active help system, while Section 3 outlines the modelling problems such a system may face. In Section 4, we turn to the issues to be dealt with as a system prepares to interrupt a user. In Section 5, we summarize our approach in building a system meant to tackle some of the problems that have been highlighted in this paper, and we also discuss the results of the automatic aid, with respect to the examples introduced in the present section. In Section 6, we give an account of recent developments in this research.
24
MARIA VIRVOU ET AL.
1.1. Motivating examples This paper arises from an empirical study undertaken on a cross-section of UNIX users at an academic site. The particular version of UNIX that was used was Berkley 4.2 on a Vax; in fact, this version makes a lot of difference to many of the examples given below, for other implementations might behave rather differently. Volunteer subjects included faculty members, research staff and postgraduate students. Each subject was automatically monitored by the shell with which s/he was interacting; in an unobtrusive manner, as users went about their normal activities on the system, each command issued was recorded in a log-file associated with that user. However, due to the volume of feedback from UNIX, it was impossible to record the replies of UNIX to the commands issued. Thus, for example, error messages and the output from commands like ls were missing from the log and had to be reconstructed, as seen below. We reconstruct four partial sessions, two from each of two users, to illustrate many of the points we wish to make about the desirability and difficulties of an active help system (hereafter called an “aid system”). The summaries given with these reconstructions are intended to enable a reader to rapidly absorb the content of what happened. For the same reason, we have also made comments next to some commands. These comments start with a hash symbol. These summaries and comments did not arise from interviews with the subjects, and merely present a single interpretation of what occurred. Even so, it is quite remarkable what a wealth of information about each user lies in these examples, waiting to be extracted by an (intelligent!) aid system. The poplog example Anna has a sub-directory tf of her home directory which she wishes to move into a new sub-directory, to be called poplog. This can be achieved by mkdir poplog; mv tf poplog. However, instead of using mv, she tries first to copy tf into poplog (commands 2–5), then checks that the copy worked (commands 6–8), and finally removes the original (commands 9–17). Unfortunately, in this implementation of UNIX, command 5 will produce a data file which describes the directory entries for tf; a very similar effect can be gained by: % cat tf > poplog/tf/tf This form of cp is intended for use by experienced users engaged in system administration functions, who are a class of users very different from Anna.
VIRTUES AND PROBLEMS OF AN ACTIVE HELP SYSTEM FOR UNIX
25
Anna does try (command 7) to check that command 5 has created a directory, suggesting she has some inkling of the limitations of cp, but the results of this test are consequent only on (commands 1–3); the tf listed here is the tf created in (command 3), not the tf copied in (command 5). Alas, she is satisfied and goes on to remove the (still unique) original. 1 2 3 4 5
% % % % %
mkdir poplog cd poplog mkdir tf cd cp tf poplog/tf
6 % cd poplog 7 % ls -al total 3 drwxr-xr-x 3 anna drwxr-xr-x 3 anna drwxr-xr-x 2 anna 8 % cd 9 % rm tf
10 11 12
13 14 15 16 17
rm: tf is a directory % rm -i tf rm: tf is a directory % man rm % rmdir tf rmdir: tf: Directory not empty % cd tf % rm house1 % rm house2 % cd % rmdir tf
#Makes a new directory ‘tf’. #This command does not do anything #sensible when applied to directories. #Unfortunately there is no error message #to alert her. #Has a new directory been created? 512 May 16 18:44 . 512 May 16 18:44 .. 512 May 16 18:44 tf #Tries to remove the old ‘tf’; but this #command does not apply to directories. #Gets an error message this time. #Tries again without success. #Turns to the manual.
#‘house1’ is lost. #‘house2’ is lost too. #The old ‘tf’ eventually removed.
26
MARIA VIRVOU ET AL.
The ht example Graham wishes to collect all his files from his current working-directory into a new sub-directory called ht. To achieve this he creates the new directory (command 1), copies everything (including ht!) into ht (command 2), enters ht to delete the data-file copy of ht itself (commands 4–8), and then neatly removes everything (except ht) from the original directory (command 10). As in the poplog example, a directory has been copied as a data-file, but in this case, the user is seen to be well aware of the behaviour of cp and rm when applied to directories rather than files. 1 % mkdir ht 2 % cp * ht 3 % ls intro ht notes 4 % cd ht 5 % ls intro ht notes 6 % rm ht 7 % cd . 8 % cd .. 9 % ls intro ht notes 10 % rm * rm: ht is a directory 11 % ls ht
#Creates a directory ‘ht’. #The newly created ‘ht’ is also matched by ‘*’. #The current working directory contains two more #files apart from the new directory ‘ht’. #Gets into ‘ht’. #The directory ‘ht’ contains a copy of the two #files as well as a data-file copy of itself. #Removes the useless data-file copy of ‘ht’. #A typing mistake. #Rectification of the previous typo.
#‘rm’ removes the files ‘intro’ and ‘notes’ #which have been copied into ‘ht’. #‘rm’ does not apply to directories. #‘ht’ is not removed.
The popcode example Graham has two directories, ccode and popcode, which he believes to contain only unwanted files. He attempts to delete each directory by entering it, removing all of its files, leaving it, and then removing it. However, in popcode he discovers (we deduce from commands 6–9) that he has a sub-directory to deal with. We note how clear it is that failure (of command 6) was not intended in this example as it was in (command 10) of the ht example.
VIRTUES AND PROBLEMS OF AN ACTIVE HELP SYSTEM FOR UNIX
1 2 3 4 5
% cd ccode % rm * % cd .. % cd popcode % ls lib zip.p 6 % rm * rm: lib is a directory 7 % cd .. 8 % cd popcode
9 % cd lib 10 % ls arcs.p nodes.p 11 % rm * 12 % cd .. 13 % rm lib rm: lib is a directory 14 % rmdir lib 15 % cd .. 16 % ls ccode popcode 17 % rmdir ccode popcode
27
#Gets into directory ‘ccode’. #Removes all the files. #Gets into the parent directory. #Gets into directory ‘popcode’.
#Removes all the files. #This time there was a directory in there #which was not removed. #Gets into the parent directory. #Gets back to the ‘popcode’ directory, #probably because (command 7) was issued #very quickly after (command 6) and before #seeing the resulting message. #Has a look at the contents of the unremoved #directory. #Removes all the files in it. #Tries to remove the directory; but command #does not apply to directories. #The right command to remove the directory.
#Removes the two empty directories.
The perquish example Anna wishes to collect two existing files, fred and fred1, from her home directory, together in a new directory called perqish, renaming them to iread.pas and drconsult.pas, respectively, in the process. However, at command 3 she mis-types perqish as perquish, an error which lies latent until command 6, and uncorrected until command 19! A version of this example is tackled in a companion paper in this volume (Jones et al. 2000).
4 % cd perqish 5 % mv fred iread.pas 6 % mv fred1 drconsult.pas mv: fred1: No such file or directory 7 % ls iread.pas 8 % cd 9 % cp fred1 perqish
MARIA VIRVOU ET AL.
#Creates a new directory ‘perqish’. #Copies ‘fred’ into the new directory #‘perqish’. #Creates a new file ‘perquish’ as a copy of #the file ‘fred1’; or was ‘perquish’ meant to #be ‘perquish’ as at the previous command?
#She expects fred1 to be in the recently #created directory ‘perqish’. #Gets an error message showing that ‘fred1’ #is not actually there. #Confirms ‘fred1’ is not in the recently #created directory ‘perqish’. #Repeats command 2 without the typo this #time.
10 % cd perqish 11 % mv fred1 drconsult #Back to the command that made her #discover the error. But (command 11) is not #identical to (command 6); an oversight? 12 % cd 13 % rm fred #Another typo; ‘fred’ should read ‘fred1’. rm: fred: No such file or directory 14 % rf fred1 #A typo again! ‘rf’ should read ‘rm’ rf: Command not found 15 % rm fred1 #Here is the correct command. 16 % ls #The accidentally created file ‘perquish’ is perqish perquish #still there. 17 % rmdir perquish #Tries to remove it unsuccessfully; rmdir: perquish: Not a directory
VIRTUES AND PROBLEMS OF AN ACTIVE HELP SYSTEM FOR UNIX
18 % rm perqish rm: perqish is a directory 19 % rm perquish 20 % ls perqish 21 % cd perqish 22 % ls drconsult iread.pas 23 % mv drconsult drconsult.pas
29
#Another attempt fails because of another #typo! #She removes the unwanted file. #Confirms the removal.
#Command 11 was not meant to be different #from (command 6) after all.
2. Some Points in Favour of Active Help There are several questions concerning automatic aid systems that seek an answer in users’ actual command sequences. First of all, is an aid system needed at all? In what way could an aid system offer help? Are there any cases where an active help system could be more suitable to help the user than a passive one? Some answers to these questions come straight out of real-life examples like those four illustrated in the previous section. 2.1. Not realising help is needed There are many cases where users type other than what they mean, without any error message being produced to alert them. This does not mean that UNIX feedback is always to blame; in fact, it is not simple to spot those errors that UNIX would not have complained about without having reasoned about the user’s actions and beliefs. Rather, this kind of reasoning is beyond the scope of UNIX or any other existing operating system, and hence, a help system would be needed instead. However, even a passive help system would not have been of much use to a user who had not been aware of a problematic situation, because this user would not have turned to it; how could one expect users to ask for help when they do not even know that they need it? This is when an aid system would be useful, since an aid system is constantly supposed to watch and reason about the user’s actions and make its own decisions, whether or not the user needs help. This can be one of the most important advantages of an aid system.
30
MARIA VIRVOU ET AL.
The sample scripts clearly illustrate this situation, where users make errors without being aware of them. This was clearly the case in the poplog example, as well as the perquish example. The consequences of such errors vary, depending on how soon the user realises there has been an error, and how difficult it is to recover from it, if this is at all possible. In the remainder of this subsection, we are going to discuss the consequences of this kind of error, in the way they occurred in the poplog and the perquish examples. We are also going to point out the fact that inefficient usage of commands may well fall into the same category of problem, where users may not be aware that they possibly need help. The popcode example will be used to illustrate this point. Catastrophic error in the poplog example A simple copy and remove plan in the poplog example ended up in the single catastrophic action remove since the copy part of the plan failed without the user having realised it. More specifically, Anna’s misconception about the way directories are copied to other directories leads her to the erroneous command 5. The lack of an error message following this command results in command 9, where she attempts to remove a directory, without it having been copied first, contrary to what she believes. Initially, she only intended to move the directory elsewhere in the file store. This is a very tricky case indeed, where the user continues issuing commands as though a previous goal has already been accomplished. She did not seek help (in the manual) before typing command 5, which was the cause of the trouble. Perhaps she thought she knew how to handle this. After which, she did not even realize her mistake, and therefore, would not have asked for help even if this had been available. This is a typical situation in which a spontaneous intervention of an aid system can save trouble. In this case, it could save the user from a catastrophic error. Considerable effort wasted in the perquish example The name of a directory called perqish is mistyped as perquish in command 3. As a result, a new file named after the mistyped name is created. At this point the user is not aware of her error. However, after command 6, she realises that there is a problem, unlike the previous example where she never even realised that something went wrong. Still, the recovery from this error proves quite expensive in terms of the number of commands. As a matter of fact, she issued 23 commands to complete her initial goal, with 10 of them (commands 7, 8, 9, 10, 11, 16, 17, 18, 19 and 23) typed in the context of recovering from the typo and consequences of command 3. In this case, almost half of the effort was wasted due to a small typing error.
VIRTUES AND PROBLEMS OF AN ACTIVE HELP SYSTEM FOR UNIX
31
Evidently, it would not have been so much if she had not made other errors at commands 11, 17 and 18. However, it may have been worse. While trying to recover from her mistake, she gets involved in new ones and this goes on for some time. This is a case where the lack of an error message from UNIX is absolutely justified, simply because the error occurs in the name of a directory or file. When new files are created, users are free to name files and directories as they please. Hence, UNIX cannot possibly complain when the user mistypes a name in a command to create a new file or directory. This kind of error could only be spotted if a system kept track of the user’s intentions and beliefs, which means that an explicit user model would be necessary. An aid system, which is supposed to provide this kind of reasoning could help the user recover quickly from an error like this. Another interesting point about this example is that this sort of typing mistake could have been made by anyone, novice or expert, suggesting that all types of users could benefit somehow from an aid system. Optimisation in the poplog and ht example Graham’s goal in the popcode example could have been obtained by only 2 commands (ls -R; rm -r ccode popcode), instead of the 17 he issued! Similarly, the effect of the first 10 commands he issued in the ht example could have been obtained by using only 2 commands (mkdir ht; mv * ht). Would he be interested in seeing these very short command sequences, so that he could use them at some later time? This is a case where the user is probably not aware of some system commands that would achieve a goal via a simpler plan than the one formed in the user’s mind. Actually, many users probably feel safer using commands they are already familiar with, and do not bother to find out more about the operating system that they are using. Instead, they try to fit their needs to their existing knowledge. An aid system could suggest an optimisation of the user’s very own actions. In this aspect, the aid system could act as a tutoring system. Again, the individuality of the user would play a significant role in deciding what help the aid system could provide. 2.2. Not knowing how to ask for help Users may not know how to ask for help. It seems that sometimes the existing manual help is not sufficient for a user who turns to it for aid. The wealth of information may confuse the non-expert, who might pick up the wrong command for their case. There are times when users need advice for their individual case, and instead, all they can get is general information. Quite
32
MARIA VIRVOU ET AL.
often, they do not even know exactly what their problem is, and therefore, they cannot form the right questions. Would they not be interested in getting an expert’s advice, tailored to the individual circumstances from which their problem occurred? In the following two subsections we first illustrate this case where the manual proves inadequate, and then we describe the case where the user is looking for the wrong kind of information. Both cases occur in the poplog example. Failure following the man command in the poplog example Anna finds out about rmdir from the rm manual entry at command 11. However, command 12, coming right after the “man rm” command fails to remove her directory as Anna then wanted. In fact, she probably wanted rm -r, but did not manage to find it in the manual. This is only an example of a case where the user finds it difficult to retrieve information from the manual. Anna was quite lucky to encounter only one failure before she used a plan which worked. There are even worse cases where some users do not even get to find the right keyword for the command that they seek. Getting help on the wrong issue in the poplog example Anna believes that she has to find out how to remove the directory tf at command 11 of the poplog example. But in fact, she would not want to remove it if she knew that it had not actually been copied first. In this case, the help she got from the manual did her no good. Ironically enough, no efficient reference facility would do any better if it did not have a user model. What Anna actually needed was a help system to tell her what was really going on, instead of just giving a reply to her question. 2.3. Trying vs. looking up Although not always recommended, users may find it quicker to actually try commands out rather than looking them up. For example, Anna probably had some idea of the existence of the option -r in the poplog example. After the failure of command 9, because tf was a directory, she decided to take another guess before she issued the man command. However, she could not remember the exact name of the option she wanted, and in this case, typed the completely irrelevant option -i. Perhaps this kind of user would be happier and safer if an aid system could guarantee interruptions at dangerous junctures; safety encourages experiment and experiment encourages learning. Bad typists, especially, would probably be happier to be interrupted, even if a facility for requesting help in (written) English were provided, because the latter would take some time to type.
VIRTUES AND PROBLEMS OF AN ACTIVE HELP SYSTEM FOR UNIX
33
2.4. More human-like interaction Frustration can result for some users when the operating system is not able to spot even the most obvious (to a human) typing error. They somehow expect the computer to have some features of humans, and may get upset when commands fail because of some small typing error. For example, a human expert would easily understand what Anna meant by command 14 in the perquish example. The whole context of the command sequence, and the similarity of the wrong command to the correct one, give sufficient clues to a human. However, intelligence is required for such an error to be spotted by the computer, especially when the typo can change the meaning of a command. This was the case in command 2 of the perquish example. A monitoring help system could make the interaction more humanlike by recognizing these errors and interrupting discreetly like a human listener.
3. Modelling Requirements In this section we address the issue of how accurately an aid system must model the user it serves. Firstly, we consider how clearly it must reason about what actually happened, using a general understanding of the interactive nature of command-driven systems like UNIX. Secondly, it is shown that in order to reason about what should have happened the aid system must maintain a model of each individual user. 3.1. What happened? For an automatic system to provide acceptable on-line aid it must show a good understanding of the user-UNIX interaction. Whatever their differences, all users interact with a shell in basically the same way; the user issues a command and UNIX may issue a reply. This notion of interaction is important; it is not enough for an aid system to ignore the flow of information to a user and concentrate merely on the file store transformation. For example, we can strip all occurrences of ls from a command sequence leaving the effect (on the machine) unchanged, but this would mean that the aid system would not model the information that the user has seen. This might lead the aid system to interventions proposing rather suspect improvements. In the popcode example, the user learned that there was a sub-directory of popcode, only after attempting to remove it. After investigation it was eventually removed, but to suggest an improved sequence “rm -r popcode” with the same effect does not capture the true sequence of events.
34
MARIA VIRVOU ET AL.
Similarly, a user’s current working-directory is an important object of manipulation, facilitating the simplified expression of (currently) non-local pathnames. If an aid system sees a command sequence cd a; cd b; cd c, should it propose an “equivalent” sequence cd a/b/c as being better? If it works it definitely is better, but the problem arises if the user is imperfect in some way (e.g. a bad typist, or forgetful about his file store environment), and then an error in the former is easier to recover from; if it does not work you will know why and be in the right place to do something about it. All users distinguish these sequences, and therefore, so must an aid system. Thus, realistic aid systems will have to employ a rich model of the interaction between a user and UNIX. To do this, it must almost certainly make some generic assumptions about users; for example, users are (reasonably) attentive, consistent, sometimes forgetful, fat-fingered, cautious, etc. Modelling UNIX is the least of our problems. 3.2. What was intended? Through an understanding of what actually happened we may hope to reason about what was intended, even though the two may be very different. If the two are significantly different, the user ought to be informed. One approach to this might be to wait for an error message to arise, and then attempt a diagnosis for the user. However, as we will see, this cannot work; some users make productive use of failure conditions, and some errors do not provoke error messages. Providing effective aid must involve modelling individual characteristics of each user, for it is easily seen that not doing so, quickly leads to confusion. What aspects of a user must we model? • We at least have to model the user’s plan; suggesting “rm -r *” as an improvement for command 10 of the ht example just to eradicate the error message would be ridiculous: the user is relying on the failure and does not want the extra effect of this proposal. • The user’s overall plan should have priority over a user’s subplan; Anna would not thank us for helping her remove tf at command 9 of the poplog example, although this is obviously what she wants there; the overall plan is very different and must be taken into account. • The individual user’s command knowledge is important too; interrupting Graham at command 13 of the popcode example would be very different from interrupting Anna at command 9 of the poplog example (in the hypothetical case where the overall plan would not conflict with her wanting to remove tf). They both attempt to remove a directory by employing the command rm without any flag. However, Graham has
VIRTUES AND PROBLEMS OF AN ACTIVE HELP SYSTEM FOR UNIX
35
generally shown a better knowledge and understanding of command rm than Anna has in the previous examples. Therefore, Graham’s mistake may have been an accidental slip, whereas Anna’s mistake may have been due to a deeper confusion of commands. An intervention at command 9 of the poplog example (or, perhaps, command 10) might be acceptable: she plainly wants to get rid of tf but does not know how. • Confusion may also arise over the environment in which a command is executed. At command 6 of the popcode example, “rm lib” fails because lib is a directory. Is this because he does not understand rm removes only files, or because he forgot lib was a directory? Knowing the user we would conclude the latter; in fact, later on in the ht example at command 10 the same user is making productive use of the same failure condition. However, if he had employed ls -F as command 5, we might be less sure. • General and individual users’ usual imperfections have to be modelled too; What would the user’s intended typing, lexical, or syntactic structure be, if not that which was actually input? Answers to such questions should be somewhere in the model of users’ usual errors. For example, the fact that Anna had mistyped perqish to perquish could have been suspected right away, only because the typing, lexical and syntactic structure of the command containing it was very similar to the previous one, and people tend to make this kind of typing error. However, as we have seen, different aspects are very heavily interrelated, and an existing model provides additional strong constraint on the sensible interpretations.
4. Interventions Given that there is a useful point that can be made to the user, the question remains as to how, when, and on what basis to intervene. How an intervention is made ought to depend on whether it proposes an optimisation or a correction; attention to a correction should be mandatory whereas offers of optimisation may be ignored. In this way corrections may be given as interruptions, like error messages, comprising a piece of English text to describe the problem and a proposed correction. Optimisations should not interrupt, but might instead alter the prompt to, say, a question-mark. If the user immediately hits return then the optimisation is given; otherwise, the user is free to issue a normal command. There is little point in proposing a correction or optimisation that is beyond the current understanding of the user; if too many new ideas are introduced, it will confuse the user and be rejected. Here again, is a role for an individual user-model: to tailor advice to suit the user.
36
MARIA VIRVOU ET AL.
When to intervene, again, depends on the nature of the proposal. Optimisations can be given as soon as they are discovered, since they rely only on what actually occurred. Corrections deal with what was intended, when it differs from what occurred; in such cases there seem to be three points at which the aid system might intervene: − When a correction first appears to be warranted; for example, command 3 in the perquish example. This may be too soon if there is not enough evidence for suspecting the command, since it seems likely that almost any command might be intended as something else. However, if there has been sufficient evidence that a problem may have occurred, then it would probably be better for the system to intervene sooner than later, to prevent further complications of this problem. − When only a correction fits all the facts; i.e., we have no user-model in which the user is not under a misconception. For example, we might expect command 6 in the perquish example to be of this nature. − When the user can no longer recover from the error if the current command is executed. There is no such point in the perquish example; in fact, she does make the recovery herself, but such a point occurs in the poplog example at command 14. The basis for each intervention can, of course, be derived only from the events preceding the intervention, and it is not envisaged that we can hope to construct a dialogue between the aid system and the user; actions speak louder than words. However, the user model may also comprise a large proportion of the basis; Graham, copying a directory, is a very different matter from Anna, doing the same. Some interventions produced by a prototype active help system for the sample scripts are presented in the following section.
5. Summary An active help system could be of much use to many users, and not only novices. This is the conclusion drawn from the analysis of the sample scripts. It could save the considerable amount of time and energy wasted in recoveries from errors. It could also act as a personal tutor, allowing the user to escape needless inefficiencies. The individuality of the user is of vital importance to the help an aid system should provide. This makes the error diagnosis more complex since both general user models and individual ones should be taken into account for a clear interpretation of the interaction leading to a good plan recognition. The latter is essential for a reasonably reliable error diagnosis. The user
VIRTUES AND PROBLEMS OF AN ACTIVE HELP SYSTEM FOR UNIX
37
modelling part should involve assumptions that the user could be imperfect in several aspects, such as having misconceptions about commands, making typing errors etc. Interventions should be kept to a minimum. Only when it is very obvious that the user needs help, should this be provided. The only aim is to help the user, and make him/her more efficient. By no means should the opposite happen in any sense.
6. Recent Developments In this section we describe briefly a prototype active help system called RESCUER (which stands for Reasoning System about Commands of UNIX using Evidence Reasonably) (Virvou 1992; 1998). RESCUER has been constructed to address users’ problems like those described in earlier sections. It is not within the scope of this paper to detail the modelling and reasoning issues of RESCUER. However, we will demonstrate the interventions produced by RESCUER when it was given, as input, the sample scripts presented in this paper. A companion paper in this volume (Jones et al. 2000) describes a prototype system addressing some of the modelling issues raised here, but none of the intervention issues. 6.1. RESCUER’s overall approach RESCUER is a prototype active help system that monitors users interacting with UNIX and offers spontaneous help when it judges that users need it. RESCUER’s design addresses three main problems that an active help system has to solve: 1. Recognition of a problematic situation. 2. Diagnosis of the cause of the problem, if there has been one. 3. Decision regarding the kind of advice and generation of response. For the recognition of a problematic situation, RESCUER uses certain peculiarity criteria in order to evaluate every command typed. These criteria include the questions whether the command was acceptable to UNIX or not, whether it was typical of its class and so on. RESCUER uses these criteria as unconfirmed symptoms of a possible problem in a command type. The existence of such symptoms results in the further examination of the command typed. As a result of this evaluation, RESCUER attaches one of three labels to the command typed, namely “expected”, “neutral”, “suspect”. The label
38
MARIA VIRVOU ET AL.
“suspect” means that RESCUER will have to generate alternative interpretations in its attempt to find one that fits the context better. The label “expected” means that the command’s effects to the file store imply some continuation of a plan from previous commands; therefore RESCUER judges that the command fits the context very well. Finally, “neutral” means that RESCUER does not get alerted by this command because it considers it unharmful (e.g. “ls”), although it is not directly involved with the effects of previous commands. The diagnosis of a problem is done by the user modelling component of RESCUER. If the user modeller can find an alternative interpretation about the user’s command, such that it is similar to what was typed, but is better than that in terms of the evaluation criteria, then RESCUER may have to adopt this alternative interpretation. The process of generating an alternative interpretation contains the information needed for the construction of an explanation of what happened. The generation of hypotheses about possible user’s misconceptions is based on a cognitive theory, called Human Plausible Reasoning theory (Collins and Michalski 1989), which was originally constructed to formalise the reasoning that people use in order to make plausible guesses about questions for which they do not have full information. Here we exploit the fact that plausible guesses can be turned to plausible human errors. Plan recognition is achieved by considering the UNIX file store as an entity that has certain properties, one of which is called “instability”. The existence of instabilities imply a possible future transition of the current state of the file store to another state, which is expected to follow at some point (e.g. from a file store state that contains an empty directory to the file store state that this directory is filled with some content). A file store is considered to be absolutely stable if it does not contain: 1. empty directories 2. directories with only one child 3. duplicate files The reason that empty directories are considered to be adding some instability to the file store is that empty directories do not seem to have any purpose if they have no contents. One would expect them to either have contents or be removed. Similarly, if a directory has only one child, it is considered to be borderline as to how useful it is to keep a directory with only one file. One would expect it to either have more contents, or the content file to be moved one level up and the directory to be removed. Finally, duplicate files also imply some transition from a file store state that contains identical duplicates to a file store state where the original copies will have been removed, or the replicated files will have been changed.
VIRTUES AND PROBLEMS OF AN ACTIVE HELP SYSTEM FOR UNIX
39
Instabilities are only used to indicate that a user may have started some plan, but do not restrict the user’s actions at all. An unstable file store does not mean that the user has made an error, but that perhaps, s/he is in a middle of a plan. A “stable file store” is considered to be every user’s final goal, denoting the completion of some plan. In this way, RESCUER manages to evaluate users’ typed actions in terms of their effects to the file store. RESCUER’s expectations, in terms of effects to the file store, are used to guide the search for possible corrections of commands. A command is “expected”, in terms of instabilities, if it adds and deletes instabilities at the same time (which means that there is some kind of continuity in the user’s actions), or if it deletes instabilities without adding any new ones. The generation of a response, if there is one, is done on the principle of preventing users from possible catastrophic errors, giving explanations that address the user’s misconceptions and suggesting alternative correct plans. However, RESCUER mainly concentrates on errors that would be “plausible” for a human observer to identify. 6.2. RESCUER’s interventions for the sample scripts In this section we demonstrate RESCUER’s responses when it was given, as input, the command sequences which were recorded in the sample scripts. The poplog example 1. % mkdir poplog 2. % cd poplog 3. % mkdir tf 4. % cd 5. % cp tf poplog|tf RESCUER: Did you mean to type: % cp -r tf poplog/tf or % mv tf poplog/tf? RESCUER has considered the fifth command as “suspect” because it is not typical of “cp” command with respect to its purpose. Since the command has been evaluated as “suspect”, RESCUER generates the two alternatives which are similar to the command typed: 1. % cp -r tf poplog/tf 2. % mv tf poplog/tf
40
MARIA VIRVOU ET AL.
They are also found to be “expected”, because they assign meaningful content to the empty directory “poplog/tf”, and are considered better than the original command because they are more typical of their class. The ht example One major point raised from this example was that the user made a constructive use of the UNIX failure to remove directories by the command “rm *” (command 10). RESCUER was smart enough to understand that command 10 was useful to the user. This command was used to remove all the original files that the user had previously copied to the directory “ht”. Command 10 had produced an error message because “*” did not apply to directories, and therefore, “ht” itself was not removed, which was precisely what the user wanted. Although command 10 was considered by RESCUER as “suspect” because it produced an error message, it was also assigned the label “expected” because of the instabilities that it removed from the file store. Therefore, RESCUER did not intervene in this case. The perquish example RESCUER intervenes at the third command as follows: 1. % mkdir perqish 2. % cp fred perqish 3. % cp fred1 perquish RESCUER: You may have made a typing mistake. Did you mean to type: ‘‘cp fred1 perqish’’ instead of ‘‘cp fred1 perquish’’? Y/N. The reasons why RESCUER has generated this response are the following: First RESCUER considers command 3 as “suspect” because it introduces new instabilities without deleting any of the previous instabilities. At this point, RESCUER starts generating similar alternatives to the command typed. In this case, it generates four alternatives: 1. 2. 3. 4.
% % % %
cp cp cp mv
fred perquish fred1 perqish -r fred1 perquish fred1 perquish
The first two are generated by changing the arguments of the action typed to other arguments (one at a time), which in this case are lexically similar.
VIRTUES AND PROBLEMS OF AN ACTIVE HELP SYSTEM FOR UNIX
41
The last two are generated by changing the command name and leaving the arguments as they are. The most similar commands to “cp” which would work for the preconditions of the semantics of this action have been found to be the commands “cp -r” and “mv”. Next, RESCUER evaluates the hypotheses in the same way as it evaluated the action typed. Two of them are found “suspect”, one of them is found “neutral”, and one is found “expected”. 1. % cp fred perquish is found “suspect” because it has a similar effect to the file store as the command originally issued. 2. % cp fred1 perqish is found “expected” because it deletes the instability of “perqish” having only one child as a content. 3. % cp -r fred1 perquish is found “suspect” because it has exactly the same effect on the file store as the command originally issued. 4. % mv fred1 perquish is found “neutral” because it neither deletes nor adds any instability (therefore suggests that it is a plan of its own). RESCUER picks the “expected” command, % cp fred1 perqish, to show the user, because at this point, it has “enough” evidence that the user may have made a typing error. There are two reasons completely independent from each other that suggest that the user may have made an error: 1. There is a command other than the command typed that is very similar to the command typed, according to the knowledge representation of RESCUER. 2. This other command is different from the command typed in the effects that it has in the file store. The “other command” is actually better than the command typed with respect to their difference. This example demonstrates the successful automatic generation of a plausible correction. RESCUER favours an earlier intervention, rather than a later one. Therefore, it suggests a correction if there is more than one reason for believing that there may have been a mistake. Naturally, RESCUER may be unnecessarily invoked (which is not the case in this example) but even so, the user can always ignore RESCUER’s suggestion.
Acknowledgements We thank Peter Norvig, Wolfgang Wahlster and Robert Wilensky for having made several helpful comments on an earlier version of this paper. We are also grateful to Stephen Hegner and Paul Mc Kevitt for their helpful comments on the current version.
42
MARIA VIRVOU ET AL.
References Breuker, J. (1988). Coaching in Help Systems. In Self, J. (ed.) Artificial Intelligence and Human Learning, 310–337. London, UK: Chapman and Hall. Collins, A. & Michalski R. (1989). The Logic of Plausible Reasoning: A Core Theory. Cognitive Science 13: 1–49. Jerrams-Smith, J. (1985). SUSI-A Smart User Interface. In Johnson, P. & Cook, S. (ed.) Proceedings of the Conference of the British Computer Society Human Computer Interaction Specialist Group. Cambridge University Press. Jones, J. G., Millington, M. & Virvou, M. (2000). An Assumption-based Truth Maintenance System in Active Aid for UNIX Users. In Hegner, S., Mc Kevitt, P., Norvig, P. & Wilensky, R. (eds.) Intelligent Help Systems for UNIX. Dordrecht, The Netherlands: Kluwer Academic Publishers (this volume). Kemke, C. (1986). The SINIX Consultant: Requirements, Design, and Implementation of an Intelligent Help System for a UNIX Derivative, Bericht Nr. 11, FR. 10.2 Informatik IV, University of Saarland, Saarbrucken, FRG. Matthews, M., Pharr, W., Biswas, G. & Neelakandan, H. (2000). USCSH: An Active Intelligent Assistance System. In Hegner, S., Mc Kevitt, P., Norvig, P. & Wilensky, R. (eds.) Intelligent Help Systems for UNIX. Dordrecht, The Netherlands: Kluwer Academic Publishers (this volume). Virvou, M. (1992). User Modelling Using a Human Plausible Reasoning Theory. Ph.D. thesis, CSRP 251, School of Cognitive and Computing Sciences, University of Sussex, Brighton BN19QH, UK. Virvou, M. (1998). RESCUER: Intelligent Help for Plausible User Errors. In Proceedings of ED-MEDIA/ED-TELECOM 98, World Conferences on Educational Multimedia and Educational Telecommunications, Vol. 2, pp. 1413–1420. Wilensky, R., Mayfield, J., Albert, A., Cox, C., Luria, M., Martin, J. & Wu, D. (1986). UC – A Progress Report, Report no. UCB/CSD 87/303, Computer Science Division (EECS), University of California at Berkeley, California 94720, USA.
The Berkeley UNIX Consultant Project ∗ ROBERT WILENSKY1, DAVID N. CHIN2, MARC LURIA3 , JAMES MARTIN4, JAMES MAYFIELD5 and DEKAI WU6 1 Division of Computer Science, Department of EECS, University of California, Berkeley, Berkeley, CA 94720, U.S.A. (E-mail: [email protected]); 2 Department of
Information and Computer Sciences, University of Hawaii, 1680 East West Rd., Honolulu, HI 96822, U.S.A. (E-mail: [email protected]); 3 NDS Technologies Israel Ltd., Jerusalem, Israel (E-mail: [email protected]); 4 Department of Computer Science, University of Colorado, Boulder, CO 80309-0430, U.S.A. (E-mail: [email protected]); 5 Computer Science and Electrical Engineering Dept., University of Maryland Baltimore County, Baltimore, MD 21228-5398, U.S.A. (E-mail: [email protected]); 6 Department of Computer Science, The Hong Kong University of Science and Technology, Clear Water Bay, Kowloon, Hong Kong, P.R. China (E-mail: [email protected])
Abstract. UC (UNIX Consultant) is an intelligent, natural-language interface that allows naive users to learn about the UNIX operating system. UC was undertaken because the task was thought to be both a fertile domain for Artificial Intelligence research and a useful application of AI work in planning, reasoning, natural language processing, and knowledge representation. The current implementation of UC comprises the following components: A language analyzer, called ALANA, that produces a representation of the content contained in an utterance; an inference component called a concretion mechanism that further refines this content; a goal analyzer, PAGAN, that hypothesizes the plans and goals under which the user is operating; an agent, called UCE go, that decides on UC’s goals and proposes plans for them; a domain planner, called KIP, that computes a plan to address the user’s request; an expression mechanism, UCE xpress, that determines the content to be communicated to the user, and a language production mechanism, UCGen, that expresses UC’s response in English. UC also contains a component called KNOME that builds a model of the user’s knowledge state with respect to UNIX. Another mechanism, UCT eacher, allows a user to add knowledge of both English vocabulary and facts about UNIX to UC’s knowledge base. This is done by interacting with the user in natural language. All these aspects of UC make use of knowledge represented in a knowledge representation system called KODIAK. KODIAK is a relation-oriented system that is intended to have wide representational range and a clear semantics, while maintaining a cognitive appeal. All of UC’s knowledge, ranging from its most general concepts to the content of a particular utterance, is represented in KODIAK. Keywords: agent, UNIX, consultant, natural language, intelligent interface, planning, knowledge representation, user modeling
1. Introduction to the UNIX Consultant (UC) Project Some time ago, we began a project called UC (UNIX Consultant). UC was to function as an intelligent, natural-language interface that would allow naive users to learn about the UNIX1 operating system by interacting with the consultant in ordinary English. We sometimes refer to UC as “an intelligent ‘help’ facility” to emphasize our intention to construct a consultation system, rather than a natural-language front-end to an operating system. Whereas front-ends generally take the place of other interfaces, UC was intended to help the user learn how to use an existing one. We had two major motivations for choosing this task. These can be summarized by saying that we believed the task to be both interesting and doable. It seemed to us that much natural-language work – indeed, much of AI research – has fallen into two largely non-intersecting categories: On the one hand, there are quite interesting and ambitious projects that have been more the fertile source of exciting speculations than of useful technology. In contrast, there are projects whose scope is severely limited, either to some intrinsically bounded, real-world task or to a laboratory micro-world. These projects result in much excitement by the production of a “working system” or successful technology. But such projects have rarely produced much in the way of progress on fundamental issues that comprise the central goals of AI researchers. Our hope was that the consultation task would require us to address fundamental problems in natural-language processing, planning and problem solving, and knowledge representation, all of which are of interest to us. We believe this to be the case because (1) the domain of an operating system is quite large and complex, (2) users’ conceptions of computer systems are often based on other domains, particularly space and containment, and (3) the structure of a consultation session requires the consultant to understand the user’s language, hypothesize the user’s intentions, reason about the user’s problem, access knowledge about the topic in question, and formulate a reasonable response. In sum, virtually all the problems of language processing and reasoning arise in some fashion. While the task is interesting, it is nevertheless limited. Arbitrary knowledge of the world is generally not required, as it may be in other naturallanguage tasks, such as text processing. Even knowledge about the domain might be limited in ways that do not compromise the overall integrity of the system. In particular, the task is intrinsically “fail-soft.” Since the system is a ‘help’ facility, it need not be capable of handling every task put to it to serve a useful function. This is probably less true of systems that are intended to be interfaces. In their case, failure to correctly process a request by the user
THE BERKELEY UNIX CONSULTANT PROJECT
45
leaves the user with little recourse. However, a consultant may be quite useful even if it cannot help all the time. Similarly, there are strategies that might be employed in a consultant task that further reduce the degree of coverage required by the system. For example, if asked a very specific question, it is not unreasonable that a consultant respond by telling the user where to look for the information. Thus, the degree of expertise of the consultation system may be circumscribed. In other words, we felt that the operating-system domain was an appropriate replacement for the “blocks world.” Building a consultant for the domain is a real task one would like to have accomplished. The domain would limit the breadth, but not the depth, of AI research required. 1.1.
UC
– science or engineering?
Our approach to AI has had a distinctly “cognitive” bent. While a lengthy exposition might be needed to define this precisely, let it suffice here to say that we are interested in modeling human beings at least to a first approximation. Thus, as far as we could, we have attempted to build a system that modeled how we believe a human consultant actually functions. In some cases, this goal meant that we would make some problems harder for ourselves than one might if one’s goals were strictly technological. For example, since many word senses are unlikely to be used when talking to a consultant, a purely engineering approach might play down the problem of ambiguity. However, it is our goal to address such problems in a general fashion. At the same time, there were many pragmatic concessions that were made in implementing UC. Some of these were forced on us by the nature of university research. For example, a process might be divided into two components for the sake of implementation, although the particular division may not be motivated otherwise. These components might even exercise two different approaches to similar subproblems, depending on the biases of their authors. Sometimes, for the sake of efficiency, we chose to implement only part of what we believed to be a larger process. Also for efficiency’s sake, and to prevent truly difficult but infrequent problems from scuttling the entire effort, we implemented some solutions that we did not believe in completely. For example, UC’s control structure is overly simplistic in ways that we understand but have not corrected. We will make note of other such situations in the text below. In general, when this was the case, the solution used took the form of checking for certain frequently occurring cases in order to preclude having to solve a general problem. Since our goals were not strictly technological, we did not feel that it was necessary or appropriate in order for our system to be considered a
46
ROBERT WILENSKY ET AL.
success to produce a product that could actually be used in a real-world setting. However, we did feel that we should show that one could develop such a system along the lines that our research suggested. This would be accomplished by developing an extendible prototype. 1.2. Reasonable agents versus intelligent interfaces Our goal in building UC is to simulate a human consultant. As a result, the system has a structure that is more complex than other so-called intelligent interfaces. Indeed, we feel that looking at such a system as an interface is misleading. Instead, we prefer the metaphor of a reasonable agent. Unlike an interface, which is a conduit through which information flows, an agent is a participant in a situation. In particular, an agent has explicit goals of its own, and a reasonable agent must be able to make obvious inferences and display judgment in making decisions. Typically, a consultant constructed along the lines of a reasonable agent will make a user’s goals its own in trying to help that user. However, a reasonable agent is not always compelled to do so. Human consultants will not obligingly give out information to which a user is not entitled or which they suspect will be put to ill use. Similarly, a good consultant might deflect a user’s request because the consultant feels that the user does not have an adequate grasp of the domain, has a particular misconception, or is lacking some particular fact. In addition, a good consultant might do something more than simply answer a question. He might take the opportunity to show the user how to do a more general task of which the user’s particular request is merely a special case. In all these situations, an action other than simply responding to a request is warranted. A reasonable agent is ideally suited to handle such a broad class of situations. It does so by deciding what its goals should be in the given situation, and then planning for them. For example, when UC is asked how to crash the system, it forms two goals, one of helping the user to know what he or she wants, and one of protecting the integrity of the system. It then realizes that these two goals are in conflict, and eventually decides the conflict in favor of the latter goal. Of course, it is possible to achieve by other means various parts of the functionality here attributed to the model of a reasonable agent. For example, one can simply build one component that tries to detect misconceptions, another that checks for requests having to do with crashing the system, yet another to capitalize on opportunities to educate the user, etc. However, the reasonable-agent framework provides a single, flexible control structure in which to accomplish all these task, and, in particular, deal with interactions between them. That is its engineering motivation. Our primary reason for
THE BERKELEY UNIX CONSULTANT PROJECT
47
adopting it is that it is our theory about how humans function in consulting situations. 1.3. Overview The structure of this report is as follows. First, we present an outline of the structure of the current version of our consultation system. We follow this with a brief description of KODIAK. The next sections constitute the bulk of this report and are essentially a detailed description of a trace of a rather simple sentence through UC’s components. In doing so, the mechanisms of those components that are primarily responsible for UC’s agent-like qualities are described. Finally, we conclude with some discussion of the deficiencies of our current design. 1.3.1. Outline of UC’s structure UC is comprised of a number of components, which are invoked in a more or less serial fashion. (1) Language Analysis (ALANA) Language analysis is that component of the understanding process that computes a representation of the content of an utterance. ALANA, written by Charles Cox, produces a KODIAK representation of the content of an utterance. This representation generally contains only what can be determined from the words and linguistic structures present in the utterance. In our theoretical framework, we call such an analysis of an utterance its primal content. The concept of primal content is related to what is usually described as the literal meaning or sentence meaning of an utterance. However, unlike literal meaning, the primal content of an utterance involves certain idiomatic interpretations (i.e., it is not necessarily composed from words and general grammatical constructions). Also, the primal content of an utterance may be rather abstract, perhaps so much so that it may not be a suitable candidate for a meaning. For example, the literal meaning of “The cat is on the mat” is generally taken to be a conventional situation in which a cat is resting upon a mat. However, the primal content of this sentence would be more abstract, where the contribution of “on” is identical to that in the primal content of “The light fixture is on the ceiling” or “The notice is on the bulletin board.” Presumably, this conveys some sort of support relation. Note that such an abstract content appears never to be in itself the meaning of such an utterance (cf. Searle 1979). In contrast to primal content is the actual content of an utterance. The actual content is context-dependent, generally requires some amount of inference based on world knowledge, and is a suitable candidate for the
48
ROBERT WILENSKY ET AL.
meaning of an utterance. For example, the actual content of “The cat is on the mat,” without a further context specified, is what the literal meaning of this sentence is generally taken to be. Computing this content from the primal content requires pragmatic knowledge about the kind of support relation a cat and a mat are likely to be in, and requires making an inference that cannot be justified by the meanings of the terms and the grammatical constructions present in the utterance. The primal/actual content distinction is elaborated on in Wilensky (1987). (2) Inference (Concretion Mechanism) The particular kind of inference needed to go from a primal content to an actual content sometimes involves a process known as concretion (Wilensky 1983). Concretion is the process of inferring a more specific interpretation of an utterance than is justified by language alone. Concretion may involve finding a more specific default interpretation or some other interpretation based on the context. For example, in the example “the cat is on the mat” above, the actual content computed is the default support relation between a cat and a mat. In some compelling context, a quite different actual content may be computed from the same primal content. (There are other possible relations between primal and actual content besides the latter being a more specific interpretation of the former. For example, a conventionalized metaphor might have a primal content that more closely resembles its literal interpretation but an actual content resembling its metaphoric interpretation. Thus, one analysis of a sentence like “John gave Mary a kiss” will have as its primal content an instance of giving, but as its actual content an instance of kissing. We will not pursue further the details of the primal/actual-content distinction here. This is largely because, in UC, the need for concretion is widespread, and our handling of other kinds of primal/actual content computations is more haphazard.) In UC, concretion is needed primarily because we need to organize knowledge about more specific interpretations of utterances than can be arrived at through linguistic knowledge alone. For example, if UC is asked the question “How can I delete a file?”, ALANA can represent that this is a question about how to delete a file. But it would not have any reason to assume that deleting a file is a specific kind of deleting. Determining that this is so is likely to be important for several reasons. For example, knowledge about how to delete a file will be found associated with the concept of “file deletion,” say, but not with the concept of deletion in general. Thus UC must infer that “deleting a file” refers to the specific kind of deletion having to do with computer storage in order to perform subsequent tasks like finding plans for accomplishing the user’s request.
THE BERKELEY UNIX CONSULTANT PROJECT
49
In UC, concretion is the function of a special mechanism designed specifically for that purpose by Dekai Wu. The output of the concretion mechanism is another KODIAK representation, generally one containing more specific concepts than that produced by ALANA. Having a specific concretion mechanism is a pragmatic concession. We feel it is unlikely that such a specific mechanism is theoretically warranted. A more justifiable position is that a general inference mechanism should be exploited here, concretion being only one of the kinds of inference such a mechanism accomplishes. A unified text-inference mechanism that accomplishes concretion as well as other forms of inference has been built (Norvig 1987). It is our belief that some mechanism akin to Norvig’s should be used in UC in place of a specialized concretion engine, but no attempt has yet been made to do so. (3) Goal Analysis (PAGAN) Having computed an actual content for an utterance, UC then tries to hypothesize the plans and goals under which the user is operating. This level of analysis is performed by PAGAN, written by James Mayfield. PAGAN performs a sort of “speech act” analysis of the utterance. The result of this analysis is a KODIAK representation of the network of plans and goals the user is using with respect to UC. Goal analysis is important in many ways for UC. As is generally wellknown, an analysis of this sort is necessary to interpret indirect speech acts, such as “Do you know how to delete a file?”, or “Could you tell me how to delete a file?”. Furthermore, goal analysis helps to provide better answers to questions such as “Does ls -r recursively list subdirectories?”. An accurate response to the literal question might simply be “No.” But a better response is “No, it reverses the order of the sort of the directory listing; ls -R recursively lists subdirectories.” To produce such a response, one needs to realize that the goal underlying the asking of this question is either to find out what ls -r does, or to find out how to recursively list subdirectories. It is the job of the goal analyzer to recognize that such goals are likely to be behind such a question. More details about PAGAN can be found in Mayfield (1989, 1992, 2000). (4) Agent (UCEgo) Having hypothesized what the user wants of it, we would expect a system like UC to do what the user requested. But, as mentioned above, this is not always appropriate. UC should not aid and abet a user trying to perform malicious mischief; it might need to correct an errant user or it might decide to supply unasked-for information to one diagnosed as not knowing an important fact.
50
ROBERT WILENSKY ET AL.
In order to deal with such situations UC is constructed as an agent. This agent reacts to users’ requests by forming goals and acting on them. The central mechanism of UC is called UCEgo, and has been developed by David Chin. In a typical transaction, UCEgo will simply adopt the goal of having the user know what the user wants to know. However, as the example above illustrates, UCEgo may adopt other goals as well, such as protecting the integrity of the system. It may also have to detect conflicts between these goals. Sometimes, UCEgo, attempting to be educational, may adopt a somewhat different goal from the user’s. Thus, if the user asks UC to actually perform some request, such as telling the user who is on the system, UC will decide to tell the user how to perform such a function, rather than do what the user requested. UCEgo implements much of the agent-like character of UC . While interfaces are generally thought of as passive conduits through which information flows, UC is an agent that listens to the user and is generally helpful. But it has its own agenda, and the requests of the user are merely a source of input to it. More details about UCEgo can be found in Chin (1987, 1991, 1998, 2000a, 2000b). (5) User Modeling (KNOME) Several of UC’s components may need information about the user to make an effective choice. For example, an expert user certainly knows how to delete a file. Thus, such a user uttering “Do you know how to delete a file?” is unlikely to be asking for this information – more likely this user is testing the consultant’s knowledge. Assessing the knowledge state of the user is the function of a usermodeling program called KNOME, developed by David Chin. It is exploited by several components, including the Expression Mechanism described below. More details about KNOME can be found in Chin (1986, 1987, 1989). (6) Domain Planner (KIP) Typically, UCEgo tries to help the user. This usually requires determining a fact that the user would like to know. This task is accomplished by KIP. KIP is a “domain planner” developed by Marc Luria. While UCEgo infers its own goals, and plans to act on them, KIP is given a task by UCEgo of determining how to accomplish what the user wants to accomplish. KIP tries to determine how to accomplish this task, using knowledge about UNIX and knowledge about the user’s likely goals. KIP returns a plan, represented in KODIAK. For example, UCEgo may give KIP the task of determining how to move a file to another machine, if this is something the user wants to know. Here, KIP
THE BERKELEY UNIX CONSULTANT PROJECT
51
would come up with the plan of copying the file to the target machine and then deleting the original. Since UCEgo is also a planner, UC in effect has two planners within it. Again, this is probably not theoretically justifiable, although the two planners have ended up focusing on rather different aspects of planning. It remains to be seen whether a single mechanism might accommodate both functions. More details about KIP can be found in Luria (1985, 1987, 1988). (7) Expression Mechanism (UCExpress) Having gotten KIP to compute a plan for the user’s request, UCEgo now tries to communicate this plan to the user. To do so, it must determine which aspects of the plan are worthy of communication and how best to communicate them. For example, if it is likely that the user knows how to use commands in general, it might be sufficient just to specify the name of the command. In contrast, it might be helpful to illustrate a general command with a specific example. UCExpress is an “expression mechanism” written by David Chin. It edits out those parts of the conceptual answer returned by KIP that, for various reasons, appear unnecessary to communicate. UCExpress may also choose to illustrate an answer in several formats. For example, it might illustrate a general answer by generating a specific example, or it might explain one command in terms of another, simpler, command. The result of UCExpress is an annotated KODIAK network, where the annotation specifies which part of the network is to be generated. More details about UCExpress can be found in Chin (1987, 1988, 2000a, 2000b). (8) Language Production (UCGen) Once UC has decided what to communicate, it has to put it into words. This is done by a generation program called UCGen. UCGen is a simple generator, programmed by Anthony Albert and Marc Luria. It takes the marked KODIAK network produced by UCExpress and, using knowledge of English, produces sentences to complete the transaction with the user. (9) Learning Mechanism (UCTeacher) Since it is intended that UC be an extensible system, a mechanism has been developed to add new knowledge to the system by talking to it in natural language. This mechanism, called UCTeacher, is the work of James Martin. UCTeacher has capabilities to extend both UC ’s knowledge base of UNIX facts as well as its knowledge of English vocabulary. More details about UCTeacher can be found in Martin (1985, 1986a, 1986b, 1987, 1988, 2000).
52
ROBERT WILENSKY ET AL.
Note that several UC components deal with goals and plans, but in rather different ways. To minimize confusion, we emphasize the different tasks that these programs perform: PAGAN hypothesizes goals of the user with respect to UC, UCEgo reacts to the user by forming goals of its own and then trying to accomplish them, and KIP is given the goal of solving some underlying UNIX problem. Thus, PAGAN is a goal analyzer, i.e., a program that attempts to determine someone else’s goals, while UCEgo and KIP are planners, i.e., programs that determine and produce plans for their own goals. UCEgo and KIP exist as two separate programs primarily for pragmatic reasons. It may be possible that a single planner could accomplish the tasks of both UCEgo and KIP. Such an integration will have to await a revision of our system. 1.4.
UC
examples
The first set of examples illustrates the scope of the current UC prototype. They are intended to show the kind of requests the system can handle, as well as its scope. The second set demonstrates some more complex capabilities. Each sentence below preceded by the prompt ‘#’ is supplied by the user. The rest of the text is a computer-generated response. Note that the system handles a variety of request forms, some direct and some indirect. Also, the responses to “general” questions will cause an example to be generated. Finally, the reader may notice that the vocabulary used to respond to a request is often different from that used in the request itself. For example, a question about file protection is answered in terms of file permission; a question about changing a directory is answered in terms of connecting to a directory. This is because the representations used to encode the system’s knowledge are kept in a canonical form, so that different expressions of the same meaning are mapped to the same internal representation. Upon generating a response, the system uses only this internal representation, which, coupled with its generating biases, causes an expression of an answer that is largely independent of the vocabulary that appeared in the request. 1.4.1. Simple examples Welcome to UC (Unix Consultant) version 3.23 To a UC ‘#’ prompt, please type in your questions about the Unix file system in English. To leave, just type a ‘ˆD’ or ‘(exit)’. Hi. How can I help you? # Can you tell me how to move a file? Use mv. For example, to move the file named foo to the file named foo1, type ‘mv foo foo1’.
THE BERKELEY UNIX CONSULTANT PROJECT
53
# Do you know how to delete a file named dekai? Type ‘rm dekai’. # Do you know how to delete a file? Use rm. For example, to delete the file named foo, type ‘rm foo’. # How can I delete the file charlie? Type ‘rm charlie’. # How can I delete the file named tony? Type ‘rm tony’. # How can I get rid of the file named boob? Type ‘rm boob’. # I want to delete a file named peter. Type ‘rm peter’. # I want to delete a file. Use rm. For example, to delete the file named foo, type ‘rm foo’. # I want to delete the file dave. Type ‘rm dave’. # How can I add read permission to a file? Use chmod. For example, to add individual read permission to the file named foo, type ‘chmod u+r foo’. # How can I add the owner execute permission to a file? Use chmod. For example, to add individual execute permission to the file named foo, type ‘chmod u+x foo’. # How can I change the group execute permission of a file? Use chmod. For example, to remove group execute permission from the file named foo, type ‘chmod g-x foo’. # How can I change the user read permission of a file? Use chmod. For example, to remove individual read permission from the file named foo, type ‘chmod u-r foo’. # How can I remove the group write permission from the file named bimbo? Type ‘chmod g-w bimbo’. # How do I change the protection of a file? Use chmod. For example, to remove individual read permission from the file named foo, type ‘chmod u-r foo’. # How can I compact a file? Use compact. For example, to compact the file named foo, type ‘compact foo’. # How can I create a directory? Use mkdir. For example, to create the directory named foo, type ‘mkdir foo’. # How can I find a file? Use whereis. For example, to find the file named foo, type ‘whereis foo’. # How can I find out how much disk space I am using? Type ‘du’. # How can I find out who is using the system? Type ‘finger’. # How can I find out wilensky’s phone number? Type ‘finger wilensky’. # How can I list a directory? Use ls. For example, to list the directory named foo, type ‘ls foo’.
54
ROBERT WILENSKY ET AL.
# How can I logout? Type ‘logout’. # Tell me how to leave. Type ‘logout’. # How can I move a file to the file named blah? Use mv. For example, to move the file named foo to the file named blah, type ‘mv foo blah’. # How can I move the file junk? Use mv. For example, to move the file named junk to the file named foo, type ‘mv junk foo’. # How do I move a file? Use mv. For example, to move the file named foo to the file named foo1, type ‘mv foo foo1’. # How do I change my directory? Use cd. For example, to connect to the directory named foo, type ‘cd foo’. # How do I copy a file? Use cp. For example, to copy the file named foo to the file named foo1, type ‘cp foo foo1’. # How do I copy the file foo to the file blah? Type ‘cp foo blah’. # How do I create a directory? Use mkdir. For example, to create the directory named foo, type ‘mkdir foo’. # How do I print a file named info? Type ‘lpr info’. # I want to print a file. Use lpr. For example, to print the file named foo, type ‘lpr foo’. # I want to change my working directory. Use cd. For example, to connect to the directory named foo, type ‘cd foo’. # I want to copy the file named rick. Use cp. For example, to copy the file named rick to the file named foo, type cp ‘rick foo’. # I want to delete a directory. Use rmdir. For example, to delete the directory named foo, type ‘rmdir foo’. # I want to remove a directory. Use rmdir. For example, to delete the directory named foo, type ‘rmdir foo’. # What’s the best way to delete a directory? Use rmdir. For example, to delete the directory named foo, type ‘rmdir foo’. # I want to display a file. Use more. For example, to display the file named foo, type ‘more foo’. # I want to edit a file. Use vi. For example, to edit the file named foo, type ‘vi foo’. # I want to find out Joe’s address. Type ‘finger joe’. # I want to remove a directory named junk. Type ‘rmdir junk’.
THE BERKELEY UNIX CONSULTANT PROJECT
55
# What’s the command to compact a file? Use compact. For example, to compact the file named foo, type ‘compact foo’. # What is a directory? A directory is a file that is used to contain files. # What is a file? A directory is a container that is used to contain text, code or files. # What is a rm? Rm is a command that is used to delete files. # What is a search path? A search path is a list of directories that is used by the csh to search for programs to execute.
1.4.2. More complex examples Welcome to UC (Unix Consultant) version 3.23 To a UC ‘#’ prompt, please type in your questions about the Unix file system in English. To leave, just type a ‘ˆD’ or ‘(exit)’. Hi. How can I help you? # Is rn used to rename files? No, rn is used to read news. To rename a file to another file, use mv. For example, to rename the file named foo to the file named foo1, type ‘mv foo foo1’.
In this example, UC hypothesized that the user asked the question in order to get an answer to some underlying question. In particular, PAGAN hypothesized possible goals of wanting to know how to rename a file and wanting to know what ‘rn’ does. Since the answer to the direct question is negative, UCEgo realizes that the underlying goals are not addressed by this answer, and, therefore, attempts to answer them. This results in the extended answer. # How can I crash the system? I’m sorry, I cannot tell you.
In this example, UC detected a conflict between its goal of helping the user and that of preserving the integrity of the system. It resolved this conflict in favor of the latter goals. However, since it could not accommodate the user’s request, it attempts to be polite by apologizing to the user. # What does ls -v do? ls does not have a -v option.
Here the user asks about a non-existent command option. Actually, the UNIX ‘ls’ command ignores options that it does not recognize, so the consultant
56
ROBERT WILENSKY ET AL.
could have answered truthfully by saying that ‘ls -v’ causes one’s directory to be listed. It did not, because PAGAN interpreted the request as a question about the function associated with the command. UCEgo reasons that, since (1) it doesn’t know about such an option, and (2) it knows all the options to all simple commands, and (3) ‘ls’ is a simple command, the user must have a misconception. It then generates the goals of correcting the misconception, which results in the answer shown. Note that UC uses an open world hypothesis together with metaknowledge to determine whether it knows some facts about UNIX. In the next example, two different sessions with two different users are represented. Welcome to UC (Unix Consultant) version 3.23 To a UC ‘#’ prompt, please type in your questions about the Unix file system in English. To leave, just type a ‘ˆD’ or ‘(exit)’. Hi. How can I help you? # How can I delete a file? Use rm. For example, to delete the file named foo, type ‘rm foo’. # What does rwho do? Rwho is used to list all users on the network, the users’ tty, the users’ login time, and the users’ idle time. Welcome to UC (Unix Consultant) version 3.23 To a UC ‘#’ prompt, please type in your questions about the Unix file system in English. To leave, just type a ‘ˆD’ or ‘(exit)’. Hi. How can I help you? # How can I find out the inode of a file? Use ls -i. # What does rwho do? Rwho is like who, except rwho lists all users on the network.
Because he asked such a simple question, the first user was diagnosed by KNOME as being a novice user. This resulted in the answer to his first question
being illustrated by an example, and the extended format of his answer to second. In contrast, the second user asked about a relatively technical aspect of UNIX. Thus, this user is likely to be a relatively sophisticated user. As a result, the answer to that user’s first question was not illustrated with an example. For the same reason, UC presumes that the second user knows about
THE BERKELEY UNIX CONSULTANT PROJECT
57
the ‘who’ command, and takes advantage of this knowledge in generating the response. Note this example also illustrates UC’s ability to generate different types of responses. This example was able to use a simile format, because the command to be generated was thought to be sufficiently close to one UC believes the user to already know. Formatting is discussed in Section 3.6. The next example illustrates some aspects of UC’s ability to generate plans. # How do I print Jim’s file on the lineprinter? Use lpr. This plan will not work if you don’t have read permission on Jim’s file.
In this example, KIP, UC’s domain planner, suspects that the plan it is suggesting might fail. This is because the plan involves printing someone else’s file, and KIP knows that the lack of appropriate permission in such cases is a likely source of plan failure. However, since it does not know a more certain plan, it delivers the suspect one, but warns the user of the possible problem. A fact about a possible problem with a plan is called a concern. Concerns are explicitly represented in UC’s knowledge base. At this stage, UC’s knowledge of concerns is fragmentary. Thus, questions above that have the same structure as this one may not cause any warnings to be generating simply because the requisite knowledge is lacking. Concerns are discussed further in Luria (1985). # Who is on the system? I’m sorry, I don’t know. Use users to find out who is on the system.
Here UC fails to answer the user’s question, because it does not have any way of running UNIX commands itself. Its failure causes it to apologize. UC then forms the goal of the user knowing how to determine the requested information. It addresses this goal, which leads to the second part of the response.
2.
KODIAK
The knowledge used by UC is represented in KODIAK, as are the various stages of the processing of an utterance. KODIAK (Keystone to Overall Design for Integration and Application of Knowledge) is an implementation of CRT (Cognitive Representation Theory), an approach to knowledge representation that bears similarities to numerous other systems, but especially
58
ROBERT WILENSKY ET AL.
Table 1. KODIAK legend. Representation
Meaning NODES
CONCEPT
# CONCEPT n
, where n is an integer
An absolute A relation CONCEPT n is an instance of CONCEPT . LINKS
The following is priority ordered (i.e. use first match for meaning): CATEGORY −α → # # is an aspectual, i.e., a relation in which CATEGORY participates. CATEGORYA −D→ CATEGORYB CATEGORYB dominates CATEGORYA . INSTANCE −I→ CATEGORY INSTANCE is an instance of CATEGORY . #−C→ CATEGORY The argument to # is constrained to be of type CATEGORY . #−V→ OBJECT The value of the argument to # is OBJECT . #−=→# The first aspectual is constrained to have the same value as the second. CATEGORY −rel→# # is an aspectual of CATEGORY , and # specializes rel, an aspectual of some concept dominating CATEGORY . CATEGORYA −rel→ CATEGORYB Each member of CATEGORYA participates in rel with some member of CATEGORYB .
those of Schank (1975), Schubert (1976), Shapiro (1979), and Brachman and Schmolze (1985). KODIAK differs from these systems in what we believe are significant ways. However, here we shall try to trivialize rather than accentuate these differences so that the reader can relate our representations to more familiar ones. The reader should consult Wilensky (1986) for a more detailed account and justification of this representation system. Table 1 summarizes the notation used in KODIAK diagrams. 2.1.
UNIX
knowledge in UC
The KODIAK knowledge representations used in UC include several rather general notions, such as state change, goal, and action, plus many specific facts about UNIX. The complete collection is too lengthy to include here. (UC is currently constructed from approximately 200 KODIAK diagrams, consisting of about 1000 absolutes and 2,000 relations. While a scope of a
THE BERKELEY UNIX CONSULTANT PROJECT
59
Figure 1. Knowledge about deletion represented in KODIAK.
diagram is to some degree arbitrary, diagrams roughly correspond to definitions of meaningful entities, like the definition a particular command.) Some more important concepts used in modeling the domain will be explained in the individual sections of this report. To facilitate understanding the KODIAK diagrams that follow, consider the representation that the UC’s knowledge base contains about the UNIX ‘rm’ command. This is used to delete a file named by its argument. Figure 1 shows how knowledge about deletion is represented in UC. The central node in this diagram is DELETE - EFFECT. DELETE - EFFECT is shown as being a kind of STATE - CHANGE that causes a something to go from existence to nonexistence. (The notions of existence and negation do play a special role in KODIAK , but also exist as ordinary states, as is the case here. From the point of view of this example, these are just like any other states.) DELETE - EFFECT specifies the “minimal” deletion event. For example, it says nothing about the cause of such an event, or who the actor of it may be. In UC in particular and in CRT in general, such state changes are the bases from which we build more elaborate concepts. For example, the action of deleting something is represented as an action that causes something to be deleted.
60
ROBERT WILENSKY ET AL.
This is encoded by showing DELETE - EFFECT to be the effect of DELETE ACTION . Using names like DELETE - EFFECT may be somewhat misleading. In particular, DELETE - EFFECT is not required to be the effect of anything – while DELETE - ACTION is defined as having DELETE - EFFECT as its result, this statement imposes a requirement on DELETE - ACTION, not on DELETE EFFECT. We call such concepts EFFECTs rather than EVENTs, say, to emphasize that we mean to include only the most essential elements of the concept, i.e., just a specialized state change. DELETE - FILE - EFFECT is a specialized version of DELETE - EFFECT in which the object deleted is constrained to be a file. DELETE - ACTION is correspondingly specialized to DELETE - FILE - ACTION. It is also shown as being a kind of TRANSITIVE - ACTION. This is a very general category denoting any action that acts upon an object. This sort of structure, in which there exists parallel effect and action hierarchies, with the effect hierarchy carrying most of the semantic weight, is typical of the representation that appear in UC. Figure 2 connects this general knowledge about deletion with knowledge about UNIX. Here we state how to achieve a DELETE - FILE - EFFECT. This accomplished by the node labeled PLANFOR 2, which points to EXECUTE UNIX - RM - COMMAND and to DELETE - FILE - EFFECT. A PLANFOR indicates that something is conceptualized as a plan for a particular goal (PLANFORs are discussed below). In other words, this notation represents the particular fact that the ‘rm’ command (i.e., the command whose name is “rm”) is used to achieve the effect of deleting a file. Again, this structure is typical of that seen in UC – most of the information about a command is represented as information about the use of that command; the intended function of the use of a command is represented by a PLANFOR between a node representing the use of the command and some effect. The rest of the diagram specifies the format of the command. In particular, the Equate link specifies that, to delete a particular file, its name must be the same as that of the argument supplied to ‘rm’.
3. A Tour Through UC The following sections describe the components of UC in more detail. To aid in understanding how these components contribute to the processing of an individual utterance, we show how each section processes the example sentence “Do you know how to print a file on the imagen?”2 In most cases, a module is capable of doing a great deal more than is required for this example, and such capabilities are attested to. However, the example is
THE BERKELEY UNIX CONSULTANT PROJECT
61
Figure 2. Knowledge about deleting files in UNIX represented in KODIAK.
useful for illustrating the kind of processing that is performed for a typical request. In order to produce a paper of reasonable length, we reduced considerably the description of some of UC’s modules. We have focused on those processes that contribute to UC’s agent-like nature, while some more conventional modules, such as the conceptual analyzer, are mentioned only in passing. References are given to descriptions of these neglected components, which have all appeared elsewhere in print, should the reader find the account herein dissatisfying. 3.1. The analyzer A conceptual analyzer maps a string of words into a meaning representation. ALANA (Augmentable LANguage Analyzer), the conceptual analyzer for UC, takes as input a sentence typed by a user, and builds a conceptual representation using the KODIAK knowledge-representation language. ALANA constructs the “primal content” of the input utterance. The primal content is the interpretation that can be computed from grammatical and lexical knowledge; it is generally rather abstract. ALANA’s results are further interpreted and refined by other parts of the system, such as the concretion mechanism, to produce an “actual content,” and the goal analyzer, to produce a representation of the intentions underlying the utterance.
62
ROBERT WILENSKY ET AL.
Figure 3. ALANA’s output for “Do you know how to print a file on the Imagen?”
ALANA is a descendent of PHRAN (Wilensky and Arens 1980), the frontend natural-language component for the original UC (Wilensky et al 1984). Like PHRAN, ALANA reads the user’s input and forms a concept that the other UC components can use for their tasks. Also like PHRAN , ALANA uses as its primitive knowledge unit the pattern-concept pair, which relates a naturallanguage structure to a conceptual structure. UC has a total of 476 patterns and knows 284 words. ALANA differs from PHRAN in its generality. ALANA generalizes on the idea of pattern-concept-pair analysis, while making it easier than it was with PHRAN for a knowledge-adder to add new patterns to the system. Since a more detailed description of ALANA can be found in (Cox 1986), we will not elaborate on it here. Instead, we merely show in Figure 3 the output produced by ALANA upon reading the sentence “Do you know how to print a file on the imagen?”. This diagram may be interpreted as follows: The entire request is summarized as ASK 11, i.e., some asking event. What is asked for is verification of some item, QUESTION 11, whose content is KNOW 3, i.e., an instance of knowing. The knower of the item is UC, and the fact is ACTION 6. ACTION 6 is interpreted as something that is the cause of a printing action PRINTACTION 0, which is itself an action whose effect (PRINT- EFFECT 0) is to cause a file (FILE 6) to be printed on an imagen printer (IMAGEN 0). Some of the nodes in this diagram point to a node labeled HYPOTHETICAL. This is a tentative convention used to indicate that the knowing and printing event, etc., are not real events, but merely hypothetical ones.
THE BERKELEY UNIX CONSULTANT PROJECT
63
3.2. The concretion mechanism As mentioned previously, our theoretical posture is that concretion is but one of a number of inference processes that can be accomplished by a single mechanism. However, in UC, for reasons of efficiency, and for pragmatic advantages, a separate concretion mechanism was implemented (by Dekai Wu). This mechanism currently does rather straightforward classification. The mechanism concretes by using information about inheritance and value constraints, as well as by considering relation information between concepts. A concept represented as an instance of a category is passed to the concretion mechanism. Its eligibility for membership in a more specific subcategory is determined by its ability to meet the constraints imposed on the subcategory by its associated relations and aspectual constraints. If all applicable conditions are met, the concept becomes an instance of the subcategory. At the same time, the relations in which the concept participates may be concreted to reflect the more specific relations of the new category of which it is inferred to be a member. Parts of the representation of printing are shown in Figure 4. Besides the printing of the contents of a computer file, PRINT- EFFECT is in principle applicable to other types of printing, such as printing a newspaper or a book. The concretion mechanism checks each of the more specific concepts dominated by PRINT- EFFECT, searching for one whose constraints can be satisfied by the input. It finds PRINT- FILE - EFFECT, who only additional constraint is that its print-object must be a file. Since PRINT- EFFECT 0 is in print-object relation with the object FILE 6, which is indeed an instance of FILE, the process can descend to this node. The concretion process will continue until it can concrete no further. Of course, it is perfectly plausible just to preclude from UC on engineering grounds interpretations of words that do not occur in the UNIX domain. As we suggested earlier, it is our preference not to do so, since we wish to address, rather than finesse, fundamental language issues. However, doing so would not really eliminate the need for concretion. Even if we do not include concepts of non-computer printing in our knowledge base, we would still have many different kinds of printing, e.g., printing ASCII files versus binary files or printing on the lineprinter versus the laser printer. A query about each of these kinds of printing requires a different response, although the term “printing” applies to all of these. A system like UC needs to concrete the concept of printing in general to the particular kinds of printing that it knows about, in order to find the knowledge needed to answer the question. Thus, eliminating interpretations that lie outside the domain simplifies the problem somewhat, but it does not change its essential nature.
64
ROBERT WILENSKY ET AL.
Figure 4. Some knowledge about printing.
In general, when concretion occurs, some node is reclassified as being an instance of a more specific category, and, in addition, the relations predicated about that node are also reclassified. For example, here we concrete PRINTEFFECT 0 to an instance of PRINT- FILE - EFFECT . At the same time, we should concrete the relation print-object predicated about it to a use of the more specific relation print-file-object. Similarly, print-dest is concreted to printfile-dest. Continuing in this fashion, the mechanism can move from PRINT- EFFECT to LASER - PRINT- EFFECT, and finally to IMAGEN - PRINT- EFFECT, since the print-dest of the input is IMAGEN 0, which is an instance of IMAGEN. At the same time, the relation print-dest is concreted to imagen-dest. In parallel with this concretion, the node PRINT- ACTION 0 gets concreted to an instance of IMAGEN - PRINT- ACTION . The final result is shown in Figure 5.
THE BERKELEY UNIX CONSULTANT PROJECT
65
Figure 5. Result of concreting “print file on the imagen.”
3.3. The goal analyzer Once an utterance has been converted to a KODIAK representation by ALANA, and has been further refined by the concretion mechanism, this internal representation is passed to PAGAN (Plan And Goal ANalyzer). PAGAN’s task is to determine what goals the speaker is addressing in making the utterance. For example, when given a representation of the utterance “Do you know how to print a file on the imagen?” asked by a naive user, PAGAN should infer that the user was using the utterance to address the goal of knowing how to print a file on the imagen. Note that PAGAN is not responsible for detecting goals that are held by the speaker but that are not conveyed by the speaker’s utterances. This problem is addressed by the ego mechanism and by the planner. To successfully do goal analysis, at least two questions must be answered. The first concerns the utterance in isolation: What kind of act does this utterance constitute? This question has traditionally fallen under the rubric of “speech-act theory” (Austin 1962; Searle 1969). For example, “Do you know how to print a file on the Imagen?” potentially has both a direct and indirect interpretation which PAGAN must choose between. The second question a goal-analysis mechanism must answer examines the role of the utterance in conversation:
66
ROBERT WILENSKY ET AL.
Figure 6. Trace of PAGAN’s processing of “Do you know how to print a file on the imagen?”
How does this utterance relate to other utterances? By virtue of being an action, an utterance always occurs within a context. This context includes such diverse factors as the identities of the speaker and of the audience, the social relationship between them, the physical locale, the task the conversation is supplementing if any, and so on. One feature of this context that is salient to goal analysis is the presence of conventional, multi-utterance sequences. Consider the exchange: 1. Do you have write permission on the parent directory? 2. Yes. The ability to understand the full meaning of (2) is contingent on the realization that it relates directly and conventionally to (1). Thus, PAGAN will require knowledge of such sequences to correctly determine the goal underlying utterances such as (2). The input to PAGAN is the structure built by the analyzer from this utterance and refined by the concretion mechanism. A trace of PAGAN as it processes this structure is shown in Figure 6. The first step performed by PAGAN is to determine whether the utterance is the continuation of a conversational plan already in effect. For this to be the case, there would need to be some previous dialogue to provide the necessary context. This dialogue would take one of two forms. It might be a plan that UC believed the user to be pursuing before the current utterance was encountered. Alternatively, it could be a plan introduced by UC that the user has adopted, that UC believes the user to be pursuing only after witnessing the current
THE BERKELEY UNIX CONSULTANT PROJECT
67
Figure 7. A plan for knowing is to ask if the hearer knows.
utterance. Since there is no previous context in the example we are tracing, neither of these possibilities is found to hold (1–2). Next, PAGAN tries to match the utterance against the first steps of plans in its planfor knowledge base. The first possibility is compared with the input structure (3), but one pair of corresponding nodes is found not to match (4–5). The second possibility, one that does match the utterance, is then compared with the input structure (6–7). This planfor corresponds to the indirect interpretation of the utterance. This is the planfor that is shown in Figure 7. A third possibility, corresponding to the direct interpretation of the utterance, also matches the input structure (8–9). An attempt to resolve this ambiguity is now made (10–11). Since neither goal matches an expected goal (12–15), the planfors are examined for plausibility. The direct interpretation is discarded, because the user model indicates that it is likely that the user knows that UC knows how to print a file on the imagen (16). (More details about the user model can be found in Chin (1986, 1989).) Thus, the planfor representing the indirect interpretation is selected (17). Once the utterance has been matched to the first step of this planfor, an instance of a PLANFOR is created with the goals determined from the input. In addition, an instance of the HAS - GOAL state is built (18). The planner of this state is the user, and the goal is the goal of the PLANFOR. This HAS -
68
ROBERT WILENSKY ET AL.
Figure 8. PAGAN output.
GOAL represents the goal that UC believes the user had in mind in making the utterance, and is returned by PAGAN as its result (19). It is shown in Figure 8. In this figure, note that PAGAN has created a node labeled ACTION 7, whose actor is the user. This represents the inference made by the goal analyzer that, if a user wants to know an action to achieve some goal, then the user intends to be the actor of that action.
3.4. The ego mechanism Thus far in the processing, UC has parsed and understood the question, and the goal analyzer has asserted that the user has the goal of knowing a plan for printing a file on the imagen. At this point, UCEgo’s processing begins. Figure 9 shows a trace of the goal-detection phase of UCEgo. In the trace, UC - HAS - GOAL 19 represents UC’s goal of helping the user (HELP 2). HAS - GOAL-ga0, which is also shown in Figure 8, is the user’s goal of knowing (KNOW-ga0) how to print a file on the imagen. PLANFOR 29 represents the fact that a plan for helping the user (HELP 2) is for UC to satisfy KNOW -ga0, which is the user knowing how to print a file on the imagen. UC - HAS - INTENTION 6 represents UC ’s intention to satisfy KNOW -ga0. UC HAS - GOAL 20 represents UC ’s goal of the user knowing how to print a file on the imagen. The user’s goal (HAS - GOAL-ga0 in the trace) combines with UC’s goal of helping the user (UC - HAS - GOAL 19, present in UC’s initial state) to activate the fact (PLANFOR 29) that a plan for helping the user is for UC to satisfy the goal of the user knowing a plan for printing a file on the imagen. Next,
THE BERKELEY UNIX CONSULTANT PROJECT
69
Figure 9. Trace of UCE go’s goal-detection process.
this planfor combines with UC’s goal of helping the user (UC - HAS - GOAL 19) to make UCEgo adopt the intention (UC - HAS - INTENTION 6) of satisfying the goal of “the user knowing a plan for printing a file on the imagen.” This intention arises as a result of UCEgo’s if-detected demon for plan selection. Finally, UCEgo adopts the user’s goal as its own. This subgoal (UC - HAS GOAL 20) is the result of UCEgo’s goal detection process. After UCEgo has detected the goal of “the user knowing a plan for printing a file on the imagen,” the plan-selection phase of UCEgo attempts to select a plan to satisfy this goal. Figure 10 shows a trace of this part of the processing. In the trace, UC - HAS - GOAL 20 is UC’s goal of knowing (KNOWga0) ACTION 7, which represents the plan part of the planfor (PLANFOR-ga1) for printing a file on the imagen (PRINT- EFFECT 0). UNIX-planner1 represents a call to KIP. PLANFOR 70 says that a plan for achieving the goal of PRINTEFFECT 0 is to use EXECUTE - UNIX - IPR - COMMAND 0, which entails using the command ‘lpr -Pip’ on the name of the file to be printed. ANSWER - FOR 1 says that an answer to the query of “how to print a file on the imagen” (ACTION 7) is PLANFOR 70. PLANFOR 30 says that a plan for achieving the goal of the user knowing how to print a file on the imagen (KNOW-ga0) is for UC to tell (TELL 4) the user PLANFOR 70. UC - HAS - INTENTION 7 represents UC’s intention of telling the user (TELL 4). UCexpress1 represents a call to UCExpress to execute TELL 4.
70
ROBERT WILENSKY ET AL.
Figure 10. A trace of UCE go’s plan selection process.
The first step of the plan is to call KIP, the planner component of UC. Figure 11 shows the if-detected demon that calls KIP. KIP is called whenever UC has the goal of knowing a plan for something In the trace, UC - HAS GOAL 20 and PLANFOR -ga1 combine to cause a call to KIP (UNIX -planner1 in the trace). KIP, as is describing is the next section, comes back with an answer (PLANFOR 70), which is an answer (ANSWER - FOR 1) to the user’s question. Then UCEgo detects the plan of telling the user the answer (PLANFOR 30 in the trace). This plan, with UC’s goal (UC - HAS - GOAL 20) of the user knowing
THE BERKELEY UNIX CONSULTANT PROJECT
71
Figure 11. If-detected demon for calling the UC planner, KIP.
the answer leads to UC’s intention to tell the user the answer (UC - HAS INTENTION 7). Finally, the intention translates into a call to UC ’s expression mechanism, UCExpress (UCexpress1 in the trace), which eventually calls UCG en to produce the answer. KIP, UCExpress, and UCG en are described in the following sections. 3.5. The planner This section describes KIP (Luria 1985), a knowledge-based, common-sense planner (Wilensky 1983). KIP includes: − a knowledge base of facts about the domain; − a planning component that uses this knowledge to: • find potential plans for problem situations; • notice potential problems with these plans; • use metaplanning knowledge (knowledge about plans) to determine which plans to suggest. KIP uses the same knowledge base as the rest of UC . In principle, it could be used to do the planning required by UCEgo. As mentioned previously, this was not attempted mostly for pragmatic reasons. In addition, the planning done by UCEgo is much more straightforward and does not require recourse to the same magnitude of knowledge as does KIP. Thus, it seems reasonable to use a much simpler planner for communicative functions. The basic structure of KIP is similar to that of UCEgo. However, KIP is a more elaborate planner. It must be able to plan for unanticipated goals of the user, and must be concerned with adverse consequences of the plans it proposes. In general, KIP has to iterate through the planning process a number of times to arrive at an adequate plan.
72
ROBERT WILENSKY ET AL.
The following are the steps of the iterative process that KIP uses: 1. Goal detection – decide what goals to work on: − Start with the goals input from UCEgo. − Detect new goals that might arise from use of proposed plans. 2. Plan selection – select a possible plan: − Try to find a stored plan that is related to the user’s goals. − Propose a new plan if necessary based on knowledge in the system. 3. Projection – test whether plan would be executed successfully: − Check for conditions that are likely not to be met. − Notice bad side effects. − Detect goals needed to address discovered problems 4. Plan evaluation – decide whether plan is acceptable: − Reject plan if impossible. − Accept if unproblematic. − Create warning if possible but problematic. The iterative structure described here is implemented by a series of metaplans (Wilensky 1983). The underlying metaplan is to find a particular plan that the user can use; these steps are parts of that process. The example being considered throughout this paper does not require much work by KIP. This is because UC’s knowledge base contains a plan for the goal of printing a file on the imagen. Also, there are no concerns associated with this particular plan. (Had the input referred to printing the file of another user, or printing on another printer, KIP would have retrieved concerns about not being about to access another persons’ files, or about the peculiarities of the type of printer, respectively.) Thus, the planner simply ends up retrieving the stored plan, making an instance of it, and passing it on. The input to the planner from UCEgo is PRINT- EFFECT 0, which we show in Figure 12 along with its most pertinent connections. Figure 13 shows an annotated trace of KIP’s processing for this example. KIP is passed a single goal, PRINT-EFFECT0. After checking for goal conflict concerns, the planner checks to see if it has achieved the exact same goal before. When it fails, as is the case here, it looks up the hierarchy. In this example, it looks first at the category IPRINT- EFFECT, of which PRINTEFFECT 0 is represented as being an instance. KIP finds in the knowledge base PLANFOR 7 (not shown in the trace) that connects IPRINT- EFFECT with EXECUTE - UNIX - IPR - COMMAND . This knowledge is shown in Figure 14. There are no concerns retrieved with this particular plan, so KIP will simply produce an instance of it and return it to UCEgo. Note that KIP’s retrieval of a plan for this goal is simplified by the concretion mechanism’s having classified the input as an instance of “imagen printing.” Originally, ALANA represented the meaning of the input as an
THE BERKELEY UNIX CONSULTANT PROJECT
73
Figure 12. Input to KIP from “Do you know how to print a file on the imagen?”: PRINT- EFFECT 0 and important links.
Figure 13. Trace of KIP’s processing.
74
ROBERT WILENSKY ET AL.
Figure 14. Representation of plan for printing a file on the imagen.
instance of printing. However, there are no plans associated with printing per se, but only with printing’s more specific subcategories. Thus, the planner would have to search for an appropriate plan had the concretion mechanism not done its job. In actuality, the planner starts at the most specific category that the input could be classified as, and works up the hierarchy. Therefore, the planner is guaranteed to find the most specific plan applicable to the situation. Note that KIP has made NAME 6 a hypothetical object, because it is the name of FILE 6, which is itself hypothetical. After KIP has made a filled-in copy of the plan, no pertinent concerns are found, and there is nothing more to do, so the plan found is returned. The planner’s output is shown in Figure 15. 3.6. The expression mechanism After UCEgo, in conjunction with KIP, has determined the proper answer, it calls UCExpress to express it to the user. The input to UCExpress is shown in Figure 16. This input conceptual network is UCEgo’s plan to tell the user the plan that KIP produced (Figure 15). If the plan were to be directly generated into English, the result might be something like: A plan for printing a file on the imagen printer is to use the lpr command with the imagen printer option. The format of the command is “lpr -Pip” and followed by the name of the file to be printed on the imagen printer.
THE BERKELEY UNIX CONSULTANT PROJECT
75
Figure 15. Output from KIP for example “Do you know how to print a file on the imagen?”
Instead, UCExpress prunes the output, and, since the user is suspected of being a novice, generates an example to illustrate the plan. The following trace illustrates the processing of UCExpress: The first phase of UCExpress is pruning, during which those concepts that the user already knows are marked so that the generator will not generate them. In this case, PRINT- EFFECT 0, the goal of PLANFOR 70, is pruned, as it is in the current context. In traversing the input conceptual network, UCExpress runs into the command-format IPR - FORMAT 0. Since this node is presumably not known to the user, UCExpress would normally not prune it. However, UCExpress specifically checks command-formats to see if they are as specific as possible, and since FILE 6 does not have a name, UCExpress schedules the creation of an example to explain the format, and prunes the original node. The rest of the network is passed without any pruning. In creating an example, UCExpress must specify all the parameters in the command format. Thus, the name of the file, which was not specified by the user, is made explicit in the example. Here, the name “foo” was chosen arbitrarily from a list. The complete example is then turned into the proposition part of a TELL (TELL 5 in the trace). Figure 18 shows the conceptual network after pruning and the addition of an example. Note that the exemplified of an EXAMPLE is automatically pruned, as it is always expressed by the expressing the generic response. The pruned and augmented conceptual network is next passed to the generator, described in the next section, which produces the following English output:
76
ROBERT WILENSKY ET AL.
Figure 16. The input to UCE xpress for example “Do you know how to print a file on the imagen?”
Figure 17. Trace of UCE xpress.
THE BERKELEY UNIX CONSULTANT PROJECT
77
Figure 18. The output of UCE xpress for the example “Do you know how to print a file on the imagen?”
Use lpr -Pip. For example, to print the file named foo on the imagen printer, type ‘lpr -Pip foo’. If the user had been judged to be at least a beginner in experience, then the command-format would also have been pruned. This is because KNOME believes that users at the beginner level and up can be assumed to know that part of the command format. In such a case, the entire output would consist of the pruned version of TELL 4, and the generated entire response would be: Use lpr -Pip.
78
ROBERT WILENSKY ET AL.
3.7. The generator After UCExpress formats an answer, the generator, UCGen, converts the conceptual response into text. The current version of UCGen has been customized to work with the types of responses that the system typically produces. It has been built to take advantage of the limited structure of these responses. To convert a KODIAK representation of a concept into text, UCGen must associate some linguistic information with the concept. This is done by attaching to a concept a pattern that represents some linguistic form. For example, a concept often expressed is PLANFOR. This concept relates a plan for achieving a goal with the goal itself. A pattern for PLANFOR is: To (gen goals) comma (gen plan). This pattern might be used to generate the sentence: To delete a file, use rm. This is somewhat akin to the pattern-concept pair construct in PHRED (Jacobs 1984), or to KING’s REF links (Jacobs 1985), although the KODIAK representation accommodates different methods for fetching patterns. Patterns mix words and punctuation with function calls. In the above example, ‘gen’ is a function that will be called with argument ‘goals’ and later with argument ‘plan’. In general, the arguments to functions that are found in generator patterns are the aspectuals associated with the concept to which the pattern is attached. In this example, the aspectuals of PLANFOR, ‘goals’ and ‘plan,’ are arguments to gen. The pattern given above for PLANFOR is the most general one for that concept. That is, it is the pattern used when both the goals and the plan are to be expressed. As described in the previous section on UCExpress, it is not always necessary to express both of these parts. For example, two answers to ‘How do I delete a file?’ are: 1. To delete a file, use rm. 2. Use rm. The expression mechanism puts a flag on each aspectual that it does not want expressed. Consequently, associated with each concept may be zero or more patterns, one for each combination of aspectuals that are to be expressed. PLANFOR is associated with the general pattern shown above, as is the pattern ‘(gen plan)’, which is applicable to the case where only the plan is to be expressed. When a concept to be output is given to the generator, those KODIAK concepts that either dominate or are categories for the concept are searched for one that has an attached generator pattern. If no pattern is found, and
THE BERKELEY UNIX CONSULTANT PROJECT
79
the concept is an aspectual, then the value for the aspectual is sent to the generator. The first pattern found is applied to the concept to be expressed to produce an English sentence. Words in the pattern are output as is. Punctuation and function calls must go through further processing. For example, in the pattern ‘To (gen goals) comma (gen plan)’, the word ‘To’ is output directly, whereas the (gen . . . ) function calls must be evaluated, and the ‘comma’ will be converted to a ‘,’. This generator is easy to understand and extend, and is well integrated with the rest of UC; it shares the KODIAK representation and concepts used by the rest of the system. Some weaknesses are that the overall structure is top down; i.e., only those concepts that are expected to exist are expressed. In general, a generator should be able to handle arbitrary permutations of conceptual relationships. Also, this generator uses little linguistic knowledge. With more complicated utterances, the simple pattern strategies employed so far would become inadequate. This section describes how the output is delivered by UC in response to the question, ‘Do you know how to print a file on the imagen?’ A diagram of some of the relevant knowledge structures is given in Figure 19. A trace produced while generating this output is given in Figure 20. The expression mechanism of UCEgo first passes TELL 4 to the generator. Only the proposition part of the TELL will be expressed, so its value, PLANFOR 70, is passed to the generator’s main routine, ‘gen.’ PLANFOR 70 is dominated by PLANFOR, so the pattern for PLANFOR is retrieved. Since the goals aspectual of PLANFOR 70 is marked to be omitted from the response by the expression mechanism, only the plan will be expressed. The pattern found is ‘(gen plan)’. The value of the plan aspectual, EXECUTE - UNIX IPR - COMMAND 0, is sent to ‘gen’. The pattern for this concept is found under EXECUTE - FILE - COMMAND and is ‘use (gen execute-command)’. The value of execute-command aspectual of EXECUTE - UNIX - IPR - COMMAND 0 is ‘lpr -Pip.’ The first response is therefore: Use lpr -Pip. Next, the generator is passed TELL 5. Once again, only the proposition is to be expressed, so EXAMPLE 0 is to be generated. The pattern, found under EXAMPLE, is ‘for example comma (gen exampler)’. This sets up a recursive call to gen with the value of the exampler relation as argument. This value is PLANFOR 31. Once again, a PLANFOR is to be generated. This time, however, both the plan and goals will be expressed. The pattern is ‘to (gen goals) comma (gen plan)’. The value of the goals aspectual is PRINT- EFFECT 1. The pattern is found under LAS - PR - EFFECT, and is ‘print (las-pr-file-obj) on the (las-prdest-obj)’.
80
ROBERT WILENSKY ET AL.
Figure 19. Knowledge pertaining to generating response to “Do you know how to print a file on the imagen?”
The las-pr-file-obj specification causes the generator to find on object in this relation to PRINT- EFFECT 1, in this case, FILE 0. One of the patterns for FILE is ‘file named (gen name)’. This pattern is applicable if the object in question has a name. Here, FILE 0 has the name ‘foo’. When UCGen is generating the name of an object, it uses a heuristic to decide which, if any article to precede the object description with. In general, UCGen will use definite articles if it can. The other part of the output from the goals aspectual is from the second half of the pattern: ‘on the (las-pr-dest-obj)’. Here the value found is IMAGEN 0. The pattern for IMAGEN 0 is just ‘imagen’. Again, the generator with supplement this description with a definite article. The value of the ‘plan’ aspectual for PLANFOR 31 is TYPE - ACTION 0. The pattern for this concept is from TYPE - ACTION and is ‘type lquote (gen typestring) rquote’. The value for the type-string aspectual of TYPE - ACTION 0 is CONCAT- STR 0. The pattern is from SEQUENCE and is ‘(gen step) (gen next)’. Here, the step is ‘lpr -Pip’ (i.e., the string1 of CONCAT- STR 0), and the next is the name of the file, ‘foo’ (the string2 of CONCAT- STR 0). Thus, the output for this call to the generator is:
THE BERKELEY UNIX CONSULTANT PROJECT
81
Figure 20. UCGen trace for the example question.
For example, to print the file named foo on the imagen, type ‘lpr -Pip foo’.
4. Problems As the preceding sections describe, there are many technical problems yet to be resolved for each component of UC. However, several problems appear to be more pervasive. One general problem is the integration of the components of the system. Control flows unidirectionally through UC. However, there are several cases in which this control structure is unsatisfactory. One such problem is the rela-
82
ROBERT WILENSKY ET AL.
tion of language analysis and inference. We believe it is cognitively correct that these components function concurrently to produce an interpretation of an utterance, whereas in UC they function serially. For example, consider again the process of understanding the sentence we have been using in our extended example: “Do you know how to print a file on the imagen?”. This utterance is syntactically ambiguous in the attachment of the prepositional phrase “on the imagen.” Syntactically, this may modify “you” or “a file” as well as “print.” UC does not deal with this ambiguity, because one of ALANA’s patterns for “print” specifically looks for “on” followed by a device. However, a more elaborate analyzer would probably not include specific information that relates this preposition to the verb, but rather would try to relate them on more general principles. In such a system, the ambiguity would be a more difficult problem. Our current approach is to build such a system and use a marker-passing algorithm (Norvig 1987) to help suggest which syntactic combination to try. For example, our knowledge about printing is such that a path between printing and a device designed for printing should be easy to find. In contrast, there would be a less obvious connection between imagen and file, or imagen and the referent of “you.” This “conceptual closeness” would suggest trying to relate printing and the imagen with a grammatical pattern, so the correct interpretation would be arrived at without other interpretations being tested. Properly done, such a marker-passing scheme would effect concretion as well. For example, to arrive at the connection between printing and the imagen, it is probable that one needs to access the node for “computerprinting.” Thus, it seems that concretion should not be a separate inference process, but one of several kinds of inference that are performed by a markerpassing mechanism. We are currently attempting to reform the analyzer and the inference mechanism in the direction described. It seems that the sort of unidirectional architecture we have employed has drawbacks elsewhere in the system. There are situations in which it seems that one component should be allowed to fail, and the failure be propagated back to another component. For example, consider processing the following query: How can I edit Joe’s file? Initially, the goal analyzer may interpret this request literally. Then the planner may fail, because the file may be protected from just such an action. It seems reasonable, however, for a consultant to suggest copying the file and editing the copy. For this to happen, control must be returned to the goal analyzer, which needs to hypothesize yet another goal underlying the goal it may have suggested initially. We are attempting to design a control structure that accommodates this flow of control.
THE BERKELEY UNIX CONSULTANT PROJECT
83
The concretion mechanism and the goal analyzer also appear to interact in important ways. For example, consider the following example: What does ls -v do? Above we showed the UC can respond appropriately to this question by uttering “There is no -v option to the ls command.” However, the question is problematic because another response to it might be “It lists the contents of the current directory.” This response is possible because, although there is no ‘-v’ option to the ‘ls’ command, it is a characteristic of this command that it ignores options it does not recognize.3 To produce the desired response, the system must recognize that the intent of the question is something like “Tell me the conventional function of the command ls -v,” and not “Tell me what actually happens when we type ls -v.” One way to phrase this is that “conventional function” and “effects occurring from” are two kinds of “doing.” There are certainly other kinds as well. For example, the same form may refer to the steps of a process. Therefore, it would appear to be the job of the concretion mechanism to select the appropriate interpretation. However, it seems that the concretion mechanism cannot choose this interpretation without some knowledge of typical user goals. For example, if a user is debugging a program, it would probably be appropriate to interpret the question as referring to the steps incurred in the process rather than to the process’s purpose. But reasoning about the user’s goals is the job of the goal analyzer, which normally is not invoked until the concretion mechanism has completed its task. The problem is avoided in the current implementation by not allowing for the other, less obvious interpretations at all. However, the example illustrates the need to have more communication between the concretion mechanism and the goal analyzer. Put more strongly, the example suggests that these distinctions between language analyzer, concretion mechanism, and goal analyzer are somewhat artificial. At this stage of our work, it is difficult to determine whether we simply want modules that interact more or a more radical control structure that integrates all these functions. There are several other more specific deficiencies of which we are aware. As we discussed previously, patterns were built into ALANA on an “as needed” basis. We are attempting to produce a more accurate language specification as we develop the inference component. Also, a mechanism for doing ellipsis, which ran in a previous version of UC, has yet to be integrated into this one. Undoubtedly, there are many deficiencies that we have not yet discovered. For example, we recently discovered that asking the same question twice resulted in no answer at all being generated for the second request. The
84
ROBERT WILENSKY ET AL.
problem turned out to be that the user model, after a question is answered, updates its model of the user to show that the user now knows this information. The second time around, this knowledge allowed the expression mechanism to prune away the entire answer, as it inferred the user already knew it. Our approach to fixing this problem is to add another demon that will detect asking for the same thing twice. Then plans for responding appropriately with this situation could be brought to bear. One important deficiency of our current system is that it still doesn’t participate in real conversations. It is our intention that UC function as a consultant and not as a front end to a data base of facts about UNIX. But our current system performs little more than this. Much of the machinery is in place, in UCEgo and PAGAN in particular, to accommodate some conversational situations. We expect much of our further development to be in this direction. Finally, although we have found that our current representation is advantageous, there are many representational issues that remain unresolved. In particular, it is difficult to express certain aspects of quantification in KODIAK. In UC, one often wants to represent facts like “all files have names” or “most directories are not empty.” We are currently working on extending KODIAK to be able to represent such notions in a cognitively plausible way.
Acknowledgements This research was sponsored in part by the Defense Advanced Research Projects Agency (DoD), ARPA order No. 4871, monitored by Space and Naval Warfare Systems Command Command under contract N00039-84-C0089, by the Office of Naval Research under contract N00014-80-C-0732, by the National Science Foundation under grant MCS79-06543, and by the Office of Naval Research under contract N00014-97-1-0578.
Notes 1 UNIX is a trademark of The Open Group. 2 A kind of laser printer used at our site. 3 This “feature” has been changed to produce an error message on some versions of UNIX .
References Allen, J. F. & Perrault, C. R. (1980). Analyzing Intention in Utterances. Artificial Intelligence 15: 143–178.
THE BERKELEY UNIX CONSULTANT PROJECT
85
Allen, J. F., Frisch, A. M. & Litman, D. J. (1982). ARGOT: The Rochester Dialogue System. In Proceedings of The Second National Conference on Artificial Intelligence, 66–70. Pittsburgh, PA: AAAI Press. Austin, J. L. (1962). How To Do Things With Words. Harvard University Press: Cambridge, MA. Brachman, R. & Schmolze, J. (1985). An Overview of the KL-ONE Knowledge Representation System. Cognitive Science 9: 171–216. Brown, J. S. & Burton, R. R. (1976). A Tutoring and Student Modelling Paradigm for Gaming Environments. In Symposium on Computer Science and Education, 236–246. Anaheim, CA. Carberry, S. (1983). Tracking User Goals in an Information-Seeking Environment. In Proceedings of Third National Conference on Artificial Intelligence, 59–63. Washington, DC: AAAI Press. Chin, D. N. (1986). User Modeling in UC, the UNIX Consultant. In Proceedings of CHI ’86. Human Factors in Computing Systems, 24–28. Boston, MA: ACM Press. Chin, D. N. (1988). Exploiting User Expertise in Answer Expression. In Proceedings of The Seventh National Conference on Artificial Intelligence, 756–760. Saint Paul, MN: AAAI Press. Chin, D. N. (1989). KNOME: Modeling What the User Knows in UC. In Kobsa, A. & Wahlster, W. (eds.) User Models in Dialog Systems, 74–107. Springer-Verlag: Berlin. Chin, D. N. (1987). Intelligent Agents as a Basis for Natural Language Interfaces. Ph.D. diss., Computer Science Division, University of California, Berkeley, CA. Also available as UCB/CSD 88/396, Computer Science Division, University of California, Berkeley, CA. Chin, D. N. (1991). Intelligent Interfaces as Agents. In Sullivan, J. W. & Tyler, S. W. (eds.) Intelligent User Interfaces, 177–206. ACM Press: Reading, MA. Chin, D. N. (1998). Intelligent Interfaces as Agents. In Maybury, M. T. & Wahlster, W. (eds.) Readings in Intelligent User Interfaces. R. Morgan-Kaufmann: Los Altos, CA. Chin, D. N. (2000a). Strategies for Expressing Concise, Helpful Answers. In Hegner, S., McKevitt, P., Norvig, P. & Wilensky, R. (eds.) Intelligent Help Systems for UNIX. Kluwer Academic Publishers: Dordrecht, The Netherlands (this volume). Chin, D. N. (2000b). Planning Intelligent Responses in a Natural Language System. In Hegner, S., McKevitt, P., Norvig, P. & Wilensky, R. (eds.) Intelligent Help Systems for UNIX. Kluwer Academic Publishers: Dordrecht, The Netherlands (this volume). Cohen, P. R. & Levesque, H. J. (1987). Persistence, Intention, and Commitment. SRI International Technical Report 415. Cox, C. A. (1986). ALANA: Augmentable LANguage Analyzer. UCB/CSD 86/283, Computer Science Division: University of California, Berkeley, CA. Davis, R. (1976). Applications of Meta Level Knowledge to the Construction, Maintenance and Use of Large Knowledge Bases. STAN-CS-76-552, Computer Science Dept.: Stanford University, Stanford, CA. Deering, M., Faletti, J. & Wilensky, R. (1982). Using the PEARL AI Package. UCB/ERL M82/19, Computer Science Division: University of California, Berkeley, CA. Ernst, G. & Newell, A. (1969). GPS: A Case Study in Generality and Problem Solving. Academic Press: New York. Fikes, R. E. & Nilsson, N. J. (1971). STRIPS: A New Approach to the Application of Theorem Proving to Problem Solving. Artificial Intelligence 2(3–4): 189–208. Grice, H. P. (1975). Logic and conversation. In Cole, P. & Morgan, J. L. (eds.) Studies in Syntax III, 41–58. Seminar Press: New York.
86
ROBERT WILENSKY ET AL.
Grosz, B. & Sidner, C. L. (1985). The Structures of Discourse Structure. CSLI–85–39, Center for the Study of Language and Information: Stanford University, Stanford, CA. Haas, N. & Hendrix, G. (1980). An Approach to Acquiring and Applying Knowledge. In Proceedings of First National Conference on Artificial Intelligence, 235–239. Stanford, CA: AAAI Press. Jacobs, P. S. (1984). PHRED: A Generator for Natural Language Interfaces. UCB/CSD 84/189, Computer Science Division: University of California, Berkeley, CA. Jacobs, P. S. (1985). A Knowledge-Based Approach to Language Production. Ph.D. diss., Computer Science Division, University of California, Berkeley, CA. Also available as UCB/CSD 86/254, Computer Science Division, University of California, Berkeley, CA. Kaplan, S. J. (1983). Cooperative Responses from a Portable Natural Language Database Query System. In Brady, M. & Berwick, R. C. (eds.) Computational Models of Discourse. MIT Press: Cambridge, MA. Lakoff, G. & Johnson, M. (1980). Metaphors We Live By. University of Chicago Press: Chicago, IL. Litman, D. J. & Allen, J. F. (1984). A Plan Recognition Model for Clarification Subdialogues. In Proceedings of Tenth International Conference on Computational Linguistics, 302–311. Palo Alto: MIT Press. Luria, M. (1982). Dividing up the Question Answering Process. In Proceedings of The Second National Conference on Artificial Intelligence, 71–74. Pittsburgh, PA: AAAI Press. Luria, M. (1985). Commonsense Planning in a Consultant System. In Proceedings of 1985 IEEE International Conference on Systems, Man, and Cybernetics, 602–606. Tucson, Arizona: IEEE Press. Luria, M. (1987). Goal Conflict Concerns. In Proceedings of Tenth International Joint Conference on Artificial Intelligence, 1025–1031. Milan, Italy: Morgan Kaufmann Publishers. Luria, M. (1988). Knowledge Intensive Planning. UCB/CSD 88/433, Computer Science Division: University of California, Berkeley, CA. Martin, J. (1985). Knowledge Acquisition through Natural Language Dialog. In Proceedings of Second Annual Conference on Artificial Intelligence Applications, 728–733. Miami, FL: AAAI Press. Martin, J. (1986a). Views from a Kill. In Proceedings of Eighth National Conference of the Cognitive Science Society, 728–733. Amherst, MA. Martin, J. (1986b). Representing and Acquiring Knowledge about Metaphors. In Proceedings of Third Workshop on Theoretical Issues in Conceptual Information Processing, 49–54. Philadelphia, PA. Martin, J. (1987). Understanding New Metaphors. In Proceedings of Tenth International Joint Conference on Artificial Intelligence, 137–139. Milan, Italy: Morgan Kaufmann Publishers. Martin, J. (1988). A Computational Theory of Metaphor. Ph.D. diss., Computer Science Division, University of California, Berkeley, CA. Also available as UCB/CSD 88/495, Computer Science Division, University of California, Berkeley, CA. Martin, J. (2000). Representing UNIX Domain Metaphors. In Hegner, S., Mc Kevitt, P., Norvig, P. & Wilensky, R. (eds.), Intelligent Helps Systems for UNIX. Dordrecht, The Netherlands: Kluwer Academic Publishers (this volume). Mayfield, J. (1989). Goal Analysis: Plan Recognition in Dialogue Systems. Ph.D. diss., Computer Science Division, University of California, Berkeley, CA. Also available as UCB/CSD 89/521, Computer Science Division, University of California, Berkeley, CA. Mayfield, J. (1992). Controlling Inference in Plan Recognition. User Modeling and UserAdapted Interaction 2(1–2): 83–115.
THE BERKELEY UNIX CONSULTANT PROJECT
87
Mayfield, J. (2000). Evaluating Plan Recognition Systems: Three Properties of a Good Explanation. In Hegner, S., Mc Kevitt, P., Norvig, P. & Wilensky, R. (eds.), Intelligent Helps Systems for UNIX. Dordrecht, The Netherlands: Kluwer Academic Publishers (this volume). Mays, E. (1980). Failures in Natural Language Systems: Applications to Data Base Query Systems. In Proceedings of First National Conference on Artificial Intelligence, 327–330. Stanford, CA: AAAI Press. McCoy, K. F. (1983). Correcting Misconceptions: What to Say when the User is Mistaken. In Proceedings of CHI ’83 Conference, 197–201. Boston, MA: ACM Press. Morik, K. & Rollinger, C-R. (1985). The Real Estate Agent – Modeling the User by Uncertain Reasoning. AI Magazine 6(2): 44–52. Newell, A. & Simon, H. A. (1972). Human Problem Solving. Prentice-Hall: Englewood Cliffs, NJ. Norvig, P. (1983). Frame Activated Inferences in a Story Understanding Program. In Proceedings of Eighth International Joint Conference on Artificial Intelligence, 624–626. Karlsruhe, West Germany: Morgan Kaufmann Publishers. Norvig, P. (1987). A Unified Theory of Inference for Text Understanding. Ph.D. diss., Computer Science Division, University of California, Berkeley, CA. Also available as UCB/CSD 87/339, Computer Science Division, University of California, Berkeley, CA. Rich, E. (1979). User Modeling via Stereotypes. Cognitive Science 3: 329–354. Rissland, E. L. (1983). Examples in Legal Reasoning: Legal Hypotheticals. In Proceedings of Eight International Joint Conference on Artificial Intelligence, 90–93. Karlsruhe, West Germany: Morgan Kaufmann Publishers. Rissland, E. L., Valcarce, E. M. & Ashley, K. D. (1984). Explaining and Arguing with Examples. In Proceedings of Fourth National Conference on Artificial Intelligence, 288–294. Austin, TX: AAAI Press. Rosch, E. (1978). Principles of Categorization. In Eleanor Rosch & Lloyd, B. B. (eds.) Cognition and Categorization. Lawrence Erlbaum: Hillsdale, NJ. Sacerdoti, E. (1974). Planning in a Hierarchy of Abstraction Spaces. Artificial Intelligence 5: 115–135. Schank, R. C. (1975). Conceptual Information Processing. North Holland: Amsterdam. Schank, R. C. & Abelson, R. P. (1977). Scripts, Plans, Goals, and Understanding. Lawrence Erlbaum: Hillsdale, NJ. Schmolze, J. G. & Lipkis, T. A. (1983). Classification in the KL-ONE Knowledge Representation System. In Proceedings of Eighth International Joint Conference on Artificial Intelligence, 330-332. Karlsruhe, West Germany: Morgan Kaufman Publishers. Schubert, L. K. (1976). Extending the Expressive Power of Semantic Networks. Artificial Intelligence 7: 163–195. Searle, J. R. (1969). Speech Acts; An Essay in the Philosophy of Language. Cambridge University Press: Cambridge, England. Searle, J. R. (1979). Literal Meaning. In Searle, J. R. (ed.) Expression and Meaning. Cambridge University Press: Cambridge, England. Shapiro, S. (1979). The SNePS Semantic Network Processing System. In Findler, N. (ed.) Associative Networks. Academic Press: New York. Sidner, C. L. (1985). Plan Parsing for Intended Response Recognition in Discourse. Computational Intelligence 1: 1–10. Teitelman, W. et al. (1978). The Interlisp Reference Manual. Xerox PARC.
88
ROBERT WILENSKY ET AL.
van Melle, W. (1980). A Domain Independent System That Aids in Constructing KnowledgeBased Consultation Programs. HPP-80-22, Heuristic Programming Project, Computer Science Department: Stanford University, Stanford, CA. Webber, B. L. & Mays, E. (1983). Varieties of User Misconceptions: Detection and Correction. In Proceedings of Eighth International Joint Conference on Artificial Intelligence, 650– 652. Karlsruhe, West Germany: Morgan Kaufman Publishers. Wilensky, R. (1983). Planning and Understanding: A Computational Approach to Human Reasoning. Addison-Wesley: Reading, MA. Wilensky, R. (1986). Some Problems and Proposals for Knowledge Representation. UCB/CSD 86/294, Computer Science Division: University of California, Berkeley, CA. Wilensky, R. (1987). Primal Content and Actual Content: An Antidote to Literal Meaning. UCB/CSD 87/365, Computer Science Division: University of California, Berkeley, CA. Wilensky, R. & Arens, Y. (1980). A Knowledge-Based Approach to Natural Language Processing. UCB/ERL M80/34, Electronic Research Laboratory: University of California, Berkeley, CA. Wilensky, R., Arens, Y. & Chin, D. N. (1984). Talking to Unix in English: An Overview of UC. Communications of the Association for Computing Machinery 27(6): 574–593.
The OSCON Operating System Consultant* PAUL MC KEVITT Center for PersonKommunikation (CPK), Fredrik Bajers Vej 7-A5, Institute of Electronic Systems (IES), Aalborg University, DK-9220, Aalborg, Denmark. E-mail: [email protected]
Abstract. OSCON (Operating System CONsultant) is an operating system consultant which gives English answers to English queries about computer operating systems. The program currently answers queries for over 40 commands from the UNIX1 and MS-DOS2 operating systems. OSCON answers a wide variation of queries that users may wish to ask. OSCON is intended to be a consultant for various types of users who may ask vague and detailed queries. OSCON is programmed in Quintus Prolog and answers queries in less than 2.5 seconds. An empirical study with the Wizard-of-Oz technique provides important data for the further development of OSCON. Keywords: intelligent help, Natural Language Processing (NLP), operating systems, UNIX
1. Introduction This paper describes the OSCON (Operating System CONsultant) program and its capabilities. OSCON is a natural language interface which answers English queries about computer operating systems. Although a comprehensive description of OSCON is given here further details are found in Mc Kevitt (1986, 1987, 1988), Mc Kevitt and Wilks (1987), Mc Kevitt and Pan (1989), and Mc Kevitt and Ogden (1989a). The user enters English queries to OSCON and the program answers them in English. The program is written in Quintus Prolog, runs on a Sun-3/ME-4 computer, and the maximum time taken to answer a given query is 2.5 seconds. OSCON answers queries for over 40 commands from the UNIX and MS-DOS operating systems. OSCON has been constructed to answer three basic types of query although recent studies indicate that there are at least nine basic query types (see Mc Kevitt and Ogden 1989a,b). OSCON will also answer queries about options on UNIX commands and complex queries about command compositions. The system is intended to be used by varying types of users with different levels of expertise. The architecture of OSCON is modular so that it is easily updated and can be easily mapped over to other domains.
90
PAUL MC KEVITT
2. Query Coverage of OSCON The problem with building effective natural language interfaces is that there are many ways of asking English queries. The system must attempt to capture all the different possibilities. One way to do this is to try and capture the basic types of queries that people ask. This gives the system the power of answering a large number of queries when it caters for each type. OSCON was designed to cater for three basic types of query. These query types were decided a priori before any data analysis was conducted. These query types are: (1) request for information (e.g. “What does rm do?”), (2) request for explanation (e.g. “What is a pipe?”), and (3) request for direction (e.g. “How do I see my file on the printer?”). Three cases of type (1) are considered important: (1) request for information[effect] (e.g. “What does rm do?”), (2) request for information[form] (e.g. “What is the syntax of cp?”), and (3) request for information[precondition] (e.g. “What is needed for rm?”). Two cases of type (2) were analyzed: (1) request for explanation[command] (e.g. “What is more?”), and (2) request for explanation[concept] (e.g. “What is a file?”), Each of these basic query types can also be asked in terms of options in the UNIX operating system. Examples are, “What option of ‘ls’ shows the number of bytes in my files?” (request for information[option]), “What does ls -l do?” (request for information[effect]), “How do I rename a file without having reported errors?” (request for direction), “What are the options on ls?” (request for information[options]), “What does the -i option normally do?” (request for information[concept]). Users can also ask queries involving command compositions. An example is, “How do I list my files and print them on the printer?” This query involves a query about listing files (request for direction) and then printing them on the printer (request for direction). The OSCON program currently answers queries constituting (1) the three basic query types, (2) queries about options, and (3) command composition queries for both the UNIX and MS-DOS operating systems. The fact that queries are of a given type aids in understanding and generating answers to them. For example, request for information queries usually include a command name. Therefore, the parser for OSCON could check for command names and if it found them, this would be an indication that the query was of type (1). Also, the generator would generate an answer, in a particular format, depending on the type of query. Rules of thumb such as these also speed up the time it takes OSCON to answer queries. Although one can add such rules of thumb into the interface it does not reflect a short-cut to natural language parsing. For example, there is no such short cut to understanding the query, “How do I print a file on the Laserwriter with no page burst?” Understanding queries is a combination of both (1) filtering the query type, and then
THE OSCON OPERATING SYSTEM CONSULTANT
91
(2) understanding the query. Examples of queries answered by OSCON are shown in Appendix A. These examples are listed by query type.
3. The Architecture of OSCON The architecture of the OSCON system is defined as six distinct modules. There are two arguments for modularizing any system: (1) it is much easier to update the system at any point, and (2) it is easier to map the system over to another domain. The six modules in OSCON are as follows: (1) ParseCon: natural language syntactic grammar parser which detects query-type, (2) MeanCon: a natural language semantic grammar which determines query meaning, (3) KnowCon: a knowledge representation for understanding, (4) DataCon: a knowledge representation for solving, (5) SolveCon: a solver for resolving query representations against knowledge base representations, and (6) GenCon: a natural language generator for generating answers in English. ParseCon consists of a set of programs which read natural language input and determine the type of query being asked by the user. We have described the three basic types of query in section 2 above. For each type of query there are tests for characteristic ways of asking that query. MeanCon consists of programs which check queries for important information. There are predicates which check for mentioned (1) command names (e.g. “ls”, “more”), (2) command-effect specifications (e.g. “see a file”), and (3) concepts (e.g. “file”, “directory”). In case (2) there are specific types of information searched for: (1) verb specifying action (e.g. “see”, “remove”), (2) object of action (e.g. “file”), (3) modifier of object (e.g. “contents”), and (4) location of object (e.g. “screen”). MeanCon also checks for option verbs (e.g. “number”) and option verb objects (e.g. “lines”). MeanCon contains a dictionary of English words that define categories such as “person”, “modifier”, “article”, “quantifier” and “prepositions”. KnowCon consists of a set of data files to represent the knowledge about the domain language used for understanding English queries. Files contain information about verbs which categorize types of command or action. Examples of categories of action are: (1) creating, (2) screenlisting, (3) printerlisting, (4) sending, (5) transferring, and (6) removing. KnowCon also contains grammar rules for operating system objects like “date”, “file” and “directory”. The grammar rules encode characteristic ways in which people talk about the objects in English. DataCon consists of a set of data files defining detailed information about operating system commands. This information is stored for the UNIX and MS-DOS operating systems. The data for UNIX is split among seven files: (1) command preconditions, (2) command effects, (3) command syntax, (4)
92
PAUL MC KEVITT
command names, (5) command precondition options, (6) command effect options, and (7) command name options. The first four files contain basic data about commands while the last three contain data for options. For MSDOS, data is contained in just four files which are similar to the first four here. SolveCon is a solver which constructs and matches representations of user queries (called formal queries) against DataCon and produces an instantiated formal query which serves as an answer for the query. SolveCon is the driver of the OSCON program because it contains the information for mapping English sentences into instantiated answers. It contains a set of complex rules which call other OSCON modules to determine (1) query type, and (2) the instantiated formal query for that query. GenCon is the natural language generator for OSCON and maps instantiated information from SolveCon into English answers. Here, there are algorithms for printing out (1) preconditions, (2) effects, and (3) syntax of commands. Also, there are routines for printing out examples of the use of commands and command compositions. The type of query asked by the user determines the information mapped to the user.
4. Knowledge Representation in OSCON One of the problems in building natural language interfaces is to organize the knowledge of the domain in some form which will be effective. There are two types of knowledge stored in OSCON: (1) knowledge about natural language, and (2) knowledge about operating systems. The knowledge about language stored includes words used to refer to command actions. For example, a user may use the words “delete”, “remove”, “get rid of”, “erase” and so on to ask a query about deleting files and directories. These words must be stored under the general category of delete. Also, there are many ways in which people ask queries about actions. For example, if you are asking about copying a file you will probably specify the file which you wish to copy. If you are asking about “displaying” you will specify what you wish to display and where you wish to display it. This type of knowledge is called “understanding knowledge” and is stored within a module of OSCON called KnowCon. More detailed knowledge about operating systems is contained in a database called DataCon. This type of knowledge includes command preconditions, command effects, command syntax, and the names of commands. Also, stored here is (1) knowledge about options for commands, (2) English descriptions of operating system concepts like “files” and “directories”, and (3) knowledge about plans or possible command combinations (e.g. ‘ls’ can
THE OSCON OPERATING SYSTEM CONSULTANT
93
precede ‘lpr’ but the converse is not true). The knowledge stored here is for the UNIX and MS-DOS operating systems. The distinction between the two types of knowledge is that one is language oriented and the other is domain oriented. One type of knowledge is used for understanding queries and the other for solving queries. This is the principle of separation of understanding and solving defined in Hegner (1988, 2000). 4.1. Knowledge for understanding (KnowCon) There are two types of understanding knowledge stored in the KnowCon module: (1) data on operating system action reference, and (2) data on descriptions of operating system objects. The first type of knowledge includes sets of words or phrases that may refer to some operating system action or command. For example, the words, “print”, “print out”, and “get a copy” would indicate that the user was referring to printing something on the printer. Such words and phrases are stored as being associated with the general concept of printing. The second type of knowledge is used for defining the ways that users refer to operating system objects. For convenience the data have been split up into two types (a) data on files and directories, and (b) data on other operating system objects. In type (a) there are grammar rules specifying how users refer to files and directories, and in type (b) there are rules for how users refer to other operating system objects. 4.1.1. Referring to actions In asking queries about operating systems users commonly use a well defined set of verbs or verb phrases. For example, if a user wants to know about removing files or directories he/she will use the following verbs and phrases: “delete”, “remove”, “get rid of”, “erase” and so on. Queries about copying may be referenced by: “copy”, “transfer”, and “move”. These phrases should be captured by the OSCON program. Phrases and words are stored under their categories in the KnowCon module of the program. It is possible to divide the set of operating systems commands into various categories. We have defined three major command categories and each of these have various subdivisions. The three categories are “listing”, “altering” and “compiling” commands. Listing commands are those which display information about the state of files in an operating system. Altering commands are used to alter the state of files in the system. Compiling commands are used to compile files in the system rather than display/alter them. There are two types of listing command, (1) screenlisting, and (2) printerlisting. Screenlisting commands are those which allow the user to see information on the screen and printerlisting will do the same for the printer. There are three subtypes of screenlisting command, (1) display-file (e.g.
94
PAUL MC KEVITT
“more”, “nroff”, “cat”), (2) display-file/directory-information (e.g. “ls”), and (3) display-system-information (e.g. “users”, “who”, “ps”, “jobs”). There is only one subtype of printerlisting command (e.g. “lpr”, “runoff”,3 “itroff”). Altering commands are of three types, (1) creating (e.g. “gemacs”, “vi”, “mkdir”), (2) removing (e.g. “rm”, “rmdir”, “kill”), and (3) transferring (e.g. “mv”, “cp”). Each of these can be applied to either files or directories. A third type of command is compiling commands. These commands cannot be categorized under altering commands as they are really utilities which do not change files but use them. An example of such a command is “run”. It is important to point out that certain commands apply to files which are not directories and others only apply to files which are directories. For example, “more” can only be used on files and not directories and “rmdir” can only be used on directories and not files. Such information can be used to specify the preconditions for commands and enables the system to detect errors in user queries and inform the user about such errors. For example, if a user asked, “How do I use “more” to display a directory?” the system could locate that the precondition for “more” is that it only works over files. Then the system could tell the user this information. The rules listed below are examples of typical action rules in KnowCon (bracketed items indicate input strings to be matched with). Rules [1] and [2] show typical verb phrasings used to reference the action of screenlisting. Rule [3] is for printerlisting and rules [4] and [5] for creating. [1] [2] [3] [4] [5]
4.1.2. Referring to objects Queries about operating systems often include reference to operating system objects like files and directories. Typically, users will refer to the object which is operated over by some command. The phrasing of the English query will dictate the object present. KnowCon has objects separated into two categories: (1) file/directory object specifications, and (2) other operating system object specifications. The rules listed below are examples of file/directory object specifications (bracketed items indicate input strings to be matched with and non bracketed items indicate non-terminals in the grammar). File object specifications include grammar rule definitions for files and directories. The first rule definition below [1] specifies that a file can be mentioned in a query by the word
THE OSCON OPERATING SYSTEM CONSULTANT
95
“file” preceded by three modifiers. The first modifier can be (1) a quantifier (e.g. “all”, “some of”), and the second (2) a possessive (e.g. “my”, “our”). Then filemod can be a modifier of type of file. Examples are “mail”, “device”, “plain”, “executable” and so on. This grammar rule will capture most of the ways that a user might refer to a file. The second rule [2] shows that a directory may have two modifiers and then the word referring to directory itself. Rule [3] shows that a file or directory could be in reference. This happens because it is ambiguous as to whether the intended referent is file or directory. The user, by using the plural of file may intend directory4 rather than the plural for file. Rule [4] defines location to be a triple: (1) preposition (prep), (2) modifier (mod), and (3) directory (dir). Phrases like “. . . in my directory”, “. . . in our directory”, and “. . . in the directory” will match here. There are more specific definitions of location for where the location is. Rule [5] is a definition of screen location in terms of (1) preposition (prep), (2) modifier (mod), and (3) screen-output (soutput). Preposition and modifier are self explanatory and output is the location of output. This could be “terminal” or “screen”. Rule [6] is the equivalent definition for a printer location. The output specification here is for a printer. Then poutput can be (1) “printer”, (2) “imagen” or (3) “laser writer”. [1] [2] [3] [4] [5] [6]
The second type of object definitions include objects other than those to do with files and directories. Some examples are shown below. Rule [1] shows the definition of a queue in terms of (1) a modifier, followed by (2) [printer, queue]. There are definitions of users by rules [2] and [3]. Rules [4] and [5] define names. The definitions of objects are simply definitions of the various ways which users refer to the objects. [1] [2] [3] [4] [5]
4.2. Knowledge for solving (DataCon) The knowledge for solving in OSCON consists of files of data that describe detailed information about operating systems. There are four types of knowledge stored here (1) basic command representation, (2) option representation, (3) concept representation, and (4) plan representation. 4.2.1. Basic command representation There are basically four types of information about any command held in the database. These are (1) preconditions, (2) effects or postconditions, (3) syntax, and (4) command names. Preconditions are lists of objects that are necessary for a command to be executed. Here are some examples of preconditions for commands from UNIX. Facts [1] and [2] show that “more” and “cat” have the precondition, “file”. The command, “mkdir” has the precondition “directory” and “cp” has no precondition. [1] [2] [3] [4]
Effects, or postconditions, are definitions of the outcome of commands. The effect is defined by a predicate which has a name and three arguments. The predicate name is the action and the arguments are (1) object, (2) object modifier, and (3) location. The facts below show some effects for UNIX commands (‘@’ acts as a slot filler where an extra slot is not required in some cases). Fact [1] shows the effect for the command “more”. The object for “more” is “file” and its modifier “contents”. The location of output of “more” is the “screen”. One case of the “cat” command [2] has the same effect as “more”. The other effect case of “cat” [3] is defined as concatenate and describes the concatenation of files together. The command “ls” will either display directory contents [4], or file information [5] on the screen. The displaying-information command “users” will display usernames on the screen. Fact [7] describes the “gemacs” command which creates files, and fact [8] the “rm” command which deletes them. [1] [2] [3] [4] [5] [6]
The syntax of commands is defined as a structure which contains the name of some command and then its syntactic definition of use. Shown below are some examples of the syntax for UNIX commands. The syntax facts are three place lists containing (1) Command name, (2) Optionname (filled in from context), and (3) Syntax description. [1] [2] [3] [4] [5]
4.2.2. Option representation DataCon also contains information for option specifications of commands. DataCon has files for (1) Option Preconditions, (2) Option Effects, (3) Option Names. There is no distinction for option syntax as this is practically the same for commands with options or not. Option preconditions are defined as three-place lists with the (1) Command Name, (2) Option Name, and (3) Precondition List. Shown below are a set of options for the various commands. The Precondition List contains the objects which must be present for the command to be executed. Facts [1], [2] and [3] show that each option for “cat” has “file” as precondition. Facts [4] and [5] show that for one option of “cp” (i) there is no precondition, and for the other option (r), “directory” is the precondition. Facts [6] and [7] show the preconditions for “ls”. [1] [2] [3] [4] [5] [6] [7]
The definition of the Option Effects are also stored as strings of English words. This is merely a convenience for generation. The definitions will enable the generator to give more detail about the specific effect of some option. Each fact here contains (1) Command Name, (2) Option Name, and (3) Option Effect with an English description.
98
PAUL MC KEVITT
[1] [2] [3] [4] [5]
[6]
opeffect(more, #, ‘‘set the window size to # lines’’). opeffect(more, c, ‘‘display each page after cleaning screen’’). opeffect(more, d, ‘‘prompt to hit space to continue at each screen’’). opeffect(cp, i, ‘‘prompt file name when overwriting’’). opeffect(cp, p, ‘‘preserve the modification information from the source’’). opeffect(cp, r, ‘‘copy each subtree rooted at that directory’’).
Option Name definitions are similar in spirit to the Command Effect specifications defined in section 4.2.1 above. However, the definitions here are for specific options. The option definitions have three arguments: (1) Option Effect defined as an action (object) specification, (2) the Command Name, and (3) Option Name. The first three facts [1], [2] and [3] show the option variations on “cat”. The various options allow the displaying of file contents in a specific manner. We also show the options for “ls” (facts [4]– [10]) and for “cp” (facts [11]–[13]). Note that a characteristic of options is that sometimes they have the same action as the main action (e.g. display and display(non-printing-characters)) while other times they have a different action (e.g. display and squeeze(blanklines)). [1] [2] [3]
4.2.3. Concept representation DataCon also contains a set of definitions of the possible concepts which a user may wish to ask about. Such concepts are considered to be operating system objects like files and directories, and concepts like piping and filtering. Concept representations are three place lists with (1) Concept name, (2) Concept name pluralized, and (3) English description of concept. We show the definitions for, “ada”, “working directory” and “unix” here. [1]
con([ada],[ada], ‘‘Ada is developed on behalf of the U.S. Department of Defense for use in embedded systems. Ada is the first practical language to bring together important features such as data abstraction, multitasking, exception handling, encapsulation and generics.’’).
[2]
con([working, directory],[working, directories], ‘‘The directory you are working in.’’).
[3]
con([unix],[unix], ‘‘UNIX Operating System manages the resources of your computer system to perform useful work on your behalf. It is composed of three major parts: the kernel, the file system, and the shell.’’).
4.2.4. Plan representation DataCon contains a list of possible Plans for command sequences which users may ask about. These command sequences are defined in terms of predicates which have five arguments. The first three arguments represent (1) an action, (2) an object, and (3) location of output. The last two arguments define the second action and its output location. We show two example rules below. Fact [1] represents displaying directories on the screen and then on the printer and fact [2] likewise for files. [1] [2]
5. Natural Language Understanding The most important part of any program which acts as a natural language interface is the natural language understander. There must be some means of
100
PAUL MC KEVITT
mapping a user query into a good meaning representation of that query. The problem, of course, is that there are very many ways of specifying queries in English. The first job of the natural language interface is to use some type of syntactic filtering which determines the type of query being asked. The filter will check queries for certain objects or phases and works as a fast mechanism for determining query type. The second job is that of determining the meaning of a query, or its content, once the type of query is determined. Let’s take a look at the two components of OSCON which tackle these two problems. 5.1. Parsing natural language queries (ParseCon) The parser called ParseCon has the job of determining the type of query present. There are characteristic ways of asking certain types of queries and the parser checks for these. The parser checks for the three basic types of query described in section 2. The ParseCon module has lists of the characteristic ways which people use to ask queries about particular query types. One of the query types checked for is request for explanation. There are a number of characteristic phrases which are used. Some of these are listed below. The first set of facts [1–4] and facts [1a–1e] define the syntax of what would come before and after some concept respectively. The second set of facts [6–11] only define possible uses of syntax before the concept but do not reflect any after-concept syntax. [1] [2] [3] [4]
wphrase([what, is, a]). wphrase([what, are]). wphrase([what, is]). wphrase([explain]). wphrase([describe]). wphrase([]).
THE OSCON OPERATING SYSTEM CONSULTANT
101
Another query type checked for is request for information[effect]. Here, again there are a number of possible characteristic phrases. We list some here [1–8]. Similarly, there are facts for (i) request for information[precondition], (ii) request for information[form], (iii) request for direction, (iv) request for explanation[command], and (v) request for explanation[concept] queries. Of course, there are syntax facts which will be compatible for all query types. However, this is not a problem because there are other characteristics of query types which separate them out. [1] [2] [3]
5.2. Determining query meaning (MeanCon) The function of the MeanCon component of OSCON is to determine query meaning. This is less of a problem for most query types than for request for direction queries. These queries involve complex phrasings of English to describe the effects which the user wishes to execute. The MeanCon component of OSCON has the function of determining the occurrence of objects in user queries. There are seven types of object searched for: (1) command name, (2) option name, (3) verb reference, (4) object, (5) object modifier, (6) object location, and (7) concept. MeanCon has a predicate called findcmd which searches for command names. The mention of command name is a good indicator of the type of query being asked. If a command name is present this indicates that the query is probably about (1) command preconditions, (2) command effects, or (3) command syntax. MeanCon also has a predicate called findopt which searches for mention of option names in queries. One of the most difficult types of user query to be understood by the system is request for direction queries. These are queries where the user
102
PAUL MC KEVITT
knows what he/she wishes to accomplish but does not know the command to do that. In these cases the user will specify, in English, some process, or effect, which he/she wants to be executed. The process/effect can be described with (1) a primary verb and (2) a secondary verb. The user will use a primary verb to describe the main action/command in question. A secondary verb may be used to describe a restriction on the main action/command. This restriction could be a definition of some option specification for the major command. Therefore, MeanCon has algorithms which check for (i) Primary Verb, (ii) Primary Verb Object, (iii) Object Modifiers, and (iv) Locations. There are also algorithms which search for (i) Secondary Verb, and (ii) Secondary Objects which usually describe option effects. In checking for request for direction queries the user will usually specify some action with a verb. This will be followed by the mention of an object such as a “file” or “directory”. There may be a modifier of the object such as “contents”. The location of the object may also be specified such as “printer” or “screen” or “directory”. There’s a predicate in MeanCon called findverb which searches for verbs in user queries. When a verb is located this will determine the major category of action/command. Therefore, the word “delete” will reference removing, “see” will reference displaying and so on. MeanCon uses the stored verbaction structures in KnowCon to find verbs in queries and their related action representations. Another predicate called findobj searches for the object of a verb. Say, for example, the user had asked, “How do I see a file?” then the findverb predicate will locate the verb “see”. The query is split up so that the phrase after the verb is checked for an object. The object “file” is located and marked. More complex queries may include modifiers and locations. Take the query, “How do I see my file contents on the screen?” In this case, the phrase “|. . . file contents on the screen?” is checked for an object by findobj. Then, “|. . . contents on the screen?” is checked for modifier by findmod. Findmod locates “contents” as a modifier. A predicate called findloc checks for locations in queries. Locations include “screen”, “printer” and “directory”. In our example, “|. . . on the screen?” is checked for location and “screen” is uncovered. MeanCon has a predicate called findcon which checks for concepts in user queries. For example, the existence for the concept, “ada” will denote a query asking about, “ada”.
THE OSCON OPERATING SYSTEM CONSULTANT
103
6. Rules for Solving (SolveCon) Now that we have defined the data available to OSCON before the system begins to process a query we move on to describe the rules which match user queries to database information. SolveCon determines the answer for a query by (1) building an uninstantiated formal query from the query, (2) matching this structure to the DataCon database, (3) retrieving data from the database, and (4) using the data to build an instantiated formal query which is passed back to the GenCon natural language generator. We shall describe the answering process in two steps: (1) the algorithm used by SolveCon to specify the query and to retrieve data from the database, and (2) the structure of instantiated Formal Queries returned by SolveCon to the GenCon generator. 6.1. The solving algorithm The Solver basically searches queries for three types of information: (1) Command Names, (2) English Descriptions of command effects, and (3) Concepts. The search process in conducted in the following order. [1] SolveCon checks to see if a command name is mentioned in the query. Then, (a) SolveCon checks if an option (e.g. -l) is mentioned. If (a) fails then (b) SolveCon checks if the query is request for direction[option]. This check is done by having ParseCon check the syntax of the query, having MeanCon check for an English Description of an option effect. If either (a) or (b) are satisfied SolveCon will retrieve from the database Option Preconditions, Option Effect, Option Syntax, and Option Name. If (a) and (b) have both failed then (c) SolveCon checks if the query is a request for information[precondition], request for information[effect] or request for information[form] query. Here, SolveCon checks the syntax again using ParseCon. If (c) fails, then (d) SolveCon checks the query for request for explanation [command] query. ParseCon is involved here too. If either (c) or (d) are satisfied SolveCon will retrieve Command Preconditions, Command Effect, Command Syntax, and Command Name from the database. If (d) fails then SolveCon moves on to step [2]. [2] SolveCon checks the query semantics. In this case the user must have asked an English query with no command names. (i) SolveCon has ParseCon check the syntax of the query. (ii) Then, SolveCon calls MeanCon to check for a Primary Verb, Verb Object, Modifier, and Location. SolveCon will retrieve Command Preconditions, Command Effect, Command Syntax, and Command Name from the database. Next, (iii) SolveCon has MeanCon check for a Secondary Verb (option action), and Secondary Verb Object. SolveCon will retrieve from the database Option Preconditions, Option Effect, and Option Syntax. If step [2] fails then SolveCon goes on to step [3]).
104
PAUL MC KEVITT
[3] SolveCon checks the query semantics. In this case the user must have asked an English query involving no command names. Also, the query must be about command combinations, or pipes, otherwise step [2] would have passed. SolveCon checks for the existence of a command combination in the user query. SolveCon has MeanCon check for the existence of a sentence connector like “and”. If this occurs then is it possible that the query involves command combination. SolveCon then calls the SolveCon algorithm again for (1) the piece of the query before the connector, and (2) the piece of the query after the connector. The data returned from (1) and (2) is integrated. If [3] fails then SolveCon tries step [4]. [4] SolveCon checks query syntax through ParseCon. Then MeanCon searches for concepts mentioned in the query. Examples of such concepts are “ada” and “protection”. 6.2. Structures returned The step of SolveCon which succeeds will return an instantiated formal query to the generator. In step [1], if cases (a) or (b) succeed, an instantiated formal query will be returned containing the following: (1) Option Preconditions, (2) Option Effect, (3) Option Syntax, (4) Option Name, and (5) Query Type. In step [1], if cases (c) or (d) succeed, the instantiated formal query contains: (1) Command Preconditions, (2) Command Effect, (3) Command Syntax, (4) Command Name, and (5) Query Type. In step [2] the formal query returned will contain slots for: (1) Command Preconditions, (2) Command Effect, (3) Command Syntax, (4) Option Preconditions, (5) Option Effect, (6) Option Syntax, and (7) Query Type. The complete structure will be instantiated when step [2] involves options. However, only parts (1), (2), (3), and (7) are instantiated when there is no mention of options. With step [3] a list containing two instantiated Formal Queries is returned. Each formal query will contain: (1) Command Preconditions, (2) Command Effect, (3) Command Syntax, (4) Command Name, and (5) Query Type. In step [4] a formal query with three pieces of information is returned. The structure contains (1) the Concept Name, (2) Concept Description, and (3) Query Type.
7. Natural Language Generation (GenCon) The final phase of the OSCON program is to map an instantiated formal representation into an English answer. There are two types of answer which may be returned to the user: (1) Stored English sentences describing some
THE OSCON OPERATING SYSTEM CONSULTANT
105
concept which are contained in the DataCon knowledge base, and (2) English sentences mapped out from instantiated Formal Queries. The natural language generator for the OSCON system is used to map instantiated formal queries into English answers. The generator has five primary components: [1] [2] [3] [4] [5]
psyntax: gives the syntax for a command peffect: gives the effect of some command pexample: gives an example on the use of some command ppre: gives the preconditions for some command ppipe: gives the commands involved in some piping example and an example of the piping
For each of the major query types various configurations of printing components are used. There are three types of request for information query: (1) request for information[precondition], (2) request for information[effect], and (3) request for information[form]. In these cases the components [1], [2], [3] and [4] are combined in order. Printing out the syntax for some command is trivial. The syntax is already stored in the DataCon knowledge base. This is just returned to the user. Printing out preconditions is quite trivial too as all GenCon has to do is to print those preconditions retrieved from the DataCon precondition information. Generation for request for information[effect] queries is more complex. GenCon will print command effects by (1) checking to see if the output should be in plan/pipe form, and if it is, then generating the answer in plan/pipe form; (2) generating the (a) Command Syntax, (b) Effect and (c) Preconditions for the command. The Effect is generated from the instantiated Formal Query produced by SolveCon which contains action, object, object modifier and object location. The latter information is generated in sentence form. Some interleaving information such as the output of prepositions between object modifier and location are handled too. For request for direction the latter algorithm is used. For request for explanation[command] components [1], [2], [3] and [4] are used. For request for explanation[concept], the answer is output from a stored piece of text. Often users ask queries about commands as concepts. The generation of these is simple as the definitions of such concepts are just stored as English descriptions in the first place. Therefore, all GenCon has to do is to map the stored sentences into English answers. We have written a simple algorithm which maps the English text into pretty format on the screen.
106
PAUL MC KEVITT
8. An Example In this section we show an example of how the query “How do I see my files with numbered lines?” is understood and answered by OSCON. First, SolveCon attempts to find out the type of query being asked. Initially, SolveCon tries step [1] to match the query as one mentioning a command and there is no match. Then step [2] is tried and a match occurs. The query is a semantic one. ParseCon is called forth and a syntactic match is found. “How do I” matches the syntactic form [how, do, i] for request for direction queries. Next, SolveCon calls MeanCon which analyzes the meaning of the sentence. (i) Findverb checks for a verb and gets “see”. From KnowCon, “display” is marked as the action. Then, (ii) findobj checks “How do I” and “my files with numbered lines” for objects. “Files” is matched as an object. Data from the query, i.e. display(file,contents,*) is matched against the DataCon database Effects and a match is found with display(file,contents,screen) This effect match from the query data to DataCon will allow SolveCon to retrieve (1) Command Preconditions ([file]), (2) Command Effect (display (file, contents, screen)), (3) Command Syntax (cat -n ), and (4) Command Name (cat). Next, the query is checked for existence of a secondary action. The findverb predicate retrieves “numbered” as a secondary action and its object is retrieved as “lines”. This representation is matched against the option database and number(lines) from the query matches number(lines) in the Option Effect definition in the database. The DataCon database is referenced and (1) Option Preconditions, (2) Option Effect, and (3) Option Name are returned. The data retrieved are integrated to form an instantiated formal query which is passed to the generator. The representation for this query is as shown below. cquery(display(file,contents,screen), ‘‘with numbered lines’’, [cat -n ], [file]), cat), -n).
Command Effect Option Effect Syntax Preconditions Command Name Option Name
The GenCon generator takes the formal query and maps it into an English answer: (i) psyntax will print out “cat -n ”, (ii) peffect will
THE OSCON OPERATING SYSTEM CONSULTANT
107
display “will display file contents on the screen.”, and (iii) popeffect will display “with numbered lines.” Therefore, the complete answer generated for this query is: ‘cat -n ’ will display file contents on the screen with numbered lines.
9. Current State of the Implementation The OSCON program can answer three major query types and replies to each in seconds: (1) request for information, (2) request for explanation, and (3) request for direction. There are three cases of type (1): request for information[precondition] request for information[effect], and (3) request for information[form] and two cases of type (2): request for explanation[command], request for explanation[concept]. In real terms this brings the query types covered up to six. The system answers the three query types with options in the case of UNIX. OSCON’s database contains information on 40 UNIX and MS-DOS commands with their respective preconditions, effects, syntax and command names. OSCON has 20 grammar rules for understanding the ways that users ask queries about categories of commands. OSCON also contains 10 plan sets for possible combinations of commands. 10. Comparison to Other Systems There are basically three other natural language consultants for operating systems. These are the Unix Consultant (UC), the Sinix Consultant (SC), and UCC/Yucca. The Unix Consultant (UC) (see Chin 1988; Wilensky et al. 1984, 1986, 1988) is a natural language consultation system for UNIX, and is in many ways similar in scope and intent to OSCON. However, there are some key differences. While OSCON is a specialized system with the sole goal of providing detailed expert help, UC has the broader goal of studying knowledge representation and planning (as well as natural language understanding) at a basic level. As such, UC takes a fundamentally different approach to knowledge representation, in that employs a much more general knowledge representation and planning mechanism, with a somewhat cognitive flavor. UC contains only one knowledge base of operating system concepts which must serve both the understanding and the solving phases of query resolution, while OSCON incorporates specialized representations for each function. Hence, UC must sacrifice some of the specialization built into OSCON.
108
PAUL MC KEVITT
Not unexpectedly, an examination of the examples presented in Wilensky et al. (1986) suggests that UC will have to perform far more extensive procedures to solve a given query, and that it may not be able to answer some of the very detailed queries that OSCON will address. UC is not intended to be a consultant which will help users migrating from one system to another. UC is a consultant for the UNIX operating system. We do not claim that the UC system could not act as a consultant on other operating systems. We claim that the Berkeley team has not concerned itself with putting information from more than one operating system in their program. It is a characteristic of computer operating systems that commands can be combined in various ways to execute complex processes. In Wilensky et al. (1986) (pp. 6–10) there are a number of examples of the queries which the UC system handles. These examples are intended to show the kinds and scope of requests answered by the system. None of the examples involve command compositions but operations of single commands. UC has no demonstrated ability for answering complex natural language queries which involve command sequencing. UC and OSCON are not truly competitors, but rather systems with different emphases. UCC is another natural language UNIX help system (see Douglass and Hegner 1982) UCC was a prototype system, implemented in Franz Lisp on a VAX-11/780. It used a simple natural language front end based on augmented transition networks. The development of a knowledge base and query solver were not advanced enough so that they could be linked with the front end. UCC generated answers to queries directly from concept case frames rather than from any particular formal language. There were two major problems with UCC: (1) as the front end included a relatively simple knowledge base it was unable to answer sophisticated queries with many constraints involving command options, and (2) the simple augmented transition network method of parsing was not sufficient enough to handle the types of queries posed by many users. A program called Yucca (Hegner and Douglass 1984) was an attempt to augment the UCC system in two ways. Yucca incorporated a much more sophisticated formal knowledge base and an improved natural language front end. OSCON’s design is based very much on principles from UCC and Yucca. The Sinix Consultant (SC) (see Hecking et al. 1988; Kemke 1986; and Kemke 1987) is a broad-based UNIX help system. It is similar to OSCON in that it is designed for the operating systems domain. Although SC contains a rich knowledge base, which reflects the technical aspects of the domain as well as the users view of the system, the focus of SC, like UC, is to provide help on the use of individual commands, rather than any interconnection of several. There appears to be no mechanism whereby the SC system
THE OSCON OPERATING SYSTEM CONSULTANT
109
will understand complex command sequences. However, it is predicted in Hecking et al. (1988) that future development of the SINIX Knowledge Base will include “combinators” for I/O-redirection and pipelining. There are several other consultation systems for operating systems, including CMS-HELP developed by Yun and Loeb (1984), TVX by Billmers and Carifio (1985), Wizard (see Shrager and Finin 1982; and Finin 1983), and USCSH by Matthews et al. (2000). Selfridge (1988) takes an expert system approach to knowledge representation. However, all of these systems appear to be either far less ambitious in scope than OSCON, or else have a totally different emphasis. In particular, all appear to employ simpler models of UNIX.
11. Conclusion and Further Work It is concluded that it is possible to build a computer program which will answer natural language user queries about operating systems. We have shown how this is done by the OSCON system. The OSCON program answers English queries for over 40 commands. Each query is answered within seconds. The development of any program like OSCON is enhanced if the program is built in a modular form where each module has a distinct function. This has been done for OSCON which is divided up into six distinct modules. This methodology allows for easy update of the program and also will allow the possibility of mapping the program over to a new domain. In comparing OSCON to the other two most related systems available today (UC, SC) OSCON turns out to have a different emphasis. The OSCON system is more concerned with the detail of operating systems and natural language processing rather than an experiment on cognitive modeling. Also, OSCON answers queries involving command combinations. Recently, some Wizard-of-Oz studies have been completed for the operating systems domain (see Mc Kevitt and Ogden 1989a). The Wizard-of-Oz technique is one where subjects interact with a computer through typed dialogue and are told that they are conversing with the computer. Subjects utterances are sent to another monitor where a “Wizard” sends back a reply to the subject monitor. Data were collected for 14 subjects who were naive UNIX users. Initial analyses of results look interesting. Seven types of queries have been discovered as opposed to the three types that were defined above a priori. Also, it has been noted that query type is strongly related to levels of user expertise. These data show that certain query types which were not decided a priori would not be handled by OSCON. Also, these query types would cause a major breakdown in communication in the natural language
110
PAUL MC KEVITT
dialogue. Further analyses of this data will be done to help determine the relationship between dialogue models and user models. There are three areas of further work proposed for OSCON which are all part of a dialogue interface to be added to the system. These are: (1) context storage mechanisms, (2) reference determination algorithms, and (3) user modeling capabilities (see Chin 1988). Dialogue management is important and it will permit the user to ask queries without having to spell the queries out in elaborate English. Initial thoughts on this research are reported in Ball et al. (1989). 12. Recent Developments There have been a number of developments with respect to the OSCON system since 1987. Analysis of Wizard-of-Oz data (see Mc Kevitt 1991; Mc Kevitt and Ogden 1989a,b) from natural language dialogues on UNIX gave nine query/intention5 types (see Table 1 below). The example UNIX queries in Table 1 are shown as they were actually typed by users. Our Wizard-of-Oz empirical data showed that expert subjects have mainly information and description intentions whereas less expert subjects have much more of the other intentions such as elaboration, explanation and repetition. Also, the data showed that there were statistically significant differences in the frequencies of intention types between expert and nonexpert subjects. Based on this grouping of intentions, and the fact that some intention sequences indicate less satisfaction than others, we proposed a weighted formula for a measure of satisfaction as follows (where [X → Y] represents the frequency of intention pairs from intention X to intention Y; ‘I’ denotes information intentions, ‘De’ denotes description intentions, etc.): 3([I → I ]) + [I → De] + [I → El] + [I → Ex] + [I → Re] +[De → I ] + [El → I ] + [Ex → I ] + [Re → I ] +3([De → De]) + [De → El] + [De → Ex] + [De → Re] +[El → De] + [Ex → De] + [Re → De] Relative dissatisfaction can be calculated by summing dissatisfaction intention pairs. The formula for a measure of dissatisfaction is: 3([El → El] + [Ex → Ex] + [Re → Re]) +2([Ex → El] + [Re → El] + [Re → Ex] + [El → Ex] +[El → Re] + [Ex → Re]) Two new components have been added to OSCON to include the capability of dialogue modelling and user modelling: DialCon and UCon. DialCon
THE OSCON OPERATING SYSTEM CONSULTANT
111
Table 1. Definitions of principal intentions for the consultancy domain Intention
General definition
UNIX domain definition
Information
An intention requesting a PLAN to achieve a specific GOAL where the GOAL is described.* E.g. “How do I cook this dish?”
An intention requesting a UNIX command to achieve a UNIX operation where the operation is described. E.g. “How do I print a file?”
Description
An intention requesting a description of an object or concept. E.g. “What is Persia?”
An intention requesting the description of UNIX concepts, objects, or commands. E.g. “what is UNIX?”
Instruction
An intention acting as an instruction to achieve a GOAL rather than the PLAN to achieve that GOAL. E.g. “Can you find out how many foreign nationals now live in Kuwait?”
An intention requesting the execution of a UNIX command. E.g. “has oscon been printed?”
Elaboration
An intention requesting more information on a PLAN or GOAL. E.g. “Could you tell me more about Iraq?” following “Where is Iraq?”
An intention requesting more information on UNIX commands, or UNIX itself. E.g. “how do i use more?” following “how do i see my file?”
Confirmation
An intention requesting confirmation of a belief or some PLAN believed to execute some GOAL. E.g. “Will sanctions stop Saddam Hussain?”
An intention requesting confirmation of a belief about the function of commands in UNIX, or the function of UNIX itself. E.g. “can i remove a directory with files in it?”
Explanation
An intention requesting explanation or clarification of an item which occurred during the execution of a PLAN for a GOAL. E.g. “Could you tell me what you mean by U.N. resolution 611?”
An intention requesting explanation of a response from the UNIX shell. E.g. “What does cp -r mean?”
Guidance
An intention requesting a PLAN for a GOAL where there is no explicit GOAL expressed. E.g. “What do I do next?”
An intention requesting help with UNIX operations, or UNIX, where there is no operation described. E.g. “I don’t understand what i’m supposed to do.”
Repetition
An intention which is a repeated request. E.g. “How many people live in the Gulf?” followed by “What number of people live in the Gulf?”
An intention repeating another intention. E.g. “How do I print a file” followed by “How do I get a print out of my file?”
Nointention
An intention which is not immediately relevant to the domain or not understood by the hearer as being relevant to the domain. E.g. “Where does Strider live?” in the domain of Economics.
An intention not understood by the system. E.g. “How do I eat a file?”
* A PLAN is defined as a set of actions to achieve some GOAL. A GOAL is defined as an operation a speaker wishes to achieve.
112
PAUL MC KEVITT
is a dialogue modelling component which uses to track query/intention sequences in dialogue. It updates an intention matrix by locating the relevant cell in the matrix representing a given intention sequence and increasing its count. DialCon indexes the cell in the matrix by pairing the current query/intention type with the previous. UCon is a user modelling component which computes the level of user satisfaction from the intention matrix and provides information for context sensitive and user sensitive natural language generation. UCon derives a binary measure of user expertise, expert and novice. UCon applies a usermodelling function, based on the weighted formulas above, to the intention matrix to determine levels of user satisfaction and dissatisfaction. Initially, the user is assumed to be an expert. Subsequent changes in the levels of satisfaction and dissatisfaction will result in changes in the level of user expertise. Such information is used by GenCon to generate context-sensitive and usersensitive natural language responses. OSCON’s capability of dialogue and user modelling is demonstrated in Appendix B. Also, we have developed a version of OSCON which acts as a command interface where after a user asks a given query then the system asks if he/she would like the resultant UNIX command executed (see Lim 1994). If the user replies “y” then the results of the execution of the command are reported.
Acknowledgements Acknowledgements are due to Louise Guthrie and Zhaoxin Pan for programming parts of OSCON. Stephen Hegner is thanked for many stimulating discussions. Yorick Wilks and the natural language processing team at the Computing Research Laboratory (CRL), New Mexico State University, USA are thanked for numerous comments on this work. Hans Brunner, Andy Parng and Scott Wolff of the Intelligent Customer Assistance Project at U S WEST Advanced Technologies are thanked for consultations on this research. This research was funded in part by U S WEST Advanced Technologies, Denver, Colorado under their Sponsored Research Program. Peter Norvig, Wolfgang Wahlster, and Robert Wilensky are thanked for comments on an earlier draft of this paper and Stephen Hegner for comments on a later draft.
Notes * This paper is effectively that already published as “Principles and practice in an operating system consultant”, in “Artificial Intelligence and Software Engineering (Vol. 1)”, Section on “AI Mechanisms and techniques in practical software”, Derek Partridge (Ed.), 1991, 439–462.
Appendix A This appendix contains a list of some of the queries which OSCON currently answers. The maximum time taken to answer any query is 2.5 seconds. The query sets have been divided into three major types (1) request for information, (2) request for explanation, and (3) request for direction. | ?- oscon. OSCON Program 1.0 (Sun-3, Sun Unix 4.2) Copyright (C) 1988, Computing Research Laboratory. All rights reserved. Dept. 3CRL, Box 30001, NMSU, Las Cruces, NM (505) 646--5466 U S WEST Advances Technology Please input your question ending with with a ‘?’ Write ‘quit.’ when you are finished ;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;; request-for-information[effect] → what does ls do? ‘ls <$directoryname>’ will display directory contents on the screen. → what happens with rm? ‘rm ’ will remove a file from your current directory. → what is the effect of date? ‘date’ will display a date on the screen. → what does more -c do? ‘more -c’ will display each page after clearing the screen. ;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;; request-for-information[form] → what is the syntax of rm? It has the syntax ‘rm ’.
114
PAUL MC KEVITT
→ How do i use cp? It has the syntax ‘cp ’. → how does he use more? It has the syntax ‘more ’. ;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;; request-for-explanation[command] → what is ls? ‘ls’ is a command. It has the syntax ‘ls ’. ‘ls <$directoryname>’ will display directory contents on the screen. → what is more? ‘more’ is a command. It has the syntax ‘more ’. ‘more ’ will display file contents on the screen. ‘more’ only takes the argument ‘file’ ;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;; request-for-explanation[concept] → what are aliases? alias is the ability to establish shorthand names for frequently used but long-winded commands. ;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;; request-for-direction → how do i see my file? ‘more ’ will display file contents on the screen. → how do i see my file on the screen? ‘more ’ will display file contents on the screen. → how do i see my file on the printer? ‘lpr ’ will display file contents on the printer. → how do i see all my files? ‘more ’ will display file contents on the screen. ‘ls ’ will display directory contents on the screen. → how do i see this file? ‘more ’ will display file contents on the screen.
THE OSCON OPERATING SYSTEM CONSULTANT
115
→ how do i see that file? ‘more ’ will display file contents on the screen. → how do i see his files? ‘more ’ will display file contents on the screen. ‘ls ’ will display directory contents on the screen. → how do i read my files? ‘more ’ will display file contents on the screen. ‘ls ’ will display directory contents on the screen. ;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;; request-for-direction → how do i list my files and print my files? ‘more ’ will display file contents on the screen. ‘lpr ’ will display file contents on the printer. To display file contents on the printer use ‘more | lpr’ → quit. over yes | ?-
Appendix B This Appendix demonstrates that the OSCON system modifys its natural language responses dynamically based on the types of utterances, and hence intention, that appear in the input. | ?- oscon. OSCON Program 1.0 (Sun-4, SunOS Release 4.1) Copyright (C) 1988, Computing Research Laboratory. All rights reserved. Dept. 3CRL, Box 30001, NMSU, Las Cruces, NM (505) 646-5466, USA. U S WEST Advances Technology Please input your question ending with with a ‘?’ Write ‘quit.’ when you are finished OSCON can answer queries for UNIX or MSDOS. Type one of these to the prompt. Do you want answers for UNIX or MSDOS? The default is UNIX [unix]: unix
116
PAUL MC KEVITT
How can I help you? → what does ls do? ‘ls ’ will display directory contents on the screen. information description elaboration explanation repetition
: : : : :
0 0 0 0 0
0 0 0 0 0
0 0 0 0 0
0 0 0 0 0
0 0 0 0 0
0 0 0 0 0
= = = = =
0 0 0 0 0
nointention : 0
0
0
0
0
0
=
0
Satisfaction = 0 Dissatisfaction = 0
== 0
OSCON displays the ‘intention matrix’ and levels of ‘satisfaction’ and ‘dissatisfaction’. Note that OSCON caters for 5 ‘real’ types of intention, and nointentions. The ‘intention matrix’ is a 2-dimensional, 6 X 6 matrix. The ‘intention matrix’ shows all intention-sequence counts to be 0 as only one intention has been entered into the program. Totals for each intention type are also shown. ‘Satisfaction’ and ‘dissatisfaction’ levels, computed from the matrix, using the user modelling function, are also 0. → how do i see my file? ‘more ’ will display file contents on the screen. information description elaboration explanation repetition
: : : : :
0 0 0 0 0
0 0 0 0 0
0 0 0 0 0
1 0 0 0 0
0 0 0 0 0
0 0 0 0 0
= = = = =
1 0 0 0 0
nointention : 0
0
0
0
0
0
=
0
Satisfaction = 1 Dissatisfaction = 0
== 1
This ‘intention matrix’ shows a total intention sequence count of 1, and the intention count for the sequence ‘explanation → information’ is set to 1. ‘Satisfaction’ also totals to 1.
THE OSCON OPERATING SYSTEM CONSULTANT
117
→ how does he see my file? ‘more ’ will display file contents on the screen. information description elaboration explanation repetition
: : : : :
0 0 0 0 1
0 0 0 0 0
0 0 0 0 0
1 0 0 0 0
0 0 0 0 0
0 0 0 0 0
= = = = =
1 0 0 0 1
nointention : 0
0
0
0
0
0
=
0
Satisfaction = 2 Dissatisfaction = 0
== 2
This ‘intention matrix’ shows a total sequence count of 2. Now, a ‘information → repetition’ sequence has occurred. Note that the syntax of this query does not have to be exactly the same as the previous query to denote a repetition. ‘Satisfaction’ increases to 2. → how do they see my file? ‘more’ is a command. It has the syntax ‘more ’. ‘more ’ will display file contents on the screen. ‘more’ only takes the argument ‘file’. information description elaboration explanation repetition
: : : : :
0 0 0 0 1
0 0 0 0 0
0 0 0 0 0
1 0 0 0 0
0 0 0 0 1
0 0 0 0 0
= = = = =
1 0 0 0 2
nointention : 0
0
0
0
0
0
=
0
Satisfaction = 2 Dissatisfaction = 3
== 3
A ‘repetition → repetition’ intention loop has occurred tilting the level of ‘dissatisfaction’ higher than ‘satisfaction’. As a reaction, more information is returned to the user. Remember that the user modelling function gives intention repetitions which are along diagonals a weight of 3.
118
PAUL MC KEVITT
References Ball, Jerry, Barnden, John A., de Ram, Sylvia Candelaria, Farwell, David, Guthrie, Louise, Guo, Cheng-Ming, Helmreich, Stephen, Mc Kevitt, Paul & Liu, Min (1989). The Need for Belief Modelling in Natural Language Processing. In Proc. of the International Conference on Cross-Cultural Communication (ICC-CC-89). San Antonio, Texas: Trinity University, March. Billmers, Meyer A. & Carifio, Michael G. (1985). Building Knowledge-based Operating System Consultants. In Proceedings of the Second Conference on Artificial Intelligence Applications, 449–454. Miami Beach, December. Chin, David (1988). Exploiting User Expertise in Answer Expression. In Proceedings of the Seventh National American Conference on Artificial Intelligence (AAAI-88), Vol. 2, 756– 760. Minnesota: St. Paul, August. Douglass, R. & Hegner, Stephen J. (1982). An Expert Consultant for the UNIX Operating System: Bridging the Gap between the User and Command Language Semantics. In Proc. of the Fourth National Conference of the Canadian Society for Computational Studies of Intelligence (CSCSI/SCEIO), 119–127. Saskatchewan, Saskatoon, May. Finin, Timothy W. (1983). Providing Help and Advice in Task Oriented Systems. In Proceedings of the Eighth International Joint Conference on Artificial Intelligence (IJCAI-83) , 176–178. Germany: Karlsruhe. Hecking, M. C., Nessen, Kemke, E., Dengler, D., Gutmann, M. & Hector, G. (1988). The SINIX Consultant – A Progress Report. Memo Nr. 28, Universitat des Saarlandes, FB 10 Informatik IV, Im Stanwald 15, D-6600 Saarbrucken 11, Germany, August. Hegner, Stephen J. (1988). Representation of Command Language Behavior for an Operating System Consultation Facility. In Proc. of the Fourth IEEE Conference on Artificial Intelligence Applications, 50–55. San Diego, CA: IEEE Computer Society. Hegner, Stephen J. (1998). Plan Realization for Complex Command Intercation in the UNIX Help Domain. In Hegner, Stephen, Mc Kevitt, Paul, Norvig, Peter & Wilensky, Robert (eds.) Intelligent Help Systems for UNIX. Dordrecht, The Netherlands: Kluwer Academic Publishers (this volume). Hegner, Stephen J. & Douglass, Robert J. (1984). Knowledge Base Design for an Operating System Expert Consultant. In Proc. of the Fifth National Conference of the Canadian Society for Computational Studies of Intelligence (CSCSI/SCEIO), 159–161. London, Ontario, May. Kemke, Christel (1986). The SINIX Consultant – Requirements, Design, and Implementation of an Intelligent Help System for a UNIX Derivative. Universitat des Saarlandes, KI-Labor (SC-Project), Bericht Nr. 11, October. Kemke, Christel (1987). Representation of Domain Knowledge in an Intelligent Help System. In Bullinger, H. J. and Shakel, B. (eds.) Human-Computer Interaction – INTERACT ’87, 215–220. Amsterdam: Elsevier Science Publications B.V. (North-Holland). Lim, Hun Keong (1994). Translation of OSCON into a Command Interface. Master’s Thesis, Department of Computer Science, University of Sheffield, Sheffield, England. Matthews, M., Pharr, W., Biswas. G. & Neelakandan, H. (2000). USCSH: An Active Intelligent Assistance System. In Hegner, S., Mc Kevitt, P., Norvig, P. & Wilensky, R. (eds.) Intelligent Helps Systems for UNIX. Dordrecht, The Netherlands: Kluwer Academic Publishers (this volume). Mc Kevitt, Paul (1986). Formalization in an English Interface to a UNIX Database. Memoranda in Computer and Cognitive Science, MCCS-86-73, Computing Research Laboratory, Dept. 3CRL, Box 30001, New Mexico State University, Las Cruces, NM 88003-0001, US.
THE OSCON OPERATING SYSTEM CONSULTANT
119
Mc Kevitt, Paul (1987). Artificial Communicators: An Operating System Consultant. Master’s Thesis, Computer Science Department, Dept. 3CU, Box 30001, New Mexico State University, Las Cruces, New Mexico 88003-0001, US. Mc Kevitt, Paul (1988). Rules of Inference in an Operating System Consultant. In Proc. of the First Irish National Conference on Artificial Intelligence and Cognitive Science (AI/CS-88), Vol. 1. Dublin, Ireland: University Industry Center, University College Dublin, September. Mc Kevitt, Paul (1991). Analysing Coherence of Intention in Natural Language Dialogue. Ph.D. Thesis, Department of Computer Science, University of Exeter, GB – EX4 4PT, England. Mc Kevitt, Paul & Wilks, Yorick (1987). Transfer Semantics in an Operating System Consultant: The Formalization of Actions Involving Object Transfer. In Proceedings of the Tenth International Joint Conference on Artificial Intelligence (IJCAI-87), Vol. 1, 569–575. Italy: Milan, August. Mc Kevitt, Paul & Pan, Zhaoxin (1989). A General Effect Representation for Operating System Commands. In Proc. of the Second Irish National Conference on Artificial Intelligence and Cognitive Science (AI/CS-89), pp. 50–65. Dublin, Ireland: School of Computer Applications, Dublin City University, September. Also in, Smeaton, Alan and McDermott, Gabriel (eds.), AI and Cognitive Science ’89, Springer-Verlag British Computer Society Workshop Series, 68–85. Heidelberg, Germany: Springer-Verlag. Mc Kevitt, Paul & Ogden, William C. (1989a). Wizard-of-Oz Dialogues for Help on Operating Systems. Memoranda in Computer and Cognitive Science, MCCS-89-167, Computing Research Laboratory, Dept. 3CRL, Box 30001, New Mexico State University, Las Cruces, NM 88003-0001, US. Mc Kevitt, Paul & Ogden, William C. (1989b). OSWIZ II: Wizard-of-Oz Dialogues in the Computer Operating Systems Domain. Memorandum in Computer and Cognitive Science, MCCS-90-181, Computing Research Laboratory, Dept. 3CRL, Box 30001, New Mexico State University, Las Cruces, NM 88003-0001, US. Selfridge, Peter G. (1988). How to Print a File: An Expert System Approach to Software Knowledge Representation. In Proceedings of the Seventh National American Conference on Artificial Intelligence (AAAI-88), Vol. 2, 380–385. Minnesota: St. Paul, August. Shrager, Jeff & Finin, Tim (1982). An Expert System that Volunteers Advice. In Proceedings of the National Conference on Artificial Intelligence (AAAI-82), 339–340. Wilensky, Robert, Arens, Yigal & Chin, David (1984). Talking to UNIX in English: An Overview of UC. Communications of the ACM 27(6): 574–593, June. Wilensky, Robert, Mayfield, Jim, Albert, Anthony, Chin, David, Cox, Charles, Luria, Marc, Martin, James & Wu, Dekai (1986). UC – A Progress Report. Report No. UCB/CSD 87/303, Computer Science Division (EECS), University of California, Berkeley, California 94720, US, July. Wilensky, Robert, Chin, David N., Luria, Marc, Martin, James, Mayfield, James & Wu, Dekai (1988). The Berkeley UNIX Consultant Project. Computational Linguistics 14(4): 35–84, December (also in this volume). Yun, David Y & Loeb, David (1984). The CMS-HELP Expert System. In Proceedings of the International Conference on Data Engineering, 459–466, Los Angeles: IEEE Computer Society.
USCSH: An Active Intelligent Assistance System MANTON MATTHEWS∗, WALTER PHARR1, GAUTAM BISWAS2 and HARISH NEELAKANDAN Department of Computer Science, University of South Carolina, Columbia, SC 29208, USA, [email protected]; 1 Current address: College of Charleston, Charleston, SC 29424, USA; 2 Current address: Vanderbilt University, Nashville, TN 37235, USA (∗ author for correspondence)
Abstract. This paper describes the knowledge sources and methods of knowledge acquisition for USCSH (University of South Carolina SHell). USCSH is an active intelligent assistance system for Unix.1 The system operates in two modes, the active mode and the intelligent mode. In the active mode USCSH monitors the user’s interactions with the system, and at appropriate times makes suggestions on how the user may better utilize the system to perform tasks. In the intelligent mode the system accepts questions in natural language and responds to them, taking into consideration the ability of the user and the context of the question. Keywords: assistance systems, shell, user-modelling
1. Introduction New software systems are constantly increasing in functionality, and the continued increase in hardware capability-to-cost ratio will serve to make the delivery of more sophisticated software possible. This makes an assistance system an essential component of these complex systems. Traditional approaches to assistance generally fall into one of several categories: command indexed systems (e.g., Unix man), hierarchical systems (e.g., VMS help), and tutorials (e.g., Unix learn). Regardless of the category these traditional approaches to assistance generally suffer from several deficiences. Responses are too verbose and are not adapted to the user’s level of knowledge of the system. On-line manuals must explain everything, and novices find them confusing, while more experienced users find it quite annoying to have to browse through a lot of irrelevant material to find the information they desire. Moreover, the systems only respond when the users realize that they need to know something, and only after they figure out the right keyword to use in the query (which places the burden of translating a request for specific help into command names on the user), or by browsing through menus (which
122
MANTON MATTHEWS ET AL.
can be quite time consuming). The problem with interactive tutorials (e.g., the Unix learn feature (Kernighan and Lesk 1976)) is that they are primarily designed for novice users, so they are rarely used for reference. In the day-today rush to get things done the user rarely finds time to devote to a “learning experience”.
2. Active and Intelligent Assistance Recently there has been substantial research addressing these deficiencies. The new approaches to assistance systems can be classified into two categories, passive and active, depending on how the interaction with the user is initiated. A passive system responds to user initiated queries, while active systems will at times initiate the interaction with the user. The idea is to emulate human consultants, who consider user proficiencies and the current context of their activities in formulating responses to queries. In addition, consultants aid a user’s learning process by indicating better and more efficient ways of getting a task done, especially when they notice the user doing things in an inefficient manner. To achieve such capabilities, online systems need to: 1. model individual users, i.e., keep track of their strengths and deficiencies, and use this information to occasionally make suggestions and help users improve their capabilities, 2. determine the context of users activities, and ensure responses cater to their individual needs and proficiencies, and 3. provide means so that user queries to the system need not be constrained by system commands and terminology. In our research, we adopt a knowledge-based approach in the design of USCSH, an active assistance interface for the Unix operating system. This system addresses the capabilities mentioned above, and therefore in many ways mirrors the capabilities of intelligent tutoring systems which explicitly structure, monitor and control the student’s learning processes (Clancey 1982). However, USCSH is designed to assist users and help improve their capabilities while they are involved in day-to-day activities, rather than only when they are in a tutorial environment. This requires that the system be unobtrusive, and not ask users too many questions, or provide unnecessary prompts while they are involved in routine activities. Therefore, our system differs from many tutoring systems (e.g., UMFE (Sleeman 1985)) which interrogate users to determine their proficiency on different system concepts while formulating responses to their original queries. This idea is reiterated in the design and implementation of computer coaching systems (Burton and Brown 1982).
THE USC SHELL
123
One of the major deficiencies of existing assistance systems is that users must know “what they need to know,” or at least must know command names or keywords that index into the information they need. Unix, as many have observed, does not use mnemonic names, and this makes it difficult and frustrating for new users to discover command names such as “cat” or “ls”. Natural language assistance systems overcome this by allowing users to phrase queries in more general terminology. The goal of such systems is to provide more flexibility, and allow the handling of terse requests such as “help on deleting lines.” There has been considerable work and progress in natural language assistance systems. With current complex systems many beginning users learn to use a minimal set of capabilities and are content to remain at this level of performance. For example, it has been observed that experienced Unix users know a small percentage of the commands available to them (Hanson et al. 1984).2 Their learning curve reaches a plateau and does not progress further because they feel that “they know how to use the system.” Their knowledge of the use of the system does not progress until it is triggered by another “nugget” of information (Matthews and Nolan 1985; Fischer et al. 1985). This trigger often occurs when observing another user, when another user makes suggestions or comments, and by casual (accidental) reading while looking for other information.
3. Criteria of System Design Two important criteria need to be discussed in assistance system design. The first criterion specified by Borenstein (1985), addresses the degree to which the system facilitates the accomplishment of a particular task by a user who does not currently know how to do it. The second criterion, which is equally important, refers to the effectiveness of the system in aiding users’ learning processes by providing mechanisms that enable them to improve their level of performance, as they are involved in their day-to-day activities on the system. Frequently the complexity of most interactive software systems (Unix, the vi editor, etc.) makes it a difficult task even for regular users to master the complete functionality of the system. Users pick up a small subset of capabilities and commands provided by the system so as to get by in performing routine activities, and because of other work pressures remain content to use the system at that level. Only when they encounter a problem that cannot be solved using their current set of known commands do they consult other sources, such as manuals and more experienced users, to learn about capabilities and features they were previously unaware off. Users may also learn additional features and capabilities by chance meetings and dialogues
124
MANTON MATTHEWS ET AL.
with more experienced users, or even by observing more experienced users at work. LEVI (LEarning VI) (Matthews and Nolan 1985; Matthews and Biswas 1985; Biswas et al. 1985), the first prototype active assistance system that we built at the University of South Carolina, addressed some of these issues. LEVI is an active assistance interface for the “vi” editor on the Unix system. It maintains models of individual users in terms of their proficiency in the usage of vi commands. The user model is an overlay model with the underlying expert model being a functional decomposition of vi capabilities. The tutoring component provides assistance to users in the form of prompts that are triggered by a set of scripts that identify some user misconceptions and inefficiencies in performing tasks, and a prompting mechanism that indicates more efficient ways of achieving the same task. The system also uses periodic prompts at regular intervals to make users aware of system capabilities they did not use in the past. Using the user model as a guide, LEVI selects periodic prompts which guide the user to new plateaus in performance and gradually would expose him to the entire domain. The goal was to have “an expert system in the interface” that would monitor the user interaction and guide the user in becoming more proficient in the use of the system. Based on these criteria and our experiences with LEVI, we feel that an effective assistance system should be developed using a framework that includes: 1. an ideal model that is used to explain system concepts and tasks to users, 2. a user model that keeps track of what the user is doing, and what he knows and does not know, 3. a natural language interface which can understand users’ queries posed in natural language, so that users are not constrained by system terminology and rigid query formats, and 4. a response generation mechanism that checks the user proficiencies and the context of the user’s activities before responding to a user query or prompting the user. The system architecture of USCSH is based on these principles.
4. Overview of USCSH The goal of the Active Assistance project at the University of South Carolina is to develop an active and intelligent user interface for the entire Unix system, which can overcome the shortcomings of current assistance systems. The aim, as with LEVI, was to have an expert system interface that would monitor the user interaction and by making appropriate suggestions guide the user in becoming more proficient in the use of the system. The second
THE USC SHELL
125
prototype system developed in this project is ‘USCSH’, the University of South Carolina SHell (Matthews and Biswas 1985). This active intelligent assistance interface, residing between the user and the Unix operating system, operates in both active and passive modes. In the active assistance mode the system continuously monitors user interactions with the Unix system, maintains user models, and at appropriate times makes useful recommendations to users about system capabilities in the context of their current activities. User commands are directly passed through to the Unix system and also posted on the assistance system working memory. The knowledge base for the overall system consists of a system description component and a comprehensive user model. The system description component contains a hierarchical organization of Unix shell (e.g., ls, chmod, cd) and utility (e.g., vi, mail, nroff) commands, typical tasks that users perform on the system (e.g., creating a document, debugging a program), and Unix system concepts (e.g., the concept of a pipe, the hierarchical nature of directories, or the role of the buffer in the vi editor). The lowest level in the hierarchy corresponds to specific commands and task names, whereas higher level nodes represent more general concepts and task descriptions. One of the key components of USCSH is the comprehensive user model. The central representational component of the user model is an overlay model (Carbonell 1970; Carr and Goldstein 1973; Clancey 1982), that is basically a collection of nodes that directly correspond to the nodes of the system capability hierarchy. A number of parameters, such as user proficiency and date last used, are associated with each node. In addition to the overlay structure, the user model contains a set of rules and procedures that are used for deriving user proficiencies when explicit information is not directly available from the user interactions. These rules are based on the Unix system hierarchy and assumptions made about a user’s typical learning process. A second component of the user model is an executable process model which represents the ideal and non-ideal models of user behavior. The ideal model contains plans that describe how more complex tasks can be executed as a sequence of simpler tasks and system commands and is represented as a hierarchy of frames (Minsky 1975). The non-ideal model (also called a buggy model by Brown and Burton (1978)) records plans that identify user errors and misconceptions which result in their performing tasks in an incorrect or inefficient manner. Based on sequences of user inputs and the (Unix) system’s response to these inputs, this model is used to infer what tasks the user is involved in, and whether he is making errors or choosing inefficient methods in executing these tasks.
126
MANTON MATTHEWS ET AL.
The active assistant is involved in two parallel but related activities as it tracks the user’s interactions with the system. First, it continually updates the dynamic user model to reflect the extent of his knowledge of the system, Second, it prompts the user on its own initiative, either periodically, or if it determines that the user is performing a task incorrectly or inefficiently. In the intelligent assistance mode the system responds to user queries on how to use the system. The first versions of the intelligent assistant were not full-fledged natural language processing systems, but used the menu/hypertext-like Screen Display System. The later version, written in LISP, uses an ATN parser to understand natural language queries. The intelligent assistance mode of USCSH is activated when the user types in the command “query”, which is treated as another C-shell command. The ATN-based parser integrates syntactic and semantic information, processes the query and translates it into an internal frame-based representation. The response generation mechanism interprets the query in the context of the user’s current activities and then generates responses that take into account the user’s proficiency in the use of the system. Response generation primarily entails matching the internal representation of the query against the frames that represent the plans of the ideal model, and retrieving a suitable frame or set of frames that contain the response to that query.
5. Active Mode There are two implementations of the active components of USCSH. One version, implemented in OPS5, runs as a separate concurrent process which communicates with the csh through a pipe. This first version illustrated the concepts but was slow (it took up to 3 minutes overhead on an NCR Tower), so we looked for ways to improve the performance. The second version is implemented by adding a “goal parser” to csh using a finite state machine implemented in C. In the active assistance system mode user commands are directly passed through to the Unix system for execution, and to the goal parser for analyzing user activities. The goal parser recognizes both efficient and inefficient plans for achieving goals. It uses the sequence of commands observed to that point in time to update its workspace of partial matches to plans. As the user types in additional commands, some partial matches are extended, others are discarded, and some are left alone. USCSH also updates the user proficiency model, incrementing nodes corresponding to observed commands, goals and concepts. After recognizing an inefficient sequence, USCSH determines if at this time it should become active and prompt the user. If the user model indicates that the user knows about what the system is about to suggest then
THE USC SHELL
127
the system will refrain from making that suggestion. This is illustrated in the third example in Section 5.1. 5.1. Examples of user interactions with USCSH (active component) Example RECOGNIZING USER INTENTIONS % % % %
vi prog.c cc -o prog prog.c prog model c_programming
After this sequence of commands USCSH recognizes the goal “c-program development.” Example RECOGNIZING WITH NOISE % % % % % % %
vi prog.c cd Maildir mail tech cd ls cc -o prog prog.c prog
Even with interspersed “noise commands” USCSH recognizes the goal “c-program development.” Example INEFFICIENCY MONITORING: FILENAME EXPANSION % rm test.o % rm test1.o % rm test2.o If the directory containing the files contained no more ‘‘.o’’ files then any appropriate prompt might be You can use the metacharacter ‘‘*’’ in the command rm *.o instead of the commands
128
MANTON MATTHEWS ET AL.
rm test.o rm test1.o rm test2.o Note that if there were more “.o” files in the directory or if the user was very familiar with the use of “*” then this suggestion would not be appropriate and would not be given. If there were more “.o” files in the directory then a different pattern might be used (if there is a sufficiently simple one). In addition to inefficiency prompts USCSH will at user-configurable intervals give periodic prompts. Periodic prompts are designed to gradually lead the user to new plateaus of performance in using the system. The idea of the sequence of plateaus of performance that the user is guided through is not unlike the “increasingly complex micro-worlds” of Fischer (1985). However, in USCSH we use both the inefficiency, error and periodic prompts to achieve this guidance where Fischer’s systems rely only on error/inefficiency messages. Our inclusion of the periodic prompt is primarily in the belief that inefficiency models can never be complete. Also, note that there may be some tasks that the user achieves via another mechanism altogether because he is unaware of capabilities of the system. An example would be using another system such as a MacIntosh to prepare transparencies, because one does not know of the capabilities of TEX and troff. 6. The User Model for USCSH The primary purpose of a user model is to represent what a user knows and what the user does not know. The user model of USCSH includes: 1. an overlay model which describes the system capabilities and concepts. 2. a process model which recognizes inefficiencies and higher level goals. Rich (1983) has presented a number of characteristics (or dimensions) that can be used to classify user models: 1. models for a single stereo-type user versus a collection of models for individual users, 2. models that are specified explicitly by users or system designers versus models inferred by the system based on the user’s interactions with the system, and 3. models of long-term user characteristics versus models of the current or more recent user activities. Sleeman (1985) added a fourth characteristic that is based on the nature and form of the information contained in the user model, and the reasoning processes used to infer that information. For USCSH to achieve the desired level of sophistication, it needs to have models for individual users and not
THE USC SHELL
129
just stereotypes. Furthermore, since it is designed to be an assistance system that aids users while they are engaged in their regular activities, the model should be derived almost entirely by observing individual user interactions with the system.
7. The Overlay Model The overlay model that is used in USCSH is an overlay on a functional decomposition of capabilities and concepts. It is basically a hierarchy of nodes, with each node representing what the user knows about the system capability or concept that corresponds to this node. The hierarchy is arranged with more abstract concepts/capabilities higher in the hierarchy and more specific concepts/capabilities further down in the hierarchy. Each Unix concept/capability has a corresponding node in the hierarchy that indicates how much the user knows or does not know about this concept/capability. Also, associated with each node in the hierarchy are two numbers that reflect the difficulty and importance of the concept/capability. The hierarchy and the importance and difficulty numbers are independent of the user and are fixed during the construction of the system. Each node of the overlay model contains a slot that stores an individual user’s proficiency in the use of the concept, command or task represented by the node. Also associated with each node are two numbers that reflect the difficulty and importance of the concept associated with the node. These numbers and the links in the hierarchy are static and independent of the user, being determined during the construction of the system. The individualized portion of the user model is the proficiency or mastery rating associated with the nodes. The proficiency ratings of the nodes for commands are updated by directly observing the use of these commands, giving an observed rating, ro . When the user is observed to be using a simple command correctly or incorrectly, the corresponding proficiency is incremented or decremented, respectively. The proficiency ratings of higher-level nodes that correspond to abstract concepts in the domain are derived using the system hierarchy and the importance and difficulty factors associated with the concepts, yielding a derived rating, rd . For a given node, rd is computed as a weighted sum of the ratings of its children. However, there exist some nodes in the hierarchy which could have both an observed and a derived rating associated with them. For example, the node associated with the concept of deleting a line in the visual editor “vi” has an observed rating which is based on the actual use of the “dd” command by the user. At the same time, it is reasonable to assume that a user who is very proficient with the visual editor also knows the concept of deleting a
130
MANTON MATTHEWS ET AL.
line, and though the system may not have observed the user performing “dd” commands, a derived rating, rd (dd) would be associated with the “dd” node. This number rd (dd) is a function of ro (vi). Ratings are integers in the range –128 to 127; 127 represents complete belief that the user knows the particular concept, –128 represents complete belief that the user does not know the concept, and 0 implies that the system has no information on the user’s proficiency in this concept or task. Also associated with each node are threshold intervals which depend on the importance and difficulty of the concept. The extent of the user’s expertise with the concept is indicated by the interval to which the rating belongs, this information being used by the assistance system to tailor its prompts and responses to the user. The first threshold interval represents a “no-information” interval, i.e., nodes whose ratings belong to this interval are nodes about which the assistance system believes it does not have sufficient information. For nodes with both ro and rd , first ro is used, and if this provides no information, then its derived rating rd is computed and used. The system, on its own initiative, periodically makes suggestions on topics that it feels the user needs to know. This is done by searching the overlay model for the “best” topic to prompt on. The best prompts, of a number of suggestions that may appear equally appropriate, are those that fill in the largest gaps in the user’s conceptual model of the system (Fischer et al. 1985), i.e., those that suggest new features and functions that the user is currently unaware of, and at the same time are related to his current tasks and activities. There are two decisions to be made in generating a periodic prompt: when to prompt, and on what to prompt. The when decision for periodic prompts is controlled via user specified parameters. The range of values this parameter can take varies from very frequent (one every half hour of connect time) to very infrequent (one every two weeks). Also, the method of delivery of the prompts can be chosen by the user. Periodic prompts may be displayed on the screen either at the start or end of a session, delivered via electronic mail, or stored in a suggestions file that users may browse at their leisure. The decision of what to prompt on is implemented using a recursive algorithm that starts at the top node of the system hierarchy and proceeds down into lower levels in a depth first manner, at each step selecting the “most promising” branch to pursue. The evaluation of each node for this purpose is based on its proficiency ratings, its importance and difficulty numbers, and the relevance of the associated concept to the user’s context of activities, which is determined by the ideal model. The first node encountered that has observed proficiency below a prespecified threshold is selected. Periodic prompting has been incorporated in USCSH, because prompting based simply on detection of incorrect or inefficient plans (which is the other
THE USC SHELL
131
occasion when the system prompts the user, and is discussed in the next section) is necessarily incomplete, and cannot cover all aspects of the assistance domain. Periodic prompts inform the user about system capabilities that he is not aware of, thus providing him a vehicle for expanding his conceptual model of the system and improving his overall performance. At the same time, periodic prompting, unless carefully controlled, could be an annoyance to any user, and it is for this reason that we leave the final decision about the prompting frequency to the user.
8. The Process Model The process model of USCSH has two components. The ideal component contains a set of rules that specify higher level goals such as “development of a C program” and “creation of a document”. The non-ideal component contains a set of rules to specify inefficient command sequences and erroneous commands. It also recognizes errors that are correct commands of other systems, such as ‘dir’ for VMS, DOS, CP/M. The ideal and non-ideal components of the process model have been developed by a detailed analysis of the day-to-day tasks that users perform on the Unix system. This can vary a great deal depending on the type of user and the kind of environment he works in. USCSH is currently tailored for users in typical office environments, who use the system mainly for document and memo preparation, communicating with other users, and probably maintaining a calendar of activities. This determines the type of Unix utilities that need to be included in the ideal model: the basic shell, file system, the visual and line-oriented editors, text formatters, the mail system and the calendar utility. The ideal component of the process model contains plans that describe how more complex tasks can be accomplished using a sequence of simpler tasks and commands, and has been implemented as a hierarchy of frames. For example, consider the task of “moving a block of lines in a file” using the visual editor. There is a frame associated with the concept of move. This frame contains a number of slots that represent the various objects in the domain which can be subjected to a move, e.g., files, and lines in a file. Each slot has a facet called command or task. A command slot means that the required action can be performed using a simple command, whereas a task slot implies that the particular task is executed as a sequence of sub-tasks. (A sub-task could be either a simple command or another sequence of sub-tasks.) The value associated with this facet represents either the simple command or the sequence of sub-tasks that have to be performed, as the case may be. In our example, the line slot has a task facet which contains the three elements
132
MANTON MATTHEWS ET AL.
(delete lines), (position cursor) and (put lines). Plainly stated, this plan says that “In order to move a block of lines in a file, delete that block, place the cursor to the desired new position, and put the block at this new position”. Now in order to determine completely how to move lines, the inferencing mechanism has to recursively examine the frames for the three new tasks, i.e., delete, position, and put, which will in turn contain the information regarding how those actions can be performed. This process thus corresponds to the traversal of a tree rooted at the concept that we started off with, viz., move, in this case, until we reach all the leaf nodes of the tree, i.e., frame-slots with only command facets. The information represented in the frames can be also used in reverse to determine what activity the user is currently involved in. For example, the frames discussed above tell us that if the user is observed to delete a block of lines, re-position the cursor, and then put that block while using the visual editor, then we can conclude that the user is moving a block of lines in a file. As another example, if a user goes through an edit-compile-run sequence of commands, then it can be inferred that he is trying to debug a program. Thus the ideal component of the process model can be used to establish the context of the user’s activities. This is important because, first, the prompting mechanism has to select a prompting topic that is most relevant to the context of the user’s current activities, and second, the intelligent assistant needs to refer to the context to resolve ambiguous references during query processing. The non-ideal component of the process model represents typical user errors and inefficiencies in executing tasks, and can be looked upon as a list or catalog of “bad” plans. Some of the user inefficiencies and errors have been derived by empirical observation; others were derived by pooling together our own intuitions and experiences in using the system.
9. The OPS-5 Version The first version of the process model of USCSH was implemented in OPS5, where the ideal and non-ideal plans are represented as OPS-5 production rules. The antecedent of a typical rule is the occurrence of a particular sequence of commands, and the corresponding conclusion is the recognition of either the context of the user’s activities, or an incorrect or inefficient usage of commands. An example of an inefficient plan to rename a file is given below: Example cp rm
THE USC SHELL
133
-> inefficient; You could have used the mv command. The above rule states that if the user makes a copy of a file, and then removes the original, then it would have been easier to use the rename command on the original file. However, this rule does not always apply, since there are occasions when the user has no option but to make a copy and then remove the original (e.g., when the user wants to change the ownership of a uucp file, or when the file indicated by the path‘file1’ has multiple links). How this possibility is handled is explained later in this section. Another inefficient plan, which pertains to the visual editor states that “if the user uses the dw command (to delete a word) immediately followed by the i command (to insert text), he could have been more efficient and used the cw command (to change a word)”. It is clear that for the above rules to fire, the commands should satisfy certain sequencing requirements depending on the rule. This sequencing information is extracted from the user’s interaction through the use of “timetags”. The OPS5 process is created as soon as the user logs on, and runs concurrently with the Unix shell. Information exchange between the shell and the OPS-5 process is accomplished via two-way pipes. Every command that the user types in is passed to the OPS-5 process as a working memory element in an “argc-argv” format that includes the command name, number of arguments, and the arguments themselves. Also associated with each command is a time-tag which indicates the recency of the command in the current session. This representation scheme thus gives us the exact sequence of commands that the user has used in the current session. The sequencing requirements that commands have to satisfy before they can qualify to participate in an inefficient plan are checked by comparing the time-tags of the commands. The comparison is done by LISP functions called from within OPS-5. Thus, the rules contain calls to external LISP functions which take the time-tags of the commands as arguments, and check if they satisfy the appropriate sequencing requirements. Other LISP functions are used to compute the length of an argument, look for arguments which have a common suffix, and identify path-names which have identical tails (e.g., /usr/accounts/grad/phil/report and report). The firing of a buggy rule implies that the user is performing a task inefficiently, and the prompting mode is activated. The user model is examined to determine whether the user is proficient and also whether he has already been prompted on this topic. In either case, the prompt is not displayed. For instance, if the user is known to be proficient at the “mv” command, and he still makes a copy and then removes the original, then he is perhaps trying to change the ownership of a uucp file, and it would be inappropriate to prompt
134
MANTON MATTHEWS ET AL.
him about “mv”. Context analysis may also help to corroborate this assumption. If a prompt is generated, then it is tailored to the level of proficiency of the user which is determined by the threshold interval in which the user’s proficiency lies. Thus we ensure that active prompting is appropriate to the user’s expertise and context of activities, maximizing the benefit derived from active prompting. The attachment of time-tags to commands provides flexibility in matching sequences of commands to goals, or users’ tasks. This is because “noise” commands that occur between commands in the actual sequence should be ignored. For example, in between the edit, compile and run sequence, the user may use a ‘ls’ command to list the contents of his directory, or use the ‘mail’ facility to communicate with other users. However, these intermediate commands are actually “noise” commands, since they do not actually alter the user’s goals, though they appear in the actual sequence. On the other hand, there are instances where intermediate commands are significant. For example, during the edit, compile and run sequence discussed above, if the user does a ‘copy’ (cp), or a ‘move’ (mv) into the current file, the conclusions of the plan sequence given above may no longer be valid. Such instances are specifically coded into the plan to inhibit its activation.
10. The Goal Parser Version Implementation of the process model as a OPS-5 production system enabled the development of a quick prototype for purposes of testing and analysis. However, as the number of plans in the ideal and deviant models kept increasing, there was a noticeable degradation in performance. For example, with approximately 50 ideal and deviant plans, it took about four to five times more time for the shell to return the prompt sign for the next command, after executing a simple “ls” or “cd” command on an NCR Tower-32 system running Unix System V with three users logged on simultaneously. This prompted the design of a more efficient match scheme for plan recognition: the use of a non-deterministic finite state automaton implemented in the C language, and directly integrated with the shell. This eliminated the communication overhead that had occurred between the shell and the OPS-5 process, which was implemented as a two-way pipe. This goal parser version of the USCSH was heavily influenced by its predecessor. The knowledge representation language is a simplified version of OPS-5, (S-OPS). The rules of S-OPS, like OPS-5, have a sequence of lefthand side elements, followed by an arrow “->”, followed by a list of actions on the right hand side. The left hand side elements are called “command instances”. Command instances consist of a command followed by a list
THE USC SHELL
135
of predicates and functions applied to the arguments of the command. The command the user types is used to determine the state transitions in the goal parser by matching it against the next command instance in partially matched plans (rules). The system will make multiple transitions if the command matches the next command instance of more than one of the partially matched plans. The actions that can appear on the right hand side of a S-OPS rule are quite limited. The only actions possible are “goal” and “inefficiency”. When either of these actions is performed the overlay model is updated and there are possible additional state transitions. In addition when a “inefficiency” action is performed, the user model is accessed to ascertain whether the user already knows the information and thus whether the prompt should really be made. To achieve the desired efficiency we needed to avoid the obvious overhead of the pipe that connected the OPS-5 and csh in the earlier version. Thus C was the clear choice of language because of the need to integrate the goal parser with the csh. It is fairly easy to update the inefficiency/goal knowledge base. The additions to the csh that vary as the knowledge base is modified are isolated in a few modules. An additional component to the makefile for the csh, makes these modules. A special tool was developed for generating the state transitions of the finite state automaton. This tool starts off with S-OPS productions as a representation of plans and converts them to the states and transitions of the finite state automaton. Normally each condition element of a S-OPS rule (i.e., a single component of a plan) corresponds to a state in the automaton. Since a number of different plans could have element subsequences that are identical, during the match process for identifying user goals or tasks, the automaton could be in many different states at the same time, and is non-deterministic in nature. For more details on the implementation of the automaton, the reader is referred to (Wang 1986). In summary, the active assistant of USCSH is like a human consultant sitting beside the user, monitoring his interactions with the system. It maintains a detailed model of the user which it updates continuously. It also keeps track of the current context of the user’s activities. Periodically, it informs the user about aspects of the system that he is unaware of. In deciding what topic to prompt the user on, it chooses one which might be most relevant to his current activities so as to maximize the benefits that accrue to the user as a result of learning about a hitherto unknown capability of the system. If it observes that the user is performing tasks incorrectly or inefficiently, it draws his attention to this fact, and presents a better method for achieving the same goal, but only after ensuring that the user does not already know about it. All these features make USCSH a fairly robust assistant, except for one capability, viz., being
136
MANTON MATTHEWS ET AL.
able to respond to specific queries of users which may be posed in natural language.
11. Intelligent Mode One of the problems with traditional assistance systems is that users often have to formulate their requests for help in terms of system commands and keywords which they may not remember or even know in the first place. This problem is made more acute in the Unix environment in which command names are not mnemonic of the actions they represent, and users find it even more difficult to remember odd-sounding command names. An effective assistant must, therefore, have the capability to interpret user queries which are posed in natural language, so that users are not burdened with the task of translating their requests in terms of system terminology. The intelligent mode of USCSH is a natural language assistance system. It was written in Franz lisp by Harish Neelakandan. The module is compiled and then is accessed as a built-in function of csh. The parsing mechanism uses an ATN framework (Charniak and McDermott 1985) with the grammar of Winograd (Winograd 1983) as a base. The queries are processed into an internal frame structure, that is then matched against a collection of method frames to produce the response. The natural language interface at this time is restricted to answering two large classes of queries; “how do I ” and “what is a /.” Note these are the deep representations that are currently handled. There are, of course, many surface variations that get translated down to these two classes of queries. For a more detailed discussion of the Intelligent Assistant component of USCSH see (Neelakandan 1987). The ATN-based parser converts user queries into an internal format using a backtracking parsing algorithm. The output of the parse is a register structure tree, which is different from the traditional phrase structure trees, and more closely related to the role structures used by Kaplan (1975). The register structure trees are represented as a collection of frames, thus, the intelligent assistant and the active assistant use a common representation scheme. One of the principal advantages of using the ATN framework is that it allows the embedding of both general and domain-specific semantic information into the syntactic parse phase without sacrificing the generality of the parser. Moreover, it can also be used to parse incomplete and, therefore, ungrammatical input, although it has traditionally been used as a syntaxdirected machine. This aspect is especially important in the domain of assistance systems, because users of assistance systems are primarily inter-
THE USC SHELL
137
ested in getting their work done, and find it frustrating to be constrained by rigid, grammatical query formats. Thus, the ATN provides a unified representation for both syntactic and semantic processing, and this is significant both from a theoretical standpoint, as well as the from point of view of realtime natural language systems. For details of how the ATN framework has been exploited to incorporate the features discussed above see (Neelakandan et al. 1987a). The parse query is first placed into one of various predetermined categories; these were compiled in (Pharr and Neelakandan 1986). The next step involves inferring the user’s goal, since goal analysis is important in the context of user assistance systems in order to understand or interpret indirect speech acts (Wilensky et al. 1986). The user’s goal is stored in a goal-frame. To resolve ambiguities or to make the goal more explicit, it may be necessary to analyze the context of the user’s current activities. The context determining mechanisms of the active assistant are then invoked through if-needed procedures associated with the goal-frame. For example, if the user asks: How do I move text? then it is possible to interpret this question in two different ways. By “text,” the user could either refer to a line or a block of lines within a file, or to an entire file itself. Each of these requires a different response, and therefore, the system differentiates them by storing them as distinct goals. In such a scenario, if the ideal component of the process model has determined that the user is currently in the edit mode, then it is likely that the user is referring to a moving lines within a file or between files, as opposed to the case of renaming a file or moving the file from one directory to another. The user model may also be able to provide some clues to resolve ambiguities. For example, if the model has evidence to show that the user is proficient in moving files, then his current question is more likely to pertain to the visual editor. Thus, both the context as well as the user model play an important role during the response generation phase of the intelligent assistant. Once it has been determined that the user is referring to moving a block of lines, the next question is whether he wants to simply relocate the position of the block within the same file or whether he wants to move this block to another file. Again, the user model can be used to make this decision. It is possible that the user model does not provide much help; in that case, the response will have to state both possibilities or prompt the user for a more specific question. The possibilities are numerous, and it is impossible to handle every particular case, but if the system is aware of different possibilities, and responds accordingly even in very general terms, users would tend to have more faith in the system. A specific follow-up question could then provide more details about what the user was actually trying to determine.
138
MANTON MATTHEWS ET AL.
After the user’s goal is established, the next step is to compute a method for achieving that goal, and this is done by matching the goal-frame to the plans of the ideal component of the process model. Considering again the question about moving lines within a file, the intelligent assistant matches the goal-frame created against the frames for move, delete, position, and put, and creates a method-frame which contains the necessary steps required to move a line. Finally, a generation component examines the method-frame and generates an English response using generic response templates. Throughout the generation process, the intelligent assistant constantly interacts with the user model and the context mechanism of the active assistant, and the Unix system itself in order to generate suitable responses to users’ queries. We do not elaborate on these aspects here (Neelakandan et al. 1987b); presents detailed examples of how the integrated system responds to users in a more intelligent manner and provides a friendlier environment for users to work in. In summary, the intelligent assistant plays the role of a human consultant who can respond to user queries on various aspects of the system. In doing so, the consultant keeps in mind various factors pertaining to the user and recent interactions in order to come up with pertinent responses to the queries. Therefore, users can pose fairly complex queries to the intelligent assistant as and when they need to get more information on specific aspects, and this helps them improve their overall performance in the use of the system.
12. Conclusions In this paper, we have described the architecture and implementation of USCSH, an active intelligent assistance system in the Unix domain. The system uses a multi-faceted user model: an ideal model that describes the preferred way of executing tasks, a non-ideal or buggy model that describes common user errors, misconceptions and inefficiencies, and a detailed overlay model that keeps track of what a user knows and what he does not know. In addition, the system keeps track of the context of the user’s activities as he is involved in his own work. Our approach is different from UC’s (Chin 1986) double stereotype scheme. We feel that our system provides a much more accurate representation of individual user proficiencies on individual components of the Unix system without an excessive increase in computational costs. Besides, our model also allows us to deal with idiosyncratic users; for example, a person who may an expert in the use of the visual editor “vi”, but a complete novice in using the “mail” utility. Another feature of USCSH is that it is far more integrated with the original shell (C shell) than similar systems that have been developed. To prevent
THE USC SHELL
139
degradation in performance with the increased computations that are required for the active assistant, we have developed an efficient matching scheme for goal and task recognition using finite state automaton techniques. The function that we currently use for computing derived proficiencies in the overlay model are ad hoc; they do satisfy some intuitive requirements, but they have not been tested extensively in real environments. Feedback obtained from users of this system will provide directions for modifying and fine tuning these functions. We are now closely monitoring the performance of the current prototype to determine it’s shortcomings and failures; these experiences will provide us with valuable feedback in expanding the capabilities of the process models. We also want to move away from a completely rule-based representation of the non-ideal component of the process model. What would be more desirable is to incorporate these rules as procedural slots in the frames that we use to represent the ideal component, thus integrating both the components of the process model into one uniform representation scheme. However, this implies that we would have to do some additional book-keeping in the processing of these procedural attachments. On the intelligent assistant side, the current implementation does not convey its own limitations to the user, and this is something which requires more work. Clearly, no natural language system is capable of parsing all natural language, but it would be helpful to the user if the system can give the user an idea of its vocabulary and scope of parsing capabilities especially when it fails to parse something, so that the user is aware of this and can adapt his querying style to maximize the percentage of successfully handled queries. Furthermore, we would like to eventually move away from the response templates and closer to natural language generation. In conclusion we feel that USCSH provides a conducive environment for users to gradually increase their proficiency in Unix, as they are involved in their day-to-day activities, and the positive help provided by the assistance system encourages them to use it as often as they desire.
13. Recent Developments Since the Berkeley workshop the team has spread out and research efforts have diversified also. The individuals of the project continue with work on modelling and intelligent agents and their applications to intelligent user interfaces. At South Carolina, Matthews has continued to lead a group working in intelligent user interfaces. With Pharr he has been worked in knowledge representation for natural language processing (NLP) and the application to dialogue systems (Pharr 1990). Matthews, Reid and Martin have worked
140
MANTON MATTHEWS ET AL.
on a hypercube based parallel parser for natural language interfaces (Matthews and Reid 1992). Matthews and Sobczak have worked with distributed intelligent agents for spectroscopy problems (Sobczak and Matthews 1990; Sobczak 1991). Biswas heads a group at Vanderbilt pursuing a number of research directions including: modelling of physical systems, (Mosterman and Biswas 1997), intelligent manufacturing and Intelligent Learning Environments.
Notes 1 Unix is a trademark of X/Open, Inc. 2 To be precise “know” should be replaced by “use”.
References Biswas, G., Matthews, C. T., Matthews, M. M., Rabon, P. & Wilhite, R. L. (1985). An Active Assistance System for vi. IEEE Intl. Conf. on Systems, Man, and Cybernetics, 746–750. Borenstein, N. S. (1985). The Design and Evaluation of On-line Help Systems. Tech. Report CMU-CS-85-151. Brown, J. S. & Burton, R. R. (1978). Diagnostic Models for Procedural Bugs in Basic Mathematical Skills. Cognitive Sciences 2: 155–192. Burton, R. R. & Brown, J. S. (1982). An Investigation of Computer Coaching for Informal Learning Activities. Intelligent Tutoring Systems, 79–98. Carbonell, J. R. (1970). AI in CAI: An Artificial Intelligence Approach to Computer Aided Instruction. IEEE Trans. on Man-Machine Systems 11: 190–202. Carr, B. & Goldstein, I. (1973). Overlays: a Theory of Modelling for Computer Aided Instruction. Int. Jour. Man-Machine Studies 5: 215–236. Charniak, E. & McDermott, D. (1985). Introduction to Artificial Intelligence. MA: AddisonWesley. Chin, D. N. (1986). A Case Study of Knowledge Representation in UC. Proc. Human Factors in Computing Systems 86. Clancey, W. J. (1982). ICAI Systems Design. Handbook of Artificial Intelligence 2: 229–235. Clancey, W. J. & Letsinger, R. (1981). Neomycin: Reconfiguring a Rule-Based Expert System for application to teaching. Proc. IJCAI, 829–836. Fischer, G., Lemke, A. & Schwab, T. (1985). Knowledge-Based Help Systems. Proc. Human Factors in Computing Systems 85: 161–168. Hanson, S. J., Kraut, R. & Farber, J. (1984). Interface Design and Multivariate Analysis of UNIX Command Use. ACM Trans. on Office Information Systems 2: 42–57. Kaplan, R. M. (1975). On Process Models for Sentence Analysis. Explorations in Cognition, 117–135. Kernighan, B. W. & Lesk, M. E. (1976). The Learn CAI systems. Unix System Documentation. Matthews, M. M. & Biswas, G. (1985). Oracle: A Knowledgeable User Interface. Proc. 9th COMPSAC Computer Software and Applications Conf., 358–363. Matthews, M. M. & Nolan, E. (1985). LEVI: A Learning Editor. Proc. 1985 USENIX Assoc., 325–331.
THE USC SHELL
141
Matthews, M. M. & Reid, R. (1992). Parallel Parsing of Ambiguous Languages on Hypercube Architectures. Lecture Notes in Computer Science 604: 539–545. Matthews, M. M. & Biswas, G. (1985). Raising User Proficiency Through Active Assistance: An Intelligent Editor. Proc. of Second Conference on Artificial Intelligence Applications, 358–363. Minsky, M. (1975). A Framework for Representing Knowledge. The Psychology of Computer Vision, 211–277. Mosterman, P. J. & Biswas, G. (1997). Formal Specifications for Hybrid Dynamical Systems. Proceedings of IJCAI-97, 211–277. Neelakandan, H. (1987). A Natural Language Interface to USCSH. MS Thesis, Department of Computer Science, Univ. of South Carolina. Neelakandan, H., Biswas, G. & Matthews, M. M. (1987a). USCSH: An Active Assistance Interface for Unix. with G. Biswas, and Harish Neelakandan, Proc. of the Intelligent Tutoring Systems 1988 (ITS-88), 334–341. Montreal, Canada. Neelakandan, H., Biswas, G. & Matthews, M. M. (1987b). An Intelligent Assistance System in the Unix Domain. Proc. Third Annual Expert Systems in Government Conference, 55–64. Pharr, W. (1990). A Knowledge Representation for Natural Language Understanding. The Third International Conference on Industrial and Engineering Applications of AI & Expert Systems, 859–865. Pharr, W. & Neelakandan, H. (1986). Natural Language Interface for Oracle. TR, Department of Computer Science, Univ. of South Carolina. Rich, Elaine A. (1983). Users are Individuals: Individualizing User Models. Int. Jour. ManMachine Studies 18: 199–214. Sleeman, D. H. (1985). UMFE: A User Modelling Front-End Subsystem. Int. J. of ManMachine Studies 23. Sobczak, R. (1991). Using a Distributed Blackboard Network for Solving Organic Spectroscopy Problems. Proceedings of the AAAI workshop on Blackboard Systems, July 1991. Sobczak, R. & Matthews, M. (1990). A Massively Parallel Expert Sytem Architecture for Chemical Structure Analysis. The Fifth Distributed Memory Computing Conference, 11– 17. Wang, Shin-Yung (1986). A Knowledge Based Model of an Active Assistance System for UNIX. MS Thesis, Department of Computer Science, Univ. of South Carolina. Wilensky, R., Mayfield, J., Albert, A., Chin, D., Cox, C., Luria, M., Martin, J. & Wu, D. (1986). UC – A Progress Report. BSD, Univ. of California at Berkeley. Winograd, T. (1983). Language as a Cognitive Process. MA: Addison-Wesley.
Editorial Intelligent Help Systems for UNIX: Natural Language Dialogue
This is the last of a series of three special issues on intelligent help systems for UNIX.1 This issue addresses natural language dialogue whereas the previous issues focussed on computational models and systems and planning and knowledge representation, respectively. In this collection, the focus is not upon issues of parsing and production per se, even though several of the systems described herein have significant capacities in this dimension. Instead, work here has evolved more intimately within the context of consultation systems, a topic seldom dealt with by other natural language systems. Nevertheless, the issues discussed are of general concern within natural language processing. In the first of two papers by Chin, the UCEgo component of UC2 is described. It implements an “active” intelligent agent with its own goals and plans to act on behalf of UC. UCEgo volunteers information, corrects user misconceptions and rejects unethical requests when appropriate. It adopts goals when it notices that the user lacks necessary knowledge, or has incorrect beliefs and then plans to volunteer information or correct the user’s misconception as appropriate. Plans are pre-stored skeletons that are indexed under the types of situations in which they are typically useful. Plan suggestion situations include the goal for the plan, plan preconditions, and appropriateness conditions. In his second paper, Chin presents the UCExpress answer-expression component of UC. Strategic issues in generation are the primary topic; specifically, the problem of expressing answers once they have been computed is addressed in detail. Following the philosophy of natural language systems that use an internal representation language, the machinations or thought processes of such systems are performed in terms of this language. Thus, in response to a user’s request, such a system formulates a set of propositions or a piece of network denoting the “conceptual response” to the user’s query. This conceptual response must then be conveyed to the user in natural language through a process of generation. However, direct translation of formulas in an internal representation language to natural language
278
EDITORIAL
is generally unwise. The propositions comprising a conceptual response are likely to contain much information that is already known to the user. In addition, there are usually several styles or formats in which a conceptual answer may be conveyed. Hence the natural language system must decide not only which conceptual response to convey, but also which format to convey it in. Chin calls this process of pruning and formatting “answer expression.” The result of answer expression is a new set of propositions to be conveyed to the user in natural language, a process of tactical generation which is not described herein. Actually, the term “answer expression” is unnecessarily restrictive. Given that an agent of any kind has something to express, the problem always arises of selecting both the appropriate portion to convey and a format in which to convey it. Thus, answer expression is not so much a process specific to answering questions in the act of consulting as it is part of a general process of deciding how to express oneself in context. Much of the analysis of answer expression in the paper appears to be formulated in a way that makes it applicable to the general case of expression, and not just the specific cases to which it is applied. Mayfield focusses on plan recognition in dialogue systems – the process of explaining why an utterance was made in terms of the plans and goals that its speaker was pursuing in making the utterance. He proposes three criteria for making judgements about the merits of explanations: applicability, grounding and completeness. Applicability refers to the needs of the system that will use the explanation, grounding refers to situating the explanation in what is known of the speaker and the dialogue, and completeness refers to how comprehensively the explanation covers the goals that motivated the production of the utterance. These criteria are applied in designing and evaluating a plan recognition algorithm and its associated knowledge base. The work of Martin is concerned with a very different natural language topic, namely, handling metaphorical language. The focus is the manner in which metaphors in the UNIX domain are represented in MIDAS (Metaphor Interpretation, Denotation, and Acquisition System), a computer program developed and based on the explicit representation of knowledge about metaphors. It may be surprising to find metaphoric language as a prominent concern of building systems that converse about operating systems. However, a compelling case is made that much of ordinary language, even that used in quasi-technical domains, is intrinsically metaphoric. In doing so, the traditional dichotomy between metaphoric and conventional language is challenged, and the notion of metaphoric word senses is introduced. Much of Martin’s work is concerned with representing the “conceptual metaphors” that underlie these word senses and which relate senses of various lexical items to one another. The most significant application of these metaphoric
EDITORIAL
279
representations is knowledge acquisition. In particular, it is shown how a system with such representations can hypothesize a new word sense of a word with which it is already familiar. Such a capability turns out to be especially useful in learning the language with which people talk about computational matters, since, in this domain, the use of word senses that are extended in precisely this manner seem to proliferate. This work is an important step in the direction of designing systems which can extend their own vocabulary, a problem that will become increasingly important as natural language systems become deployed in the real world. Quilici investigates how users’ mistaken beliefs about UNIX commands can be corrected. He shows how knowledge about specific UNIX-related plans and abstract knowledge about how beliefs can be justified can be represented and how it can be used to form justifications for advisor beliefs and to understand justifications given for user beliefs. Knowledge is captured in justification patterns, which are domain-independent knowledge structures that are similar to the abstract knowledge structures used to understand the point behind a story. Justification patterns enable the advisor to understand and formulate novel belief justifications, giving the advisor the ability to recognise and respond to novel misconceptions. To sum up, the problem of how to enable a consultant to become “active” and intervening when it believes the user to have misconceptions is one which must be addressed. It is illuminating to compare this requirement of an “active” consultant to that embodied in active (as opposed to passive) help systems, as described in papers in the first special issue in this series. The papers in the current issue also bring to the forefront the importance of planning in terms of both recognition and production for understanding and responding to the user’s utterances. Finally, we see how phenomena such as metaphor in language can even seep into specific domains like UNIX, and that for full understanding it needs to be processed. The articles in this issue are derived from papers originally presented at a workshop entitled Knowledge representation in the UNIX help domain, organised by Peter Norvig, Wolfgang Wahlster and Robert Wilensky at the University of California, Berkeley, USA, in December, 1987. The workshop was funded, in part, by the International Computer Science Institute and all participants were invited. The area of intelligent help systems for UNIX provides such a fruitful example domain of application for many AI techniques that we have decided to publish this work, which is still timely, more widely and particularly now where we have many spoken dialogue systems applied to such fixed domains (see Bernsen et al. 1998, BusinessWeek 1998).
280
EDITORIAL
Each article here has been reviewed by the editors and has been subsequently revised; furthermore, all authors have been asked to include a section on recent developments on their work. Related work which may be of interest to the reader can be found in Kobsa and Wahlster (1988) which focusses on user modelling and in Maybury and Wahlster (1998) which is a comprehensive publication on intelligent user interfaces. Members of the Kluwer editorial and publishing staff are to be thanked for their help in producing this issue. It would not have been possible without Melanie Willow (Journals Editorial Office), André Diepenbroek (Desk Editor, Editorial Department), and Vanessa Nijweide (Kluwer Publishing Editor, Humanities and Social Sciences). Melanie and André have worked very hard with authors and reviewers to enable this to happen. The Editors, October, 2000. Stephen J. Hegner Umeå University Sweden Paul Mc Kevitt University of Ulster (Magee), Derry Northern Ireland Peter Norvig NASA Ames Research Center California, USA and Robert Wilensky University of California at Berkeley USA
Notes 1. UNIX is a trademark of X/Open, Inc. 2. For an overview of UC (UNIX Consultant), see the paper of Wilensky et al. in the first special issue of this three-part series.
EDITORIAL
281
References Bernsen, N-O., Dybkjær, H. & Dybkjær, L. (1998). Designing Interactive Speech Systems: From First Ideas to User Testing Springer-Verlag: New York, NY. BusinessWeek (1998). Speech Technology is the Next Big Thing in Computing, Special Report. BusinessWeek, Feb. 23. New York: McGraw-Hill. Kobsa, A. & Wahlster, W. (1988). User Models in Dialog Systems. Springer-Verlag: Berlin. Maybury, M. & Wahlster, W. (eds.) (1998). Readings in Intelligent User Interfaces. Morgan Kaufmann Publishers: Los Altos, CA.
Planning Intelligent Responses in a Natural Language System DAVID N. CHIN Department of Information and Computer Sciences, University of Hawaii, 1680 East West Rd., Honolulu, HI 96822, U.S.A. E-mail: [email protected] Abstract. Intelligent help systems cannot merely respond passively to the user’s commands and queries. They need to be able to volunteer information, correct user misconceptions, and reject unethical requests when appropriate. In order to do these things, a system must be designed as an intelligent agent. That is, a system needs to have its own goals and then plan for these goals. A system which did not have its own goals would never refuse to help users perform unethical actions. Such an intelligent agent has been implemented in the UCEgo component of UC (Wilensky et al. 1984; Wilensky et al. 1988) (UNIX Consultant), a natural language system that helps the user solve problems in using the UNIX operating system. UCEgo provides UC with its own goals and plans. By adopting different goals in different situations, UCEgo creates and executes different plans, enabling it to interact appropriately with the user. UCEgo adopts goals when it notices that the user either lacks necessary knowledge, or has incorrect beliefs. In these cases, UCEgo plans to volunteer information or correct the user’s misconception as appropriate. These plans are pre-stored skeletal plans that are indexed under the types of situations in which they are typically useful. Plan suggestion situations include the goal which the plan is used to achieve, the preconditions of the plan, and appropriateness conditions for the plan. Indexing plans by situations improves efficiency and allows UC to respond appropriately to the user in real time. Detecting situations in which a plan should be suggested or a goal adopted is implemented using if-detected daemons. These daemons provide a single mechanism which can be used both for detecting goals and suggesting plans. Different methodologies for the efficient implementation of if-detected daemons are discussed. Keywords: agent, daemon, intention, meta-goal, meta-planning, planning, speech act, UNIX
1. Introduction Consider the problem of building a program that simulates a human consultant. A user would be able to come up to such a program and obtain advice in the program’s domain of expertise by entering queries in English (or some other natural language). The consultant program would then provide solutions in English. A user might ask for advice about how to do things, for definitions of terminology, or for advice in solving problems. In short, this program would behave like a real human consultant. In order to build such a system, one needs to satisfy at least three requirements. First, the computer system needs to be able to understand the user’s
284
DAVID N. CHIN
queries. Next, the program must be able to solve the user’s problems and formulate an answer. Finally, the system must be able to convey the solution to the user in a clear, concise manner. Needless to say, there are many difficult and unsolved problems in each of these areas. The first requirement, understanding the user, involves the whole of natural language understanding, a difficult area of artificial intelligence research. The second requirement, problem solving, has a long continuing history of research in AI. The last requirement, communicating the answer to the user, has a shorter history of research in AI, but is no less difficult a problem. However, even if all of the problems in each of the three areas were to be solved, and one could build a natural language consultation system that did each of the three tasks perfectly, that would still not be enough for a good natural language consultation system. A good consultation system also needs to be able to take the initiative in a dialog with the user, rather than always responding passively to the user. For instance, consider the following user interaction with a hypothetical program that provides advice on using the UNIX1 operating system: User: What does ls -v do? Program: It lists the contents of your current directory. The hypothetical program gives an answer that is literally correct, since the ls command actually ignores inappropriate flags such as -v. However, a consultant that provides only the above answer has failed to correct the user’s incorrect preconception that the ls command has a -v flag. So, although the user did not specifically ask whether ls has a -v flag, a good consultant would not fail to provide the information that in fact ls does not have such a flag. Such a response is shown in the next dialog: User: What does ls -v do? Program: There is no -v option for ls. This is a better answer even though it literally does not answer the user’s question. In deciding to ignore the user’s direct question and shift its attention to the user’s misconception, the program has taken the initiative in the dialog. This shows that a good consultant cannot just passively respond to the user; rather, it must have its own agenda. In the previous dialog, the better answer requires that the program realize that the user has a misconception. To do this, the system must first infer that the user believes that ls has a -v option and then realize that the user’s belief conflicts with the program’s knowledge. So, in general, a good consultant system must also model the user’s knowledge and beliefs. Even after a consultant system realizes that the user has a misconception, it must decide how to deal with the misconception. In the above example, the
PLANNING INTELLIGENT RESPONSES IN A NATURAL LANGUAGE SYSTEM
285
system decides that it should inform the user of the facts in order to correct the user’s misconception. In other cases, the system may choose to ignore the misconception, as in the following scenario: User:
How can I delete someone else’s file when I don’t have write permission on the file?
Program: I will not help you delete someone else’s file because that is unethical. In the user’s statement above, the user has assumed that one needs write permission on the file to delete it. This is not true. Rather, one needs write permission on the parent directory to delete the file. Regardless of what is the correct precondition, the program decides not to help the user because of ethical considerations. This also means that the program decides not to correct the user’s misconception, so as to avoid helping the user delete someone else’s file. This is an example of a decision by a consultant program to be uncooperative. Of course a good consultant program cannot arbitrarily decide to be uncooperative. In the previous case, the decision to be uncooperative was based on the fact that the user’s goal of deleting someone else’s file conflicts with the program’s goal of preserving all users’ files. In this case, the program’s goal of preserving files wins out over the program’s desire to help the user who asked the question. These sorts of goals and goal interactions are needed to guide a consultant system properly.
2. UC UC (UNIX Consultant) (Wilensky et al. 1984; Wilensky et al. 1988), is a natural language consultation program that answers the user’s questions on how to use the UNIX operating system. After understanding the user’s query, the UCEgo component determines how UC will respond to the user by detecting appropriate goals, planning for those goals, and then executing the plan. Detecting appropriate goals is described in more detail in (Chin 1987; Chin 1991). This paper describes the process of planning and execution in UCEgo.
3. Planning Speech Acts Natural language systems act primarily by communicating with the user. These communicative actions are called speech acts (Austin 1962; Searle 1969). A planner that produces plans consisting of speech acts has somewhat
286
DAVID N. CHIN
different requirements than other types of planners. First of all, speech act planners need to perform in real time in order to carry out a dialog with the user. This implies that such planners need to avoid inefficient search and backtracking by using real world knowledge to guide the planning process. 3.1. Other planners Planning has a long history in AI, starting from its origins as search within a problem space. In the GPS means-ends analysis formalism of (Newell and Simon 1963), a planner searches for a sequence of operators that allows the planner to move from an initial state in the problem space to the goal state of the planner. STRIPS (Fikes and Nilsson 1971) is an early example of a planner based on means-ends analysis. ABSTRIPS (Sacerdoti 1974) extended the formalism to work in hierarchical problem spaces. ABSTRIPS broke down a planning problem into a hierarchy of sub-problems and solved each sub-problem independently. Since sub-problems may not actually be independent, planners were developed by (Sussman 1975; Tate 1975; Warren 1974; Waldinger 1977; Sacerdoti 1977; Stefik 1980) and others that could handle planning when the ordering of plan steps is critical to the success of a plan. KAMP (Appelt 1981) applied this type of planner to the problem of planning natural language utterances. KAMP planned English sentences from the speech act level down to selection of actual words. The previous types of planners develop plans from scratch. This presents a problem, since such planning is computationally expensive. For example, it was not unusual for KAMP to take several hours to plan a complex utterance. Indeed, Appelt developed the TELEGRAM unification grammar (Appelt 1983) to improve the efficiency and modularity of planning at the linguistic level. Developing plans from scratch using “weak methods” such as search and theorem-proving leads to inefficient back-tracking and computationally expensive checking of preconditions for every plan step. These methods do not take advantage of available domain knowledge about the types of plans that are applicable in different situations. Another problem with general purpose planners that use “weak methods” is that their full computational power is not needed in planning speech acts. The OSCAR speech act planner (Cohen 1978) showed that a very simple planner that did not backtrack was sufficient for planning basic speech acts. Although OSCAR did not actually produce natural language output, it did plan speech acts at the conceptual level in enough detail to demonstrate the computational theory of speech act generation devised by Cohen and Perrault 1979). However, since OSCAR did not have to produce speech acts in real time in order to sustain a dialog with a user, it did not worry about how to plan efficiently. Given a goal, OSCAR merely looped through an ordered list of all
PLANNING INTELLIGENT RESPONSES IN A NATURAL LANGUAGE SYSTEM
287
potential actions until it found one whose effects matched the goal. Only after deciding upon an action would OSCAR test the preconditions of the plan and adopt as sub-goals any preconditions that OSCAR did not believe to already be true. If OSCAR could not satisfy a precondition sub-goal, then it failed in planning since it did not backtrack to try different actions. The fact that OSCAR worked fairly well despite this seemingly severe limitation shows that planning speech acts does not usually require complex planning. An alternative to planning from scratch using “weak methods” is presented by (Schank and Abelson 1977) in their theory of scripts and plans. In their theory, a script consists of a very specific series of steps, while a plan is more abstract and includes other components, such as preconditions on the use of the plan. The TALE-SPIN story generator (Meehan 1976; Meehan 1981) implemented this theory to produce plans for the characters in its stories. (Friedland 1980) and (Friedland and Iwasaki 1985) describe the MOLGEN and SPEX planners, which extended the idea of scripts and plans into a hierarchy of skeletal plans. Skeletal plans are pre-stored plans whose plan-steps may vary in generality from specific actions as in scripts to abstract sub-goals. They are similar to the MACROPs of STRIPS and the chunks of (Rosenbloom and Newell 1982). However, the latter systems emphasized learning chunks or MACROPs rather than the selection and instantiation of abstract plans. In a similar vein, (Carbonell 1986; Kolodner et al. 1985; Alterman 1986; Hammond 1986) have worked on adapting previous plans to new situations using techniques such as analogical reasoning, casebased reasoning, and utilization of a knowledge hierarchy to generalize old plan-steps and then respecify them to form new plan-steps. Using pre-stored skeletal plans makes for a much more efficient planner than one that has to plan from scratch using weak methods. However, the use of pre-stored plans presents a different efficiency problem: how to find the right plan for a specific situation. TALE-SPIN indexed scripts and plans under the goal of the script/plan and then looped through all possibilities, checking the most specific scripts first, until a workable plan was found. Similarly, MOLGEN and SPEX only looked at the UTILITY slot, i.e., the goal, of the skeletal plan. Since skeletal plans of MOLGEN and SPEX did not have preconditions, the planners could not even consider a plan’s preconditions to eliminate unsuitable plans. Instead, the planners considered all skeletal plans that fit the goal and returned all proper instantiations of those plans for the user’s consideration. (Hendler 1985) describes SCRAPS, a planner that used marker-passing to help make more efficient choices during planning. The marker-passing mechanism detected plan choices that might result in intra-plan sub-goal conflicts early in the planning process. The planner could then avoid the potential
288
DAVID N. CHIN
conflict by choosing an alternative and hence avoid inefficient backtracking. For example, SCRAPS was able to avoid backtracking while planning in the following situation: You are on a business trip (in a distant city). You wish to purchase a cleaver. By passing markers from BUYING, *ME*, and CLEAVER-27, SCRAPS found the following intersection path: BUYING ⇒ TAKE-TRIP ⇒ PLANE ⇒ BOARDING ⇒ WEAPONSCHECK ⇒ IF you go through WEAPONS-CHECK with a WEAPON, you get arrested ⇐ WEAPON ⇐ CLEAVER ⇐ CLEAVER-27 This path was then evaluated to determine that it represents a negative interaction, because it is a “bad thing” to get arrested. As a result, SCRAPS ruled out the choice of taking a PLANE (taking a BUS or TRAIN instead), and avoids the backtracking that would be required if the planner had chosen to take a PLANE. One of the problems with such a scheme is the length of the marker-passing path needed to detect important intersections. As longer paths are considered, there are more and more spurious intersections. If a marker-passer were to consider only short paths, then would it run the risk of missing important longer path-length intersections. With a path length of eight as in the previous example, any reasonably large knowledge base would produce a very large number of intersections, most of which would be spurious. Even if marker-passing were implemented in parallel, and all of the resulting intersections checked in parallel, it is uncertain whether it would be more efficient to plan using marker-passing or simply to plan in parallel (e.g., a planner could consider traveling by PLANE, BUS, and TRAIN in the previous example in parallel, and then evaluate the best plan later). With current serial machines, marker-passing would undoubtedly be less efficient. Another problem with SCRAPS is that it ignores other criteria for making choices besides potential negative sub-goal interactions. For example, choosing among the various travel plans in Hendler’s example should depend on much more than buying a cleaver. One would not want to take a bus or train if the business trip in the distant city were on the opposite coast of the United States. Choice criteria such as the distance of the trip are not preconditions in the sense that one could still take a bus or train for long distance travel, although one would not want to do so, unless one suffered from fear of flying or lacked the money for a plane ticket. In fact, most people would rather abandon the goal of buying a cleaver, rather than abandon the plan of taking the plane home from a distant city. Of course, most people in that situation would fix their plans by mailing the cleaver home or putting it in
PLANNING INTELLIGENT RESPONSES IN A NATURAL LANGUAGE SYSTEM
289
checked baggage (to avoid going through weapons-check with the cleaver). However, the latter is not a fair criticism of SCRAPS, since SCRAPS was not programmed with the knowledge needed to make such plan fixes. 3.2. Planning in UCEgo UCEgo attacks the problem of efficient planning of speech acts in several ways. First of all, like OSCAR, UCEgo uses a very simple planner that avoids inefficient backtracking. Secondly, like Meehan’s TALE-SPIN, Friedland’s MOLGEN planner, and Iwasaki’s SPEX, UCEgo uses pre-stored skeletal plans that efficiently encode knowledge about planning for dialog. However, unlike those planners, UCEgo selects for consideration only those skeletal plans that are appropriate to the current situation. A plan is judged appropriate if the goal of the plan is currently a goal of the system, and also if the preconditions and appropriateness conditions for the plan are satisfied by being true in the current situation. UCEgo indexes plans according to the type of situation in which the plans are useful. As a result, UCEgo does not waste resources in considering pre-stored plans that are inappropriate and hence will fail. Another difference between UCEgo and other planners is that UCEgo uses the idea of meta-planning developed by (Wilensky 1983) and first implemented in part by (Faletti 1982). Through meta-planning, UCEgo is able to handle positive and negative interactions among UCEgo’s goals and plans. This notion of meta-planning is different from that of (Stefik 1981), who used “meta-planning” in reference to his MOLGEN planner. Our notion of meta-planning involves the recursive application of a planner to solve its own planning problems (problems such as interactions among goals or plans). On the other hand, (Stefik 1981) used “meta-planning” to refer to a multilayered control structure that allows a planner to schedule its own planning tasks using a variety of strategies, such as least-commitment or application of heuristics. When planning problems arose, MOLGEN could only backtrack. It could not apply itself recursively to create and execute plans for handling the planning problems. The rest of this paper describes the processes of planning and plan execution in UCEgo. Section 3.3 describes plan selection in UCEgo, giving details on the types of situations in which different types of plans are suggested. Plan execution and other types of simple reasoning in UCEgo are described in Section 3.4.
290
DAVID N. CHIN
3.3. Plan selection UCEgo selects plans based on the current situation. Every plan in UCEgo has one or more associated situation classes. When the situation class associated with a plan matches the current situation, that plan is suggested to UCEgo. The suggestion of plans based on situations is implemented using if-detected daemons. If-detected daemons can be considered tiny inference engines that look for particular classes of situations. When the current situation matches the situation class of a daemon, it performs appropriate actions such as suggesting a plan. A simple example of an if-detected daemon used to suggest plans is shown in Figure 1. This daemon suggests the plan (PLANFOR1) of having UC exit (UC-exit1) whenever UC has the goal (UC-HAS-GOAL1) of exiting. The arrow(s) from the double-concentric circles labeled “if-detected” point to the detection-net of the daemon and the arrows pointing to the doubleconcentric if-detected circles represent the addition-net. The detection-net is somewhat like the antecedent of a rule and the addition-net is somewhat like the consequent of a rule. 3.3.1. Situation types Situations that suggest plans consist of many different types of information. A plan situation always includes the main goal of the plan. It may also include preconditions and other appropriateness conditions. For example, the plan suggestion situation for a USE-CHAIN-SAW plan might include the following appropriateness condition: need to cut thick branch (over 1 inch diameter). This appropriateness condition is not a precondition, since chain saws can be used to cut smaller branches. Indeed, when a user has already started using a chain saw to cut some thick branches, the user will often use the chain saw for smaller branches. However, if one had only small branches to trim, one would not think of using a chain saw (hedge shears are more appropriate). Adding such appropriateness conditions to a plan suggestion situation prevents the suggestion of inappropriate plans. The plan suggestion situations in UC can be divided into four main categories. These are situation classes that suggest: 1. inform-plans 2. request-plans 3. social-plans 4. meta-plans Situations that suggest inform-plans comprise those situations in which the planner wishes to inform the user of some information. Request-planning situations are those in which the planner wishes to request information from the user. Situations that invoke social-plans include salutations and apolo-
PLANNING INTELLIGENT RESPONSES IN A NATURAL LANGUAGE SYSTEM
291
if-detected
PLANFOR1? plan
UC-exit1?
goals
UC-HAS-GOAL1? goal
EXIT1?
Figure 1. Suggest plan of executing the UC-exit procedure when UC wants to exit.
gies. Meta-planning situations involve suggesting meta-plans for dealing with meta-goals. Each of these situation classes and the plans that are suggested to deal with those classes of situation are described in the following sections. 3.3.2. Inform-plans UCEgo suggests inform-plans whenever UC has the goal of having the user know something. There are two situation classes in which inform-plans are detected. The two classes are distinguished by the type of information that UC wants the user to know. If the knowledge is a network of real concepts, then UCEgo simply suggests the plan of communicating those concepts to the user. On the other hand, if UC wants the user to know something that is only a description of some concept(s), then UC should communicate to the user the concept(s) that is the referent of the description. For example, if UC wants the user to know “how to delete files,” then UC should inform the user that “the rm command is used to delete files.” “How to delete files” is a description of the concepts, “the rm command is used to delete files.” If UC were to communicate just the description to the user, that would not achieve UC’s goal of having the user know how to delete a file. UC needs to communicate the referent of the description to the user. As a result, UC needs to compute the referent before informing the user in situations where the type of information is a description. The two situation classes that suggest inform-plans are summarized in Table 1. The first part of the situation in both situation classes is the planner’s goal of having the user know something. The second part of the situation in the first class represents the precondition that the information should not be
292
DAVID N. CHIN
a description. The opposite is the case in the second class. Also, the second class of situation has the additional precondition that the description must have an identified referent. The actual if-detected daemons that detect inform-plan situations and suggest the plans are shown in Figures 2 and 3. Figure 2 shows the if-detected daemon for detecting situations in which UC has the goal (UC-HAS-GOAL3) of having the user know (KNOW1) something (SOMETHING1) that is not a description. Since descriptions in UC are always hypothetical (i.e., they cannot serve as referents), they can be detected by checking to make sure that the concept to be communicated is not hypothetical. Since hypothetical concepts are marked as being dominated by HYPOTHETICAL, this means that the if-detected daemon should check to make sure that whatever matches SOMETHING1 is not dominated by HYPOTHETICAL. This check is indicated by the NOT link from DOMINATE1 in Figure 2. In the second class of situations, UC wants the user to know a description for which UC has identified a referent. Figure 3 shows the if-detected daemon for detecting situations in which UC has the goal (UC-HAS-GOAL2) of having the user know (KNOW1) something (SOMETHING1) that is a description. The referent of the description is indicated by the ANSWERFOR relation between SOMETHING1 (the description) and SOMETHING2 (the referent). There is no explicit check to make sure that SOMETHING1 is indeed a description, because only descriptions participate in the ANSWERFOR relation. 3.3.3. Request-plans The request-plan is suggested by UCEgo whenever UC wants to know something, ?x, and additionally, UC believes that the user is likely to know ?x. The latter condition is not a precondition in the classical sense, since UC can still use the plan of asking the user even when UC does not believe that it is likely to work. Also, the fact that UC does not believe that the user knows the information does not mean that the plan will necessarily fail, since UC may be mistaken in its beliefs about what the user knows. In fact, when they have no better alternatives (or are just too lazy to try other alternatives), people will often ask someone else for information even when they believe that it is very unlikely that person knows the information. Thus the condition that one only asks someone one believes knows the information should not preclude use of this plan. This contradicts Schank and Abelson’s 1977 approach in which this is termed an uncontrollable precondition, meaning that this plan is always aborted, unless this precondition is true. Nevertheless, one does not normally ask someone for information when one does not believe that this person possesses the information. This is an example of an appropriateness condition
PLANNING INTELLIGENT RESPONSES IN A NATURAL LANGUAGE SYSTEM
293
Table 1. Situations that suggest inform-plans. Situation
Suggested Plan
Planner wants the user to know ?x, and ?x is not a description
tell the user ?x
Planner wants the user to know ?x, and ?x is a description, and ?y is the referent of ?x
tell the user ?y
if-detected
NOT PLANFOR1?
UC-HAS-GOAL3?
goals
goal
DOMINATE1?
KNOW1? knower dominated dominator
fact
plan
PERSON1? listener
SOMETHING1?
proposition
TELL1?
speaker MAYBE1?
UC
Figure 2. Suggest plan of telling user when UC wants the user to know a real concept.
for the use of a plan. UCEgo will not suggest the plan of asking someone for information unless UC believes that the person knows the information sought by UC. Whether or not the user knows the information sought by UC is modeled by KNOME, UC’s user modeling component (Chin 1987; Chin 1989). Since such information is often not represented explicitly in UC’s knowledge base, but instead is inferable from the user’s level of expertise, a call to KNOME is needed to determine whether or not the user knows something. Hence in
Figure 3. Suggest plan of telling the user the referent of a description.
the if-detected daemon for suggesting the request-plan, the appropriateness condition is coded as a test involving a call to the KNOME procedure, doesuser-know?. This is shown in Figure 4. 3.3.4. Social-plans Social-plans consist of salutations and apologies. Common to all situation classes that suggest social-plans is the planner’s goal of being polite to the user. If the planner did not wish to be polite, then there would be no point to either greeting the user or apologizing to the user. Salutations include greetings and farewells. UCEgo suggests the greeting plan, whenever UC first encounters someone (a precondition), and UC has the goal of being polite to this person. The plan of saying good-bye is suggested, whenever UC has the goal of being polite and also has the goal of exiting. Although there are two UC-goals in the good-bye plan’s suggestion situation, only one goal is satisfied by the good-bye plan. The good-bye plan is only a plan for being polite, since UC cannot exit merely by means of saying goodbye to the user. The goal of exiting serves as an appropriateness condition for
PLANNING INTELLIGENT RESPONSES IN A NATURAL LANGUAGE SYSTEM
SOMETHING1? Figure 4. Suggest the plan of asking, when it is likely that the user knows.
suggesting the plan of exiting. It is not a precondition, because the planner cannot plan to achieve the precondition before using this plan. It is not even an uncontrollable precondition, since it is a condition under the planner’s control. After all, if a planner has the goal of being polite to the user, then it might try to use the good-bye plan, and then decide to exit in order to satisfy this precondition of the good-bye plan. The if-detected daemon that suggests the plan of greeting the user is shown in Figure 5. This daemon is triggered, whenever UC has the goal of being polite to someone, and UC encounters this person for the first time. The daemon that suggests the plan of saying good-bye to the user is shown in Figure 6. The situations that trigger this daemon are those in which UC has the goal of being polite to someone and also has the goal of exiting. Social goals involving apologies are suggested when UC cannot fulfill its obligations as a consultant. This occurs either when UC cannot tell the user the solution to the user’s problem because UC does not know the answer to the user’s query, or when UC does not want to tell the user the answer. In the first case, UC apologizes to the user for not knowing. In the second case, UC apologizes to the user for not being able to tell the user (this is really a canned figure of speech, since UC actually is able to tell the user but just does not want to do so). The situations that suggest these plans of apology are summarized in Table 2. The actual if-detected daemons that detect situations calling for UC to apologize to the user are shown in Figures 7 and 8. In the first daemon, the
296
DAVID N. CHIN
PLANFOR1?
plan
goals UC-HAS-GOAL2? goal
if-detected
GREET1?
BE-POLITE1? listener
polite-to
proposition speaker
PERSON1?
is-polite
HELLO UC Figure 5. Suggest plan of greeting the user when encountering a new user.
PLANFOR1? plan UC-HAS-GOAL1?
goals UC-HAS-GOAL2?
goal
goal
EXIT1?
BE-POLITE1?
if-detected GREET1?
polite-to
listener proposition speaker
GOODBYE1?
PERSON1?
is-polite
UC
Figure 6. Suggest plan of saying good-bye to the user when exiting.
PLANNING INTELLIGENT RESPONSES IN A NATURAL LANGUAGE SYSTEM
297
Table 2. Situations that suggest plans of apology. Situation
Suggested Plan
Planner has goal of being polite to user, User has goal of knowing something, ?x, Planner does not know ?x
apologize to user for not knowing ?x
Planner has goal of being polite to user, User asked UC a question about something, ?x, Planner wants to prevent the user knowing ?x
apologize to user for not being able to tell user ?x
fact that UC does not know something has two possible sources. First, this fact may already be in UC’s knowledge base. Secondly, UCEgo may add such knowledge to UC’s knowledge base after one of UC’s other components (e.g., UC’s domain planner) has tried to solve the user’s problem and reports a failure. For the second daemon, the fact that UC wants to prevent the user from knowing something is usually the result of a preservation goal. For example, when the user asks UC how to delete UC, this will trigger the goal of preserving the UC program and hence the goal of preventing the user from knowing how to delete UC. This leads to a goal conflict for UC between wanting to tell the user in order to help the user, and wanting to prevent the user from knowing. In this case, UCEgo resolves the conflict by abandoning the goal of wanting to tell the user. The plan for resolving the conflict is described later.
3.3.5. Meta-plans Meta-plans are just like any other plans in UC. The only difference is that meta-plans tend to be useful for achieving meta-goals. An example of a metaplan is the plan of calling the procedure, UC-merge-goals, in order to satisfy the meta-goal of MERGE-REDUNDANT-GOALS. The if-detected daemon that suggests this plan is shown in Figure 9. The UC-merge-goals procedure takes two similar goals and merges them. UC-merge-goals first matches the two goals to see if they are identical. If so, the goals can be merged by simply discarding any one of the goals. A more complex case is when one of the goals is contained by the other goal. In such a case, UC-merge-goals discards the contained goal. For example, if the user asks, “Is compact used to compact files?” then UC adopts the following three similar goals:
298
DAVID N. CHIN
USER1? listener APOLOGIZE1? speaker UC plan
apology
if-detected
FALSE knower truth-val
HAS-GOAL1?
PLANFOR1? goals
goal
KNOW1?
goal
UC-HAS-GOAL2?
planner
BE-POLITE1?
fact KNOW2? fact
knower
SOMETHING1?
polite-to
is-polite
I USER1?
UC
MAYBE
Figure 7. Suggest plan of apologizing when UC does not know the answer.
UC-HAS-GOAL1? UC-HAS-GOAL2? Figure 9. Suggest plan of merging redundant goals.
1. UC wants the user know whether compact is used to compact files ⇒ UC wants the user to know that yes, compact is used to compact files. 2. UC wants the user to know the effects of the compact command ⇒ UC wants the user to know that compact is used to compact files. 3. UC wants the user to know how to compact files ⇒ UC wants the user to know that to compact a file, use compact. The similarity among the goals does not become apparent until after UC deduces the referent of the descriptions in the original goals. Although theoretically the order of merging goals does not make any difference in the final result, in actual practice the referents of the descriptions of the first two goals are found before the third, so the first two goals listed above are the first to be merged. In merging the first two goals, the second goal is contained by the first, so the goals are merged by simply abandoning the second goal. Next, after UC identifies the referent of the third goal, UCEgo notices that it is similar to the first goal (a similarity with the second goal is not detected, since the second goal has already been abandoned at this point). Once again, the third goal is approximately contained by the first goal (approximate in that “to compact a file, use compact” is represented as a PLANFOR relation, which is similar to but not identical to the HAS-EFFECT relation that is used to represent, “compact is used to compact files), so the two goals are merged by abandoning the third goal. These two merges leave only the first goal, which leads to UC’s answer of “Yes.” The propositional part of this answer is
300
DAVID N. CHIN
pruned by UCExpress, the component of UC that refines UC’s answer to the user. Another of UCEgo’s meta-plans is suggested when UCEgo detects a goal conflict and adopts the meta-goal of resolving the conflict. The appropriate meta-plan is suggested by the if-detected daemon shown in Figure 10. This meta-plan represents a call to the procedure, UC-resolve-conflict, which resolves the conflict by abandoning the less important of the two conflicting goals. To determine which goal is less important, UC-resolveconflict first searches for a direct precedence relationship (represented by a HAS-PRECEDENCE relation) between the two goals. If such a relation does not exist, then UC-resolve-conflict expands the search to include the causal parents of the goals. The search continues until the ultimate sources of the goals, which are usually UC themes, are included in the check for relative precedence relations. Since goal conflicts usually involve goals that originate from different UC themes, and, because all of UC’s themes have a relative precedence, UC-resolve-conflict is almost always able to decide which goal to abandon in order to resolve the conflict. An example of resolving a goal conflict is shown in the trace of a UC session shown in Figure 11(a–d). In this dialog, the user asks UC how to crash the system, which leads UC to adopt the following two conflicting goals: 1. UC wants the user know how to crash the system (UC-HAS-GOAL66). 2. UC wants to prevent the user from knowing how to crash the system (UC-HAS-GOAL67). The first goal is a sub-goal of UC’s goal of helping the user, which in turn originates from UC’s consultant role theme. The second goal is a sub-goal of UC’s goal of preserving the system, which in turn originates from UC’s staying alive life theme. UCEgo detects the fact that these two goals conflict, since UC both wants to achieve some state and prevent the achievement of that state. To resolve the goal conflict, UCEgo calls the UC-resolve-conflict procedure, which checks the relative precedence of the two conflicting goals and abandons the less important goal. The search for precedence terminates at UC’s Stay-Alive life theme and UC’s Consultant role theme. Since UC’s life theme has greater precedence than UC’s role theme, the UC-resolve-conflict procedure resolves the conflict by abandoning the goal of having the user know how to crash the system. Although UCEgo has abandoned the goal of having the user know how to crash the system, UCEgo still has the goal of being polite to the user. This leads UCEgo to the plan of apologizing to the user for UC’s inability to help the user. UCEgo suggests this plan in a situation where someone asks UC a question, UC wants to be polite to this person, and UC want to prevent that person from knowing the answer to the query. Similar plans calling for UC
PLANNING INTELLIGENT RESPONSES IN A NATURAL LANGUAGE SYSTEM
SOMETHING1? SOMETHING2? Figure 10. Suggest plan of resolving the conflict.
to apologize in order to be polite are suggested when UC does not know the answer and when UC cannot perform actions that the user requests. More details on these and other social plans can be found in Section 4.1.2. 3.4. Plan execution After UCEgo has suggested a plan for satisfying a goal, it must decide whether or not to execute that plan. UCEgo needs to decide to execute a plan, rather than always executing any suggested plan, because UCEgo might have to choose among several alternative plans that have been suggested.2 Also, UCEgo may have to change or even abandon a plan that interacts with another of UCEgo’s active plans. In order to find such plan interactions and correct them before it is too late, UCEgo separates planning and plan execution into two distinct phases of processing. The planning process, especially planning fairly simple plans such as those in UCEgo, can be considered a simple reasoning process. Other simple reasoning processes include figuring out which UNIX command to use for a particular purpose, recalling the effects of a particular UNIX command, or remembering the definition of a term. In UCEgo, each type of reasoning is initiated in the appropriate situation by an if-detected daemon. These are described below.
302
DAVID N. CHIN
# How can I crash the system? The parser produces: (ASK10 (listener10 = UC) (speaker10 = *USER*) (asked-for10 = (QUESTION10 (what-is10 = (ACTION14? (actor14 = *USER*)))))) (CRASH-ACTION0? (del-effect0 = (CRASH-EFFECT0? (crash-object0 = UNIX-SYSTEM))) (actor0-1 = *USER*) (cause0-0 = (ACTION14? ...)))
UC’s parser understands the user’s input as a question about a way to crash the UNIX system. The goal analyzer produces: ((HAS-GOAL-ga0 (planner-ga0 = *USER*) (goal-ga0 = (KNOW-ga0? (knower-ga0 = *USER*) (fact-ga0 = (ACTION14? ...)))))) UCEgo: suggesting the plan: (PLANFOR71 (goals71 = (HELP5 (helpee5 = *USER*) (helper5 = UC))) (plan71 = (SATISFY6 (need6 = (KNOW-ga0? ...)) (actor6 = UC)))) based on the situation: (UC-HAS-GOAL63 (status63 = ACTIVE) (goal63 = (HELP5 ...))) (HAS-GOAL-ga0 ...)
Since UC wants to help the user (UC-HAS-GOAL63), and the user wants (HAS-GOAL-ga0) to know (KNOW-ga0) something, UCEgo suggests the plan of satisfying the user’s goal of knowing.
Figure 11. (a). First part of UC dialog showing the meta-goal of resolving a goal conflict.
3.4.1. Intentions In UCEgo’s first phase of processing, it detects goals, suggests plans for achieving its goals, and adopts the intention of executing those plans. The intention of executing a plan means that UCEgo has scheduled the plan for execution during its second phase of processing, plan execution. There is one exception to this: when the intended plan is a sub-goal (i.e., the plan is to SATISFY some state), then UCEgo immediately adopts the desired state as a sub-goal in order to continue planning. The fact that UCEgo has adopted an intention does not mean that it cannot abandon that intention later. For example UCEgo may abandon an intention to carry out a plan if later UCEgo decides to abandon the goal which that plan is meant to achieve. UCEgo’s notion of intention is similar to Cohen and Levesque’s 1987a, 1987b usage of intention as a persistent (i.e., a commitment over time) goal
PLANNING INTELLIGENT RESPONSES IN A NATURAL LANGUAGE SYSTEM
Since the user wants to know how to alter (crash is a kind of altering) something that UC wants (UC-HAS-GOAL59) to preserve, UCEgo suggests the plan of preventing the user from knowing how to crash the system. UCEgo: detected the goal: (UC-HAS-GOAL66 (goal66 = (KNOW-ga0? ...))) from the situation: (UC-HAS-INTENTION10 (intention10 = (SATISFY6 ...)) (status10 = ACTIVE)) UCEgo: detected the goal: (UC-HAS-GOAL67 (goal67 = (PREVENT5 ...))) from the situation: (UC-HAS-INTENTION9 (intention9 = (SATISFY7 ...)) (status9 = ACTIVE))
UCEgo adopts the sub-goals of having the user know how to crash the system (UC-HAS-GOAL66) and preventing the user from knowing (UCHAS-GOAL67). UCEgo: detected the goal: (UC-HAS-GOAL68 (goal68 = (RESOLVE-GOAL-CONFLICT2 (conflict-goal-A2 = (UC-HAS-GOAL67 ...)) (conflict-goal-B2 = (UC-HAS-GOAL66 ...))))) from the situation: (UC-HAS-GOAL66 ...) (UC-HAS-GOAL67 ...)
UCEgo detects the meta-goal of resolving the conflict. UCEgo: suggesting the plan: (PLANFOR73 (goals73 = (RESOLVE-GOAL-CONFLICT2 ...)) (plan73 = (UC-resolve-conflict1 (goal-A1 = (UC-HAS-GOAL67 ...)) (goal-B1 = (UC-HAS-GOAL66 ...))))) based on the situation: (UC-HAS-GOAL68 ...)
UCEgo suggests the meta-plan of calling the UC-resolve-conflict procedure to resolve the goal conflict.
Figure 11. (b). Second part of UC dialog showing the meta-goal of resolving a goal conflict.
Since UC has the goal of preventing the user from knowing something that the user asked about, and UC has the goal of being polite to the user, UCEgo suggests the plan of apologizing to the user for not being able to tell the user in order to be polite. UCEgo: detected conflicting goals: (UC-HAS-GOAL67 (goal67 = (PREVENT5 (preventer5 = UC) (prevent-state5 = (KNOW-ga0? (knower-ga0 = *USER*) (fact-ga0 = (ACTION14? (actor14 = *USER*)))))))) (UC-HAS-GOAL66 (goal66 = (KNOW-ga0? (knower-ga0 = *USER*) (fact-ga0 = (ACTION14? (actor14 = *USER*)))))) UCEgo: goal, UC-HAS-GOAL67, has higher precedence, so resolving goal conflict by discarding the goal, UC-HAS-GOAL66 The planner is passed: ((CRASH-EFFECT0? ...)) The planner produces: nil
As it turns out, UC does not in fact know how to crash the system (the planner does not return a plan to achieve CRASH-EFFECT0). However, even if UC did know how, it would not tell the user, since it has abandoned that goal. Unfortunately, UC did not abandon the goal before it had already decided to call the UNIX domain planner, otherwise UC might save that step. In a sense, UC is thinking in parallel. On the one hand, it is continuing to plan for both conflicting goals, while at the same time it is planning how to resolve the conflict between the two goals. Potentially, any one of these processes might finish first. So, the planner might return a plan for crashing the system and UC might even adopt the plan of telling this to the user, before UC has resolved the goal conflict. However, since UCEgo separates planning and plan execution into two distinct stages, UCEgo will always abandon the goal (and its sub-goals) before it executes any plans such as telling the user how to crash the system. Then UC will not execute the plan, since its goal has been abandoned by UC. This separation of the planning and plan execution phases ensures that such subtle timing effects among parallel lines of thought will not present a problem for UCEgo. Figure 11. (c). Third part of UC dialog showing the meta-goal of resolving a goal conflict.
PLANNING INTELLIGENT RESPONSES IN A NATURAL LANGUAGE SYSTEM
305
The generator is passed: (APOLOGIZE2 ...) I’m sorry, I cannot tell you. UCEgo: do not know a single planfor the foreground goal: (UC-HAS-GOAL67 ...) so adding the meta-goal: (UC-HAS-GOAL69 (goal69 = (KNOW60? (knower60 = UC) (fact60 = ACTION15?)))) (PLANFOR75? (goals75 = (PREVENT5 ...)) (plan75 = ACTION15?))
The present version of UCEgo does not know how to prevent the user from knowing, so it adopts the meta-goal of finding out a plan for preventing the user from knowing. A more advanced version of UCEgo that did have plans for preventing the user from knowing (e.g., lying to the user, threatening the user, etc.) would not need to adopt the meta-goal of finding out such a plan. The planner is passed: ((PREVENT5 ...)) The planner produces: nil
Figure 11. (d). Fourth part of UC dialog showing the meta-goal of resolving a goal conflict.
to do an action. As in their notion of relativized intention, UCEgo abandons an intention when the motivation for the intention no longer holds. However, unlike their definition of intention, UCEgo does not worry about its own beliefs concerning commitment of the action. Cohen and Levesque’s 1987a, 1987b theoretical treatment of intention needed to be concerned about the beliefs of the agent since they wanted to be able to rule out the possibility that an agent might intend to doing something accidentally or unknowingly. In a real system, such as UCEgo, intentions are adopted as part of the planning process, so it would never accidentally or unknowingly adopt an intention to perform an action. Such concerns are more relevant to analyzing the intentions of other agents. Figure 12 shows the if-detected daemon that adopts intentions. Whenever UC has a goal (UC-HAS-GOAL1), there is a plan for that goal (PLANFOR1), and that PLANFOR is real and not hypothetical (implemented by the NOT DOMINATE1), then this daemon asserts that UC should adopt the intention of carrying out the plan. Unlike other systems that need to instantiate the abstract plans that are selected by the system, in UCEgo plans are automatically instantiated by
306
DAVID N. CHIN
PLANFOR1? goals1 plan1 dominated ACTION1?
SOMETHING1?
DOMINATE1? NOT dominator
goal
intention1 if-detected MAYBE1? UC-HAS-INTENTION1? status
UC-HAS-GOAL1? status
STATUS1? Figure 12. If-detected daemon that adopts the intention of executing a plan.
the if-detected daemons that suggested the plans. This is possible, because all information relevant to the plan, especially information needed to fully instantiate a plan, are encoded as part of the situation class in which UCEgo suggests the plan. For example, consider what happens when the if-detected daemon shown in Figure 13 is activated. This daemon suggests the plan of adopting the sub-goal (SATISFY1) of preventing (PREVENT1) the altering (ALTER-EFFECT1) of something (SOMETHING1) in situations where: 1. UC wants (UC-HAS-GOAL1) to preserve (PRESERVE1) that something (SOMETHING1). 2. someone else (checked by the NOT DOMINATE1 with dominator UCHAS-GOAL1) wants (HAS-GOAL2) to alter it. If the user tells UC, “I want to delete UC,” then this is interpreted as “the user has the goal of deleting the UC-program.” Since UC has the goal of preserving the UC-program, this daemon is activated. As a result, it creates a new instance of PLANFOR with goals being the goal of preserving the UCprogram and with plan being a new instance of SATISFY. This in turn has need being a new instance of PREVENT with preventer being UC and with prevent-state being deleting the UC-program. The final result is a completely specified version of the abstract plan stored under the if-detected daemon. So, since the plan suggested by the daemon is already completely specified, UCEgo does not need to further instantiate the abstract plan.
PLANNING INTELLIGENT RESPONSES IN A NATURAL LANGUAGE SYSTEM
307
DOMINATE1? NOT if-detected dominator dominated PLANFOR1? UC-HAS-GOAL1? plan goal
SOMETHING1? Figure 13. Suggest plan of preventing the altering of what UC wants preserved.
After there are no more goals to be detected, plans to be suggested, intentions to be adopted, or inferences to be made – that is, after there are no more if-detected daemons to activate – UCEgo proceeds to its next phase, executing those intentions that are still active. Since UC can only perform communicative actions, UCEgo only has to worry about producing output to the user. It does this simply by taking the concepts that it wants to communicate to the user and passing them to the UCExpress component. 3.4.2. Simple reasoning Besides planning for goals and executing the plans, UCEgo also performs other types of reasoning in certain situations. For example, when UCEgo has the goal of having someone (usually UC or the user) know a plan, it calls the UNIX domain planner component of UC. The if-detected daemon that does this is shown in Figure 14. Calling the domain planner to compute a plan for doing something in UNIX can be viewed in two ways. One might think of this as part of the plan for satisfying UC’s goal of having the user know how to do something in UNIX. In this view, the plan would consist of two steps: figuring out the answer, and then informing the user of this answer. This is technically correct, but it does not seem cognitively valid that a consultant has to do planning in order to figure out the answer, especially for the fairly simple queries that
308
DAVID N. CHIN
if-detected
UNIX-planner1?
UC-HAS-GOAL1? PLANFOR1?
goal1
user-goals1
goals1 KNOW1? fact
plan1 SOMETHING2?
EVENT1? Figure 14. Daemon that calls the UNIX domain planner component of UC.
UC can handle. When a human UNIX consultant is asked, “How can I delete a file?” it does not seem as if the consultant thinks, “I will figure out the answer and then tell the user.” Rather, the consultant seems to retrieve the answer from memory instinctively and then plans to inform the user of this answer. So, when a human consultant is told, “Don’t think about how to delete a file,” it is very hard for the consultant to stop the thought processes that lead to recall of the rm command. If humans had to plan to figure out this information, then it should be fairly easy to not execute the plan and so not think about how to delete a file. UCEgo takes the view that such simple thought processes are unplanned. That is, UCEgo does not plan to think and then think; rather, it always performs simple thought processes in appropriate situations. Since these simple thought processes do not lead directly to actions on the part of UC, they do not interfere with UCEgo’s planning process. Another example of a procedure that implements a simple thought process for UC is the recall of the definition of a term. The UC-define procedure is called by the if-detected daemon of Figure 15, whenever UC wants someone to know the definition of a term. Similarly, when UC wants someone to know the effects of some UNIX command, the if-detected daemon of Figure 16 calls the UC-find-effects procedure. When UC wants someone to know whether something is a plan for something else, UCEgo calls the UC-is-planfor procedure as shown in Figure 17. Finally, whenever UC wants someone to know whether some state holds, UCEgo calls the UC-is-state procedure shown in Figure 18.
PLANNING INTELLIGENT RESPONSES IN A NATURAL LANGUAGE SYSTEM
309
if-detected
UC-define1? UC-HAS-GOAL1? DEFINITION-OF1? goal1 unknown-term term definition KNOW1? fact
SOMETHING2?
SOMETHING1? Figure 15. Daemon for finding out the definition of a term.
Figure 16. Daemon for finding out the effects of a command.
4. If-detected Daemons There are two main problems in recognizing situations. First of all, situations are difficult to detect, because they consist of arbitrary collections of external and internal state. (Wilensky 1983) suggests the use of if-added daemons in detecting situations, but pure if-added daemons are problematic, because they can only detect a change in a single state. This is fine for situations that comprise only a single state. However, situations that consist of many states in conjunction are much harder to detect, because the various states are
310
DAVID N. CHIN
if-detected
UC-HAS-GOAL1?
UC-is-planfor1?
goal1 is-planfor KNOW1? fact PLANFOR1? Figure 17. Daemon for finding out whether some action is the plan for some goal.
if-detected
UC-HAS-GOAL1?
UC-is-state1?
goal1 is-state KNOW1? fact STATE1? Figure 18. Daemon for finding out whether some state holds.
usually not realized simultaneously. Because the different states that comprise a situation become true at different times, an if-added daemon that was activated by the addition of one particular state would always need to check for the co-occurrence of the other states. Also, to detect a multi-state situation, one would need as many if-added daemons as states. Each if-added daemon would be slightly different, since each would need to check for a slightly different subset of states after activation. The other problem in recognizing situations is how to do it efficiently. In any reasonably complex system, there are a very large number of possible
PLANNING INTELLIGENT RESPONSES IN A NATURAL LANGUAGE SYSTEM
311
internal and external states. Looking for certain situation types becomes combinatorially more expensive as there are more possible states and more situation types. Parallel processing would help, but parallel machines are not yet widely available. Even with parallel machines, optimization techniques can still be used to reduce the computational complexity considerably. This section describes how if-detected daemons can recognize multi-state situation classes and how they are implemented in an efficient manner in UC. 4.1. Structure of the daemon Like all daemons (Charniak 1972), if-detected daemons are composed of two parts: a pattern and an action. For if-detected daemons, these are called the detection-net and the addition-net respectively, since both the pattern and action in if-detected daemons are composed of a semantic network in the KODIAK representation language (Wilensky 1987). These daemons work by constantly looking in UC’s knowledge base for a KODIAK network that will match its detection-net. When a match is first found, the daemon adds a copy of its addition-net to UC’s knowledge base. An if-detected daemon is said to be activated when it finds a match for its detection-net among any of the KODIAK networks in UC’s knowledge base. Any particular KODIAK network is allowed to activate a daemon only once. This avoids the problem of a daemon being repeatedly activated by the same KODIAK network. The KODIAK networks of the detection-net and addition-net are not distinct, but rather may share concepts/nodes in their networks. In such cases, the if-detected daemon does not copy the shared node in the addition-net, but instead uses the concept that matched the shared node. A simple example of an if-detected daemon whose detection-net and addition-net share nodes is shown in Figure 19. Figure 19 shows the actual form of the daemon as it is entered into UC using the KODIAK graphic interface. This daemon is activated whenever UC has a background goal that is a goal sequence. In such cases, UC adopts as a new background goal the first step of the goal sequence. The detectionnet of the daemon is composed of those parts of the network that have arrows leading into the double circle labeled “if-detected” plus all concepts that are either its aspectual-values3 (i.e., the values of its aspectuals) or the aspectual-values of those concepts, recursively. In KODIAK diagrams, this corresponds to all nodes that have arrows pointing to the double circle or that can be reached by following arrows away from those concepts. This daemon’s detection-net consists of the concepts: UC-HAS-GOAL3, GOALSEQUENCE2, STATUS2, and SOMETHING2. The addition-net is similarly depicted, except that the arrow points from the double-circle toward the initial nodes. In this case, the addition-net consists of the nodes: UC-HAS-GOAL4,
312
DAVID N. CHIN
BACKGROUND-GOAL if-detected I
BACKGROUND-GOAL I
UC-HAS-GOAL4? status goal
UC-HAS-GOAL3? status goal STATUS2? GOAL-SEQUENCE2? goal-step
SOMETHING2? Figure 19. If-detected daemon for handling background goal sequences.
STATUS2, and SOMETHING2. Note that SOMETHING2 and STATUS2 are shared by both the detection-net and the addition-net. So, when a match is found, the daemon will create a new copy of UC-HAS-GOAL4 that will have as its goal whatever matched SOMETHING2 and as its status whatever matched STATUS2. 4.1.1. Comparing other daemons Although if-detected daemons look for the presence of particular configurations of KODIAK network in UC’s knowledge base, these configurations come into being predominantly4 when new concepts are created and added to UC’s knowledge base, rather than when pre-existing concepts are reconfigured (e.g., by changing the value of an aspectual). In this sense, if-detected daemons are similar to if-added daemons (Charniak 1972) that are activated when adding information to a data-base. The difference is that if-added daemons look only for the addition of simple patterns to the data-base, whereas if-detected daemons can handle arbitrary conjunctions5 of patterns. So, an if-detected daemon may be activated when concepts matching only a small portion of its detection-net are added to the data-base, provided that the rest of the detection-net is already matched by concepts already present in UC’s knowledge base. Another consequence of handling arbitrary conjunctions is that an ifdetected daemon may be activated many times by the addition of only one datum to the data-base. Such cases occur when that part of the detectionnet that is not matched by the added concept matches several distinct sets of concepts in UC’s knowledge base. For example, multiple activations
PLANNING INTELLIGENT RESPONSES IN A NATURAL LANGUAGE SYSTEM
313
can occur with a detection-net consisting of a conjunction of independent networks that we will refer to as net-A and net-B. Suppose that there are several conceptual networks in the data-base that match net-A, called A1, A2, and A3. Then, when a conceptual network, B1, matching net-B is added to the data-base, the if-detected daemon will activate three times, once each for A1 & B1, A2 & B1, and A3 & B1. If-detected daemons can also handle negations. This means that the daemon is activated by the absence of data matching the pattern that is negated. Usually, only a part of the daemon’s detection-net is negated. In such cases, the daemon looks for the presence of concepts matching that part of the detection-net that is not negated, and then for the absence of concepts matching that part of the detection-net that is negated. Since the detectionnet and addition-net of if-detected daemons are both KODIAK networks, the negated parts of the detection-net may shared concepts/nodes with the nonnegated parts. In such cases, the shared nodes serve as additional constraints on the negated parts of the detection net in that the daemon need only detect the absence of KODIAK network where the shared nodes have been replaced by their matches. Although if-detected daemons can handle both conjunctions and negations and so should be able to detect any situation, it is still useful to have procedural attachment for if-detected daemons. This is because not all knowledge is represented explicitly in knowledge bases; some knowledge is only inferable from the knowledge bases. Such inference procedures are often complex, so it is often undesirable to encode the procedures as daemons. An example of a daemon with an attached procedure is shown in Figure 4. This daemon detects the plan of having UC ask someone a question about something, whenever UC believes that the person knows what UC wants to know. The arrow labeled “TEST” indicates a procedure attached to the daemon. In this case, the procedure is an instance of the does-user-know? procedure, which represents a call to KNOME. This call is necessary, because whether or not some user knows some fact may not be explicitly represented in the knowledge base, but may instead be inferable from the user’s level of expertise. Such inferences are made by the does-user-know? procedure of KNOME. After the daemon has detected that UC has the goal of knowing something and that there is someone present, then KNOME is called via the procedure to see if that person knows what UC wants to know. If so, then the test completes the activation of the daemon, and the plan of asking that person in order to find out what UC wants to know is added to UC’s knowledge base. Besides calls to procedures that test for input, if-detected daemons also allow calls to procedures in their output, i.e., in their addition-nets. An example of this is shown in the if-detected daemon of Figure 14. This
314
DAVID N. CHIN
if-detected daemon is used to call the UNIX Planner component of UC whenever UC wants to know some way to do something. UNIX-planner1 is a kind of procedure (i.e., it is an instance of the PROCEDURE category in KODIAK terminology), so the daemon knows that it should not just copy the node, but should also call the procedure UNIX-planner with the arguments being whatever matched SOMETHING1. This capability of if-detected daemons makes them less like pure daemons, which only add information to their data-base, and makes them more like production systems. The essential difference is that if-detected daemons are embedded in a full hierarchical conceptual network representation system, namely KODIAK, whereas most production systems allow only first-order predicate logic representations. 4.1.2. An example The following example will show in detail how if-detected daemons work. Consider the if-detected daemon shown in Figure 7. This daemon is activated whenever: 1. A user wants to know something; and 2. UC does not know it; and 3. UC wants to be polite to the user. In such situations, the daemon will add the fact that a plan for being polite to the user is for UC to apologize to the user for not knowing. The detectionnet of the daemon encodes the situation and consists of the concepts: HAS-GOAL1, KNOW2, SOMETHING1, KNOW1, UC, FALSE, UC-HASGOAL2, TRUE, BE-POLITE1, and USER1. The addition-net consists of the concepts: PLANFOR1, APOLOGIZE1, UC, USER1, KNOW1, and SOMETHING1. This daemon might be activated when the user asks UC, “What does du -r do?” Although UC does know what du does, it does not know what du -r does. Moreover, thanks to UC’s meta-knowledge (knowledge about what UC itself knows), UC knows that it does not have any knowledge about the options of du. To be polite, UC apologizes to the user for not knowing what du -r does. Figure 20 shows the state of affairs after the user has asked UC the question and UC’s goal analyzer has determined the user’s goal. The relevant concepts include the fact that UC has the goal of being polite to the user and the fact that the user has the goal of knowing the effects of du -r. This by itself is not enough to cause the activation of the daemon, since part of the detection-net does not have a match, namely that UC does not know the effects of du -r. After UC has tried to find out the effects of du -r and failed, the process responsible notes the failure by adding the fact that UC does not know the effects to UC’s knowledge base. The relevant concepts are shown in Figure 21. This completes the match of the daemon’s detection net. UC-HAS-
PLANNING INTELLIGENT RESPONSES IN A NATURAL LANGUAGE SYSTEM
315
HAS-GOAL-ga0 UC-HAS-GOAL47 goal
status
KNOW-ga0 knower-ga0 fact-ga0
TRUE
BE-POLITE5 is-polite5
goal-ga0
STATE11?
polite-to5 *USER*
UC
effect-of-command20 HAS-EFFECT20?
-r-OPTION0
command-of-effect20
UNIX-DU-COMMAND0 option3 command-of-option3 HAS-OPTION3 Figure 20. Relevant concepts leading up to activation of the daemon.
GOAL2 is matched by UC-HAS-GOAL47; BE-POLITE1? is matched by BE-POLITE5; USER1? is matched by *USER*; HAS-GOAL1? is matched by HAS-GOAL-ga0; KNOW2? is matched by KNOW-ga0; SOMETHING1? is matched by STATE11?; and KNOW1? is matched by KNOW47. In matching, a hypothetical concept (i.e., a concept without a referent) is allowed to match any concept that is a member of the same categories as the hypothetical concept. The matching concept is also allowed to be a member of more categories than the hypothetical concept (since KODIAK has multiple inheritance), and is also allowed to be a member of more specific sub-categories than the hypothetical concept. For example, the hypothetical concept SOMETHING1? can be matched by STATE11?, because STATE is a more specific sub-category of the SOMETHING category. Concepts such as UC, TRUE, and FALSE in the detection-net that are not hypothetical are treated as constants instead of as variables. A non-hypothetical concept can only match itself. For example, the value of the truth-val aspectual of whatever matches KNOW1? must be FALSE, because FALSE is not a hypothetical concept. One disadvantage of using the hypothetical marker for variables is that it is hard to specify that the matching concept must be a hypothetical concept. This problem is solved by adding the new marker MAYBE exclusively for this purpose. Thus SOMETHING1? is marked as dominated by MAYBE in
316
DAVID N. CHIN
UC FALSE knower47 status47-0 KNOW47 fact
STATE11? Figure 21. Relevant concepts completing the activation of the daemon.
the detection-net of the daemon. This adds the requirement that whatever matches SOMETHING1? must also be hypothetical. So, STATE11? can match SOMETHING1?, only because it is indeed hypothetical. After the match, the daemon adds a copy of its addition-net to UC’s knowledge base. The output of this daemon in this example is shown in Figure 22. Concepts that are shared between the addition-net and the detection-net are not copied. Rather, the corresponding matching concept is used instead. An example of a shared concept is BE-POLITE1?, which was matched by BEPOLITE5. The copy of the addition-net shown in Figure 22 shows that BE-POLITE5 is used directly. Hypothetical concepts that are not shared are copied, and non-hypothetical concepts are used directly. Copying hypothetical concepts in the addition-net means creating new concepts that are dominated by the same categories as the old concepts except for the hypothetical marker. In those cases where one desires the new copy to also be hypothetical, the MAYBE marker can be used to mean that the copy should also be made hypothetical. This is analogous to the use of MAYBE in detection-nets. 4.2. Implementation strategies The simplest way to activate daemons is the simple production system method, which loops through all the daemons and performs matching to determine which daemons should be activated. This scheme takes increasingly more processing time as the number of daemons increases and as the size of the data-base increases. Theoretically, every daemon’s pattern would
PLANNING INTELLIGENT RESPONSES IN A NATURAL LANGUAGE SYSTEM
Figure 22. Output from the daemon: a copy of its addition-net for UC’s knowledge base.
need to be matched against every piece of information in the data-base. The processing cost for if-detected daemons is especially high, because ifdetected daemons have complex detection-nets that consist of combinations of possibly independent concepts. For if-detected daemons, each independent concept in the detection-net needs to be matched against every entry in the data-base. The processing cost for large numbers of if-detected daemons and very large data-bases becomes prohibitive when real-time response is needed as in UC. Actual production systems have addressed the problem of efficiency with a variety of methods. These methods cannot be directly applied to if-detected daemons, because daemons differ in several important aspects from the rules in most production systems (some of the ideas can be modified to apply, and these are described later). First, if-detected daemons use a semantic network representation (KODIAK), whereas most production systems do not (an exception is described by (Duda et al. 1978)). As a result, ifdetected daemons can take advantage of the multiple inheritance taxonomies of semantic network representations and can more easily use the same relation in several different patterns and actions. Also, instead of variables, if-detected daemons use the hypothetical marker, which allows nodes in the detection-
318
DAVID N. CHIN
net to match any concept in the knowledge base that is lower in the KODIAK hierarchy. Since ordinary production systems only allow specific tokens at the top-level of their patterns, they would need many more rules to encode the same information as one if-detected daemon. Finally, if-detected daemons are designed to operate in parallel, whereas most production systems require a conflict resolution mechanism to determine which of several conflicting rules should be activated. Since if-detected daemons are designed to activate in parallel, the best solution to the problem of efficiency would be to perform the match testing of different daemons in parallel. Unfortunately, parallel machines that run LISP (the implementation language for UC) are not yet readily available. Even with parallel LISP machines, some optimizations are still useful for improving speed and efficiency. This section will discuss the variety of such optimizations and how they might be implemented to considerably improve the performance of if-detected daemons. One possible optimization in the processing of if-detected daemons involves taking advantage of the organization of the data-base to limit the search for matches. This is called the data-base retrieval optimization. For AI knowledge bases that are organized in inheritance hierarchies, this means restricting candidates for matching to only those concepts that are in the same part of the inheritance hierarchy as the concepts in the detection-net. For example, when looking for a match for HAS-GOAL1?, the matcher need only look at instances of HAS-GOAL, and instances of HAS-GOAL’s subcategories (which in this case includes only UC-HAS-GOAL). This simple optimization, which is commonly used in data-base retrieval, considerably restricts the size of the initial set of candidates. 4.2.1. Distributed data-driven activation Another possible optimization for the implementation of if-detected daemons is to perform the match testing for only those daemons that are probable candidates for activation. This may seem impossible, since it would be hard to tell whether a daemon is a probable candidate for activation without looking at it first. However, the data-base retrieval optimization can be used in reverse. Rather than looking in the knowledge base hierarchy for candidates, one can look in the hierarchy for matching detection-net concepts, when one changes the knowledge-base. This technique is called distributed data-driven activation. It is data-driven, since one looks for daemons to activate as data is changed (i.e., added, deleted, or modified). It is distributed, since any particular piece of newly changed data may only match part of the detection-net of a daemon. The rest is matched by either previously changed data or subsequent changes to the data-base.
PLANNING INTELLIGENT RESPONSES IN A NATURAL LANGUAGE SYSTEM
319
Distributed data-driven activation is similar to techniques used in production systems to increase their efficiency. The Rete Match Algorithm that is used in OPS5 (Forgy 1982) extracts features from the patterns of rules and forms a discrimination net of features that is used for matching patterns. When elements are added to or removed from working memory, OPS5 uses this precompiled discrimination net to match production rules. (McDermott et al. 1978) showed that by using pattern features to index into rules, the estimated cost of running a production system can be improved so that the run time is almost independent of the number of productions and number or working memory elements. Distributed data-driven activation is different from these schemes in that it uses the multiple inheritance hierarchy of KODIAK to index into if-detected daemons. Nevertheless, the main idea of distributed data-driven activation is similar to production system methods like the Rete Algorithm. To see how distributed data-driven activation works, consider what might happen when a new instance of HAS-GOAL, HAS-GOAL1, is added to a knowledge base. This new instance can only cause the activation of those daemons that have detection-nets that might match HAS-GOAL1. Detectionnet concepts that might match HAS-GOAL1 include hypothetical instances of HAS-GOAL or hypothetical instances of any of the parent categories of HAS-GOAL (i.e., M-POSSESS, STATE, and SOMETHING). This is just the reverse of the process used in the data-base retrieval optimization. In that case, one starts from the detection-net and looks down the conceptual hierarchy for possible matches, whereas here one starts from the potential match and looks up the conceptual hierarchy for hypothetical concepts that are in detectionnets. A small optimization for speeding up the lookup is to precompile a list for every category of those instances that are part of some detection-net. This way, whenever a new instance is added, one can just look in the list to see which daemons might possibly be affected. Such precompilation can be done when daemons are first defined in the system. Another small optimization is to check for a match only when all the nodes of a detection-net have been primed; that is, marked as having potential matches. This way, when a new concept primes one node of a detection-net, one can check to see if all of the other nodes have been primed before trying to match the entire detection-net. If not all of the nodes of the detection-net have been primed, no matching is needed yet, since there will be nothing in the knowledge base that will match the unprimed nodes. If there were potential matches for these unprimed nodes then they would have been primed when the potential matches were added to the knowledge base. This optimization works well when a system is just starting up. However, as more concepts are
320
DAVID N. CHIN
created, more of the nodes of a detection-net will have potential matches, and so more daemons will become fully primed (i.e., all of the nodes of its detection-net have been primed). Once a daemon becomes fully primed, any single new concept that primes a detection-net node will require matching. It is not possible to reset the primes after activation, because it is always possible that a new concept in conjunction with many old concepts might cause the activation of a daemon. This optimization is worthwhile in systems such as UC where sessions with users are brief enough so that many daemons remain unprimed for a significant part of the session. One of the advantages of distributed data-driven activation is that it does away with the some of the bookkeeping needed in the production system loop method. Since daemons can only be activated by changes in the knowledge base, the search can no longer find something that was a previous match. Thus, the system no longer needs to keep around a list of previous matches to avoid multiple activations of a daemon on the same concepts. 4.2.2. Delayed matching Another optimization technique involves reducing the frequency of the activation process. In the simple production system loop, the processing costs can be reduced by performing the loop less frequently. For example, rather than executing the loop immediately whenever something changes in the knowledge base, the system can wait and execute the loop at fixed times. This way, one loop through the daemons can catch many different activations. This delaying tactic does not work if the system expects the daemons to be activated immediately. However in many applications such as UC, immediate activation of daemons at an atomic level is not needed. For example, in UC the activation of daemons can wait until after UC’s parser/understander finishes creating the KODIAK network that represents the user’s input. It is not necessary to activate daemons as soon as the understander creates another KODIAK concepts, because there are no daemons that influence the understander. Activating daemons at the end of the parsing/understanding process is good enough for the other components of UC. The same delaying optimization can be applied to the distributed datadriven activation scheme. Instead of testing the detection-net of a daemon for a match as soon as its nodes have been primed, the testing for a match can be delayed provided that the system remembers the priming concepts. Then all the matching can be performed at a later time to save work. By delaying the matching as long as possible, the system is given time to complete the match. For example, consider the case of a detection-net that consists of a single relation, R1?, that relates two concepts, A1? and B1?. Suppose further that this daemon is fully primed, that is there are potential matches for R1?,
PLANNING INTELLIGENT RESPONSES IN A NATURAL LANGUAGE SYSTEM
321
A1?, and B1?. Then suppose that the system creates the matching concepts A2, B2 and R2 where R2 relates A2 to B2. If the system adds each of these concepts to the knowledge base at separate times (which is not unlikely), then the system will have to try to match the detection-net after adding each concept. This is necessary because the new concept could potentially match the detection-net in conjunction with other older concepts that primed the other nodes of the detection-net. For example is A2 is added first, then the system will have to try matching A1? to A2, B1? to its old primes and R2? to its old primes. Since none of the old primes of R2? will relate A2, the match will fail. This will be repeated again when B2 is added and when R2 is added. Thus the system will have to try matching the detection net as many times as priming concepts are added to the knowledge base. However, if matching can be delayed until all of the pertinent concepts have been added, then the system will have to go through the matching process only once. In practice, the delaying optimization saves considerable work. However there is some minor additional bookkeeping needed. The system needs to keep track of which concepts have primed which detection-net nodes since the last time matching was done. The system also needs to keep track of which daemons with fully primed detection-nets have been primed since the last matching cycle. Since the system already keeps track of the new priming concepts, it becomes easy to keep a list of old primes also. This way, the system no longer needs to look in the conceptual hierarchy for potential matches (the data-base retrieval optimization). This optimization is a spacetime tradeoff, since keeping a list of old primes takes up more space while looking in the hierarchy takes more time. 4.3. UC’s implementation The actual implementation of if-detected daemons in UC uses a distributed data-driven activation scheme with delayed matching. When UC is created, the if-detected daemons are entered into UC after all KODIAK categories have been defined in UC. Preprocessing of daemons involves creating a fastaccess list for each category (except the SOMETHING category) consisting of those detection-net nodes that are hypothetical and that are members of that category. These lists are stored under the categories’ property lists and are used to speed up access when priming the detection-net nodes. The SOMETHING category includes everything in UC’s knowledge base, so the fast-access list for the SOMETHING category is simply a pointer to the list of all concepts in the knowledge base. During the execution of UC, processing of if-detected daemons occurs in the two distinct phases in a delayed matching scheme. The two phases are priming and matching. Each phase is described below.
322
DAVID N. CHIN
4.3.1. Priming Priming of detection-net nodes is performed whenever concepts are created or modified in UC. Since all KODIAK concepts in UC are stored in UC’s knowledge base, there is no distinction made between creating concepts and adding concepts to the knowledge base. When a concept is created or modified, it primes all matching detection-net nodes. Detection-net nodes are found by looking in the fast-access lists stored under the concept’s immediate categories and all their dominating categories up the conceptual hierarchy. A special case is made for those concepts that are modified by concreting them, that is making them members of more specific categories than their previous categories. In these cases, the modified concept will already have primed its old categories (and their dominating categories) at the time that the modified concept was first created or last modified. Hence the modified concept should not prime these old categories to avoid multiple primings. Priming involves storing the new/modified concept under the primed node’s list of priming concepts (kept on the primed node’s property list). After priming a node of a daemon’s detection-net, the priming process checks to see if all of the other nodes of that daemon’s detection-net have been primed. If so, the fully primed daemon is added to a global list of daemons to be checked during the next matching phase. The initial version of UC’s priming mechanism was coded by Lisa Rau who has since applied a form of priming and matching to information retrieval from story data-bases in the SCISOR system (Rau 1987a; Rau1987b). Unlike UC, SCISOR’s priming system is a true marker passing scheme, and matching in SCISOR is used to rate the similarity of the retrieved networks to the priming network. In SCISOR, there is no sense of additional inferences beyond unification of the matched networks such as those in the addition-nets of if-detected daemons. 4.3.2. Matching The matching phase in UC occurs at distinct points in UC’s processing: before UC’s parser/understander, before UC’s goal analyzer, and after the goal analyzer. Each matching phase is actually a loop that goes through the global list of fully-primed daemons (collected during the priming phase) and tests them for matches. After testing every daemon for matches, the additionnets of the successfully matched daemons are copied and added to UC’s knowledge base. Theoretically, matching for each daemon can be done in parallel, although in practice UC runs on sequential machines. Likewise, copying the addition-nets can be done in parallel. As the addition-nets are copied and added to UC’s knowledge base (actually an atomic operation, since all KODIAK concepts are added to UC’s knowledge base as soon as they are created), priming may occur, because
PLANNING INTELLIGENT RESPONSES IN A NATURAL LANGUAGE SYSTEM
323
the knowledge base is being modified. These copies of addition-nets can in turn (possibly in conjunction with older concepts) cause more daemons to become fully primed. Hence after copying all of the appropriate additionnets, the matching process begins anew. This loop continues until there are no more daemons that have been fully primed waiting to be matched. Testing for matches in UC involves three phases. First, the non-negated parts of the detection-net are matched. If matching is successful, then the negated parts of the detection-net are checked. Finally if both previous steps succeed, the daemon’s procedural tests are examined. If all three phases succeed, then the assoc list of detection-net nodes and their matches are stored for later use in copying the addition-net. The addition-nets of activated daemons are not copied until the system has finished the match testing for all fully primed daemons. In theory, this prevents a copy of the addition-net of one daemon from invalidating the match of another daemon. In practice, the situations encoded in UC’s daemons do not have such interaction problems. Copying addition-nets is fairly straightforward. The detection-net is traversed and nodes are processed as follows: 1. Nodes found in the assoc list that was created during matching are replaced by their matches. 2. Nodes that are hypothetical, but not in the assoc list, are replaced by a copy. 3. Nodes that are non-hypothetical are replaced by themselves. After copying the detection-net, those nodes that are procedures (i.e., instances of PROCEDURE or a sub-category of PROCEDURE) are also executed. The name of the lisp function to call is given by the named of the procedure node, and the arguments are given by its aspectuals. Some of these procedures include calls to UC’s UNIX planner component, calls to UC’s generator component, and calls to exit UC. 4.3.3. An example A simple example will show how if-detected daemons are actually processed in UC. The if-detected daemon shown in Figure 23 is used to call KNOME via the procedure user-knows, whenever UC encounters the situation where some person (PERSON1) wants (HAS-GOAL1) to know (KNOW1) something (SOMETHING1), and that person is not UC (implemented by the NOT test which checks to make sure that HAS-GOAL1 is not a UC-HAS-GOAL). This daemon is typically activated when UC’s goal analysis component determines that the user has the goal of knowing something. The arguments of the userknows procedure include the user, what the user wanted to know, and FALSE, which indicates that KNOME should infer that the user does not know.
324
DAVID N. CHIN
UC-HAS-GOAL1? dominator DOMINATE1? NOT if-detected
dominated
HAS-GOAL1? goal1
user-knows1? KNOW1?
uk-fact
fact1 SOMETHING1?
uk-truth-val
planner1 knower1
uk-user
FALSE
PERSON1? Figure 23. Daemon16: call KNOME when someone wants to know something.
Figure 24 shows a trace of a UC session in which this daemon is activated. When the user asks, “How can I delete a file?” UC’s goal analyzer determines that the user has the goal (HAS-GOAL-ga0) of knowing (KNOW-ga0) how to delete a file (ACTION12). When UC’s goal analyzer creates the concepts that encode this inference, the concepts prime other related concepts in the detection-net of if-detected daemons. HAS-GOAL-ga0 primes a number of concepts in if-detected daemons. Among these is HAS-GOAL1, which is in the detection net of the daemon shown in Figure 23. When HAS-GOAL1 is primed by HAS-GOAL-ga0, HAS-GOAL-ga0 is added to the list of primers of HAS-GOAL1 that is stored under HASGOAL1’s property list. The reason why the trace message about priming occurs before the trace message about the goal analyzer’s output is because priming is an atomic operation integrated within creation of KODIAK concepts. As the goal analyzer creates concepts, priming occurs and trace messages about priming are output. The trace message about what the goal analyzer produces is not output until after the goal analyzer has finished creating concepts.
PLANNING INTELLIGENT RESPONSES IN A NATURAL LANGUAGE SYSTEM
325
..
# How can I delete a file? . Marking HAS-GOAL4 as primed by HAS-GOAL-ga0 Marking daemon32 as fully primed Marking HAS-GOAL3 as primed by HAS-GOAL-ga0 Marking daemon24 as fully primed Marking HAS-GOAL2 as primed by HAS-GOAL-ga0 Marking daemon23 as fully primed Marking HAS-GOAL1 as primed by HAS-GOAL-ga0 Marking daemon16 as fully primed
Daemon16 is the daemon shown in Figure 23. Marking HAS-GOAL0 as primed by HAS-GOAL-ga0 Marking daemon0 as fully primed}
UCEgo detects the following concepts: (HAS-GOAL-ga0 &) and asserts the following concept into the database: (user-knows8 (uk-user8 = *USER*) (uk-truth-val8 = FALSE) (uk-fact8 = (ACTION12? &))) KNOME: Asserting *USER* does not know ACTION12?
.. .
Use rm. For example, to delete the file named foo, type ‘rm foo’.
Figure 24. Trace of concept priming leading to the activation of a daemon.
The priming of HAS-GOAL1 completes the priming of its daemon, which is labeled daemon16. Daemon16 is added to the global list of fully primed daemons for processing during the delayed matching phase. The first such matching phase occurs after UC’s goal analyzer has finished. In the matching phase, the detection nets of all fully primed daemons are checked for matches. Daemon16 is one of these, so HAS-GOAL1 is matched against HAS-GOAL-ga0. Since both are instances of HAS-GOAL, the two match at the top level, so matching continues with their aspectuals. HAS-GOAL1’s goal1 aspectual has the value KNOW1, which is matched against the value of HAS-GOAL-ga0’s goal aspectual, KNOW-ga0. Both are instances of KNOW, so their aspectuals are checked. PERSON1, the knower of KNOW1, matches *USER*, the knower of KNOW-ga0; and SOMETHING1, the fact
326
DAVID N. CHIN
of KNOW1, matches ACTION12, the fact of KNOW-ga0. Finally the planner aspectual of HAS-GOAL1 is matched against the planner aspectual of HASGOAL-ga0. In this case, PERSON1 has already been matched with *USER*, so the planner of KNOW-ga0 must also be *USER* for a proper match. This is indeed the case, so the detection-net of daemon16 is completely matched and daemon16 is activated. Daemon16 is activated by creating a copy of its addition-net, which consists of user-knows1 with aspectuals and values: uk-fact1 = SOMETHING1, uk-user1 = PERSON1, and uk-truth-val1 = FALSE. Since userknows1 is hypothetical, a new copy of user-knows1 is created. This is shown in the trace as user-knows8. Next its aspectuals are copied. The uk-fact1 aspectual has the value SOMETHING1, which is also hypothetical. However, SOMETHING1 was previously unified with ACTION12, so instead of creating a new copy of SOMETHING1, the unified concept, ACTION12 is used instead. Similarly, PERSON1 was unified with *USER*, so uk-user8 gets the value *USER*. On the other hand, the value of uk-truthval1 is not hypothetical, so its value, FALSE, is used directly for the value of uk-truth-val8. In the trace, the new copy of user-knows1, user-knows8, is noted as being asserted into the database. Since user-knows is a procedure (i.e., it is dominated by the PROCEDURE category), UCEgo next calls the user-knows procedure with arguments *USER*, ACTION12, and FALSE. User-knows is an entry to the KNOME component for inferring a user’s knowledge state. In this case, KNOME asserts that the user does not know how to delete a file (ACTION12). Later (not shown) after UC’s UNIX planner has determined that a plan for deleting a file is to use the rm command, KNOME will figure out that the user does not know rm. Finally, after more priming and matching, UC produces its answer, and tells the user to use rm (usually giving an example of using rm also).
5. Conclusion 5.1. Summary The main issue addressed by UCEgo’s planner is efficient planning. As the main dialog planner for the interactive UC system, UCEgo needs to plan efficiently in order to be able to respond to the user in real time. This is in direct contrast to most other AI planners, which did not have this constraint and so could afford to plan inefficiently. I approached this problem of efficient planning in two ways. First, UCEgo incorporates a very simple planner that takes advantage of knowledge about typical speech acts encoded in prestored skeletal plans to completely avoid inefficient weak methods. Secondly,
PLANNING INTELLIGENT RESPONSES IN A NATURAL LANGUAGE SYSTEM
327
UCEgo avoids inefficient backtracking by selecting plans according to the situation. UCEgo encodes knowledge about which plans are typically useful in different types of situations by adding appropriateness conditions to plans. These appropriateness conditions are not preconditions, because plans can be used even when their appropriateness conditions are violated (sometimes even successfully). Appropriateness conditions encode when it is appropriate to use a plan, in contrast to preconditions, which encode when it is possible to use a plan. By encoding the appropriateness conditions along with the preconditions and the goal of a plan into a situation class, UCEgo can suggest the plan whenever it encounters a situation that fits the situation class. These situation classes are represented using if-detected daemons, which suggest the plan associated with the situation class whenever the daemon detects a matching situation. By selecting among only appropriate plans as opposed to all possible plans, UCEgo avoids inefficient backtracking during planning. UCEgo’s success shows that a very simple planner that is based on prestored skeletal plans and that does not backtrack can be used successfully to plan speech acts. UCEgo also shows that it is possible to plan speech acts without having to worry about mutual beliefs to the extent that the OSCAR system (Cohen 1978) did. For example, to produce a simple inform type speech act, UCEgo worries only about having the user believe the proposition, whereas OSCAR worried about having the user believe that the system believes the proposition, and then this belief convincing the user to believe the proposition. In everyday usage, when the system does not have any a priori reason to believe that the user might disagree with the system (such as during argumentation), such complex reasoning about mutual beliefs is not absolutely necessary for planning speech acts. Even when the system fails to convince the user by simply informing the user of the proposition, it can still notice the user’s incredulity and correct the situation by providing additional support for the proposition. 5.2. Problems One potential shortcoming of planning as implemented in UCEgo is that UCEgo does not have the capability to fall back on planning from first principles when it fails to find a pre-stored plan. In one sense, this shows that UCEgo’s approach is superior since UCEgo never needs to fall back on inefficient planning from first principles to plan any of the speech acts needed for UC to respond to the user. This agrees with people’s intuitions that they are not planning from scratch in everyday conversation. On the other hand, people do fall back on planning from first principles occasionally (perhaps
328
DAVID N. CHIN
more frequently when writing than when speaking). So, to be complete, UCEgo really should have such a capability. Unlike skeletal plans (Friedland 1980; Friedland and Iwasaki 1985), the UCEgo’s plans are not organized into an abstraction hierarchy, but are encoded at a single level of abstraction. For planning speech acts, this is not a real problem, because UCEgo’s single level of plan abstraction matches the single level of communication abstraction that is represented by speech acts. The lower levels of communication abstraction, choice of expressions and words, are handled by UC’s expression mechanism (UCExpress) and tactical generator. The higher levels of abstraction (i.e., paragraphs and larger units) are not addressed by UCEgo. If UCEgo were to be extended to handle real world actions besides speech acts, or if UCEgo were to be extended to plan larger communicative units than speech acts, then UCEgo would need to organize plans into an abstraction hierarchy. In order to organize plans into an abstraction hierarchy, one should also organize the situations that suggest plans into an abstraction hierarchy of situation classes. Currently, UCEgo does not organize situation classes into an abstraction hierarchy, although such a hierarchy would also be useful for other tasks such as detecting goals.
6. Recent Developments The UCEgo component described in this paper only reacts to the current situation and does not carry out multi-step plans, which may be required for longer dialogs. For example, arguing with the user about how to best do something in UNIX requires more complex dialog planning. UCEgo cannot just address the user’s most recent argument without taking into account the previous arguments (both those put forth by the user and those given by UC). There is some question as to whether a reactive planner like UCEgo can exhibit longer dialog coherence and if so, what kinds of skeletal plans and situations might be needed to implement dialog planning.
Acknowledgements The work described in this paper was done at the University of California, Berkeley as part of my Ph.D. thesis. I wish to thank Robert Wilensky who supervised this work. I also wish to thank the members of BAIR (Berkeley Artificial Intelligence Research) who have contributed to the UC project. This research was sponsored in part by the Defense Advanced Research Projects Agency (DoD), ARPA order No. 4871, monitored by Space and
PLANNING INTELLIGENT RESPONSES IN A NATURAL LANGUAGE SYSTEM
329
Naval Warfare Systems Command Command under contract N00039-84-C0089, by the Office of Naval Research under contract N00014-80-C-0732, by the National Science Foundation under grant MCS79-06543, and by the Office of Naval Research under contract N00014-97-1-0578.
Notes 1. UNIX is a trademark of X/Open, Inc. 2. Actually, this version of UCEgo never does suggest two different plans for the same goal. However, it is possible, so UCEgo was designed to handle such contingencies. 3. In the KODIAK semantic network knowledge representation language, aspectuals are the arguments of relations, such as HAS-PART, CONTAINS, and HAS-INTENTION. For example, the aspectuals of HAS-PART are whole and part. In Figure 19, the UC-HASGOAL relation has aspectuals goal and status, which are denoted as labels on arrows leading out from the UC-HAS-GOAL relations. 4. It was found that the particular if-detected daemons used in UC were not being activated by changes in the values of aspectuals, so UC was optimized to not look for this type of activation. Other types of network reconfiguration, such as when individual concepts are concreted (i.e., made instances of more specific concepts down the hierarchy), were more common. 5. Disjunctions can be handled by both types of daemons simply by splitting the disjunction into two daemons.
References Alterman, R. (1986). An Adaptive Planner. In Proceedings of The Fifth National Conference on Artificial Intelligence 1, 65–69. Philadelphia, PA: AAAI Press. Appelt, D. E. (1981). Planning Natural Language Utterances to Satisfy Multiple Goals. Ph.D. diss., Computer Science Department, Stanford University, Stanford, CA. Also available as 259, Artificial Intelligence Center, SRI International, Menlo Park, CA. Appelt, D. E. (1983). TELEGRAM: A Grammar Formalism for Language Planning. In Proceedings of The Eight International Joint Conference on Artificial Intelligence 1, 595–599. Karlsruhe, Germany: Morgan Kaufmann Publishers. Austin, J. L. (1962). How to Do Things with Words. London: Oxford University Press. Carbonell, J. G. (1986). A Theory of Reconstructive Problem Solving and Expertise Acquisition. In Michalski, R. S., Carbonell, J. G. & Mitchell, T. M. (eds.) Machine Learning II. Los Altos, CA: Morgan Kaufmann. Charniak, E. (1972). Towards a Model of Children’s Story Comprehension. TR-266, Artificial Intelligence Laboratory, Massachusetts Institute of Technology, Cambridge, MA. Chin, D. N. (1987). Intelligent Agents as a Basis for Natural Language Interfaces. Ph.D. diss., Computer Science Division, University of California, Berkeley, CA. Also available as UCB/CSD 88/396, Computer Science Division, University of California, Berkeley, CA. Chin, D. N. (1989). KNOME: Modeling What the User Knows in UC. In Kobsa, A. & Wahlster, W. (eds.) User Models in Dialog Systems, 74–107. Berlin: Springer-Verlag.
330
DAVID N. CHIN
Chin, D. N. (1991). Intelligent Interfaces as Agents. In Sullivan, J. W. & Tyler, S. W. (eds.) Intelligent User Interfaces, 177–206. Addison-Wesley: Reading, MA. Cohen, P. R. (1978). On Knowing What to Say: Planning Speech Acts. Ph.D. diss., University of Toronto, Toronto, Canada. Also available as 118, University of Toronto, Toronto, Canada. Cohen, P. R. & Levesque, H. J. (1987a). Persistence, Intention, and Commitment. 415, Artificial Intelligence Center, SRI International, Menlo Park, CA. Cohen, P. R. & Levesque, H. J. (1987b). Rational Interaction as the Basis for Communication. 89, Stanford University Center for the Study of Language and Information. Cohen, P. R. & Perrault, C. R. (1979). Elements of a Plan-based Theory of Speech Acts. Cognitive Science 3: 177–212. Duda, R. O., Hart, P. E., Nilsson, N. J. & Sutherland, G. L. (1978). Semantic Network Representations in Rule-Based Inference Systems. In Waterman, D. A. & Hayes-Roth, F. (eds.) Pattern-Directed Inference Systems, 155–176. New York: Academic Press. Faletti, J. (1982). PANDORA – A Program for Doing Commonsense Planning in Complex Situations. In Proceedings of The Second National Conference on Artificial Intelligence, 185–188. Pittsburg, PA: AAAI Press. Fikes, R. E. & Nilsson, N. J. (1971). STRIPS: A New Approach to the Application of Theorem Proving to Problem Solving. Artificial Intelligence 2(3–4): 189–208. Forgy, C. L. (1982). Rete: A Fast Algorithm for the Many Pattern/Many Object Pattern Match Problem. Artificial Intelligence 19: 17–37. Friedland, P. E. (1980). Knowledge-Based Experiment Design in Molecular Genetics. Ph.D. diss., Computer Science Department, Stanford University, Stanford, CA. Friedland, P. E. & Iwasaki, Y. (1985). The Concept and Implementation of Skeletal Plans. Journal of Automated Reasoning I: 161–208. Hammond, K. J. (1986). CHEF: A Model of Case-based Planning. In Proceedings of The Fifth National Conference on Artificial Intelligence 1, 267–271. Philadelphia, PA: AAAI Press. Hendler, J. A. (1985). Integrating Marker-Passing and Problem-Solving. Ph.D. diss., Computer Science Department, Brown University, Providence, RI. Also available as CS-85-08, Computer Science Department, Brown University, Providence, RI. Kolodner, J. L., Simpson, R. L. & Sycara-Cyranski, K. (1985). Model of Case-Based Reasoning in Problem Solving. In Proceedings of The Ninth International Joint Conference on Artificial Intelligence, 284–290. Los Angeles, CA: Morgan Kaufmann Publishers. McDermott, J., Newell, A. & Moore, J. (1978). The Efficiency of Certain Production System Implementations. In Waterman, D. A. & Hayes-Roth, F. (eds.) Pattern-Directed Inference Systems, 155–176. New York: Academic Press. Meehan, J. R. (1976). The Metanovel: Writing Stories by Computer. Ph.D. diss., Department of Computer Science, Yale University, New Haven, CT. Also available as tr074, Department of Computer Science, Yale University and through Garland Publishing, New York. Meehan, J. R. (1981). TALE-SPIN. In Schank, R. C. & Riesbeck, C. K. (eds.) Inside Computer Understanding, 197–226. Hillsdale, NJ: Lawrence Erlbaum. Newell, A. & Simon, H. A. (1963). GPS, a Program that Simulates Human Thought. In Feigenbaum, E. A. & Feldman, J. (eds.) Computers and Thought. New York: McGraw Hill. Rau, M. (1987a). Information Retrieval from Never-ending Stories. In Proceedings of The Sixth National Conference on Artificial Intelligence 1, 317–321. Seattle, WA: AAAI Press.
PLANNING INTELLIGENT RESPONSES IN A NATURAL LANGUAGE SYSTEM
331
Rau, M. (1987b). Spontaneous Retrieval in a Conceptual Information System. In Proceedings of The Tenth International Joint Conference on Artificial Intelligence, 155–162. Milano, Italy: Morgan Kaufmann Publishers. Rosenbloom, P. S. & Newell, A. (1982). Learning by Chunking: Summary of a Task and a Model. In Proceedings of The Second National Conference on Artificial Intelligence, 255–257. Pittsburgh, PA: AAAI Press. Sacerdoti, E. D. (1974). Planning in a Hierarchy of Abstraction Spaces. Artificial Intelligence 5(2): 115–135. Sacerdoti, E. D. (1977). A Structure for Plans and Behavior. Amsterdam: Elsevier NorthHolland. Schank, R. C. & Abelson, R. P. (1977). Scripts, Plans, Goals, and Understanding. Hillsdale, NJ: Lawrence Erlbaum. Searle, J. R. (1969). Speech Acts; An Essay in the Philosophy of Language. Cambridge, England: Cambridge University Press. Stefik, M. (1980). Planning with Constraints. Ph.D. diss., Computer Science Department, Stanford University, Stanford, CA. Also available as 80–784, Computer Science Department, Stanford University, Stanford, CA. Stefik, M. (1981). Planning and Meta-Planning (MOLGEN: Part 2). Artificial Intelligence 16: 141–170. Sussman, G. J. (1975). A Computer Model of Skill Acquisition. New York: American Elsevier. Tate, A. (1975). Interacting Goals and their Use. In Proceedings of The Fourth International Joint Conference on Artificial Intelligence, 215–218. Tbilisi, Georgia: Morgan Kaufmann Publishers. Waldinger, R. (1977). Achieving Several Goals Simultaneously. In Elcock, E. W. & Michie, D. (eds.) Machine Intelligence 8. New York: Halstead/Wiley. Warren, D. H. D. (1974). WARPLAN: A System for Generating Plans. Memo 76, Department of Computational Logic, School of Artificial Intelligence, University of Edinburgh. Edinburgh, United Kingdom. Wilensky, R. (1983). Planning and Understanding: A Computational Approach to Human Reasoning. Reading, MA: Addison-Wesley. Wilensky, R. (1987). Some Problems and Proposals for Knowledge Representation. UCB/CSD 87/351, Computer Science Division, University of California, Berkeley, CA. Wilensky, R., Arens, Y. & Chin, D. N. (1984). Talking to UNIX in English: An Overview of UC. Communications of the ACM 27(6): 574–593. Wilensky, R., Chin, D. N., Luria, M., Martin, J., Mayfield, J. & Wu, D. (1988). The Berkeley UNIX Consultant Project. Computational Linguistics 14(4): 35–84.
Strategies for Expressing Concise, Helpful Answers DAVID N. CHIN Department of Information and Computer Sciences, University of Hawaii, 1680 East West Rd., Honolulu, HI 96822, U.S.A. E-mail: [email protected]
Abstract. An intelligent help system needs to take into account the user’s knowledge when formulating answers. This allows the system to provide more concise answers, because it can avoid telling users things that they already know. Since these concise answers concentrate exclusively on pertinent new information, they are also easier to understand. Information about the user’s knowledge also allows the system to take advantage of the user’s prior knowledge in formulating explanations. The system can provide better answers by referring to the user’s prior knowledge in the explanation (e.g., through use of similes). This process of refining answers is called answer expression. The process of answer expression has been implemented in the UCExpress component of UC (UNIX Consultant), a natural language system that helps the user solve problems in using the UNIX operating system. UCExpress separates answer expression into two phases: pruning and formatting. In the pruning phase, subconcepts of the answer are pruned by being marked as already known by the user (and hence do not need to be generated), or marked as candidates for generating anaphora or ellipsis (since they are part of the conversational context). In the formatting phase, UCExpress uses information about the user’s prior domain knowledge to select among specialized expository formats, such as similes and examples, for expressing information to the user. These formats allow UCExpress to present different types of information to the user in a clear, concise manner. The result of UCExpress’ answer expression process is an internal form that a tactical level generator can easily use to produce good English. Keywords: answer expression, formatting, generation, planning, pruning, user modeling
1. Introduction UC (UNIX Consultant) (Wilensky et al. 1984; Wilensky et al. 1988), is a natural language consultation program that answers the user’s questions on how to use the UNIX1 operating system. After understanding the user’s query, UC produces a plan consisting of communicative acts. This plan must be further refined because the concepts that UC wants to communicate to the user are often not organized in easily understood formats and often complete to the point of verbosity. The process of refining communicative actions is called answer expression (Luria 1982). The subcomponent of UC that does answer expression is called UCExpress. This paper describes how UCEx-
334
DAVID N. CHIN
press refines a communicative plan to produce a clear, concise answer for expression to the user. To see why answer expression is necessary, consider the following example: User: What is a directory? A1: A directory is a file. A2: A directory is a file that is used to contain files. A3: A directory is a file. Only empty directories can be deleted. Directories cannot be edited. Directories contain files. Directories form a tree-like structure. Directories always contain themselves and their parents. A plan for listing a directory is to use the ls command. The different replies are all correct, but contain differing amounts of information. The first answer does not give enough information to distinguish between files and directories. The second answer (what UC actually produces) provides just enough information in a concise form and is the best answer in this situation. The third answer is too verbose and overwhelms the user with information. An answer like A3 would be more appropriate in cases where the user had requested, “Tell me all you know about directories.” This example illustrates the problem of determining how much of the answer to express to the user. Another problem in answer expression is deciding what format to use to present the answer. Consider the following scenario: User: How can I move a file to another machine? A1: To move a file to another machine, type ‘rcp’ followed by one or more spaces or tabs followed by the name of the file to be moved followed by one or more spaces or tabs followed by the name of the machine followed by a colon followed by the new name of the file on the other machine followed by a carriage return followed by ‘rm’ followed by one or more spaces or tabs followed by the name of the file. A2: Use rcp to copy the file to another machine and then use rm to delete it. For example, to move the file foo to the file foo1 on machine dali, type ‘rcp foo dali:foo1’. The first answer is correct and quite general, but it is so verbose that it is indecipherable. On the other hand, the second answer is succinct and gives the user information in an easily readable form, but it is considerably less general. In fact the second answer is somewhat inaccurate, since it applies only to copying a file named foo to a file named foo1. It is up to the reader to use analogous reasoning to apply this to other cases. Despite this lack of generality, the second answer form is clearly superior to the first. Note that for a program to format the answer in the second form requires additional computation to transform the general solution of A1 into an example.
STRATEGIES FOR EXPRESSING CONCISE, HELPFUL ANSWERS
335
A natural language system needs to incorporate knowledge about when and how to use special presentation formats like examples to more clearly convey information to the user. These concerns about how much information to present to the user and about what format to use can be viewed as corresponding respectively to Grice’s Maxims of Quantity and Quality (Grice 1975). Although such considerations can be considered part of generation, there are sufficient differences in both the necessary knowledge and the processing to separate such strategic concerns from the more tactical problems of generation such as agreement and word selection. These strategic problems are the domain of an expression mechanism such as UCExpress.
2. User modeling Because answer expression is concerned with how much information to present to the user, an important consideration for answer expression is: what does the user already know? Given a model of the user’s knowledge and the current conversational context, an answer expression mechanism like UCExpress can reduce the amount of information presented to the user by pruning information already known to the user. Also, the level of expertise of the user may predict which presentation strategies will be most effective. For example, consider the actual UC sessions shown in Figures 1 and 2. In session 1, UC gives an example of how to use the rm command whereas in session 2, UC does not give an example of how to use ls -i. This is because in the first session, the user is a novice, so UC cannot assume that the user knows the format of the rm command. However, in session 2, the user is an intermediate, so UC can assume that the user would know how to use ls -i. Also, in session 2, UC uses a simile to explain what ruptime does in terms of what uptime does. This simile is shorter and clearer than the full answer given by UC in session 1. However, this simile is only useful if the user already knows what uptime does. UC can assume this for the intermediate user of session 2, but cannot do so for the novice user of session 1. These examples show how a model of the user’s knowledge can be used to improve the process of answer expression. In UC, the KNOME (KNOwledge Model of Expertise) component models what the user knows about UNIX. More details can be found in Chin (1986, 1987, 1989), so this section will only give enough information so that the reader can understand how KNOME is used by UCExpress. KNOME uses a stereotype approach (Rich 1979) where the characteristics of classes of users are organized under stereotypes. KNOME separates users into four levels of expertise (stereotypes): novice, beginner, intermediate,
336
DAVID N. CHIN
# How can I delete a file? Use rm. For example, to delete the file named foo, type ‘rm foo’.
# What does ruptime do? ruptime is used to list the uptime of all machines on the network, list the number of all users on them and list their load average.
Figure 1. UC session 1 with a novice user.
# How can I find out the inode of a file? Use ls -i.
# What does ruptime do? ruptime is like uptime, except ruptime is for all machines on the network.
Figure 2. UC session 2 with an intermediate user.
and expert. Individual users are classified as belonging to one of the above stereotype levels and inherit the characteristics of the stereotype. However, particular facts about the particular user override inheritance, so individual users differ from their stereotypes, which serve as reference points (Rosch 1978). Besides stereotypes for users, KNOME also has stereotype levels for UNIX facts. This feature is termed a double stereotype system (Chin 1986; Chin 1989). Stereotype levels for UNIX facts include simple, mundane, complex, and esoteric. Examples of simple information are the rm, ls, and cat commands, the technical term “file,” and the simple file command format (the name of the command followed by the name of the file to be operated upon). The mundane category includes the vi, diff and spell commands, the technical term “working directory,” and the -l option of ls, while the complex category includes the grep, chmod, and tset commands, the term “inode,” and the fact that write permission on the containing directory is a precondition for using the rm command for deleting a file. The esoteric category consists of information which is not in the mainstream usage of UNIX, but instead serves special needs. A good example is the spice program, that is useful only for people interested in semiconductor circuit simulations. Thanks to the additional stereotype classification of UNIX information encoded in UC, it becomes extremely easy and space efficient to encode the
STRATEGIES FOR EXPRESSING CONCISE, HELPFUL ANSWERS
337
Table 1. Relation between user stereotypes and knowledge difficulty levels. User stereotype
Knowledge difficulty level Simple Mundane Complex
Esoteric
Expert Intermediate Beginner Novice
ALL ALL MOST AFEW
– – – NONE
ALL MOST AFEW NONE
MOST AFEW NONE NONE
relation between user stereotypes and their knowledge of UNIX. The core of this knowledge is shown in Table I. Table I indicates that the novice user in session 1 (see Figure 1) likely does not know the format for the rm command, which is a simple fact, and definitely does not know the uptime command, which is a mundane fact. On the other hand, the intermediate user in session 2 (see Figure 2) definitely knows the format for the ls -i command, which is a simple fact, and is likely to know the uptime command.
3. UCExpress After other components of UC have identified a response to the user, this is passed to UCExpress, which decides how much of the response to present to the user and how to format it. The separation of this process of deciding how much of the answer to express from the process of figuring out the answer was first suggested by Luria (1982) who applied this distinction to a question answering system for story understanding. His system first found the causal chain that represented the answer, then used answer expression to decide how much of the causal chain to express to the user. The response passed to UCExpress is in the form of a conceptual network in the KODIAK representation language (Wilensky 1987). UCExpress, operates on this input in two phases, pruning and formatting. During pruning, UCExpress prunes common knowledge from the answer using information about what the user knows based on the conversational context and a model of the user’s knowledge. Next the answer is formatted using specialized expository formats for clarity and brevity. The final result is an augmented KODIAK conceptual network that is ready for direct generation into natural language using a tactical level generator such as KING (Jacobs 1986).
338
DAVID N. CHIN
4. Pruning When UCExpress is passed a set of concepts to communicate to the user, the first stage of processing prunes them by marking any extraneous concepts, so that later the generator will not generate them. The pruning is done by marking rather than actual modification of the conceptual network, since information about the node may be needed to generate appropriate anaphora for the pruned concept. The guiding principle in pruning is to not tell the user anything that the user already knows. Currently UC models two classes of information that the user may already know. The first class of information is episodic knowledge from a model of the conversational context. The current conversational context is tracked by marking those concepts that have been communicated in the current session. The second class of information concerns the user’s knowledge of UNIX related facts. Such user knowledge is modeled by KNOME. Thus any concept that is already present in the conversational context or that KNOME indicates is likely to be known to the user is marked and is not communicated to the user. 4.1. An example trace To see how pruning works in detail, consider the trace of a UC session shown in Figure 3. The trace shows UCExpress’ processing of the question, “How can I print a file on the laser printer?” The answer given by UC is, “Use lpr -Plz,” along with a reminder to file the printer output in the boxes. The actual KODIAK conceptual network that is passed to UCExpress, shown in Figure 4 is not nearly as succinct, because it contains all of the details of the command that are needed for planning. If the KODIAK network passed to UCExpress were to be generated directly into English, it might look like the following: To print a file on the laser printer, use the lpr -Plz command. The command-format of the lpr -Plz command is “lpr” followed by concatenating “-P” with “lz” followed by the name of the file to be printed on the laser printer. This literal paraphrase is harder to understand than UC’s more concise answer. To see how UCExpress prunes the network to arrive at the actual answer, consider the division of the concepts into the following three subnetworks: PLANFOR260: A plan for PRINT-EFFECT0 is UNIX-LPR-Plz-COMMAND0
STRATEGIES FOR EXPRESSING CONCISE, HELPFUL ANSWERS
339
# How can I print a file on the laser printer? The parser produces: (ASK10 (listener10 = UC) (speaker10 = *USER*) (asked-for10 = (QUESTION10 (what-is10 = (ACTION14? (actor14 = *USER*)))))) (PRINT-ACTION0? (pr-effect0 = PRINT-EFFECT0?) (actor0-1 = *USER*) (cause0-0 = (ACTION14? ...))) (HAS-PRINT-DEST0 (pr-dest0 = LASER-PRINTER0) (pr-dest-obj0 = PRINT-EFFECT0?)) (HAS-PRINT-OBJECT1 (pr-object1 = FILE3?) (pr-obj-obj1 = PRINT-EFFECT0?)) The goal analyzer produces: ((HAS-GOAL-ga0 (planner-ga0 = *USER*) (goal-ga0 = (KNOW-ga0? (knower-ga0 = *USER*) (fact-ga0 = (ACTION14? ...)))))) The planner is passed: (PRINT-EFFECT0?) The planner produces: (PLANFOR260 (goals260 = PRINT-EFFECT0?) (plan260 = (UNIX-LPR-Plz-COMMAND0 (lpr-plz-file0 = FILE3?) (UNIX-LPR-Plz-COMMAND-effect0 = PRINT-EFFECT0?)))) (HAS-FILE-NAME18 (named-file18 = FILE3?) (file-name18 = (lisp = nil))) (LPR-Plz-HAS-FORMAT0 (LPR-Plz-HAS-FORMAT-command0 = (UNIX-LPR-Plz-COMMAND0 ...)) (LPR-Plz-HAS-FORMAT-format0 = (LPR-Plz-FORMAT1 (lpr-plz-file-arg1 = (file-name18 = aspectual-of (HAS-FILE-NAME18 ...))) (LPR-Plz-FORMAT-step1 = (SEQUENCE10 (step10 = lpr) (next10 = (CONCAT00 (concat-step00 = -P) (concat-next00 = lz)))))))) (HAS-COMMAND-NAME30 (HAS-COMMAND-NAME-named-obj30 = (UNIX-LPR-Plz-COMMAND0 ...)) (HAS-COMMAND-NAME-name30 = (SEQUENCE10 ...))) Express: now expressing the PLANFOR: (PLANFOR260 ...) Express: not expressing the format of the command, UNIX-LPR-Plz-COMMAND0, since the user already knows it. Express: not expressing PRINT-EFFECT0?, since it is already in the context. The generator is passed: (TELL7 (listener7-0 = *USER*) (speaker7-0 = UC) (proposition7 = (PLANFOR260 ...)) (effect7 = (STATE-CHANGE1 (final-state1 = (KNOW-ga0? ...))))) The generator is passed: (TELL8 (speaker8 = UC) (listener8 = *USER*) (proposition8 = (REMINDER10 ...))) Use lpr -Plz. Don’t forget to file the printer output in the boxes.
Figure 3. UC session with an intermediate user showing trace of UCExpress.
340
DAVID N. CHIN
Figure 4. KODIAK representation of the lpr -Plz plan for printing.
PRINT-EFFECT0: Printing a file on the laser printer LPR-Plz-HAS-FORMAT0: The command-format of the UNIX-LPR-Plz-COMMAND0 is “lpr -Plz ” These three subnetworks are depicted in Figure 4 as regions enclosed in double lines. In traversing this network, UCExpress prunes LAS-PRINTEFFECT0, because “printing a file on the laser printer” is already a part of the context (it is part of the user’s question). Also, the command-format (LPR-Plz-HAS-FORMAT0) is pruned from UC’s actual answer based on information from KNOME. In this case, KNOME was able to deduce that, since the user was not a novice, the user already knew the UNIX-LPR-PlzFORMAT, which is an instance of the SIMPLE-FILE-FORMAT (the name of the command followed by the name of the file to be operated upon), which all non-novice users know. Finally what is left unpruned is the plan
STRATEGIES FOR EXPRESSING CONCISE, HELPFUL ANSWERS
341
part of PLANFOR260, UNIX-LPR-Plz-COMMAND0, which the generator translates as “Use lpr -Plz.” If the user was just a novice, then UC could not assume that the user already knew the command-format and instead would provide the following answer that includes an example of the lpr -Plz command-format: Use lpr -Plz. For example, to print the file foo on the laser printer, type ‘lpr -Plz foo’. Pruning is similar to the “msg-elmt” realization stage of McDonald’s MUMBLE (McDonald 1984), which was used to generate pronouns when a concept had been previously mentioned by MUMBLE. However, since MUMBLE did not have access to a model of the user, it was not able to avoid expressing those concepts which a user model would indicate that the user already knows. Another approach is used by Appelt’s KAMP system (Appelt 1985) in planning referring expressions. KAMP used mutual knowledge as a criterion for planning pronominal and anaphoric noun phrases. It would be very difficult to adapt such an approach to do pruning since KAMP does not deal with the uncertainty that is inherent in user models like KNOME that reason from stereotypes.
5. Formatting After pruning, UCExpress enters the formatting phase, during which it tries to apply different expository formats to express concepts in a clearer manner. UCExpress’ expository formats include example, definition, and simile. Each expository format is used to express different types of information. Formats are triggered by encountering particular concept types in the answer network. After triggering, the procedural component of the expository format is called to transform the concept into the corresponding format. The formats are not simple templates that can be filled in with readily available information. A fair amount of additional processing is needed to transform the information into the right format. 5.1. Example format The example format is used in expressing general knowledge about complex (i.e., multi-step) procedures such as UNIX commands. In UC’s representation of UNIX commands, every command has an associated command format. When expressing a command, UCExpress checks to see if it should also express the format of the command. If KNOME believes that the user already knows the format of the command, then there is no need to express the format.
342
DAVID N. CHIN
Next, UCExpress checks to see if the format of the command is completely specified. If so, UCExpress collapses the command and format into a single statement as in the following UC dialog: # How can I add general write protection to the file personal? Type ‘chmod o-w personal’.
An English rendition of the conceptual network passed to UCExpress for the above example might be something like: A plan for adding general write protection to the file personal is to use the chmod command with format ‘chmod’ followed by concatenating ‘o’ with ‘-’ with ‘w’ followed by ‘personal’. Since the command is completely specified, the format of the command is combined with the command to form a shorter and more easily understood answer. If the command is not completely specified, then UCExpress uses an example format to express the format of the command to the user. The key principle in producing examples is to be explicit. So, UCExpress first steps through a copy of the general procedure to transform any general information into specific instances. In cases where the under-specified part of the procedure has a limited range of options, an arbitrary member that is compatible with the rest of the procedure and with previous UCExpress choices is selected. Next, the new, completely specified copy of the format is combined with a copy of the command, much as in the above UC dialog. Finally the new plan is encapsulated in an example shell (which tells the generator to produce “For example,”). To see the algorithm in more detail, consider the UC dialog shown in Figures 5 and 6. The conceptual answer that is passed to UCExpress in the dialog can be paraphrased in English as: A plan for changing the read permission of a file is to use the chmod command with format ‘chmod’ followed by concatenating with with ‘r’ followed by . In stepping through the above format, is underspecified. In order to give an example, a particular value is needed, so UCExpress arbitrarily chooses a value from the list of possible fillers (user, group, other, or all). The same is done for . In the case of ‘r’, this is already a fully specified value for protection-access-type,
STRATEGIES FOR EXPRESSING CONCISE, HELPFUL ANSWERS
343
# How can I change the read permission of a file? The parser produces: (ASK10 (listener10 = UC) (speaker10 = *USER*) (asked-for10 = (QUESTION10 (what-is10 = (ACTION14? (actor14 = *USER*)))))) (CHANGE-PROT-FILE-ACTION0? (ch-prot-effect0 = (CHANGE-PROT-FILE-EFFECT0? (change-prot0 = FILE-PROTECTION1) (change-file0 = FILE3?))) (actor0-1 = *USER*) (cause0-0 = (ACTION14? ...))) (HAS-FILE-PROTECTION2 (prot-file2 = FILE3?) (file-prot2 = FILE-PROTECTION1)) (HAS-ACCESS-TYPE1 (access-protection-type1 = READ-PROT) (prot-type-arg1 = FILE-PROTECTION1)) The goal analyzer produces: ((HAS-GOAL-ga0 (planner-ga0 = *USER*) (goal-ga0 = (KNOW-ga0? (knower-ga0 = *USER*) (fact-ga0 = (ACTION14? ...)))))) The planner is passed: ((CHANGE-PROT-FILE-EFFECT0? ...))
Figure 5. First half of UC session showing an answer that contains an example.
so UCExpress maintains the selection. However, with , there is no list of possible fillers. Instead, UCExpress calls a special procedure for selecting names. This naming procedure chooses names for files starting with ‘foo’ and continuing in each session with ‘foo1’, ‘foo2’, etc. Other types of names are selected in order from lists of those name types (e.g., machine names are chosen from a list of local machine names). By selecting the names in order, name conflicts (e.g., two different files with the same name) can be avoided. Another consideration in creating examples is that new names must be introduced before their use. Thus ‘foo’ should be introduced as a file before it appears in ‘chmod g+r foo’. This is done implicitly by passing the entire PLANFOR as the example, so that the generator will produce ‘to add group read permission to the file named foo’ as well as the actual plan. 5.2. Definition format The definition format is used to express definitions of terminology. The UCdefine procedure first collects the information that will be expressed in the definition. Collecting the right amount of information involves satisfying the Gricean Maxim of Quantity (Grice 1975). The usual procedure is to collect the information that the term has some semantic category, and then add the primary usage of the term. In rare cases where the node does not have a
344
DAVID N. CHIN
The planner produces: (PLANFOR330 (goals330 = (CHANGE-PROT-FILE-EFFECT0? ...)) (plan330 = (UNIX-CHMOD-COMMAND0 (chmod-file0 = FILE3?) (chmod-protection0 = FILE-PROTECTION1) (UNIX-CHMOD-COMMAND-effect0 = (CHANGE-PROT-FILE-EFFECT0? ...))))) (HAS-FILE-NAME19 (named-file19 = FILE3?) (file-name19 = (lisp = nil))) (HAS-PROT-VALUE1 (prot-type-arg1-1 = FILE-PROTECTION1) (value-protection-type1 = (lisp = nil))) (HAS-USER-TYPE1 (prot-type-arg1-0 = FILE-PROTECTION1) (user-protection-type1 = (lisp = nil))) (CHMOD-HAS-FORMAT0 (CHMOD-HAS-FORMAT-command0 = (UNIX-CHMOD-COMMAND0 ...)) (CHMOD-HAS-FORMAT-format0 = (CHMOD-FORMAT0 (CHMOD-FORMAT-step0 = chmod) (CHMOD-FORMAT-args0 = ...)))) (HAS-COMMAND-NAME80 (HAS-COMMAND-NAME-named-obj80 = (UNIX-CHMOD-COMMAND0 ...)) (HAS-COMMAND-NAME-name80 = chmod)) Express: now expressing the PLANFOR: (PLANFOR330 ...) Express: creating an example for the incomplete plan, CHMOD-FORMAT0 Express: choosing a name, foo, for an example file. Express: selecting USER-PROT -- print name, u, to fill in a parameter of the example. Express: selecting ADD-STATUS -- print name, +, to fill in a parameter of the example. Express: created the example(s): ((TELL7 (speaker7-0 = UC) (listener7-0 = *USER*) (proposition7 = (EXAMPLE0 (example0 = (PLANFOR330-0 (goals330-0 = (CHANGE-PROT-FILE-EFFECT0-0? (change-prot0-0 = FILE-PROTECTION1-0) (change-file0-0 = FILE3-0?))) (plan330-0 = (TYPE-ACTION0 (speaker0-4 = *USER*) (type-string0 = (CHMOD-FORMAT0-0 (CHMOD-FORMAT-step0-0 = chmod) (CHMOD-FORMAT-args0-0 = (CHMOD-TWO-ARG-SEQ0-0 (chmod-file-arg0-0 = ... foo) (CHMOD-TWO-ARG-SEQ-step0-0 = (PROT-ARG-SEQ0-0 (user-bit0-0 = ... u) (PROT-ARG-SEQ-concat-next0-0 = (ARG-SEQ0-0 (value-bit0-0 = ... +) (access-bit0-0 = r))))))))))))))))) Express: not expressing CHANGE-PROT-FILE-EFFECT0?, since it is already in the context. The generator is passed: (TELL6 (effect6 = (STATE-CHANGE1 (final-state1 = (KNOW-ga0? ...)))) (listener6-0 = *USER*) (speaker6-0 = UC) (proposition6 = (PLANFOR330 ...))) The generator is passed: (TELL7 ...) Use chmod. For example, to add group read permission to the file named foo, type ‘chmod g+r foo’.
Figure 6. Second half of UC session showing an answer that contains an example.
STRATEGIES FOR EXPRESSING CONCISE, HELPFUL ANSWERS
345
usage, some other property of the node is chosen. For example, a definition of a directory would include the information: 1. directories are files 2. directories are used to contain files After such information is collected, it must be transformed into a definition format. This involves creating instances of both the term and its category and then combining the two pieces of information into one coherent statement. The latter task requires an attachment inversion where the distinguishing information is reattached to the term’s category rather than to the term itself. For example, consider creating a definition for the term “directory,” which has the category “file” (in UNIX, a directory is simply a special type of file). The information that distinguishes directories from other types of files is that directories are used to contain files (as opposed to documents, movies, spreadsheets, etc.). To create the definition, the distinguishing information is reattached from the term directory to its category, file to create the following definition. User: What is a directory? UC: A directory is a file that is used to contain files. This attachment inversion is not specific to English but seems to be a general universal linguistic phenomenon in the expression of definitions. Here are some other examples of the definition format: User: What is a file? UC: A file is a container that is used to contain text, code, or files. User: What is a container? UC: A container is an object that is used to contain objects. User: What is rm? UC: Rm is a command that is used to delete files. User: What is a search path? UC: A search path is a list of directories that is used by the csh to search for programs to execute. 5.3. Simile format The simile format is used by UCExpress to provide explanations of what a command does in terms of other commands already known to the user. This format is invoked when UCExpress attempts to explain a command that has a sibling or a parent in the command hierarchy that the user already knows (as modeled in KNOME). An example is explaining what ruptime does in terms of the command uptime. A trace of UC’s processing is shown in Figure 7.
346
DAVID N. CHIN
# What does ruptime do? The parser produces: (ASK10 (listener10 = UC) (speaker10 = *USER*) (asked-for10 = (QUESTION10 (what-is10 = STATE13?)))) (HAS-EFFECT21? (effect-of-command21 = STATE13?) (command-of-effect21 = UNIX-RUPTIME-COMMAND0)) The goal analyzer produces: ((HAS-GOAL-ga0 (planner-ga0 = *USER*) (goal-ga0 = (KNOW-ga0? (knower-ga0 = *USER*) (fact-ga0 = STATE13?))))) UCEgo: trying to find effects for UNIX-RUPTIME-COMMAND0 the effects are: ((HAS-EFFECT6-0 (command-of-effect6-0 = (UNIX-RUPTIME-COMMAND0 ...)) (effect-of-command6-0 = (LIST-ACTION3-0 (list-loc3-0 = TERMINAL1-0) (list-objs3-0 = UP-TIME1-0)))) (HAS-EFFECT7-0 (command-of-effect7-0 = (UNIX-RUPTIME-COMMAND0 ...)) (effect-of-command7-0 = (LIST-ACTION4-0 (list-loc4-0 = TERMINAL1-0) (list-objs4-0 = NUMBER1-0)))) (HAS-EFFECT8-0 (command-of-effect8-0 = (UNIX-RUPTIME-COMMAND0 ...)) (effect-of-command8-0 = (LIST-ACTION5-0 (list-loc5-0 = TERMINAL1-0) (list-objs5-0 = LOAD-AVERAGE1-0))))) UCExpress: Found a related command, so creating a comparison between UNIX-RUPTIME-COMMAND2 and UNIX-UPTIME-COMMAND0 Express: not expressing UNIX-RUPTIME-COMMAND0, since it is already in the context. The generator is passed: (TELL5 (effect5 = (STATE-CHANGE1 (final-state1 = (KNOW-ga0? ...)))) (listener5-0 = *USER*) (speaker5-0 = UC) (proposition5 = (HAS-EFFECT24 (command-of-effect24 = (UNIX-RUPTIME-COMMAND0 ...)) (effect-of-command24 = (AND0 (step0-0 = (LIST-ACTION3-0 (list-loc3-0 = TERMINAL1-0) (list-objs3-0 = UP-TIME1-0))) (next0-0 = (AND1 (step1-0 = (LIST-ACTION4-0 (list-loc4-0 = TERMINAL1-0) (list-objs4-0 = NUMBER1-0))) (next1-0 = (LIST-ACTION5-0 (list-loc5-0 = TERMINAL1-0) (list-objs5-0 = LOAD-AVERAGE1-0)))))))))) ruptime is like uptime, except ruptime is for all machines on the network.
Figure 7. UC session showing the simile format.
The processing involves comparing the effects of the two commands and noting where they differ. In the above example, the effects of uptime are to list the uptime of the user’s machine, list the number of all users on it, and list its load average. The effects of ruptime are similar except it is for all machines on the user’s network. The comparison algorithm does a network comparison of the effects of the two commands. A collection of differences is
STRATEGIES FOR EXPRESSING CONCISE, HELPFUL ANSWERS
347
generated, and the cost of expressing these differences (measured in number of concepts) is compared with the cost of simply stating the effects of the command. If expressing the differences is more costly, then the simile format is not used. On the other hand, if expressing the differences is less costly, then the differences are combined into a shell of the form “ is like , except [ also ...] [and] [ does not ...] [and] ...”
6. Conclusion 6.1. A comparison McKeown’s TEXT system (1985) is perhaps the closest in spirit to UCExpress. TEXT provided definitions using an identification schema. This is similar to UCExpress’ definition format except TEXT did not worry about how much information to convey. TEXT was designed to produce only paragraph length descriptions, hence it was not overly concerned with how much information to provide. The definition format requires more knowledge about the domain in order to select the most relevant information for a short description. TEXT also used a compare and contrast schema to answer questions about the differences between objects in a database. This is similar to UCExpress’ simile format except that the compare and contrast schema was not used for giving descriptions of an object in terms of another that the user already knew. Since TEXT did not have a complete model of the user, it was unable to determine if the user already knew another object that could be contrasted with the requested object. This lack of a user model was also evident in the fact that TEXT did not provide anything similar to the pruning phase of UCExpress. Pruning is probably more relevant in a conversational context such as UC as contrasted with a paragraph generation context such as TEXT. On the other hand, TEXT was able to keep track of the conversational focus much better than UC. Focus does not seem to be quite as essential for a system like UC that gives brief answers. Other related research includes work on using examples for explanation and for argument in a legal domain (Rissland 1983; Rissland et al. 1984). The difference between those examples and the examples created by UCExpress is that Rissland’s examples are pre-formed and stored in a database of examples whereas UCExpress creates examples interactively, taking into account user provided parameters. Rissland’s HELP system dealt only with help about particular subjects or commands rather than arbitrary English questions like UC, thus HELP did not have to deal with questions such as how to print on a particular printer. Also by using pre-stored text, HELP was not concerned
348
DAVID N. CHIN
with the problem of transforming knowledge useful for internal computation in a planner to a format usable by a generator. The TAILOR system (Paris 1989) used an idea of user expertise similar to KNOME’s to tailor explanations to the user’s level of expertise. TAILOR concentrated on higher level strategies for explanation than UCExpress. For example, TAILOR used notions of the user’s level of expertise to choose among process-oriented or parts-oriented description strategies in building up a paragraph. TAILOR could also mix the two types of strategies within a paragraph to explain different aspects of a system. Such considerations are more important when generating longer explanations as in TAILOR, than when generating brief explanations as in UCExpress. 6.2. Summary UC separates the realization of speech acts into two processes: deciding how to express the speech act in UCExpress, and deciding which phrases and words to use in UC’s tactical level generator. Through this separation, the pragmatic knowledge needed by expression is separated from the grammatical knowledge needed by generation. UCExpress makes decisions on pragmatic grounds such as the conversational context, the user’s knowledge, and the ease of understanding of various expository formats. These decisions serve to constrain the generator’s choice of words and grammatical constructions. Of course, it is sometimes impossible to realize all pragmatic constraints. For example, UCExpress may specify that a pronoun should be used to refer to some concept since this concept is part of the conversational context, but this may not be realizable in a particular language because using a pronoun in that case may interfere with a previous pronoun (in another language with stronger typed pronouns, there may not be any interference). In such cases, the generator needs to be able to relax the constraints. By passing the generator all of the conceptual network along with additional pragmatic markings on the network UCExpress allows the generator to relax constraints as needed. This way, the generator has access to any information needed to relax the constraints added by UCExpress.
7. Recent developments An abbreviated version of this paper was previously presented at the 1988 AAAI Conference (Chin 88). Answer expression is currently being extended for multi-media output in the MC (Maintenance Consultant) system. A
STRATEGIES FOR EXPRESSING CONCISE, HELPFUL ANSWERS
349
description of of the multi-media input and user modeling aspects of MC can be found in (Chin et al. 94).
Acknowledgements The work described in this paper was done at the University of California, Berkeley as part of my Ph.D. thesis. I wish to thank Robert Wilensky who supervised this work. I also wish to thank the members of BAIR (Berkeley Artificial Intelligence Research) who have contributed to the UC project. This research was sponsored in part by the Defense Advanced Research Projects Agency (DoD), ARPA order No. 4871, monitored by Space and Naval Warfare Systems Command Command under contract N00039-84-C0089, by the Office of Naval Research under contract N00014-80-C-0732, by the National Science Foundation under grant MCS79-06543, and by the Office of Naval Research under contract N00014-97-1-0578.
Note 1 UNIX is a trademark of X/Open, Inc.
References Appelt, D. E. (1985). Planning English Sentences. Cambridge: Cambridge University Press. Chin, D. N. (1986). User Modeling in UC, the UNIX Consultant. In Proceedings of The CHI86 Conference, 24–28. Boston, MA: Association for Computing Machinery. Chin, D. N. (1987). Intelligent Agents as a Basis for Natural Language Interfaces. Ph.D. diss., Computer Science Division, University of California, Berkeley, CA. Also available as UCB/CSD 88/396, Computer Science Division, University of California, Berkeley, CA. Chin, D. N. (1988). Exploiting User Expertise in Answer Expression. In Proceedings of The Seventh National Conference on Artificial Intelligence, 756–760. Saint Paul, MN: AAAI Press. Chin, D. N. (1989). KNOME: Modeling What the User Knows in UC. In Kobsa, A. & Wahlster, W. (eds.) User Models in Dialog Systems, 74–107. Berlin: Springer-Verlag. Chin, D. N., Inaba, M., Pareek, H., Nemoto, K., Wasson, M. & Miyamoto, I. (1994). Multi-Dimensional User Models for Multi-media I/O in the Maintenance Consultant. In Proceedings of Fourth International Conference on User Modeling, 139–144. Hyannis, MA: User Modeling, Inc. Grice, H. P. (1975). Logic and Conversation. In Cole, P. & Morgan, J. L. (eds.), Studies in Syntax III, 41–58. New York: Seminar Press. Jacobs, P. S. (1986). A Knowledge-Based Approach to Language Production. Ph.D. diss., University of California, Berkeley, CA. Also available as UCB/CSD 86/254, Computer Science Division, University of California, Berkeley, CA.
350
DAVID N. CHIN
Luria, M. (1982). Dividing up the Question Answering Process. In Proceedings of The Second National Conference on Artificial Intelligence, 71–74. Pittsburgh, PA: AAAI Press. McDonald, D. D. (1984). Natural Language Generation as a Computational Problem: an Introduction. In Brady, M. & Berwick, R. C. (eds.), Computational Models of Discourse, 209–265. Cambridge, MA: MIT Press. McKeown, K. R. (1985). Discourse Strategies for Generating Natural-Language Text. Artificial Intelligence 27: 1–41. Paris, C. L. (1989). Tailoring Object Descriptions to a User’s Level of Expertise. In Kobsa, A. & Wahlster, W. (eds.) User Models in Dialog Systems, 200–232. Berlin: Springer-Verlag. Rich, E. (1979). User Modeling via Stereotypes. Cognitive Science 3: 329–354. Rissland, E. L. (1983). Examples in Legal Reasoning: Legal Hypotheticals. In Proceedings of The Eighth International Joint Conference on Artificial Intelligence 1, 90–93. Karlsruhe, Germany: Morgan Kaufmann Publishers. Rissland, E. L., Valcarce, E. M. & Ashley, K. D. (1984). Explaining and Arguing with Examples. In Proceedings of The Fourth National Conference on Artificial Intelligence, 288–294. Austin, TX: AAAI Press. Rosch, E. (1978). Principles of Categorization. In Rosch, E. & Lloyd, B. B. (eds.) Cognition and Categorization. Hillsdale, NJ: Lawrence Erlbaum. Wilensky, R. (1987). Some Problems and Proposals for Knowledge Representation. UCB/CSD 87/351, Computer Science Division, University of California, Berkeley, CA. Wilensky, R., Arens, Y. & Chin, D. N. (1984). Talking to UNIX in English: An Overview of UC. Communications of the ACM 27(6): 574–593. Wilensky, R., Chin, D. N., Luria, M., Martin, J., Mayfield, J. & Wu, D. (1988). The Berkeley UNIX Consultant Project. Computational Linguistics 14(4): 35–84.
Evaluating Plan Recognition Systems: Three Properties of a Good Explanation JAMES MAYFIELD The Johns Hopkins University Applied Physics Laboratory, 11100 Johns Hopkins Road, Laurel, MD 20723-6099 USA. E-mail: [email protected] Abstract. Plan recognition in a dialogue system is the process of explaining why an utterance was made, in terms of the plans and goals that its speaker was pursuing in making the utterance. I present a theory of how such an explanation of an utterance may be judged as to its merits as an explanation. I propose three criteria for making such judgments: applicability, grounding, and completeness. The first criterion is the applicability of the explanation to the needs of the system that will use it. The second criterion is the grounding of the explanation in what is already known of the speaker and of the dialogue. Finally, the third criterion is the completeness of the explanation’s coverage of the goals that motivated the production of the utterance. An explanation of an utterance is a good explanation of that utterance to the extent that it meets these three criteria. In addition to forming the basis of a method for evaluating the merit of an explanation, these criteria are useful in designing and evaluating a plan recognition algorithm and its associated knowledge base. Keywords: dialogue systems, natural language processing, evaluation, explanation, plan recognition
1. Introduction In its broadest interpretation, an explanation of an event is a set of conditions that allow or cause the event to occur. There is a huge number of such explanatory conditions for any given event. To be useful in understanding or in formulating a response to an observed event, an explanation of that event must contain only a subset of these conditions. Some of the more important types of conditions are: 1. States of the world 2. Causal relationships 3. Beliefs, intentions, and affect of agents 4. Other events, including actions 5. Physical laws The types of conditions selected to be part of an explanation dictate the character of the explanation. For example, the following are potential explanations for the overwriting of a particular file:
352
JAMES MAYFIELD
1. The Emacs editor overwrote it 2. Melissa edited it 3. Melissa was feeling malicious 4. It wasn’t write-protected At first, this final statement seems to be less an explanation of the event than an enabling condition of it. However, it seems to be a perfectly good explanation of the event relative to the question ‘How could it have been overwritten?’ Other types of explanations include scientific explanations, political explanations, affective explanations, etc. Schank (1986) even suggests explanations composed of questions. These examples demonstrate that there are many different types of explanations for a single event that might be useful for a particular task. This paper details what it means for an explanation of an utterance (which is one kind of event) to be a good one. I formalize the concept of ‘goodness’ by suggesting three criteria by which the quality of an explanation of an utterance may be judged. While these criteria are intended to apply to explanations of utterances, they are also useful for a broad range of explanation-based tasks. There are three reasons that it is useful to study criteria for evaluating explanations. First, they can be of help in the design of a knowledge base, because they indicate what types of representations will lead to the desired explanations. Secondly, they assist in the design of a plan recognition algorithm by placing constraints on the output of such an algorithm. Finally, such criteria provide a way to evaluate the output of a plan recognizer, once it has been implemented. To demonstrate the usefulness of these criteria in the evaluation of plan recognition systems, I apply them to the output of the plan recognizer for the Unix1 Consultant (UC) project (Wilensky et al. 1984; Wilensky et al. 1988). UC serves as a consultant for novice users of the UNIX operating system. UC’s plan recognition component is called PAGAN (for Plan And Goal ANalyzer). PAGAN’s job is to build explanations of the user’s utterances.2 That is, PAGAN must infer an interrelated set of plans and goals held by the user that led her to make the utterance. The representation of plans and goals that PAGAN uses, as well as its algorithm for constructing explanations, are described in Section 6. 2. Criteria for Evaluating Explanations A good explanation of an utterance meets the following criteria: 1. Applicability 2. Grounding 3. Completeness
EVALUATING PLAN RECOGNITION SYSTEMS
353
The applicability criterion states that a good explanation of an utterance is applicable to the needs of the system that will use that explanation. The grounding criterion states that a good explanation of an utterance is grounded in what is already known of the speaker and of the dialogue. The completeness criterion states that a good explanation of an utterance covers every aspect of the utterance in depth; it leaves no portion of the utterance unexplained. These criteria apply to the explanation proper, and not to the algorithm used to construct it. That is, they are criteria that one applies to an explanation to determine how good it is, not to an algorithm to see how well the algorithm creates explanations (although they can of course be applied to the output of a plan recognition algorithm). In the following sections, I describe each of these criteria in more detail.
3. The Principle of Applicability The principle of applicability states that a good explanation of an utterance is applicable to the needs of the system that will use it. That is, no matter how good an explanation might be in other respects, if it doesn’t give the system that will use it what that system needs to do its job, then it is not a good explanation. An assumption that underlies the principle of applicability is that a system that is trying to explain an utterance has interests of its own, which understanding the utterance might help to further. Thus, this principle implies that the operation of a plan recognizer cannot be independent of the system in which it is embedded, and therefore the concept of a domain-independent plan recognizer is impractical. Of course, it might be possible to isolate the portions of a plan recognizer that rely on the particular task to be performed, and represent those portions declaratively. The remaining inference engine would be domain-independent, but the plan recognizer as a whole would not. The reader might have noticed that I have not included accuracy among the criteria for evaluating explanations. There are two reasons for this. First, accuracy is in general subsumed by applicability. That is, in most cases an explanation of an utterance that does not accurately reflect the speaker’s intentions will not be applicable to the needs of the system. Secondly, in some cases, inaccurate explanations are perfectly acceptable. A system designed to simulate a paranoid schizophrenic for example might prefer a delusional interpretation of an utterance to an accurate one. Closely related to the issue of accuracy is the issue of user misconceptions. It is possible for the user’s beliefs to be at odds with the system’s beliefs. In such cases, the plan recognizer should usually faithfully model the user’s beliefs, even when they are incorrect. Much research has been devoted
354
JAMES MAYFIELD
to the recognition of misconceptions, including that of Calistri-Yeh (1991), Chin (1988), Eller and Carberry (1992), Pollack (1984), Quilici (1989), and Retz-Schmidt (1991). There are three dimensions of an explanation along which applicability may be assessed: 1. Composition 2. Content 3. Granularity Applicability of composition concerns the type of element out of which explanations should be constructed. Applicability of content deals with the particular choice of elements that compose an explanation. Finally, applicability of granularity covers the level of generality of the elements of a particular explanation. These dimensions are discussed in the following sections. 3.1. Applicability of composition The principle of applicability of composition holds that the type of the elements that compose an explanation must be applicable to the needs of the system. The composition of a good explanation is largely a reflection of how that explanation will be put to use; different tasks require different types of explanations. For example, a system whose task is to build a model of a user’s knowledge will need to build explanations that are composed of facts about the user that allowed the user to produce an utterance. In such a system, an explanation of: U SER: Can you tell me how to delete a file? might be composed of facts such as: 1. The user does not know how to delete a file. 2. The user believes that the system knows how to delete a file. 3. The user believes that the system will cooperate with the plan. 4. The user knows what a file is, and what it means to delete one. 5. The user understands the task of the system. 6. The user understands English. For a consulting system such as UC, the knowledge that is applicable to the system’s task is knowledge of the plans and goals of the user. This is because the purpose of such a system is to address the user’s goals. Thus, an explanation of this question for UC’s purposes might be composed of facts such as: 1. The user wants to know how to delete a file. 2. The user expects to be told how to delete a file.
EVALUATING PLAN RECOGNITION SYSTEMS
355
3. The user wants to delete a file. 4. The user wants to release disk space. Notice that these facts include both goals that the speaker holds and actions that the speaker expects the hearer to carry out. Because the understanding of a speaker’s plans and goals is crucial to so many domains that might benefit from a natural-language interface, this paper focuses on explanations composed of plans and goals. 3.2. Applicability of content The principle of applicability of content holds that each of the components of a particular explanation should be applicable to the needs of the system. A typical utterance is rarely aimed at achieving a single isolated goal, but rather sits at the tip of a whole chain of motivating goals. Not all such goals will be applicable to a given system though. A good explanation of an utterance will not include all of these goals, but will include only those that are applicable to the purposes of the system. For example, the novice who asks: U SER: How can I print a file on the laser printer? might have asked the question so as to find out how to get a printout, so as to obtain a printout of the file, so as to check its contents for accuracy, so as to turn in an accurate report, so as to get a good grade, so as to graduate with a high grade point average, so as to get a good job, and so on. Only a portion of this chain of motivating goals is likely to be applicable to the purposes of a particular system, and a good explanation will include only that portion. UC’s contract with its users for example is to provide information on the use of the UNIX operating system. Thus, in the UC context, a good explanation will not include goals that go beyond the use of UNIX. In this example, the last few goals in the list of motivating goals are beyond UC’s purview, and therefore should not be a part of an explanation designed for use by UC. At the other end of the spectrum, a good explanation of an utterance must include at least one goal that is within the system’s domain of expertise. Thus, if an explanation of the utterance: U SER: I have a problem with ls. includes only the goal of informing the hearer of the problem or the goal of the hearer knowing about the problem, that explanation is an inadequate one for the UC domain. An adequate explanation of this statement for UC’s purposes would indicate that the user is attempting to solicit help from UC, since this is a goal that is applicable to UC’s task. In practice, adherence to the principle of completeness (described in Section 5) ensures that an explanation will have at least this desired depth.
356
JAMES MAYFIELD
3.3. Applicability of granularity The third dimension along which applicability can be assessed is the granularity of the explanation. The principle of applicability of granularity holds that a good explanation contains the right amount of detail. It is often possible to divide a single action into a number of subactions. The extent to which an explanation is divided in this way is its level of granularity. Different levels of granularity may be more or less applicable to a given system. For example, suppose a user tells UC: U SER: I want to add public read permission to my file called blatz. An explanation of this statement that indicates that the speaker probably wants to use a UNIX command has a reasonable level of granularity for UC. On the other hand, an explanation that states that the speaker probably wants to type each letter of the name of a UNIX command and each letter of the file name is not a reasonable one for UC’s purposes. While it is likely to be correct, this explanation is nevertheless too fine-grained to be of use. Of course, if UC is concerned that the user might not know how to spell the command name, then the latter explanation above may be quite applicable. This points out that the type of explanation that is applicable can change as the short-term goals of the system change. Thus, such goals must always be taken into account when judging the merits of an explanation.
4. The Principle of Grounding The second criterion for evaluating an explanation is the principle of grounding. This principle states that a good explanation of an utterance relates what is inferred from that utterance to what is already known about the speaker and the dialogue. Typically, before hearing an utterance, a system will already have some knowledge that is applicable to the processing of that utterance. This knowledge is of two types: 1. Knowledge of the dialogue 2. Knowledge of the speaker First, the system might have already engaged in dialogue with the speaker. Secondly, the system might have some stored knowledge about the user, or be able to make informed guesses about what the speaker knows or believes. It follows from the existence of these two kinds of knowledge that there are two ways that an explanation can be grounded in existing knowledge: 1. By relating it to knowledge of the dialogue 2. By relating it to knowledge of the speaker
EVALUATING PLAN RECOGNITION SYSTEMS
357
The first way that an explanation can be grounded in existing knowledge is by relating it to knowledge of the dialogue. For example, it is difficult to imagine a good explanation for the utterance: U SER: It’s called crazy-fingers. that does not relate the utterance to some preceding dialogue. The main reason for this is that the subject of the sentence is a pronoun with no clear referent. Consider now the exchange: UC: What’s the name of the file? U SER: It’s called crazy-fingers. In this example, UC’s question provides a background against which the user’s statement can be understood. A good explanation of the user’s utterance must relate the utterance to this previous dialogue. A second way that an explanation can be grounded in existing knowledge is by lending credence to, or by casting doubt on, something that the system already believes of the user. In this example, the user’s response in the exchange: UC: What’s the name of the file? U SER: It’s called crazy-fingers. might lend credence to UC’s belief that the user knows the ‘ls’ command (which lists file names in UNIX), thereby grounding the statement in knowledge that UC previously held about the speaker (in addition to grounding it in UC’s knowledge of the dialogue). This kind of grounding lies in areas of user modeling outside of plan recognition, and I will not discuss it further. See Kass and Finin (1988) for an introduction to the broad range of components of a general user model. An assumption that motivates the principle of grounding is that the utterance to be explained is part of a larger rhetorical structure such as a dialogue or text. The significance of this assumption is that interactive systems that don’t engage in dialogue, such as simple question-answering systems, can largely ignore the principle of grounding. However, such systems are limited in their usefulness as natural language systems; dialogue is an important part of language. There are two important aspects of an explanation that is grounded in knowledge of preceding dialogue: 1. The type of concept to which the explanation is attached 2. The type of attachment First, it is useful to categorize the type of knowledge to which the utterance is connected. There are many such concepts; I am concerned only with plans and goals. An explanation may attach to either of these components of the
358
JAMES MAYFIELD
dialogue model, or possibly to both of them. Secondly, the type of attachment to a concept is important, and can be categorized. The following sections are divided according to the type of knowledge to which an explanation is connected. Within each section, I discuss the ways that an explanation can be attached to a concept of that type. 4.1. How an explanation attaches to a plan There are three ways that an utterance can connect to a plan that a speaker is already pursuing: 1. Plan continuation 2. Plan delay 3. Plan rejection These types of grounding reflect the status of an existing plan relative to the event to be explained. Whenever there is such an active plan, each new utterance to be explained will relate to that plan in one of these three ways. As there is usually one or more active plans at any point in a dialogue, this aspect of grounding is widespread. The following sections describe each of these types of grounding in detail. 4.1.1. Plan continuation A good explanation of an event includes every plan in which the event is a step. If such a plan is part of the system’s existing model of the user’s plans and goals, then the explanation is thereby grounded in prior knowledge. For example, consider the exchange: UC: Is the file in your directory? U SER: Yes. UC has initiated the plan of first asking a question then receiving a response, so as to achieve the goal of knowing whether the file under discussion is in the user’s directory. Prior to the processing of the user’s utterance, the second step of this plan (a yes/no response by the user) has not yet been observed. Therefore, UC’s question has set up a specific expectation about how the user will proceed, namely that she will provide a yes/no answer. A good explanation of the user’s utterance in this example will include UC’s plan, because the uttering of an affirmative response is a continuation of that plan. It is important to note that this type of grounding is applicable to any goal that has been inferred to help explain an utterance, not just the lowest-level one. For instance, if the user had instead responded to the above question: UC: Is the file in your directory? U SER: Is the Pope Catholic?
EVALUATING PLAN RECOGNITION SYSTEMS
359
the expectation of a yes/no response could not be used directly to ground the explanation that the speaker wants to ask a rhetorical question about the Pope’s religion. However, asking such a question is a plan for the goal of conveying an affirmative response. This goal can be grounded in the expectation of a yes/no response by plan continuation. The explanation that the speaker wants to ask a rhetorical question is therefore indirectly grounded in the expectation. Thus, a good explanation of the user’s utterance in this example would indicate that the question was asked as a plan for the goal of conveying an affirmative answer, and that conveying an affirmative answer was done as a plan for the goal of UC knowing the answer to its question. In dialogue, plan continuation may be signaled by a participant before it occurs: UC: What is the name of the file? U SER: Hold on, let me check. Here, the user has initiated a plan of informing UC about an upcoming plan continuation. The remaining steps of the user’s plan are to obtain the requested information, then to inform UC. This plan as a whole constitutes a continuation of UC’s plan. 4.1.2. Plan delay When one participant in a dialogue (or in fact in any cooperative venture) initiates a plan that requires the participation of other agents, it does not follow that the other participants will readily accept the plan and agree to continue it. They may instead decide that for some reason they should take another action. There are two general categories into which phenomena of this type may be placed: plan delay, and plan rejection. Plan delay, an interruption in the execution of a plan, is the subject of this section. Plan rejection, a termination of the execution of a plan by one of its participants, is discussed in the next section. Plan delay occurs when a plan participant wishes to delay the execution of the plan until some later time. There are three types of plan delay: 1. Clarification delay 2. Skepticism delay 3. Extra-schematic delay A clarification delay is a delay designed to fill a gap in an agent’s knowledge of the plan or goal. A skepticism delay is one in which the agent fully understands the plan or goal, but questions its validity. Finally, an extra-schematic delay is one that arises due to circumstances outside of the plan being delayed. These three types of plan delay are the subject of the following sections.
360
JAMES MAYFIELD
Clarification delay Clarification delay is delay designed to fill a gap in a participant’s knowledge of the plan. Consider the exchange: UC: What is the name of the file? U SER: Do you mean the file I want to copy? In this example, the user is not sure what question is being asked. Rather than selecting a particular interpretation of the question and answering it, thereby continuing the plan, the user decides to get clarification about the meaning of the question from UC. This delays UC’s plan until the clarification is made. Clarification delay is terminated (that is, the original plan continues) when the subplan is completed via plan continuation (see above), or when it is discarded via plan rejection (see below). In this example, once UC indicates the intended meaning of the question, the user can continue UC’s original plan. Skepticism delay Skepticism delay arises when a plan participant fully understands the plan being delayed, but is skeptical about its efficacy or efficiency: UC: What is the name of the file? U SER: I’m not sure I have it. Here, the user is questioning whether UC’s plan will work at all. The user can also suggest that there might be a more efficient plan: UC: What is the name of the file? U SER: Can’t you look for it? In this example, the user is delaying UC’s plan by questioning whether it wouldn’t be more efficient for UC to search out the information itself. Skepticism delay is completed, and the delayed plan is continued, when the skepticism is allayed. Since it is possible that the skepticism might not be allayed, the system must have the ability to infer that plan delay has been converted to plan rejection. Plan rejection is discussed below. Extra-schematic delay The third kind of plan delay is extra-schematic delay. Extra-schematic delay is plan delay that arises due to concerns outside of the plan being delayed: UC: What is the name of the file? U SER: That reminds me of the one about the file and the Stanford student ...
EVALUATING PLAN RECOGNITION SYSTEMS
361
In this example the plan delay is triggered by a component of UC’s plan, but the plan introduced by the user does not itself bear on UC’s plan. 4.1.3. Plan rejection The other way that an utterance may fail to continue an existing plan is by rejecting the plan as flawed in some way. As with skepticism delay, plan rejection may stem either from efficacy or efficiency considerations. A plan participant may reject a plan for reasons of efficacy either because the plan cannot be carried out, or because one of the effects of the plan is undesired. For example, a participant might be unable to answer a question: UC: What is the name of the file? U SER: I don’t know. Here, UC has asked the user for the name of some file under discussion. UC now expects the user to tell it the name. Since the user doesn’t know what the name is, she rejects the plan by indicating her inability to provide the answer. A plan participant may reject a plan for reasons of efficiency either because there is a better way to achieve the desired outcome, or because the execution of the selected plan would be too expensive in an absolute sense. For example, the participant might indicate that another agent would be better able to perform a portion of the plan: UC: What is the name of the file? U SER: You should ask Marcia. Here, a new plan is suggested by the user, thereby rejecting UC’s initial plan. 4.1.4. The relationship between plan delay and plan rejection Although it is useful to treat plan delay and plan rejection as completely separate phenomena, they are in fact closely related. This is because there is never a guarantee that a delayed plan will be resumed. Thus, when the system detects plan delay, it must have the ability to treat it as plan rejection. Furthermore, plan rejection will never be absolute, because the speaker always has the option of taking it up again. Thus the distinction between plan delay and plan rejection is one of degree. 4.2. Connecting an explanation to a goal There are three ways that an event may attach to a goal. These ways parallel the ways that an event can attach to a plan: 1. Goal achievement 2. Goal scrutiny 3. Goal rejection
362
JAMES MAYFIELD
These three types of attachment to a goal are described in the following sections. 4.2.1. Goal achievement An action can achieve a goal, either by continuing an existing plan, or by introducing a new plan for an existing goal (cf. Wilensky (1983)). The former case occurs when an action is grounded by plan continuation. The latter case occurs when an action does not conform to a specific expectation, and instead addresses a higher-level goal (that is, a goal that motivated the plan containing the expected action). Suppose that instead of responding “yes,” as in an earlier example, the user engages in the following dialogue: UC: Is the file in your directory? U SER: It’s in /tmp. Many accounts of yes/no questions have tried to view a response such as this one as an answer to the yes/no question asked. Such approaches require a refinement of the notion of what it means to be a yes/no question. For example, Hirschberg (1984) represents yes/no questions as scalar queries rather than as questions that take simple yes/no answers. In contrast to such positions, I believe it is important to view yes/no questions themselves as questions that can only take yes/no answers or answers that can be inferred to indicate yes/no answers independent of the context of the question (as in the response ‘Is the Pope Catholic?’). The real complexity inherent in yes/no questions arises because yes/no questions are typically used in service of higher-level goals. A response to a yes/no question can address such a higher-level goal, but an answer to a yes/no question can only be yes or no. The above example is a case in point. There is no way to construe this response as either an affirmative answer or a negative answer to the question.3 Thus, the lowest-level expectation is not met. However, the goal UC was addressing in asking the question was to know the location of the file. This goal is itself an expectation, because it is a step of a higher-level plan that has not yet been completed. By informing UC of the file’s location, the response succeeds in addressing this goal, independent of UC’s original plan of receiving a yes/no response to its question; the utterance therefore matches the expectation that the user will address the goal. Thus, a response such as this one is grounded both by goal achievement, and by plan rejection. In a query such as the one given below, an explanation of the user response ‘no’ cannot be generated without a misconception detection component, because such a response is malformed: UC: Is the file in your directory? U SER: No.
EVALUATING PLAN RECOGNITION SYSTEMS
363
This is because such a question is usually in service of the goal of knowing the location of the file, and whereas an affirmative response satisfies this higherlevel goal, a negative response does not. However, there are cases where a negative response is perfectly appropriate. For example, there are some UNIX utilities that require that certain files are located in the user’s directory. If the user were inquiring as to why such a utility wasn’t working, then the above exchange is perfectly reasonable. The point here is that it is not a plan recognizer’s responsibility to determine whether a response such as this one is malformed. Rather, this task should fall to a separate component that detects user misconceptions [cf. discussion in Section 3]. 4.2.2. Goal scrutiny The second way that an event can be grounded in a goal is by questioning that goal in some way. Consider for example the exchange: UC: What is the name of the file? U SER: Why do you want to know? Here, the user is skeptical of UC’s need for the information it is requesting, and so questions whether UC’s goal is a legitimate one. Notice that in addition to questioning UC’s goal, this response also delays UC’s plan. 4.2.3. Goal rejection The third way that an event can be grounded in a goal is by rejecting that goal. This may occur either because the goal should not be achieved, or because it cannot be achieved. For example, in the following exchange the user indicates that there is a reason that UC’s goal should not be fulfilled: UC: What is the name of the file? U SER: Sharon doesn’t want anyone else to know. Thus, UC’s goal is rejected. The other reason that a participant may reject a goal is if the participant believes that the goal cannot be achieved. For example, the goal might not be achievable if some premise of the goal is incorrect. Consider the exchange: UC: What is the name of the file? U SER: I haven’t typed the data into a file yet. Here, UC has been led to believe that the data are contained in a UNIX file. This premise is incorrect, and consequently it is impossible for the user to answer the question. Instead, the user indicates the reason that the question is flawed. The extreme case of this phenomenon is when the user’s overall goal has been incorrectly analyzed:
364
JAMES MAYFIELD
UC: What’s the name of the file you want to delete? U SER: I don’t want to delete anything. Here, UC has incorrectly inferred that the user desires to delete a file, and has initiated a plan to determine which file is to be deleted. Since the user never intended that a file be deleted, she rejects UC’s goal.
5. The Principle of Completeness The principle of completeness states that a good explanation of an utterance covers every aspect of the utterance; it leaves no portion of the utterance, or of the explanation itself, unexplained. There are two kinds of completeness: 1. Depth Completeness 2. Breadth Completeness Depth completeness is completeness of a single line of explanation of an utterance. An explanation of an utterance exhibiting depth completeness includes a goal that motivated the production of the utterance, and a goal or theme to explain each inferred goal. Breadth completeness is coverage of all aspects of the utterance. An explanation that exhibits breadth completeness includes every goal that motivated the production of the utterance. The following sections describe these two kinds of completeness. 5.1. Depth completeness When an explanation of an utterance is inferred, that explanation itself may be subject to explanation. For example, the goal of requesting that UC tell the user how to compress a file might explain the utterance: U SER: Can you tell me how to compress a file? That goal itself can be explained by the user’s goal of knowing how to compress a file, which might in turn be explained by the user’s goal of compressing a file, etc. The depth completeness criterion states that a good explanation includes an explanation of every goal inferred in this way. Of course, if the only type of explanation for a goal were another goal, strict adherence to the depth completeness criterion would require infinite explanations. There are two ways around this apparent quandary. First, a non-intentional explanation (Schank and Abelson 1977; Wilensky 1983) can be used to explain a goal, thereby terminating the chain of explanation. Secondly, the principle of applicability can be called into play. The principle of completeness is often at odds with the principle of applicability. Where the applicability criterion dictates that a particular portion of an explanation should be omitted, the completeness criterion dictates that it should remain a
EVALUATING PLAN RECOGNITION SYSTEMS
365
part of the explanation. In such cases, the applicability criterion takes precedence; there is little sense in having a complete explanation of an utterance if it isn’t applicable to the task at hand. The modified depth completeness criterion is then that a good explanation includes an explanation of every inferred goal in the domain of discourse. 5.2. Breadth completeness The breadth completeness criterion requires that a good explanation of an utterance or of an inferred goal include every motivating goal of that utterance or inferred goal. It is derivative of Wilensky’s notion of exhaustion (Wilensky 1983). The breadth completeness criterion is based on a general principle of planning, namely that it is often possible to address more than one goal with a single action. Appelt (1985) and Norvig (1988) describe this phenomenon. For example, a person can both ask what ‘vi’ does, and indicate a suspicion that it is an editor, with a question like: U SER: Is vi an editor? The most obvious way to address two goals with a single utterance is by explicitly listing two requests, as in: U SER: I want to know how to rename a file and how to copy a file. One might claim that such a statement is no more than a weak association of two separate statements within the same sentence. However, until a principled way is available to determine how a given input should be divided before it is processed, sentences will remain the natural processing unit. Thus, an utterance such as this one must be handled as a single request, since it uses a single sentence. A complete explanation of this statement must then include both the goal of knowing how to rename a file, and the goal of knowing how to copy a file. The incorporation of multiple goals into a single utterance need not be so straightforward. For example, a single utterance may be used both to request something of the hearer, and to inform the hearer of a particular fact. The question: U SER: How do I delete the file ‘binkle’ in my top-level directory? both requests that the hearer say how to delete the file, and informs the hearer of the whereabouts of the file. A good explanation of this question must include both of these as goals of the speaker. It is possible to use a single utterance to address virtually any pair of goals that can themselves be addressed linguistically. Thus a taxonomy of pairs of goals that might be addressed by a single utterance is no easier to come by
366
JAMES MAYFIELD
than a taxonomy of goals themselves. Consequently, I will not expound on such a taxonomy beyond providing a number of examples. The interested reader is referred to Appelt (1985) for a description of this problem from a generation perspective. To see the importance of recognizing multiple goals, consider this example: U SER: I want to delete a directory and all the files in it. The user here has expressed two goals explicitly in a single utterance. To successfully respond to this utterance, the system must recognize both goals. If the system recognizes only the goal of deleting the directory, it may come up with the plan of moving its files elsewhere and then deleting it; if it recognizes only the goal of deleting the files, the plan it chooses will almost certainly leave the directory intact. Thus, it is important for PAGAN to infer all the goals that motivate the production of an utterance. Consider another example: U SER: How can I prevent anyone from reading my file without deleting it? The user has two goals in making this utterance: to prevent other people from reading the file, and to preserve the file. Once again, the goals are distinct and non-conflicting, and both must be addressed to adequately respond to the question. The first goal is the user’s main goal; the second is called an adjunct goal (Wilensky 1983). However, the method for inferring these goals, given a representation of the utterance, is less straightforward than in the previous example, since they are not mentioned as a statement of goals but rather as a question. Addressing multiple goals with a single action is not limited to goals in the domain of discourse. Discourse goals themselves can be pursued in this manner: U SER: My last question is how to save my file once I’ve edited it. The speaker of this utterance has the domain goal of finding out about how to save a file while in the editor.4 She also has the discourse goal of indicating that she expects the dialogue to terminate after this problem has been resolved. While awareness of this latter goal may not be useful to a system that simply performs rote question answering, it will be helpful to a more advanced system in planning its own utterances. Finally, consider the statement: U SER: I have a problem with ls.
EVALUATING PLAN RECOGNITION SYSTEMS
367
The user who initiates a conversation with UC in this way probably has several goals in mind, including to initiate a dialogue with UC, to solicit help from UC, and to inform UC that the ‘ls’ command is the topic of the problem. In many systems, goals such as these are not explicitly inferred. Rather, the system embodies assumptions that its authors have made about the types of higher-level goals that the system’s users will have. Encoding this type of knowledge procedurally is not necessarily bad. However, the system should have the ability to represent and treat this knowledge declaratively should the need to do so arise. The validity of the breadth completeness criterion is based on the assumption that the interests of the system that will use the explanation are not so constrained as to be single-minded. If the system is single-minded about its goals, then this criterion isn’t useful. For example, a simplistic database lookup program will generally only need to extract a database query from the user’s utterance. It won’t matter to such a program whether a full understanding of an utterance is achieved, only that a database query is selected. For systems with broader applicability though, this is an important criterion.
6. System Evaluation: Explanations in UC As an example of the use of these criteria in system evaluation, in this section I will apply them to the output of UC’s plan recognition component PAGAN. Two factors allow PAGAN to build explanations that meet the criteria discussed above: 1. PAGAN’s representation of plans and goals 2. PAGAN’s algorithm These factors are the subjects of the next two sections. 6.1. Representation of plans and goals There have been two main overlapping approaches to the representation of knowledge about plans and goals. Operators (as used in STRIPS (Fikes and Nilsson 1971) and subsequent systems, notably that of Allen and Perrault 1980) capture knowledge about the effects of actions. They are useful for planning, but are less useful in plan recognition because they do not highlight the outcome the planner typically aims to achieve in using the operator. Scripts are an alternative representation. Scripts (Schank and Abelson 1977) allow knowledge of the actions of multiple agents to be collected into a single representation. They are useful in understanding stereotypical situations, but are less useful for handling novel situations.
368
JAMES MAYFIELD
The structure that PAGAN uses to represent knowledge about plans and goals is called a planfor (cf. Wilensky 1983). A planfor is a relation between a type of goal and a hypothetical event (called a plan) that constitutes a possible method of achieving a goal of that type. Typically, such plans represent compound events composed of a partially ordered set of hypothetical actions. Planfors unite the idea of a script as a way to group related actions of multiple agents with the notion of a plan that is being pursued by a single actor in order to achieve a particular goal. Planfors highlight the notion that a plan is a way to achieve a goal, as opposed to an abstract structure bearing no direct explicit relation to a goal. They also allow that a planner may intend that another agent perform some action as a part of the plan, without being in a position to control the other agent’s performance of the action. Planfors are described in detail in Mayfield (1989). The use of planfors allows PAGAN to build explanations that meet the applicability of composition criterion. UC’s task is in part to address the user’s goals. By emphasizing the importance of motivating goals in plan recognition, planfors give UC exactly the information it needs to provide the user with intelligent responses. In another type of system, the relationship between a plan and its motivating goal might be less important. For example, Quilici et al. (1985) are interested in producing explanations where causal relationships are as important as intentional ones. Their system therefore has a wide variety of link types that are used in the construction of an explanation. Applicability of granularity is facilitated by the choice of an appropriate set of planfors for inclusion in a particular system. In PAGAN, as in most plan recognition systems, planning knowledge is hand-coded. This leads to explanations that better meet the applicability of granularity criterion, but at the expense of producing a large body of planning knowledge in an ad hoc manner. For large systems, the process of creating planfors will need to be automated. Results in explanation-based learning (cf. Braverman and Russell 1988) show promise in this regard. 6.2. PAGAN’s algorithm Plan recognition has received much attention in recent years. Wilensky’s PAM system (Wilensky 1978) demonstrated that a simple recognition algorithm can perform sophisticated analysis of action given the appropriate knowledge about plans and goals. Allen (1979) showed how the plan recognition paradigm can be applied to language understanding. More recent work has expanded on these basic ideas (for example, Appelt and Pollack (1992), Carberry (1986), Eller and Carberry (1992), Grosz and Sidner (1985), Litman (1985), and Raskutti and Zukerman (1991)).
EVALUATING PLAN RECOGNITION SYSTEMS
369
PAGAN’s brand of plan recognition is called goal analysis, to stress the importance of the user’s goals to the overall plan structure. Before goal analysis can begin, the user’s utterance must be read and converted (via parsing and semantic interpretation) to an internal representation. This process produces one or more interpretations of the user’s utterance. These interpretations are marked as being mutually incompatible before being handed to PAGAN for goal analysis. This allows PAGAN to address ambiguities that prior components of the system have been unable to resolve. The first step of the goal analysis algorithm is to find one or more potential explanations of the utterance. There are three places where PAGAN might find an explanation for an utterance: 1. Among its expectations of speaker actions 2. Among the abstract planfors in its knowledge base 3. Among the non-intentional explanations in its knowledge base First, an explanation might be found among the expectations that are held about future actions of the speaker. Such expectation-matching builds explanations that meet the grounding criterion, because they relate the utterance to the previous dialogue structure. For example, if the system were expecting an answer to a yes/no question, the statement: U SER: No. could easily be matched against that expectation. Secondly, an explanation might be found among the collection of planfors stored in the long-term knowledge base. The importance of the planfor representation to the production of explanations that meet the applicability of composition and applicability of granularity criteria was described in Section 6.1. Finally, an explanation might be found among the set of non-intentional explanations in the knowledge base. For example, the non-intentional explanation ‘computer users often want to edit files’ might be used to explain why a particular user wants to edit a particular file. The use of a non-intentional explanation does not necessarily assist in building explanations that meet the criteria discussed above, but it does offer a method for terminating a sequence of inferences about an utterance. The second main step of the goal analysis algorithm is to handle any ambiguities that have arisen. Ambiguity can arise because the sentence used in the utterance is ambiguous, because the use of the sentence is ambiguous, or because the utterance as a whole (or one the goals inferred to explain the utterance) might be part of more than one plan. The sources and resolution of ambiguity are described fully in Mayfield (1989).
370
JAMES MAYFIELD
Sometimes, an utterance may legitimately have more than one explanation. When PAGAN produces two explanations that are compatible with one another, both are carried forward. The ability to produce multiple explanations for a single event allows PAGAN to meet the breadth completeness criterion. The third step of the goal analysis algorithm is to determine whether the inferred explanation for the utterance should itself be explained. This decision is based on an assessment of the depth completeness of the explanation chain. An explanation that is complete is not extended any further. An explanation is complete when it meets some expectation instantiated by previous dialogue, or when the most recently inferred goal lies beyond the system’s domain of expertise. An explanation that is incomplete is subjected to further analysis by the algorithm. An explanation is incomplete when the inferred goal is always used in service of some other goal, or when the dialogue model contains a strong expectation that has not yet been met. Thus, the third step of the algorithm helps to build explanations that meet the applicability of content and depth completeness criteria. If neither of these conditions holds, then it is not immediately clear whether the explanation is complete. In such cases, two additional assessments are made to determine whether processing should continue. First, the system determines its level of curiosity about the explanation. This is compared against an estimate of the difficulty of continuing the analysis. If the level of difficulty is higher, processing is terminated. In such cases, the production of a good explanation is sacrificed in order to conserve system resources. On the other hand, if the level of curiosity about the explanation is higher, processing continues, and an explanation that better meets the depth completeness criterion is produced. See Mayfield (1992) for a complete discussion of PAGAN’s chaining algorithm.
7. Computer Example The following trace shows PAGAN’s processing of the user’s utterance in the exchange: UC: Is the file in your directory? U SER: The file is in /tmp. The trace shows how PAGAN is able to produce explanations that meet the applicability, grounding, and depth completeness criteria. While the explanation produced by PAGAN meets the breadth completeness criterion, it does so only because there is just one explanation chain to be derived. An utterance that addressed more than one compatible goal would have led PAGAN to
EVALUATING PLAN RECOGNITION SYSTEMS
371
build multiple compatible explanations, better illustrating compliance with the breadth completeness criterion. The following conventions are used in the traces: concept nodes in KODIAK (the semantic net representation language used in UC (Wilensky 1987)) are written in capital letters. Descriptions produced by PAGAN as it runs are written in roman, and comments added by hand after-the-fact are written in italics. Names that end with a number represent subconcepts of the concept represented by the root name. For example, LOCATION–OF8 is a specialization of the LOCATION–OF concept. Other than deletion of extraneous trace information after-the-fact, and the introduction of comments in italics, the only changes made to PAGAN’s trace output by hand are changes in appearance, such as the introduction of font information and the addition of white space.
UC: Is the file in your directory? U SER: The file is in /tmp.
First, the parser produces an explanation of the user’s utterance, rooted at TELL–ACTION3.
Interpretation produced by the parser: TELL–ACTION3
PAGAN begins its processing by determining whether it really has any work to do. Determining whether to chain on TELL–ACTION3. Determining whether TELL–ACTION3 is a complete explanation of (THE FILE IS IN /TMP). Determining whether TELL–ACTION3 is outside of UC’s domain of expertise. TELL–ACTION3 is not outside of UC’s domain of expertise. The explanation is not necessarily complete. Determining whether TELL–ACTION3 is an incomplete explanation of (THE FILE IS IN /TMP). Determining whether TELL–ACTION3 is an instrumental goal. TELL–ACTION3 is an instrumental goal. The explanation is incomplete. Chaining should be done on TELL–ACTION3.
372
JAMES MAYFIELD
PAGAN has decided that it should try to explain the user’s utterance, so it searches for an initial explanation.
Attempting to find an explanation for TELL–ACTION3. Determining whether TELL–ACTION3 is grounded in the preceding dialogue.
Here, PAGAN tries to determine whether TELL–ACTION3 is a continuation of UC’s plan, which calls for a yes/no answer. It does so by asserting that it is, then catching the error that results.
Trying to ground TELL–ACTION3 in INDICATE–YES–OR–NO. Concreting TELL–ACTION3 to be INDICATE–YES–OR–NO. TELL–ACTION3 was not grounded in INDICATE–YES–OR–NO because: Can’t concrete UTTERANCE3 to UTTERANCE–OF–INDICATE– YES–OR–NO because the ranges differ.
Next, PAGAN checks whether TELL–ACTION3 constitutes a new plan for UC’s existing goal of knowing whether the file is in the user’s directory. It does this in the same way that it checked for plan continuation above.
Determining whether TELL–ACTION3 is a new plan for the existing goal of KNOWING–WHETHER1 TELL–ACTION3 is not a new plan for the existing goal of KNOWING–WHETHER1 because: Attempt to concrete incompatibly from TELL–ACTION3 to KNOWING–WHETHER1. TELL–ACTION3 was not grounded in the preceding dialogue.
Now, PAGAN looks for a planfor explanation of the utterance. It finds one, and infers that the user has the goal of UC knowing whether the file is in the directory. The use of this planfor leads to an explanation that meets the applicability of composition criterion (because UC is concerned with plans and goals) and the applicability of granularity criterion (because the planfor was created to reflect the level of granularity expected by UC).
EVALUATING PLAN RECOGNITION SYSTEMS
373
Trying to find planfor explanations for TELL–ACTION3. Found PLANFOR53 as an explanation for TELL–ACTION3. Synopsis: Goal: hearer know whether X; Step 1: Tell hearer that X. Inferred goal is KNOWING–WHETHER2. Explanation found for TELL–ACTION3.
PAGAN now determines whether it should continue its processing. Determining whether to chain on KNOWING–WHETHER2. Determining whether KNOWING–WHETHER2 is a complete explanation of TELL–ACTION3. Determining whether KNOWING–WHETHER2 is outside of UC’s domain of expertise. KNOWING–WHETHER2 is not outside of UC’s domain of expertise. The explanation is not necessarily complete. Determining whether KNOWING–WHETHER2 is an incomplete explanation of TELL–ACTION3. Determining whether KNOWING–WHETHER2 is an instrumental goal. KNOWING–WHETHER2 is an instrumental goal. The explanation is incomplete. Chaining should be done on KNOWING–WHETHER2.
PAGAN has decided that although the explanation as it stands meets the applicability of content criterion, it does not meet the depth completeness criterion; PAGAN therefore continues its processing. Attempting to find an explanation for KNOWING–WHETHER2. Determining whether KNOWING–WHETHER2 is grounded in the preceding dialogue.
Once again, PAGAN checks for plan continuation. KNOWING– WHETHER2 is not a type of indicating yes or no, so the check fails. Trying to ground KNOWING–WHETHER2 in INDICATE–YES–OR–NO. KNOWING–WHETHER2 was not grounded in INDICATE–YES–OR–NO because: Attempt to concrete incompatibly from KNOWING–WHETHER2 to INDICATE–YES–OR–NO.
374
JAMES MAYFIELD
Now, PAGAN checks whether KNOWING–WHETHER2 represents a new plan for UC’s existing goal. It does, so the explanation is grounded in the representation of the dialogue so far. The old plan is rejected, and processing terminates. Determining whether KNOWING–WHETHER2 is a new plan for the existing goal of KNOWING–WHETHER1 KNOWING–WHETHER2 is a new plan for the existing goal of KNOWING–WHETHER1 Existing plan PLAN107 has been rejected; new plan is PLAN111. Explanation found for KNOWING–WHETHER2.
8. Recent Developments The work reported herein was part of a general approach to plan recognition developed for the Unix Consultant system (Mayfield 1989). In addition to exploring properties of good explanations, the work examined the representation of plans and goals, sources of ambiguity, ambiguity resolution, and control of inference in plan recognition (Mayfield 1992). The Unix Consultant project wound down not long after the work was completed.
9. Conclusions This paper introduces the following criteria for good explanations of utterances: 1. Principle of Applicability − Applicability of Composition − Applicability of Content − Applicability of Granularity 2. Principle of Grounding 3. Principle of Completeness − Depth Completeness − Breadth Completeness The principle of applicability, states that a good explanation of an utterance is one that is applicable to the needs of the system that will use that explanation. The second criterion, the principle of grounding, states that a good explanation of an utterance relates what is inferred from that utterance to what is already known about the speaker and the dialogue. The last criterion, the principle of completeness, states that a good explanation of an utterance covers
EVALUATING PLAN RECOGNITION SYSTEMS
375
every aspect of the utterance. Together, these three criteria provide a means to determine the quality of an explanation of an utterance. They are useful because they can be used to evaluate the effectiveness of a plan recognition algorithm at a high level, and to suggest how such an algorithm might be improved.
Notes 1 Unix is a trademark of X/Open, Inc. 2 This is not the same as explaining a concept to the user. The creation of this latter sort of
explanation is an act of generation rather than one of understanding, and is performed in UC by the planner and the language generator. 3 In most cases, this response implies a negative answer, but only by virtue of the analysis described here. 4 Of course, the speaker also has the discourse goal of asking how to save the file. However, this goal is subservient to the domain goal. I am concerned in this section only with multiple goals that are not causally related to one another.
References Allen, J.F. (1979). A Plan-Based Approach to Speech Act Recognition. Ph.D. thesis, Technical Report 131/79, Computer Science Department, University of Toronto. Allen, J.F. & Perrault, C.R. (1980). Analyzing Intention in Utterances. Artificial Intelligence 15(3): 143–178. Appelt, D.E. (1985). Planning English Referring Expressions. Artificial Intelligence 26(1): 1–33. Appelt, D.E. & Pollack, M.E. (1992). Weighted Abduction for Plan Ascription. User Modeling and User-Adapted Interaction 2(1–2): 1–25. Braverman, M.S. & Russell, S.J. (1988). IMEX: Overcoming Intractability in Explanation Based Learning. In Proceedings of the National Conference on Artificial Intelligence, 575–579. Los Altos, CA: Morgan Kaufmann. Calistri-Yeh, R.J. (1991). Utilizing User Models to Handle Ambiguity and Misconceptions in Robust Plan Recognition. User Modeling and User-Adapted Interaction 1(4): 289–322. Carberry, S. (1986). TRACK: Toward a Robust Natural Language Interface. In Proceedings of the Canadian National Conference on Artificial Intelligence, 84–88. Chin, D. (1988). Intelligent Agents as a Basis for Natural Language Interfaces. Ph.D. thesis, Technical Report UCB/CSD 88/396, Computer Science Department, University of California, Berkeley, California. Eller, R. & Carberry, S. (1992). A Meta-rule Approach to Flexible Plan Recognition in Dialogue. User Modeling and User-Adapted Interaction 2(1–2): 27–53. Fikes, R.E. & Nilsson, N.J. (1971). STRIPS: A New Approach to the Application of Theorem Proving to Problem Solving. Artificial Intelligence 2: 189–208. Grosz, B. & Sidner, C.L. (1985). The Structures of Discourse Structure. Technical Report CSLI-85-39, Center for the Study of Language and Information, Stanford University, Palo Alto, California.
376
JAMES MAYFIELD
Hirschberg, J. (1984). Toward a Redefinition of Yes/no Questions. In Proceedings of the Tenth International Conference on Computational Linguistics, 48–51. Palo Alto: International Committee on Computational Linguistics. Kass, R. & Finin, T. (1988). Modeling the User in Natural Language Systems. Computational Linguistics 14(3): 5–22. Litman, D.J. (1985). Plan Recognition and Discourse Analysis: An Integrated Approach for Understanding Dialogues. Ph.D. thesis, Technical Report TR170, Department of Computer Science, University of Rochester. Mayfield, J. (1989). Goal Analysis: Plan Recognition in Dialogue Systems. Ph.D. thesis, Technical Report UCB 89/521, Computer Science Division (EECS), University of California, Berkeley, California. Mayfield, J. (1992). Controlling Inference in Plan Recognition. User Modeling and UserAdapted Interaction 2(1–2): 83–115. Norvig, P. (1988). Multiple Simultaneous Interpretations of Ambiguous Sentences. In Program of the Tenth Annual Conference of the Cognitive Science Society. Pollack, M.E. (1984). Good Answers to Bad Questions: Goal Inference in Expert Advicegiving. Technical Report MS-CIS-84-15, Computer Science Department, University of Pennsylvania, Philadelphia, Pennsylvania. Quilici, A.E. (1989). Detecting and Responding to Plan-oriented Misconceptions. In Kobsa, A. and Wahlster, W. (eds.) User Models in Dialog Systems, 108–132. Springer Verlag: Berlin. Quilici, A.E., Dyer, M.G. & Flowers, M. (1985). Understanding and Advice Giving in AQUA. Technical Report UCLA-AI-85-19, Computer Science Department, University of California, Los Angeles, California. Raskutti, B. & Zukerman, I. (1991). Generation and Selection of Likely Interpretation During Plan Recognition in Task-oriented Consultation Systems. User Modeling and User-Adapted Interaction 1(4): 323–353. Retz-Schmidt, G. (1991). Recognizing Intentions, Interactions, and Causes of Plan Failures. User Modeling and User-Adapted Interaction 1(2): 173–202. Schank, R. & Abelson, R. (1977). Scripts, Plans, Goals and Understanding. Lawrence Erlbaum: Hillsdale, NJ. Schank, R.C. (1986). Explanation Patterns: Understanding Mechanically and Creatively. Lawrence Erlbaum Associates: Hillsdale, NJ. Wilensky, R. (1978). Understanding Goal-Based Stories. Ph.D. thesis, Research Report 140, Computer Science Department, Yale University, New Haven, Connecticut. Wilensky, R. (1983). Planning and Understanding: A Computational Approach to Human Reasoning. Addison-Wesley: Reading, MA. Wilensky, R. (1987). Some Problems and Proposals for Knowledge Representation. Memorandum UCB/CSD 87/351, University of California, Berkeley, California. Wilensky, R., Arens, Y. & Chin, D. (1984). Talking to UNIX in English: An Overview of UC. Communications of the ACM 27(6): 575–593. Wilensky, R., Chin, D., Luria, M., Martin, J., Mayfield, J. & Wu, D. (1988). The Berkeley UNIX Consultant Project. Computational Linguistics 14(4): 35–84.
Representing UNIX Domain Metaphors JAMES H. MARTIN Department of Computer Science, University of Colorado at Boulder, ECOT 7-7, Campus Box 430, Boulder, CO 80309-0430, USA; E-mail: [email protected] Abstract. The language used to describe technical domains like UNIX is filled with metaphor. An approach to metaphor, based on the explicit representation of knowledge about metaphors, has been developed. MIDAS (Metaphor Interpretation, Denotation, and Acquisition System) is a computer program that that has been developed based upon this approach. MIDAS can be used to represent knowledge about conventional metaphors, interpret metaphoric language by applying this knowledge, and dynamically learn new metaphors as they are encountered during normal processing. Keywords: knowledge representation, metaphor, natural language processing
1. Introduction The language used to describe technical domains like UNIX is filled with metaphor. A consultant system that is going to accept natural language input from users, and provide appropriate natural language advice, must be prepared to handle such metaphorical language. Consider the following UNIX examples. (1) (2) (3) (4) (5)
How can I kill a process? I am in emacs. How do I get out of emacs? You can enter emacs by typing emacs to the shell. I want to give someone permission to read a file.
The italicized words in each of these examples are being used to metaphorically refer to concepts that are quite distinct from those that might be considered the normal meanings of the words. Consider the use of enter in Example 4. Enter is being used, in this example, to refer to the actions on a computer system that result in the activation of a program. This use is clearly different from what might be called the ordinary or basic meaning of the word that has to do with the actions that result in an agent entering an enclosure. While the word enter is used metaphorically in Example 4, this metaphor is neither novel nor poetic. Instead, the metaphorical use of enter results from
378
JAMES H. MARTIN
a conventional, systematic, conceptual metaphor that allows computer processes to be viewed as enclosures. The various actions and states that have to do with containment are used to refer to actions and states that have to do with the activation, deactivation, and use of these computer processes. This conceptual metaphor, structuring processes as enclosures, underlies the normal conventional way of speaking about these processes. Therefore, the uses of the words enter in Example 4, in in 2 and get out of in 3 are ordinary conventional ways of expressing these concepts that nevertheless involve this enclosure metaphor. The approach to conventional metaphor described in this chapter is a knowledge-intensive one. This approach was inspired by the work of Lakoff and Johnson (1980), and builds on the computational approaches begun in (Jacobs 1985; Martin 1986, 1987; Norvig 1987) and (Wilensky 1986). This approach asserts that the interpretation of conventional metaphoric language proceeds through the direct application of specific knowledge about the metaphors in the language. The interpretation of novel metaphors is accomplished through the systematic extension, elaboration, and combination of already well-understood metaphors. The proper way to approach the topic of metaphor, therefore, is to study the details of both individual metaphors and the system of metaphors in the language. This approach has been embodied in MIDAS (Metaphor Interpretation, Denotation, and Acquisition System) (Martin 1988). MIDAS is a set of computer programs that can be used to perform the following tasks: explicitly represent knowledge about conventional metaphors, use this knowledge to interpret metaphoric language, and learn new metaphors as they are encountered. In order to make the problem of understanding metaphors more concrete, consider the following session with the UNIX Consultant (Wilensky et al. 1988), involving an enclosure metaphor. > (do-sentence) Interpreting sentence: How can I get into lisp? Applying conventional metaphor Enter-Process. UC: You can get into lisp by typing lisp to the shell. In this example, the user has employed the conventional metaphor, described above, that entails that programs can be viewed as enclosures or environments. The action of entering such an enclosure, underlying the phrase get into, corresponds to the action that begins the use of the program.
REPRESENTING UNIX DOMAIN METAPHORS
379
In order to appropriately handle this example, UC must be able to access and apply specific knowledge about this conventional metaphor. UC handles this kind of metaphoric language by calling upon MIDAS. In this example, UC calls upon MIDAS to find a coherent interpretation for this use of get into. MIDAS finds and applies the conventional metaphor that allows the invocation of a program to be viewed as an entering. Section 3 focuses on the structure of individual metaphors and the systematicities evident among some of the important core metaphors in UNIX. Section 4 describes how these systematic metaphors can be represented using the KODIAK representation language (Wilensky 1986). Section 5 describes how MIDAS can use this metaphoric knowledge to interpret known conventional metaphors. Finally, Section 6 shows how MIDAS can use the systematic structure of known metaphors to dynamically learn new metaphors as they are encountered.
2. Previous Computational Approaches to Metaphor The metaphoric knowledge approach, described here, is an attempt to fuse the notion of a systematic language convention with the notion of analogical systematicity that has been at the core of the most previous computational work on metaphor (Carbonell 1981; DeJong and Waltz 1983; Fass 1988; Gentner et al. 1988; Indurkhya 1987). These approaches assert that metaphors like those cited above arise solely from an underlying conceptual similarity or analogy between the concepts representing the literal meaning of the words and the concepts underlying the ultimate meaning of the utterance. The task of interpreting metaphoric language is usually accomplished through the use of an analogical transfer process that directly matches the concepts from the two domains. Most importantly, these processes make no use of explicit knowledge about metaphors that are a conventional part of the language. The goal of MIDAS has been to account for the analogical systematicity of these metaphors while at the same time representing their conventionality in a way that permits efficient processing.
3. Representing UNIX Metaphors A detailed analysis of what needs to be represented necessarily precedes the task of constructing a knowledge base. This analysis should reveal the phenomena that need to be captured and suggest certain requirements for how these phenomena should be captured. The analysis of some conventional UNIX metaphors, given here, will reveal some of the salient characteristics
380
JAMES H. MARTIN
that need to be captured. This analysis will only touch upon some of most important characteristics. For a more in-depth discussion of metaphorical systematicities see (Lakoff and Johnson 1980), and for a complete discussion of how these systematicities can be represented see (Martin 1988). Section 4 on representation will present some of the details of how these characteristics are captured using KODIAK. Consider Example 2 again. The metaphorical use of the word in reflects a systematic metaphorical structuring of processes as enclosures. Metaphors like this may be said to consist of two sets of component concepts, a source component and a target component. The target consists of the concepts to which the words are actually referring. The source refers to the concepts in terms of which the intended target concepts are being viewed. In this example, the target concepts are those representing the state of currently using a computer process, in this case EMACS. The target concepts are those that involve the state of being contained within an enclosure. The approach taken here is to explicitly represent conventional metaphors as sets of associations between source and target concepts. The metaphor specifies how the source concepts correspond to various target concepts. In this case, the metaphor consists of associations that specify that the state of being enclosed represents the idea of currently using the editor, the user plays the role of the enclosed thing, and the Emacs process plays the role of the enclosure. Therefore, the first requirement for the representation language is to be able to capture these sets of metaphorical associations between concepts from diverse conceptual domains. This first requirement for the representation is summarized as follows: Fundamental Representational Requirement: Conventional metaphors must be explicitly represented as coherent sets of associations between source and target concepts. It is, however, clearly not sufficient to merely represent the metaphor underlying Example 2 independently of the metaphors underlying Examples 3 and 4. The metaphor that a process can be viewed as an enclosure lies at the core of each of these uses. This Enclosure metaphor is extended in Examples 3 and 4 in ways that are predictable from the semantics of the source and target domains. The common shared component metaphor among a group of related metaphors will be referred to as a core metaphor. Correspondingly an extended metaphor is a metaphor that includes all the associations of a core metaphor and adds new associations that coherently extend the core metaphor. Table 1 gives some examples of common UNIX core metaphors and various extensions to them.
You can edit files when you are in the editor. You can get into lisp by typing lisp to the shell. How can I get out of lisp?
For example, consider the core metaphor Process-As-Living-Thing. This core metaphor is the basis for the extended metaphors Creator-As-Parent, Created-As-Child, Terminating-As-Killing and Termination-As-Death. In general, the Process-As-Living-Thing metaphor is used to structure the actions that processes perform and the actions that are performed upon them. The Termination-As-Death metaphor structures the target concept of termination in terms of the death of a living thing. The corresponding TerminatingAs-Killing metaphor allows the actions that cause the termination to be viewed as a killing (an action that causes a death). The Process-As-Enclosure metaphor is a distinct core metaphor that also structures some kinds of processes. It primarily structures the target concepts of actively using a process, starting to use a process, and finishing the use, in terms of entering, being enclosed within, and exiting. In each of these cases, the extended-metaphor contains the core-metaphor, and extends it with the addition of further associations. The representation of the metaphorical associations that comprise the core metaphors must be capable of capturing the fact that these associations may be shared by several other extended metaphors. This second representational requirement can be stated as follows: Extended Metaphor Requirement: The representation must facilitate the sharing of component metaphorical associations from core metaphors to related extended metaphors.
382
JAMES H. MARTIN
Now consider the following metaphor examples from outside the domain. (6) (7) (8)
UNIX
John can kill a conversation by walking into a room. Tom couldn’t wait to get into the game. The manager took Tom out of the game after one inning.
Each of these examples contains a metaphor whose overall structure strongly resembles the structure of the UNIX metaphors given in Examples 1 through 5. In Example 6, a conversation is being viewed as a living thing, motivating the metaphor that an action that causes its termination can be viewed as a killing. This is similar to the structure of Example 1 where the termination of an abstract concept (a process) is also viewed as a killing. Examples 7 and 8 are further illustrations of this phenomenon. Enclosure or Container metaphors occur across many diverse domains. The commonality among these metaphors is that an ongoing process or activity is often viewed as an enclosure where participation in the process is structured as a containment. Correspondingly the concepts of entering and exiting denote the concepts of starting and finishing participation. These examples illustrate that, far from being unique to UNIX, the metaphors used in Examples 1 through 5 are simply specializations of abstract metaphors that are already a widespread conventional part of English. The representation chosen to capture individual core metaphors and their extensions should also capture this overall similarity of structure among metaphors from diverse domains. This final requirement for the representation can be stated as follows: Similarity Requirement: Representations of conventional metaphors must capture the similarity relationships among metaphors from diverse conceptual domains.
4. Representing Conventional Metaphors The following sections show how the Fundamental Representational Requirement, Extended Metaphor Requirement, and Similarity Requirement can all be fulfilled through the straightforward use of KODIAK ’ S structured association and inheritance facilities.
REPRESENTING UNIX DOMAIN METAPHORS
383
4.1. Individual conventional metaphors The first requirement for the representation is to represent metaphors as concepts consisting of sets of associations between source and target concepts. Consider Example 9. (9)
How can I kill a process?
This example, from the UNIX domain, involves the conventional metaphor that to kill an ongoing process means to terminate it. The target concepts involve computer processes and the actions that terminate them. The source concept is that of the action of causing a living thing to die. The metaphor consists of the source, target, and the set of associations linking them. Figure 1 shows the KODIAK representation of the source domain from Example 9. It states that a killing is a kind of action with a result that is a death-event which is in turn an event. The kill-victim of the killing is an inherited role from action indicating that the kill-victim is effected by the action. The kill-victim is constrained to be a living-thing and the killer must be an animate-agent. Finally the equate links require that the kill-victim must be the same as the dier, the role in the death-event representing the deceased. Figure 2 shows the corresponding concepts from the target domain. It states that a terminate-process-action is a terminate-action which is a kind of action. The terminated-process role is an inherited role specifying the patient of the action. The result of the action is a terminate-process-effect which is a kind of terminate-event. Finally, the terminated-process is equated to the terminated-process-event of the terminate-process-effect. This is analogous to the relationship between the kill-victim and the dier shown in Figure 1. What is needed is a way of associating the appropriate source and target concepts. Such an association is realized in KODIAK by using a relation called a metaphor-map. A metaphor-map is simply a relation whose roles specify the needed source and target concepts. Metaphor maps are needed to link all the core source concepts in Figure 1 to their counterparts in the target domain. In particular, the killing maps to the terminate-action, the kill-victim maps to the terminated-process, the killer maps to the actor of the terminate-action, and the result of the killing maps to the result of theterminating. Figure 3 shows the complete set of maps underlying Example 9. It is the co-occurrence of all these maps that constitutes the conventional metaphor that terminating something can be viewed as a killing. This co-occurrence of a set of more primitive inheritance relations is the definition of a structured association. Therefore, a kind of structured
384
JAMES H. MARTIN
D
Action
Event
S D
Actor
D S
Kill Result
Killing D
S
C Death−Event
Patient
S Killer
D
S
S
=
Kill Victim
Dier
C C
C
D Animate
Living−Thing
Figure 1. A Kodiak definition for the concept Killing.
D Action
Event
D
S Patient
D
Terminate−Action
S
Terminate
C
D
Terminate−Event
S
Result
D
Terminated Event
D
Terminate Process−Action
S
Terminate Process Result
Terminate Process−Effect
C
S
D
S Terminated Process Event
=
Terminated Process C
C
Process
Figure 2. A Kodiak definition for the concept Terminating.
385
REPRESENTING UNIX DOMAIN METAPHORS Source
Killing
S Kill Result
S Kill Victim
Source
Source
Target
Killing−Terminate
Kill−Terminate Result
Killed−Process
Target
Terminate−Process Action
Terminate Process Result
S
S
Target Terminated Process
S
S Source Killer
Killer−of−Process
Target
Agent of Terminate
Figure 3. Kill-terminate-maps.
association called a metaphor-sense is introduced to capture this meaningful co-occurrence of metaphor-maps. These maps tie together concepts that are highly inter-connected in the source and target domains. In Example 9, the metaphor-maps tie a single concept and its attendant roles on the source side to an equivalent set on the target side. A metaphor-sense is, therefore, a structured association that ties together sets of component metaphor-maps that together constitute a meaningful conventional metaphor. A metaphor-sense represents a meaningful unit in the same way that the concept killing and its relations taken together form a meaningful unit. Figure 4 shows the abbreviated notation for illustrating metaphor-senses. The sense itself is represented as the box enclosing the individual maps. To a significant extent, metaphor-senses are the minimal meaning-bearing unit of conventional metaphors. Metaphor-maps represent the building blocks out of which meaningful metaphor-senses are constructed. The metaphorsense represents the level at which one would say that there is a conventional metaphor that to terminate something is to kill it. This level of representation will frequently correspond to a single metaphorical word sense. 4.2. Extended metaphors Consider the problem of capturing the relationships among extended and core metaphors. As described in Section 3 the fundamental phenomenon is that the metaphorical associations that constitute a core metaphor play a central role in metaphors that are the extensions to the core metaphor. This is captured simply in KODIAK since the metaphor-maps that constitute the core metaphor exist explicitly in the system as independent concepts. They are shared as the
386
JAMES H. MARTIN
Terminate−Process Action
Killing
S
Kill Result
Killing−Terminate Kill−Terminate Event
S
Kill Victim
Killed−Process
Terminate Process Result
Terminated Process
S
S
Killer−of−Process S
Agent Of Terminate
Killer
S
Figure 4. Kill-Terminate-Sense Abbreviated.
fundamental building blocks in all the metaphors that extend from the core metaphor. An extended metaphor uses the maps from the core and adds more of its own to constitute an extension. Consider again the following Enclosure metaphors. (10) (11) (12)
I am in emacs. How can I get out of emacs? You can enter emacs by typing emacs to the shell.
The metaphor-sense underlying the metaphor in Example 10 consists of three metaphor-maps: in-emacs-map, enclosed-user and enclosing-used. These three metaphor-maps link the concepts of the state of being enclosed, the enclosed thing and the enclosure, to the to the concepts of the state of using an EMACS process, a user of the process and the EMACS process being used. The metaphors underlying Examples 11 and 12 are clearly extensions to the In-Emacs metaphor-sense. They both refer to changes of state with respect to the same source and target concepts that make up the In-Emacs metaphor. They, therefore, both share the metaphor-maps that make up the core and add new maps to elaborate it. For example, the Enter-Emacs metaphor-sense underlying Example 12 adds two new maps to the core metaphor. The first map associates the source concept of entering with the target concept of invoking an emacs process. The second adds an association linking the source and target initial states of such an action. The maps that make up the core metaphor specify the final state of the action.
REPRESENTING UNIX DOMAIN METAPHORS
387
4.3. Capturing similarity Section 3 noted that many UNIX metaphors bear a strong resemblance to conventional metaphors that are already a normal part of English. This systematic relationship among metaphors is captured in KODIAK through the use of its inheritance mechanisms. Metaphor-maps and metaphor-senses are both full-fledged KODIAK concepts, and can therefore be arranged in abstraction hierarchies. Hierarchies are the primary mechanism used to account for the similarities and differences among conventional metaphors. Consider again the metaphor underlying Example 1. As discussed above, the metaphor-sense underlying this example contains a metaphor-map linking the source concept of a living-thing with the target concept computer-process. This is an instantiation of a more general metaphor that allows us to view non-living things in terms of living things for the purpose of explaining or understanding their behavior in terms of living things. Examples 14 through 17, from (Lakoff and Johnson 1980), all contain specialized instances of this general metaphor. (13) (14) (15) (16) (17)
How can I kill a process? Inflation is eating up our savings. Those ideas died a long time ago. He is the father of modern quantum mechanics. Those ideas won’t yield any fruit.
Example 14 is motivated by the metaphor that the reduction in savings, caused by inflation, can be viewed as inflation consuming the savings. Inflation is viewed as an animal that can consume things. Example 15 contains a metaphor dealing with the duration of ideas. When an idea is no longer held or believed it has died. Example 16 contains the notion that the creation of an idea is a birth-event, and that the originator of the idea plays the role of the father in the birth event with the created idea playing role of the child. Once again, in this metaphor, there is a metaphor-map from a concept that is not a living thing (the created idea) to a role that must be one (the child being born). This metaphor-map, however, is more specific since the constraint is not only to be a living thing but to be human. Finally, Example 17 contains the notion that an idea can produce new ideas. This is metaphorically structured as a plant producing new fruit. In this case, an idea is viewed as a specific kind of living thing, a plant. What these examples all have in common is the idea that an abstract concept like a process or idea can be viewed as a living thing to explain some aspect of its nature. They differ in the particular kind of living-thing
388
JAMES H. MARTIN
that is used and in the role that it plays. These similarities and differences result in specific metaphor-maps in each particular case. What is needed is a mechanism that can capture the commonalities and differences among these various metaphor-maps. This mechanism is provided by the general inheritance mechanisms provided by KODIAK. Metaphor-maps that share properties are dominated by more abstract parent maps that capture the commonalities among the children. The source and target roles of the parent map are constrained by concepts that are more abstract than, and dominate, the constrainers on the children’s source and target roles. Figure 5 illustrates this situation with the maps from Examples 13 through 17. The top half of Figure 5 shows the hierarchical relationships among the maps underlying the above examples. They all converge on the abstract metaphor-map representing the idea of viewing a non-living-thing as a living-thing. The two metaphor-maps in the dotted box are expanded in the bottom half of the diagram to show the exact details of the inheritance links. In this expansion, we can see that the idea-as-living-thing metaphor-map dominates the forgotten-idea-as-dier map. In the parent map, the idea-as-target role is constrained to be an idea. The living-thing-as-source role is constrained to be a living-thing. In the forgotten-idea-as-dier map, we see that the inherited source role is specialized by being constrained by the dier role of the death-event. The inherited target role is further specialized by being constrained by the forgotten-idea role. As mentioned above, metaphor-senses may also be included in these hierarchies. When it appears that an entire set of metaphor-maps is being used repeatedly in specialized domains then a more abstract metaphor-sense can be created from a set of abstract metaphor-senses. This is more fully discussed in (Martin 1988). This use of an abstraction hierarchy provides the link from specialized metaphors in the UNIX domain to the existing conventional metaphors in English. Section 6 will show how these hierarchies can be exploited to dynamically induce new metaphors in the UNIX domain by analogy to known metaphors from other domains.
5. Conventional Metaphor Interpretation The main thrust of the MIDAS approach to metaphor interpretation is that normal processing of metaphoric language proceeds through the direct application of specific knowledge about the metaphors in the language. The interpretation of sentences containing metaphoric language is a two-step
389
REPRESENTING UNIX DOMAIN METAPHORS Non−Living−Thing As Living−Thing
process. The first step in the interpretation of an input sentence is the production of a syntactic parse and a preliminary semantic representation. In the second step, this preliminary representation is replaced by the most specific interpretation that can coherently account for the input. This interpretation may be a literal one or one of a number of conventional metaphorical interpretations. This general interpretation process has been implemented in the Metaphor Interpretation System (MIS) component of MIDAS. The MIS examines the initial primal representation in an attempt to detect and resolve uses
390
JAMES H. MARTIN
of conventional UNIX metaphors. In the following UC example, a user has posed a question involving the conventional metaphor structuring processes as enclosures. The MIS component finds and resolves this metaphor. The resolution produces an instantiation of a coherent target concept representing the correct conventional meaning of the utterance. In the following examples, trace output is interspersed with a running commentary shown in normal Roman font. > (do-sentence) Interpreting sentence: How can I get into lisp? Interpreting primal input. (A Entering50 (ISA Entering) (agent597 (ISA agent) (A I203 (ISA I))) (patient562 (ISA patient) (A Lisp58 (ISA Lisp)))) The input phrase get into is treated as a phrasal unit with a conventional meaning corresponding to Entering. The preliminary semantic representation produced in this step is called the primal representation (Wilensky 1987). The primal representation produced by the parser represents concepts derivable from knowledge of the grammar and lexicon available to the parser. In particular, the primary task accomplished in this phase is the appropriate assignment of filled case roles to the concept underlying the head of a phrase. This primal representation represents a level of interpretation that is explicitly in need of further semantic processing. Concreting input relations. Concreting patient to entered. Concreting agent to enterer. The patient and agent roles, with their respective filler concepts I203 and Lisp58, were derived solely from the verb class that enter belongs to, and the syntax of the sentence. In this next step of processing, these generic roles are replaced by the more specific semantic roles that are actually attached to the Entering concept. Interpreting concreted input. (A Entering50 (ISA Entering) (enterer50 (ISA enterer) (A I203 (ISA I))) (entered50 (ISA entered) (A Lisp58 (ISA Lisp))))
REPRESENTING UNIX DOMAIN METAPHORS
391
Failed interpretation: Entering50 as Entering. Failed interpretation: Entering50 as Enter-Association. The literal interpretation and one of the other known Entering metaphors are rejected before the correct metaphor is found and applied. These interpretations are rejected because the input concepts filling the roles of enterer and entered do not match the requirements for these roles in these interpretations. In particular, the interpretation as an actual Entering requires that the entered concept must be a kind of enclosure. The filler of the entered role in the input, Lisp58, fails this requirement, therefore this interpretation is rejected. Similarly the Enter-Association metaphor specifies that the entered concept must be a kind of Association. Again, Lisp58 fails to satisfy this constraint and causes the rejection of the metaphoric interpretation posing this constraint. Note that the fact that the system considers the literal interpretation first is an artifact of the search procedure. It does not indicate any reliance on attempting the literal meaning first as was the case in previous approaches. All the conventional metaphorical uses have equal status with the known literal concept, Entering. Valid known metaphorical interpretation. Applying conventional metaphor Enter-Lisp. (A Enter-Lisp (ISA Container-Metaphor Metaphor-Schema) (enter-lisp-res enter-res lisp-invoke-result) (lisp-enterer enterer lisp-invoker) (entered-lisp entered lisp-invoked) (enter-lisp-map Entering Invoke-Lisp)) Mapping input concept Entering50 to concept Invoke-Lisp30 Mapping input role enterer50 with filler I203 to target role lisp-invoker30 Mapping input role entered50 with filler Lisp58 to target role lisp-invoked30 Yielding interpretation: (A Invoke-Lisp30 (ISA Invoke-Lisp) (lisp-invoked30 (ISA lisp-invoked) (A Lisp58 (ISA Lisp))) (lisp-invoker30 (ISA lisp-invoker) (A I203 (ISA I))))
392
JAMES H. MARTIN
The Enter-Lisp metaphor has been found and applied to the given input concepts. The main source concept is interpreted as an instance of the Invoke-Lisp concept according to the enter-lisp-map. The input roles enterer and entered are interpreted as the target concepts lisp-invoker and lisp-invoked respectively. This interpretation of the Entering concept is then used to fill the role of the topic role of the How-Question that constitutes the representation of the rest of the sentence. Final interpretation of input: (A How-Q207 (ISA How-Q) (topic206 (ISA topic) (A Invoke-Lisp30 (ISA Invoke-Lisp) (lisp-invoked30 (ISA lisp-invoked) (A Lisp58 (ISA Lisp))) (lisp-invoker30 (ISA lisp-invoker) (A I203 (ISA I)))))) This how-question, with the reinterpreted topic concept, is then passed along to the next stage of UC processing. UC then prints the answer as follows. Calling UC on input: (A How-Q207 (ISA How-Q) (topic206 (ISA topic) (A Invoke-Lisp30 (ISA Invoke-Lisp) (lisp-invoked30 (ISA lisp-invoked) (A Lisp58 (ISA Lisp))) (lisp-invoker30 (ISA lisp-invoker) (A I203 (ISA I)))))) UC: You can get into lisp by typing lisp to the shell. Note that when a conventional metaphor has been employed by the user in asking a question, UC’s natural language generator uses the same metaphor in producing the answer. In this example, the system uses the same enclosure metaphor employed by the user to express the plan.
REPRESENTING UNIX DOMAIN METAPHORS
393
6. Interpreting New Metaphors MIDAS will inevitably face the situation where a metaphor is encountered for which none of the known metaphors provides an adequate explanation. This situation may result from the existence of a gap in the system’s knowledge-base of conventional metaphors, or from an encounter with a novel metaphor. In either case, the system must be prepared to handle the situation. The approach taken by MIDAS to understanding new or unknown metaphors is called the Metaphor Extension Approach. This approach asserts that a new metaphor can best be understood by extending an existing metaphor in a systematic fashion. The basis for this approach is the belief that the known set of conventional metaphors constitutes the best source of information to use in understanding new metaphors. The basic strategy is to first find a known metaphor that is systematically related to the new example. This candidate metaphor is then applied to the new example in an attempt to produce an appropriate target meaning. The process of applying the candidate metaphor to the new example is dependent upon the kind of semantic connection between the candidate and the new example. Three kinds of connections are recognized, yielding three kinds of extension inferences: similarity extension, core-extension and combined-extension. Once the intended target meaning of the new example has been determined, a new metaphor is created and stored away for future use. When metaphors of this type are encountered again the system can interpret them directly. This strategy is realized in the Metaphor Extension System (MES) component of MIDAS. When no coherent explanation can be found for a given primal input it is passed along to the MES. The basic steps of MES algorithm are given in the following here. These steps will then be made more concrete in terms of a detailed trace from the MES. Step 1: Characterize the new input. Partial source and target components of a new metaphor are extracted from the primal representation accepted as input. The terms current source and current target will be used to refer to the source and target concepts derived from the input example. Step 2: Search for related metaphors. This step searches for any known metaphors that are potentially related to this new use. The search consists of an attempt to find a path or paths through the network from the current source to the current target concepts that contains a known metaphor. A metaphor contained in such a path is judged to be relevant. Step 3: Evaluate the set of candidate metaphors found in Step 2. The purpose of this step is to select a metaphor from the set found in Step 2 for
394
JAMES H. MARTIN
further processing. This choice is based on a set of criteria to determine the metaphor that is closest conceptually to the current example. Step 4: Apply this previously understood metaphor to the current example. The candidate metaphor is applied to the current target based on the relationship between the candidate mapping and the current example. Depending on this relationship, either a similarity, core, or combined extension inference is performed. Step 5: Store the new metaphor. Create and store a new metaphor consisting of the source and target concepts identified in the above steps along with appropriate associations between them. This new metaphor will be used directly when future instances of this metaphor are encountered. Consider the processing of the following example. In this example, UC encounters a metaphor that it has not seen before and has no direct knowledge of. MIDAS makes use of its knowledge of a related metaphor to determine the likely meaning of this new use and creates a new metaphor to be used directly in the future. > (do-sentence) Interpreting sentence: How can I kill a process? Interpreting primal input. (A Killing16 (ISA Killing) (agent87 (ISA agent) (A I46 (ISA I))) (patient76 (ISA patient) (A Computer-Process10 (ISA Computer-Process)))) The parser accepts the input sentence, as specified by the user, and produces a primal representation of the input in the form of KODIAK concepts. Concreting input relations. Concreting patient to kill-victim. Concreting agent to killer. Interpreting concreted input. (A Killing16 (ISA Killing) (killer16 (ISA killer) (A I46 (ISA I))) (kill-victim16 (ISA kill-victim) (A Computer-Process10 (ISA Computer-Process))))
No valid interpretations. Attempting to extend existing metaphor. Once the concreted representation has been created the system attempts to determine if the given input is consistent with any of the known conventional interpretations, literal or metaphorical. In this case, the input is not consistent with either the literal Killing concept or any of the three known metaphorical uses of kill. At this point, all the possible conventional interpretations of the primal input have been eliminated as potential readings. The input is now passed to the Metaphor Extension System in an attempt to extend an existing metaphor to cover this new use and determine the intended meaning. Entering Metaphor Extension System Searching for related known metaphors. Metaphors found: Kill-Conversation Kill-Delete-Line Kill-Sports-Defeat The first step in the extension step is to collect all the relevant known metaphors that might be related to this new use. This initial search scans through all the metaphors directly attached to the input concept, and also at all the metaphors attached to concepts that are core-related to the input concept. In this case, the system has knowledge of three metaphors that share the same source concept with the current use. Selecting metaphor Kill-Conversation to extend from. (A Kill-Conversation (ISA Kill-Metaphor Metaphor-Schema) (kill-c-res kill-result conv-t-result) (killed-conv kill-victim conv-termed) (killer-terminator killer conv-termer) (kill-term Killing Terminate-Conversation)) The candidate metaphors are ranked according to a “conceptual distance” metric. This is a measure of how close the candidate metaphors are to the new example. The primary factor contributing to this metric is a measure
396
JAMES H. MARTIN
of similarity between the target concepts of the candidate metaphor and the input filler concepts. The candidate metaphor that is judged to be closest to the input example according to this metric is chosen as the candidate metaphor for further processing. The selected metaphor is classified for further processing according to its relationship to the input example. In this case, the candidate metaphor is in a similarity relationship to the input metaphor. Attempting a similarity extension inference. Extending similar metaphor Kill-Conversation with target concept Terminate-Conversation. Abstracting Terminate-Conversation to ancestor concept Terminating producing abstract target meaning: (A Terminating3 (ISA Terminating) (terminated3 (ISA terminated) (A Computer-Process10 (ISA Computer-Process))) (terminator3 (ISA terminator) (A I46 (ISA I)))) The first step in the processing of a similarity extension inference is to identify the concepts specified in the input example with their corresponding target concepts. In this example, the concept Computer-Process10 is identified with the target role of terminated-conversation, and the role of I46 is identified with the target role of conversation-terminator. The constrainers of these concepts, however, are too specific to accept these input concepts. In this example, there is a mismatch between the input concept Computer-Process and the candidate target concept Conversation. The next step, therefore, is to abstract the target concept of the candidate to the first concept that can accept the concepts specified in the input. In this case, the concept Terminate-Conversation is abstracted to its ancestor concept Terminating. The ancestor of the terminated-conversation role has as a constrainer the abstract concept Process, which can constrain the more specific concept Computer-Process. Concreting target concept Terminating to Terminate-Computer-Process producing concreted meaning: (A Terminate-Computer-Process10 (ISA Terminate-Computer-Process) (c-proc-termer10 (ISA c-proc-termer) (A I46 (ISA I)))
REPRESENTING UNIX DOMAIN METAPHORS
397
(c-proc-termed10 (ISA c-proc-termed) (A Computer-Process10 (ISA Computer-Process)))) The next step in the similarity extension inference is to look down the hierarchy from Terminating to see if there are any more specific concepts beneath this one that can adequately accommodate the input concepts. The specific existing concept Terminate-Computer-Process10 is found. The concept c-proc-termed10 is a more specific concept than terminated and can still accept the input concept Computer-Process10 as a filler, since the constraining concept on the concept c-proc-termed is a Computer-Process. Creating new metaphor: Mapping main source concept Killing to main target concept Terminate-Computer-Process. Mapping source role killer to target role c-proc-termer. Mapping source role kill-victim to target role c-proc-termed. (A Killing-Terminate-Computer-Process (ISA Kill-Metaphor) (killing-terminate-computer-process-map Killing Terminate-Computer-Process)) (killer-c-proc-termer-map killer c-proc-termer) (kill-victim-c-proc-termed-map kill-victim c-proc-termed) The next stage of processing creates a new metaphor that represents this newly learned use. The role correspondences from the input example to the target concepts of the candidate metaphor form the basis for a new set of metaphoric associations that make up the new metaphorsense. In this case, the main source concept, Killing, is mapped to the intended target concept Terminate-Computer-Process. The source roles killer and kill-victim are mapped to the concepts c-proc-termer and c-proc-termed, respectively. In each case, a new metaphor-map is created to connect the source and target concept in the knowledge base. The map is then classified properly in the hierarchy of existing maps and connected to the newly created metaphor-sense representing this new metaphor. In the case of a similarity extension inference, the newly created metaphor-maps are made siblings (children of the same parent) of the corresponding metaphor-maps from the candidate metaphor used. The
398
JAMES H. MARTIN
newly created metaphor-sense is also made a sibling of candidate metaphor. In the current example, the newly created metaphor-sense, KillTerminate-Computer-Process, is made a sibling of the candidate metaphor Kill-Conversation. Final interpretation of input: (A How-Q46 (ISA How-Q) (topic46 (ISA topic) (A Terminate-Computer-Process10 (ISA Terminate-Computer-Process) (c-proc-termer10 (ISA c-proc-termer) (A I46 (ISA I))) (c-proc-termed10 (ISA c-proc-termed) (A Computer-Process10 (ISA Computer-Process)))))) The final representation of the input sentence now contains the intended target concept, Terminate-Computer-Process, as the topic of the user’s original how-question. Calling UC on input: (A How-Q46 (ISA How-Q) (topic46 (ISA topic) (A Terminate-Computer-Process10 (ISA Terminate-Computer-Process) (c-proc-termer10 (ISA c-proc-termer) (A I46 (ISA I))) (c-proc-termed10 (ISA c-proc-termed) (A Computer-Process10 (ISA Computer-Process)))))) UC: You can kill a computer process by typing ^C to the shell. The following session demonstrates the altered processing by the system now that the Killing-Terminate-Computer-Process metaphor has been acquired. The same question is again posed to the system. > (do-sentence) Interpreting sentence: How can I kill a process?
REPRESENTING UNIX DOMAIN METAPHORS
399
Interpreting primal input. (A Killing17 (ISA Killing) (agent88 (ISA agent) (A I47 (ISA I))) (patient77 (ISA patient) (A Computer-Process11 (ISA Computer-Process)))) Concreting input relations. Concreting patient to kill-victim. Concreting agent to killer. Interpreting concreted input. (A Killing17 (ISA Killing) (killer17 (ISA killer) (A I47 (ISA I))) (kill-victim17 (ISA kill-victim) (A Computer-Process11 (ISA Computer-Process)))) Failed interpretation: Killing17 as Killing. Valid known metaphorical interpretation. Applying conventional metaphor Killing-Terminate-Computer-Process. (A Killing-Terminate-Computer-Process (ISA Kill-Metaphor) (kill-victim-c-proc-termed-map kill-victim c-proc-termed) (killer-c-proc-termer-map killer c-proc-termer) (killing-terminate-computer-process-map Killing Terminate-Computer-Process)) The application of this known metaphor immediately yields the intended interpretation. Yielding interpretation: (A Terminate-Computer-Process11 (ISA Terminate-Computer-Process) (c-proc-termed11 (ISA c-proc-termed) (A Computer-Process11 (ISA Computer-Process))) (c-proc-termer11 (ISA c-proc-termer) (A I47 (ISA I)))) As in the previous example, all the conventional meanings are attempted before an interpretation is settled upon.
400
JAMES H. MARTIN
Failed interpretation: Killing17 as Kill-Delete-Line. Failed interpretation: Killing17 as Kill-Sports-Defeat. Failed interpretation: Killing17 as Kill-Conversation. Final interpretation: (A How-Q47 (ISA How-Q) (topic47 (ISA topic) (A Terminate-Computer-Process11 (ISA Terminate-Computer-Process) (c-proc-termed11 (ISA c-proc-termed) (A Computer-Process11 (ISA Computer-Process))) (c-proc-termer11 (ISA c-proc-termer) (A I47 (ISA I)))))) UC: You can kill a computer process by typing ^C to the shell.
7. Summary Consultant systems in technical domains must be capable of dealing with the conventional metaphors that structure those domains. The MIDAS system provides this capability for the UNIX Consultant. In particular, MIDAS has been used to systematically represent knowledge about the conventional metaphors in UNIX, interpret metaphorical language based on these metaphors, and finally to learn new UNIX domain metaphors as they are encountered by UC during normal processing.
8. Recent Developments While the MIDAS project (Martin 1990, 1992) demonstrated some significant results, it nevertheless had a major shortcoming. The effectiveness of the approach for language interpretation, generation and acquisition was obviously dependent on the size and the correctness of the knowledge-base of non-literal conventions. Unfortunately, the knowledge-base used by MIDAS did not have any kind of real coverage, nor did it have an empirically verifiable basis. Our more recent empirical work (Martin 1994) has been an attempt to address some of these problems. Specifically, we have been
REPRESENTING UNIX DOMAIN METAPHORS
401
developing corpus-based techniques to identify and analyze the conventional metaphors that tend to occur within large text collections genres. The results of these empirically-based analyses then forms the basis for the creation of a knowledge-base. References Carbonell, J. (1981). Invariance Hierarchies in Metaphor Interpretation. In Proceedings of the Third Meeting of the Cognitive Science Society, 292–295. Berkeley, CA: Cognitive Science Society. DeJong, G. F. & Waltz, D. L. (1983). Understanding Novel Language. Computers and Mathematics with Applications 9(1): 131–147. Fass, D. (1988). Collative Semantics: A Semantics for Natural Language. Ph.D. diss., New Mexico State University, Las Cruces, New Mexico. CRL Report No. MCCS-88-118. Gentner, D., Falkenhainer B. & Skorstad J. (1988). Viewing Metaphor as Analogy. In Helman, D. (ed.) Analogical Reasoning. Dordrecht: Kluwer Academic Publishers. Indurkhya, B. (1987). Approximate Semantic Transference: A Computational Theory of Metaphors and Analogy. Cognitive Science 11(4): 445–480. Jacobs, P. S. (1985). A Knowledge-Based Approach to Language Production. Ph.D. diss., University of California, Berkeley, Computer Science Department, Berkeley, CA. Report No. UCB/CSD 86/254. Lakoff, G. & Johnson, M. (1980). Metaphors We Live By. Chicago, Illinois: University of Chicago Press. Martin, J. H. (1986). The Acquisition of Polysemy. In The Proceedings of the Fourth International Conference on Machine Learning, 198–204. Irvine, CA, Martin, J. H. (1987). Understanding New Metaphors. In The Proceedings of the Tenth International Joint Conference on Artificial Intelligence, 137–139. Milan, Italy. Martin, J. H. (1988). A Computational Theory of Metaphor. Ph.D. diss., University of California, Berkeley, Computer Science Department, Berkeley, CA. Report No. UCB/CSD 88–465. Martin, J. H. (1990). A Compututational Model of Metaphor Interpretation. Cambridge, MA: Academic Press. Martin, J. H. (1992). Computer Understanding of Conventional Metaphoric Language. Cognitive Science 16(2): 233–270. Martin, J. H. (1994). Metabank: A Knowledge-base of Metaphoric Language Conventions. Computational Intelligence 10(2): 134–149. Norvig, P. (1987). A Unified Theory of Inference for Text Understanding. Ph.D. diss., University of California, Berkeley, Computer Science Department, Berkeley, CA. Report No. UCB/CSD 87–339. Wilensky, R. (1986). Some Problems and Proposals for Knowledge Representation. Technical Report UCB/CSD 86/294, University of California, Berkeley, Computer Science Division. Wilensky, R. (1987). Primal Content and Actual Content: An Antidote to Literal Meaning. Technical Report UCB/CSD 87/365, University of California, Berkeley, Computer Science Division. Wilensky, R., Chin D., Luria M., Martin J., Mayfield J., & Wu D. (1988). The Berkeley UNIX Consultant Project. Computational Linguistics 14(4): 35–84.
Using Justification Patterns to Advise Novice UNIX Users ALEX QUILICI∗ Department of Electrical Engineering, University of Hawaii at Manoa, 2540 Dole St, Holmes 483, Honolulu, HI, 96822, USA (E-mail: [email protected])
Abstract. Novice UNIX users have many incorrect beliefs about UNIX commands. An intelligent advisory system for UNIX should provide explanatory responses that correct these mistaken beliefs. To do so, the system must be able to understand how the user is justifying these beliefs, and it must be able to provide justifications for its own beliefs. These tasks not only require knowledge about specific UNIX-related plans but also abstract knowledge about how beliefs can be justified. This paper shows how this knowledge can be represented and sketches how it can be used to form justifications for advisor beliefs and to understand justifications given for user beliefs. Knowledge about belief justification is captured by justification patterns, domain-independent knowledge structures that are similar to the abstract knowledge structures used to understand the point behind a story. These justification patterns allow the advisor to understand and formulate novel belief justifications, giving the advisor the ability to recognize and respond to novel misconceptions. Keywords: belief inference, plans, user modeling
1. The Problem An intelligent UNIX help system should be able to provide explanations that correct mistaken plan-oriented user beliefs. Consider this dialog between a novice UNIX user and a more experienced UNIX advisor. (1) U SER: I accidentally removed a file using “rm”. How can I restore it now? (2) A DVISOR: You can’t. It takes several days to restore a file. (3) U SER: But I need to access the file now. (4) A DVISOR: You should use “rm -i” to remove files. That way you’re asked before any files are removed. (5) U SER: But I don’t want to be asked questions. (6) A DVISOR: You could move unwanted files into “/tmp”. (7) U SER: But that’s going to fill up “/tmp”. (8) A DVISOR: “/tmp” is automatically cleaned up every few hours, so filling it up isn’t a problem.
404
ALEX QUILICI
This user has this set of mistaken beliefs that need to be corrected: − There is a plan that can restore files immediately. − “rm” is an acceptable plan for removing files. − Removing files to “/tmp” is not an acceptable plan for removing files. − Removing files to “/tmp” fills up “/tmp”. The advisor’s responses explain why these user beliefs are mistaken. The advisor explains that: − There is no way to recover files today because the action for recovering files takes at least two days. − “rm -i” should be used instead of “rm” because it asks before removing the file. − Removing files to “/tmp” is an acceptable plan for removing files because because it does not fill up “/tmp”. − Removing files to “/tmp” does not fill up “/tmp” because some automatic action removes the files in that directory. To provide cooperative explanations like these, the advisor must be able to construct a model of the user’s beliefs and their justifications, and the advisor must be able to formulate justifications for why various user beliefs are incorrect. This paper is concerned with the knowledge the advisor needs to construct this user model and how this knowledge is represented and used. It assumes that the advisor is initially presented with a representation for the beliefs underlying a given user’s utterance. This assumption implies that the user’s plans and goals have been recognized, a task addressed by many current systems (Carberry 1989; Kautz and Allen 1986; Wilensky 1983).
2. What the Advisor Needs to Know A UNIX advisory system needs to represent two distinct classes of knowledge. The first involves specific plan-oriented user and advisor beliefs. In our dialog, for example, the user and advisor provide a sizeable set of beliefs about the various plans available for removing and recovering files. These beliefs are concerned with which plan is most appropriate for a given goal, whether or not there is a plan that achieves a particular goal, whether a particular state is an enablement or effect of a plan, what goals the user has, and so on. The other involves knowledge about how to justify plan-oriented beliefs. The advisor needs this knowledge to construct the belief justifications that are presented to the user, such as the advisor’s reason for why there is no plan that achieves the goal of recovering a file today. Furthermore, the advisor needs this knowledge to understand the user’s belief justifications, such as the user’s reasons for why “rm -i” and moving files to “/tmp” are not appropriate plans
USING JUSTIFICATION PATTERNS
405
Table 1. The planning relationships relevant to our example dialog. R ELATIONSHIP
S EMANTICS
A hasgoal E E applies E0 E causes E0 E interferes E0 E enables E0 E preferred E0
E is a goal of actor A E is an acceptable way to achieve E E has E0 as one of its effects E0 cannot occur if E does E is necessary for E0 to occur E is more desirable than E0
for the goal of removing a file. It is clearly possible to capture some of this knowledge in specific belief justifications. The advisor could, for example, possess concrete knowledge that “rm -i” is an appropriate plan for removing a file because it asks the user before removing the file. Furthermore, the advisor might possess the knowledge that “rm -i” might be considered inappropriate because users don’t want to be asked a question. The advisor, however, cannot be expected to possess all possible specific belief justifications in advance. To form these justifications the advisor clearly needs some general knowledge about what constitutes an appropriate justification. 2.1. Representing plan-oriented beliefs We represent beliefs in the same way as do most other systems that deal with the possibly contradictory beliefs of multiple dialog participants (Flowers et al. 1982; Pollack 1986a, 1986b). There is a relationship, belief(A, R), that indicates that an actor A (either the user or the advisor) believes a planning relationship R holds. Our representation for planning relationships combines elements used to represent plan effects and enablements found in other systems dealing with plan-oriented misconceptions (Pollack 1986a, 1986b) and elements needed to represent the goals and intentions of the user (Dyer 1983). We are concerned with the planning relationships shown in Table 1. There, A denotes an actor and E denotes an event (an actor’s execution of a particular plan or an action that takes place as a result of a plan’s execution). These relationships provide a way to represent beliefs about a plan having a particular state as an enablement or effect, a plan being appropriate for a particular situation, and so on. Each of these relationships also has a corresponding negated relationship, not-R, which is used to represent beliefs such as a plan not having a particular enablement or effect. These sorts of beliefs
406
ALEX QUILICI
are frequently found in advice-seeking dialogs. The advisor, for example, believes that the file recovery plan does not causes the file to be restored immediately, and he user believes that “rm” does not lead to his being asked questions. This small set of planning relationships can be used to represent the explicitly provided beliefs found in our example dialog.1 Each dialog utterance corresponds to a small set of beliefs. − In (1), the user provides an initial set of beliefs. One is that executing “rm” causes a file to be removed. causes represents a plan effect: a state change or action that results from a plan’s execution. Another user belief is that there is some plan that causes the file to be recovered immediately. The last user belief is that the user hasgoal of recovering the file immediately. hasgoal represents an actor desiring that a particular state hold or a particular event take place. − In (2), the advisor’s response provides a pair of beliefs. One is that there is no plan that causes the user’s file to be recovered immediately. The other is that there is some plan that causes the file to be recovered eventually. − In (3), the user provides one new belief: that the user hasgoal of accessing the file today. − In (4), the advisor again provides a pair of beliefs. One is that using “rm -i” applies to the goal of removing a file. applies represents the notion that a plan is an acceptable way to achieve a particular goal. The other belief is that “rm -i” causes a question that precedes the file’s being removed. − In (5), the user follows up with a belief that the user hasgoal of not being asked questions. − In (6), the advisor provides a single belief that moving the file to “/tmp” applies to the goal of removing a file. − In (7), the user provides a single belief that moving files to “/tmp” causes “/tmp” to fill up. − In (8), the advisor provides one final pair of beliefs. One is that moving files to “/tmp” not-causes “/tmp” to fill up. The other is that there is an action that interferes with “/tmp” filling up. The interferes relationship represents an action or effect preventing another action or effect from occurring. The user and advisor also appear to hold other, related beliefs that are not explicitly provided as part of the dialog. In particular, several of the user and advisor responses implicitly suggest other beliefs the user holds.
USING JUSTIFICATION PATTERNS
407
Table 2. The JPs that appear in our example dialog. R ELATIONSHIP
J USTIFICATION PATTERN
not-exists some E0 , where E0 causes E A hasgoal E E applies E0 E not-applies E0 E preferred E0 E interferes E0 E not-causes E0
JP:C LASS -ACTION -T HWARTS -E FFECT JP:E NABLEMENT-F OR -G OAL JP:P REFERRED -A LTERNATIVE JP:V IOLATES -OTHER -G OAL JP:ACHIEVES -A DDITIONAL -G OAL JP:H AS -T HWARTING -E FFECT JP:T HWARTED -B Y-OTHER -ACTION
− In (3), the user appears to believe that restoring a file enables accessing the file. The enables relationship represents one action being necessary for another to occur. − In (4), the advisor appears to believe that “rm -i” is preferred to “rm”. The preferred relationship represents one action being more desirable than another action. − In (5), the user appears to be believe that “rm -i” not-applies to the goal of removing a file. − In (7), the user appears to believe that removing a file to “/tmp” notapplies to the goal of removing a file and that he hasgoal of not filling up “/tmp”. 2.2. Representing abstract knowledge about belief justification We represent abstract knowledge about how to justify different kinds of beliefs using a set of Justification Patterns (JPs). A JP is an abstract configuration of planning relationships that captures one class of belief justification for one type of planning relationship. Different types of planning relationships, such as one plan being preferred to another, or a plan having a particular effect, are associated with different JPs. Instantiated JPs form the justifications for holding specific beliefs. Our example dialog makes use of a set of JPs for a variety of different belief types. Each response, in fact, makes use of at least one JP. Table 2 summarizes the different JPs used and the belief classes they justify. Here, we run through those responses and show how each of those JPs are represented and where each is used. − In (1), the user is simply describing his situation and not trying to justify any beliefs, so no JPs appear.
408
ALEX QUILICI
− In (2), the advisor uses a pair of JPs. The first, JP:C LASS -ACTION T HWARTS -E FFECT, justifies the advisor’s belief that there is no plan for immediately recovering a file. In general, this JP provides a way to justify a belief that there is no plan that causes a particular effect E (restoring the file today). The justification is that an action that causes a more general effect (restoring the file) somehow interferes with E (restoring the file immediately). JP:C LASS -ACTION -T HWARTS -E FFECT not-exists E0 , where E0 causes E justifies
E isa X P causes X P interferes E The other, JP:H AS -T HWARTING -E FFECT, is used to justify the advisor’s belief that a file recovery plan interferes with recovering the file today (the final belief in the advisor’s instantiation of the preceding JP). In general, this JP is a way to justify a belief that executing a plan P (the recovery plan) interferes with an effect E (recovering the file today). The justification is that this plan P has another effect E0 (taking two days to recover the file) that keeps E from occurring. JP:H AS -T HWARTING -E FFECT P interferes E justifies
P causes E0 E0 interferes E − In (3), the user uses a single JP, JP:E NABLEMENT-F OR -G OAL, to justify his belief that he wants to recover the file immediately. In general, this JP can be used to justify a belief that an actor A (the user) has a goal G (recovering a file immediately). The justification is that G is an enabling condition for some other goal G0 (accessing the file today). JP:E NABLEMENT-F OR -G OAL A hasgoal G justifies
G enables G0 A hasgoal G0
USING JUSTIFICATION PATTERNS
409
− In (4), the advisor again uses a pair of JPs. The first, JP:P REFERRED A LTERNATIVE, justifies the advisor belief that “rm -i” should be used to remove files. In general, this JP is used to justify a belief that executing a plan P (“rm -i”) applies to a goal G (removing a file). The justification is that P is preferred to another plan P0 (“rm”) that also achieves G. JP:P REFERRED -A LTERNATIVE P applies G justifies
P causes G P0 causes G P preferred P0 The other, JP:ACHIEVES -A DDITIONAL -G OAL, is used to justify the advisor’s belief that “rm -i” is preferred to “rm” (the final belief in the advisor’s instantiation of the preceding JP). In general, the JP is used to justify a belief that an action P (“rm -i”) is preferred over another action P0 (“rm”). The justification is that P achieves another goal G (asking questions before removing the file) that P0 does not. JP:ACHIEVES -A DDITIONAL -G OAL P preferred P0 justifies
A hasgoal G P causes G P0 not-causes G − In (5), the user uses another new JP, JP:V IOLATES -OTHER -G OAL, to justify a belief that “rm -i” is not applicable to the user’s goal of removing a file. In general, this JP is used to justify a belief that a plan P (“rm -i”) is not applicable to a goal G (removing a file). The user’s justification is that P0 (“rm -i”) has an effect E (asking questions) that the user has a goal to avoid. JP:V IOLATES -OTHER -G OAL P not-applies G justifies
P causes G P causes E A hasgoal not E − In (6), the advisor simply provides a new alternative plan for the goal, with no justification.
410
ALEX QUILICI
− In (7), the user uses the same JP, JP:V IOLATES -OTHER -G OAL, used in (5), but this time uses it to justify a belief that removing a file by moving it to “/tmp” is an unacceptable plan. In particular, removing a file to “/tmp” fills up “/tmp”, an action the user has a goal to avoid. − In (8), the advisor uses one final JP, JP:T HWARTED -B Y-OTHER ACTION. This JP justifies his belief that moving files to “/tmp” does not cause “/tmp” to fill up. In general, this JP is used to justify the belief a plan P (moving files to “/tmp”) does not have an effect E (filling up “/tmp”). The justification is that there is some other action (the clean-up program) that interferes with E. JP:T HWARTED -B Y-OTHER -ACTION P not-causes E justifies
E0 interferes E exists P0 causes E0
3. Using Justification Patterns Why are justification patterns important? JPs capture knowledge about belief justifications that is independent of specific plans and can be used in a variety of situations, allowing novel belief justifications to be formulated and understood without the need for a large collection of specific belief justifications. To see how this knowledge is useful, consider the advisor’s processing of the beliefs in (5). There, the advisor is presented with a single user belief: namely, that the user has a goal not to be asked questions. The advisor’s first task is to understand why the user is providing these beliefs. That involves figuring out which belief, if any, these beliefs justify, and which other user beliefs form an unstated component of this justification. In particular, the advisor must infer that the user is justifying a belief that “rm -i” is not appropriate for removing files, and the advisor must infer the unstated user belief in this justification: that “rm -i” leads to the user being asked questions. How can the advisor can make those inferences? One way is to find the JP that captures the user’s stated belief and that somehow relates this to previous user and advisor beliefs. The advisor can then assume that the user is trying to justify the belief justified by that JP and that the user holds the other beliefs within that JP. The advisor’s other task is to find an appropriate justification for any advisor beliefs that differ from the user’s. For example, in (4), the advisor
USING JUSTIFICATION PATTERNS
411
presents “rm -i” as being preferred to “rm” for removing files and justifies it by noting that it asks the user before actually removing the file. How can the advisor construct this justification? By using one of the JPs for a belief that one plan is preferred to another. The JP guides the advisor’s memory search for appropriate justifying beliefs. When the advisor finds a set of beliefs that can successfully instantiate the JP, those beliefs constitute the advisor’s response to the user. 3.1. Comprehending belief justifications How exactly are belief justifications understood? The process of comprehending the user’s belief justifications breaks into several parts: selecting and instantiating a likely-to-be-relevant JP, relating it to a belief already in the dialog, and confirming that it is reasonable for the user to have used this JP in forming his response. First, the advisor finds a candidate JP by running through each of the JPs that contain a relationship corresponding to one of the user’s newly-stated beliefs. In (5), for example, the advisor is presented with the belief that the user has a goal of not being asked questions. This belief matches a belief in JP:E NABLEMENT-F OR -G OAL, resulting in this instantiation of that JP: “rm -i” not-applies G justifies
“rm -i” causes G “rm -i” causes ask question user hasgoal not asked question (stated user belief) Second, the advisor has to relate this candidate JP to some belief that has already been stated or inferred from the dialog. The advisor must see if this JP justifies a belief that matches a known user belief, negates a known advisor belief, or appears in a JP that justifies one of those beliefs. At this point, as far as the advisor knows, the user holds no matching belief. However, the negation of the belief justified by this JP matches this previously stated advisor belief: “rm -i” applies remove-file As a result, the advisor can instantiate the JP with this new information. “rm -i” not-applies remove-file (contradicts stated advisor belief) justifies
“rm -i” causes remove-file “rm -i” causes ask question user hasgoal not asked question (stated user belief)
412
ALEX QUILICI
Finally, the advisor must confirm that it is reasonable for the user to have used the candidate JP. That is, the advisor must verify that the user holds any beliefs in this JP that were not explicitly provided in the current response. This verification of a belief is done by either: − Finding that the user provided those beliefs earlier in the dialog, or − Determining that the advisor holds those beliefs and the user has provided no explicit contradiction. Here, the advisor is trying to find user beliefs that “rm -i” causes file removal, that it causes a question to be asked, and that the user has a goal not to be asked questions. The user provides the last explicitly, and the advisor believes the first two and the user has provided no beliefs that contradict them. As a result, the advisor assumes the user holds these beliefs and has confirmed that the user is using this JP, and the advisor is now aware of which belief the user is justifying. 3.2. Constructing belief justifications How exactly are belief justifications constructed? The process of constructing belief justifications has several parts: selecting a candidate JP, instantiating it with information from the belief being justified, and then verifying that the advisor holds the beliefs in this partiallyinstantiated JP. First, the advisor finds a candidate JP by classifying the belief to be justified and selecting one of the JPs corresponding to that class of belief. In our example, the advisor must construct a justification for the belief that “rm -i” should be used to remove a file. The advisor classifies this belief as a plan applying to a goal and selects one of the JPs associated with that type of belief: JP:P REFERRED -A LTERNATIVE. Second, the advisor instantiates the candidate JP with information from the belief he is trying to justify. Here, the advisor instantiates the JP with P as “rm -i” and G as removing a file. rm -i applies remove file (advisor belief to justify) justifies
rm -i causes remove file P0 causes remove file rm -i preferred P0 Finally, the advisor then tries to confirm that the beliefs in this candidate JP form a suitable justification. The advisor does this by repeatedly selecting a belief in the JP, trying to verify it, and then instantiating the JP with any new information gleaned from the verification process. To verify a particular belief, the advisor searches memory for a matching belief (or specific
USING JUSTIFICATION PATTERNS
413
instances of it) and, if that fails, tries to justify the belief using additional JPs. This process stops when the JP is successfully instantiated and confirmed or when memory search fails to yield new verifying beliefs. For the above JP, the advisor first tries to verify that “rm -i” removes a file. Memory search yields the confirming belief that it does. The advisor then tries to find a match for the belief that there is some other plan that can remove a file. In this case, the advisor locates the belief that “rm” does so and and the advisor instantiates the JP with this information. rm -i applies remove file (advisor belief to justify) justifies
rm -i causes remove file (stated advisor belief) rm causes remove file (stated user belief) rm -i preferred rm That leaves one belief to verify, that “rm -i” is preferred to “rm” for removing files. Unfortunately, the advisor doesn’t locate this belief in memory. As a result, the advisor must try to justify this belief using JPs. In this case, the advisor uses JP:ACHIEVES -A DDITIONAL -G OAL to try to justify this belief, which the advisor instantiates as: rm -i preferred rm (advisor belief to justify) justifies
user hasgoal G rm -i causes G rm not-causes G This leaves the advisor several beliefs to try to instantiate and verify. In this case, the advisor locates the belief that rm -i causes a question to be asked before file removal, resulting in this JP being instantiated as: rm -i preferred rm (advisor belief to justify) justifies
user hasgoal asked before remove rm -i causes asked before remove (stored advisor belief) rm not-causes asked before remove The advisor now locates the remaining beliefs in memory: that rm does not ask before removing the file and that the user has a goal of being asked before the file is removed. The result of the confirmation process is a fully instantiated and verified JP.
414
ALEX QUILICI
4. Implementation Status The model discussed in this chapter has been implemented in a Prolog program: The Correction Machine. The program’s current domain of expertise is the basic UNIX commands needed to remove, recover, and rename files. It possesses the seven justification patterns discussed here, along with ten others that we have found useful in processing variants of our example dialog. The program itself consists of a COMPREHENDER and a CONSTRUCTOR. The COMPREHENDER ’ S input is a representation for a set of user beliefs. Its output is the beliefs justified by these beliefs and the instantiated justification patterns to which these beliefs belong. In addition, the output shows the relationships between these beliefs and other beliefs already specified in the dialog. The CONSTRUCTOR ’ S input is a particular advisor belief. Its output is a set of beliefs that justify holding this belief, along with a description of the particular justification patterns the advisor used to produce that explanation. The program can construct and comprehend all the belief justifications in the file removal debate. Currently, we are trying to answer several questions about our model. First, how well do the justification patterns described here account for responses to misconceptions in domains other than those of novice computer users? To test their domain-independence, we are extending the program to give advice about simple day-to-day planning. Second, how sufficient is our set of justification patterns for providing UNIX advice? We are now studying many different user/advisor dialogs, searching for the presence of other useful justification patterns. We are also working on improving our model’s performance, particularly on the task of selecting potentially useful JPs, as this process is potentially time-consuming. During comprehension, for example, an input belief may correspond to a variety of different JPs, many of which are eventually ruled out because they can’t be related to other beliefs in the dialog or happen to contain beliefs that contradict known user beliefs. Also, during construction, many possibly-useful JPs are eventually ruled out because the advisor can’t verify that he holds a belief the JP contains. There are two ways to speed up the process; both of which lead to unanswered questions. One is to save instantiated justification patterns for later use, but then how are these specific JPs organized and retrieved? The other is to first try the JPs that are most likely to prove useful, but then how can the most appropriate JP be selected? Finally, we are trying to extend the model toward being a more complete dialog participant, a task that raises one other important question: How is the particular belief to justify selected? The advisor quickly faces a large collection of beliefs from which he must choose a particular belief to justify. Our model, however, simply assumes that this belief has already been chosen.
USING JUSTIFICATION PATTERNS
415
5. Comparison with Related Work There are several classes of related systems. The first includes systems that try to provide advice to novice UNIX users. Existing UNIX advisors such as UC (Wilensky et al. 1988) and SC (Hecking et al. 1988) do not attempt to explain mistaken beliefs. Instead, they assume the user’s problem is incomplete knowledge, and focus on filling in the gaps indicated by questions such as “How do I remove a file?”, “What does ‘rm -i’ do?”, and so on. The other classes are systems that try to correct user misconceptions and systems that try to understand or participate in arguments. 5.1. Explaining user misconceptions Our own earlier effort, AQUA (Quilici 1989a; Quilici et al. 1988; Quilici et al. 1986) tried to correct plan-oriented user misconceptions. AQUA worked by taking a single user belief (that a plan is applicable to a goal, or that a state is an enablement or effect of a plan) and used strategies that try to justify not holding the belief. A serious problem with AQUA was that it had strategies only for why people do not believe something, and not for why they do. It had no knowledge about how to justify a belief that a plan is preferred to another plan or has a particular state as an enablement or effect. This made it impossible for AQUA to understand the user’s belief justifications, and it severely limited AQUA’s ability to construct belief justifications. ROMPER (McCoy 1989) tried to explain a different class of user misconceptions, mistaken user beliefs involving misclassifications or misattributions involving objects. ROMPER applied an approach similar to AQUA’s, using a set of strategies for justifying its beliefs to the user. However, ROMPER’s strategies differed in that they combined justifications for why the user might have held a belief with justifications for why the system did not. Because of this lack of separation, and because it possessed only a few strategies for only two kinds of mistaken beliefs, ROMPER suffered from the same drawbacks as AQUA. SPIRIT (Pollack 1986a, 1986b) tried to detect and correct the mistaken plans of users of a computer mail program. Although SPIRIT’s overall task was similar to ours, it took a completely different approach. Rather than trying to infer which beliefs the user was attempting to justify, it tried to infer the user beliefs underlying the user’s mistaken beliefs. In addition, rather than trying to justify its own beliefs to correct user misconceptions, it simply pointed out which of the user’s underlying beliefs were incorrect. Finally, there’s a large collection of tutoring systems that attempt to correct and explain user misconceptions (Sleeman and Brown 1982). Generally, these systems locate mistaken beliefs in a database of domain-specific error-
416
ALEX QUILICI
explanation pairs and provide the associated explanation. This approach has several drawbacks. Because the explanations are domain-specific, having the tutor provide explanations for mistakes in a new domain involves finding a new set of error-explanation pairs. Moreover, because these systems simply retrieve explanations, they can handle only those misconceptions they know about in advance. 5.2. Systems that process arguments Two other systems considered the problems of constructing or comprehending belief justifications in the context of participating or understanding editorial arguments. The first, ABDUL/ILANA (Flowers et al. 1982), argues about responsibility for historical events. Its knowledge about belief justification is also in rules similar to our JPs. One rule is: X did not attack first if Y preceded X’s attack with an attack act. The problem with these rules is that they are tied to specific types of events, such as an attack, that are found in just a few domains. JPs, on the other hand, are tied only to abstract planning relationships independent of a particular domain. The other system, OpEd (Alvarado et al. 1986), recognizes plan-oriented belief justifications in economic editorials. Its primary knowledge about belief justification is in its argument units (AUs). One AU is AU-OPPOSITEEFFECT, which says that one can believe P is bad if one’s opponent believes that P achieves a goal G, but one does not believe P achieves G because one believes P thwarts G. The problem with AUs is that they’re tied to knowledge about argument structure (such as the connections between the beliefs of different participants) and not limited to knowledge about belief justification (a plan not achieving a goal because it somehow thwarts the goal). This makes it difficult to use them for constructing justifications, a task not considered by OpEd.
6. Conclusions This paper has presented a model of the process of providing corrective explanatory responses to misconceptions made by novice UNIX users. To provide these responses, the advisor must be able to formulate justifications for its own beliefs and to understand the belief justifications of the user. Both tasks require abstract knowledge about belief justification. This knowledge is represented in a collection of justification patterns that capture knowledge about possible ways to justify different types of plan-oriented beliefs. This approach differs from earlier efforts in two ways. First, its knowledge about
USING JUSTIFICATION PATTERNS
417
belief justification depends only on the abstract planning structure of the different types of beliefs being justified, and not on the specific domain (knowledge about events) or the specific task (arguing or correcting misconceptions). Second, it demonstrates how the same knowledge about belief justification can be used to build a model of the user’s beliefs, and to provide an appropriate advisor response, not for just one or the other. Justification patterns are important because they represent knowledge about an entire class of justifications. This knowledge allows novel belief justifications to be formed and understood, so long as they fall into one of the known belief classes. It also eliminates the need for large collections of specific justifications. Justification patterns potentially benefit any system, such as an intelligent user interface or tutor, that wishes to recognize and respond to the mistaken beliefs and belief justifications of its users.
7. Recent Developments The work reported in the original version of this paper was the starting point for a considerable amount of subsequent research. 7.1. Our subsequent research efforts Our initial focus was to provide a more detailed model of the process of forming advisor responses (Quilici 1989b). In particular, this work elaborated the process of providing an appropriate belief justification. It showed how justification patterns could be used to control the search for the relevant advisor beliefs to provide as a response, as well as how the advisor’s response could provide a belief justification that took advantage of previously stated user beliefs (and was therefore more likely to be accepted by the user than a canned justification that didn’t take these beliefs into account). Our next step was to determine whether our model could be generalized beyond simple advice-giving dialogs in the UNIX domain. In particular, we explored how to apply our approach to a restricted class of debates about everyday planning: debates in which each dialog response presents a belief, each belief addresses some perceived disagreement, and each belief is part of a justification for some other belief related to the dialog (Quilici 1992; Quilici 1991; Quilici 1990). These efforts showed that justification patterns contributed toward addressing two major problems in discourse processing: understanding the underlying connections between successive dialog utterances and producing coherent dialog responses. Our efforts then turned back to the UNIX advising world, and we developed a considerably more complete model of the advice-giving pro-
418
ALEX QUILICI
cess (Quilici 1994). In particular, this work showed how focus heuristics could effectively direct the search for the particular justifications that related user beliefs to beliefs provided earlier in the dialog. In addition, we showed how many advisor dialog responses could be modeled as directly addressing unstated user beliefs that were inferred from the justification patterns used to relate user responses to earlier dialog responses. 7.2. Current open problems Despite all of this research effort, several important open problems remain, and we are currently working to address them. The first is that while our justification patterns are useful, they are clearly not complete. For example, we currently can’t represent responses involving class-based or example-based justifications, nor can we deal with responses involving probabilistic justifications. We also have no mechanism for representing responses that include meta-justifications: reasons for why the justifications for executing one plan outweigh the justifications for executing another plan. As a result, our representation for belief justifications needs to be extended considerably. The second is our advisor’s ignorance of the purpose of the advisory dialog. All along, we have simply modeled the advisor’s role as a process of detecting and responding to incorrect user beliefs. However, users participate in a dialog for a purpose, such as to achieve particular knowledge goals (e.g., knowing an acceptable plan to achieve a domain goal). It appears as though these knowledge goals can help the advisor choose which user beliefs to respond to, as well as when to provide responses that do not simply justify a belief (e.g., providing an alternate plan for a goal). As a result, we need a more sophisticated model of the advisor’s response processing that takes into account user knowledge goals. The third is that our approach considers only those user beliefs necessary to relate the user’s stated belief to the dialog. While this constrains the belief inference process, it means that the advisor’s responses cannot address the underlying reasons why a user holds a stated belief (e.g., why the user does not want to be asked questions). Fortunately, justification patterns suggest a mechanism to use to infer these user beliefs; namely, trying to find instantiated JPs that can justify them. As a result, our model must be extended to determine exactly when to use justification patterns to try to infer justifications for why users hold the beliefs they state. Finally, our advisor is currently passive: it doesn’t actively seek out information from the user. However, it’s reasonable for an advisor to ask the user questions as part of the advice-giving process (e.g., asking “why?” after the user specifies that answering questions is undesirable). This active
USING JUSTIFICATION PATTERNS
419
understanding can help the advisor determine the underlying reasons why the user holds beliefs that conflict with the advisor’s beliefs, as well as provide other needed information that helps the advisor determine the most appropriate response. As a result, our model must be extended to include a mechanism for determining when it is desirable to ask the user questions, as well as a mechanism for understanding the user’s answers. Despite all of the open problems that must be addressed before we can have an on-line UNIX advisory system, it’s encouraging that there are potential solution paths that build on our previous work. This suggests that our original paper may eventually prove to be have been an important first step.
Notes ∗ This paper primarily reports work done when the author was a graduate student in the
Computer Science Department of the University of California, Los Angeles. It also describes later work that was supported by NSF Research Initiation Award #9309795. 1 By “explicitly provided beliefs” we refer to beliefs that the user has stated explicitly or that can be readily inferred from the user’s questions. For example, if the user asks “How do I do X?”, we can automatically infer that the user has X as a goal and that the user believes that there is some plan that achieves X.
References Alvarado, S., Dyer, M. G. & Flowers, M. (1986). Editorial Comprehension in OpEd through Argument Units. In Proceedings of the Sixth National Conference on Artificial Intelligence. Philadelphia, PA. Carberry, S. (1989). Modeling the User’s Plans and Goals. In Kobsa, A. & Wahlster, W. (eds.) User Modeling and Dialog Systems. New York, NY: Springer Verlag. Dyer, M. G. (1983). In-depth Understanding: A Computer Model of Narrative Comprehension. Cambridge, MA: MIT Press. Flowers, M., McGuire, R. & Birnbaum, L. (1982). Adversary Arguments and the Logic of Personal Attacks. In Lehnert, W. & Ringle, M. (eds.) Strategies for Natural Language Processing. Hillsdale, NJ: Lawrence Erlbaum. Hecking, M., Kemke, C., Nessen, E., Dengler, D., Gutmann, M. & Hector, G. (1988). The SINIX Consultant – A Progress Report, Technical Memo 28. University of Saarbrücken. Kautz, H. & Allen, J. (1986). Generalized Plan Recognition. In Proceedings of the Sixth National Conference on Artificial Intelligence. Philadelphia, PA. McCoy, K. (1989). Reasoning on a Highlighted User Model to Respond to Misconceptions. In Kobsa, A. & Wahlster, W. (eds.) User Modeling and Dialog Systems. New York, NY: Springer Verlag. Pollack, M. (1986a). A Model of Plan Inference that Distinguishes between the Beliefs of Actors and Observers. In Proceedings of 24th meeting of the Association of Computational Linguistics. New York, NY. Pollack, M. (1986b). Inferring Domain Plans in Question-answering, Ph.D. Thesis, Department of Computer Science, University of Pennsylvania.
420
ALEX QUILICI
Quilici, A., Dyer, M. G. & Flowers, M. (1986). AQUA: An Intelligent UNIX Advisor. In Proceedings of the Seventh European Conference on Artificial Intelligence. Brighton, England. Quilici, A., Dyer, M. G. & Flowers, M. (1988). Recognizing and Responding to Plan-oriented Misconceptions. Computational Linguistics 14(3): 38–51. Quilici, A. (1989a). AQUA: A System that Detects and Responds to User Misconceptions In Kobsa, A. & Wahlster, W. (eds.) User Modeling and Dialog Systems. New York, NY: Springer Verlag. Quilici, A. (1989b). The Correction Machine: Formulating Explanations for User Misconceptions. In Proceedings of the 11th International Joint Conference on Artificial Intelligence. Detroit, MI. Quilici, A. (1990). Participating in Plan-oriented Dialogs. In Proceedings of the 12th Annual Conference of the Cognitive Science Society. Boston, MA. Quilici, A. (1991). The Correction Machine: A Computer Model of Recognizing and Producing Belief Justifications in Argumentative Dialogs. Ph.D. Thesis, Department of Computer Science, University of California. Quilici, A. (1992). Arguing about Plan Alternatives. In Proceedings of the 17th Annual Meeting of the Computational Linguistics Society. Nantes, France. Quilici, A. (1994). Forming User Models by Understanding User Feedback. User Modeling and User Adapted Interaction 3(4): 321–358. Sleeman, D. & Brown, J. S. (eds.) (1982). Intelligent Tutoring Systems. Orlando, FL: Academic Press. Wilensky, R., Chin, D., Luria, M., Martin, J., Mayfield, J. & Wu, D. (1988). The Berkeley UNIX Consultant Project. In Computational Linguistics 14(4): 35–84. Wilensky, R. (1983). Planning and Understanding. Reading, MA: Addison Wesley.
Artificial Intelligence Review 14: 145–147, 2000.
145
About the Authors
Matthias Hecking is 41 years old. He received his Master’s degree in computer science (Dipl.-Inform.) at the University of Hamburg in 1984. During and after the studies Matthias developed commercial software for different software houses. From 1985 to 1988 he worked at the SINIX Consultant (SC) at the chair of Prof. Wahlster at the University of the Saarland. Matthias joined the German Research Center for Artificial Intelligence (DFKI). From 1988 to 1993 he worked on the PHI project. He received his Ph.D. in computer science at the University of the Saarland in 1993. Then Matthias became a house husband. During this time he worked on the Jesko and Jascha projects. Since 1996 Matthias works as a free-lancer software engineer in the Microsoft world. Stephen J. Hegner received the Ph.D. degree in Computer and Information Science from the University of Massachusetts at Amherst in 1977. Currently, he is a senior lecturer in the Department of Computing Science at Umeå University, Sweden. His areas of professional interest focus on algorithmic aspects of knowledge representation and inference in database and knowledgebase systems. John Jones was born in 1956 in England and died in 1994. He achieved a BSc First Class degree in Mathematics from
the University of Nottingham in 1980. He completed his Ph.D. in Mathematical Logic at the Department of Mathematics, University of Nottingham. He was a Research Fellow in Artificial Intelligence at the University of Edinburgh from October 1983 until September 1986. He joined the Department of Computer Science at the University of Hull as a Lecturer in October 1986 and remained there until he died. Christel Kemke received her Master’s Degree in Computer Science from the University of Dortmund, Germany, with a thesis on Finite State Machines and Neural Network Modeling, and a Ph.D. from the Technical Faculty at the University of Bielefeld, Germany, with a thesis on Neural Network Models for Natural Language Processing. She also completed a B.Sc. honours degree in Psychology at the Open University, Milton Keynes, England. From 1984 to 1988 she has been working on the SINIX Consultant Project at the University of Saarbruecken, Germany. Then, she conducted research in the ASL project at the DFKI in Saarbruecken on the application of Neural Networks for Speech and Language Processing which she continued during a 6 months stay at the International Computer Science Institute in Berkeley, CA, USA. She worked as a Research Computer Scientist at the University College Dublin, Ireland, in the
146
ABOUT THE AUTHORS
EU funded QUALIT Project on Knowledge Engineering and Representation. She has conducted several university seminars and conference tutorials on Neural Network Modeling, Natural Language Processing, and Knowledge Representation, in Germany and Ireland, and has supervised 12 Master’s Degree students. She was elected Co-Chair of the German Interest Group on Neural Networks and is member of the German Computer Science Society and the Cognitive Science Society. Currently, she is visiting professor of the Computer Science Department at New Mexico State University, Las Cruces, New Mexico, USA. Paul Mc Kevitt is 36 and from Dún Na nGall (Donegal), Ireland. He is Associate Professor (Senior Lecturer) in the School of Computer Science at The Queen’s University of Belfast, Northern Ireland. Previously, he has been Visiting Professor of Intelligent MultiMedia Computing in the Institute of Electronic Systems at Aalborg University, Denmark and a British EPSRC (Engineering and Physical Sciences Research Council) Advanced Fellow in the Department of Computer Science at the University of Sheffield, England. The Fellowship, commenced in 1994, and released him from his Associate Professorship (tenured Lecturership) for 5 years to conduct full-time research on the integration of natural language, speech and vision processing. He completed a Master’s degree in Education (M.Ed.) at the University of Sheffield in 1999. He completed his Ph.D. in Computer Science at the University of Exeter, England in 1991. His Master’s degree in Computer Science (M.S.) was obtained from New Mexico State University, New Mexico,
USA in 1988 and his Bachelor’s degree in Computer Science (B.Sc., Hons.) from University College Dublin (UCD), Ireland in 1985. His primary research interests are in Natural Language Processing (NLP) including the processing of pragmatics, beliefs and intentions in dialogue. He is also interested in Philosophy, MultiMedia and the general area of Artificial Intelligence. Mark Millington is 39 and from Chester, England. He is a consultant specialising in finance and telecommunications applications, working in the Chicago area. He completed his Ph.D. in Computer Science in 1985 at the University of Edinburgh, Scotland, and his Bachelor’s degree in Computer Science and Mathematics in 1980 at the University of Essex, England. Peter Norvig received a B.S. in Applied Mathematics from Brown University and a Ph.D. in Computer Science from the University of California at Berkeley. He has been a Professor at the University of Southern California and a Research Faculty Member at Berkeley. He has over forty publications in various areas of Computer Science, concentrating on Artificial Intelligence, Natural Language Processing and Software Engineering. He has also served as a Senior or Chief Scientist at Sun Microsystems Labs, Harlequin Inc. and Junglee Corp., and is currently Chief of the Computational Sciences Division at NASA Ames Research Center. Maria Virvou is 34, from Athens, Greece. She has been a lecturer in the Department of Computer Science at the University of Piraeus, Greece since 1994. She completed her Ph.D. in Artifi-
ABOUT THE AUTHORS
cial Intelligence and Computer Science at the University of Sussex, England in 1992. She received her M.Sc. degree in Computer Science from University College London in 1987. Her first degree in Mathematics was obtained from the University of Athens, Greece in 1986. Her current research interests are in user modelling, Artificial Intelligence in Education, distance learning and adaptive user interfaces. Robert Wilensky received his B.A. and his Ph.D. from Yale University. In 1978, he joined the faculty of the University of California at Berkeley, where he is now
147
Professor in the Division of Computer Science, and in the School of Information Management and Systems. He has served as Chair of the Computer Science Division, the director of BAIR, the Berkeley Artificial Intelligence Research Project, and the director of Berkeley Cognitive Science Program. Professor Wilensky has published numerous articles and books in the area of artificial intelligence, planning, knowledge representation, natural language processing, and digital information systems. He is currently Principal Investigator of UC Berkeley’s Digital Library Project.
Editorial Intelligent Help Systems for UNIX: Planning and Knowledge Representation STEPHEN J. HEGNER1, PAUL MC KEVITT2, PETER NORVIG3 and ROBERT WILENSKY4 1 Umeå University, Sweden; 2 University of Ulster, Northern Ireland; 3 NASA Ames Research Center, California, USA; 4 University of California at Berkeley, USA
This is the second of a series of three special issues on intelligent help systems for UNIX.1 This issue addresses planning and knowledge representation whereas the first issue focussed on computational models and systems and the next will be on natural language dialogue. The papers in this issue are concerned with discovering what the user wants to do, and figuring out a way to do it as well as representing the knowledge needed to do so. A passive consultant accepts questions from the user, but is otherwise ignorant of the user’s goals. An active consultant continually monitors the user’s actions, and tries to discover the user’s goals from these actions. The contribution of Hecking is a theoretical analysis of the problem that an active consultant faces, the plan recognition problem. He describes two modules, one to recognize plans, and the other to advise the user of possible alternative plans that would achieve the same goal more efficiently. Along the way, a representation of plans, subplans, goals, actions, properties and time intervals is developed. In sum, all the results from the planning subfield of artificial intelligence (AI) are applicable to the UNIX consultant task. Hegner discusses plan realisation in the UNIX help domain, as embodied in the Yucca-* help system. The primary goal of this system is to provide detailed expert advice on the use of the UNIX command language for complex queries, the solution of which may involve interconnection of several commands, each with multiple options. This requires not only the representation of dynamic knowledge about command behaviour at a level of detail sufficient to support solution of the query, but also a planning mechanism capable of interconnecting such knowledge into a cohesive solution. A key to realising such behavior is the observation that the form of knowledge representation necessary to support modelling of communication with the user is quite different than that which is appropriate for the representation of details about the behavior of an operating system. Yucca-* therefore embodies a two-
150
STEPHEN J. HEGNER ET AL.
level architecture, comprised of an understander unit for interfacing with the human, and a formal knowledge and solver unit for detailed knowledge about UNIX. The paper focusses on the design of the formal knowledge and solver unit, with particular emphasis on aspects of the problem unique to the UNIX domain. It is argued that very general techniques (weak methods) are not sufficient in complex domains. Domain-specific shortcuts, such as clichés, are often needed to gain acceptable performance. Again, the UNIX consultant task provides an example of this AI truism. Jones et al. describe a mechanism for identifying the explicit assumptions about the user which are necessary to account for the various hypotheses concerning what the user is thinking at every stage of interaction with an intelligent help system. The consistency of the assumptions is managed by an Assumption-based Truth Maintenance System (ATMS). Selection among different user models is based on the information which is extracted from the feedback that UNIX provides in response to user actions. The mechanism is used by the user modelling component of such a help system. First-order logical representations, such as those employed by Hecking and Hegner, are not well suited in cases where properties involve default values. Typically, the logic must be extended with some kind of non-monotonic component. One might think that such an extension is not necessary in the UNIX domain, since all commands have a well-defined semantics. However, while the result of an action on a given state of the world is well-defined, knowledge about the state of the world is not. The user may have misconceptions about how commands work, or about what is true in the current state. The work of Jones et al. searches for a model of the user which is consistent with the current state, while requiring the fewest number of non-default assumptions about the user’s knowledge. Kemke describes knowledge representation in the SINIX Consultant, an intelligent help system for the SINIX operating system, a UNIX derivative developed by Siemens AG. The SINIX Consultant answers natural language questions about SINIX concepts and commands and also gives unsolicited advice. The knowledge base is a taxonomical hierarchy of SINIX concepts, divided into objects and actions operating on these objects. A single concept in the knowledge base is described by a set of attributes reflecting structural or syntactical features, the use, application and purpose of the command or object, and additional information for explaining the concept to the user. In comparison to Yucca-*, the SINIX Consultant embodies much more detailed knowledge about the interactive behavior of complex individual commands, such as mailers, while the emphasis of Yucca-* is upon knowledge which allows one to combine relatively simple independent commands (e.g., ls, cat) together to form complex solutions.
EDITORIAL
151
The fundamental lesson which may be extracted from the papers of this issue is that the form of knowledge representation which is required depends heavily upon the goals of the system. In particular, those systems which have, as a goal, the delivery of complex technical information about UNIX (e.g., Yucca-* and the SINIX consultant) must embody a design fundamentally different from those which focus primarily upon modelling the user (e.g., the system of Jones et al., as well as a number of components of the UC system described in the previous issue.) The articles in this issue are derived from papers originally presented at a workshop entitled Knowledge representation in the UNIX help domain, organised by Peter Norvig, Wolfgang Wahlster and Robert Wilensky at the University of California, Berkeley, USA, in December, 1987. The workshop was funded, in part, by the International Computer Science Institute and all participants were invited. The area of intelligent help systems for UNIX provides such a fruitful example domain of application for many AI techniques that we have decided to publish this work, which is still timely, more widely and particularly now where we have many spoken dialogue systems applied to such fixed domains (see Bernsen et al. 1998; BusinessWeek 1998). Each article here has been reviewed by the editors and has been subsequently revised; furthermore, all authors have been asked to include a section on recent developments on their work. Related work which may be of interest to the reader can be found in Kobsa and Wahlster (1988) which focusses on user modelling and in Maybury and Wahlster (1998) which is a comprehensive publication on intelligent user interfaces. We regret John Jones passed away in 1994 in a climbing accident in Peru. Members of the Kluwer editorial and publishing staff are to be thanked for their help in producing this issue. It would not have been possible without Melanie Willow (Journals Editorial Office), André Diepenbroek (Desk Editor, Editorial Department), and Vanessa Nijweide (Kluwer Publishing Editor, Humanities and Social Sciences). Melanie and André have worked very hard with authors and reviewers to enable this to happen. The Editors, June 2000 Stephen J. Jegner Umeå University Sweden Paul Mc Kevitt University of Ulster Northern Ireland, UK
152
STEPHEN J. HEGNER ET AL.
Peter Norvig NASA Ames Research Center California, USA and Robert Wilensky University of California at Berkeley USA
Note 1 UNIX is a trademark of X/Open, Inc.
References Bernsen, N-O., Dybkjær, H. and Dybkjær, L. (1998). Designing Interactive Speech Systems: from First Ideas to User Testing. Springer-Verlag: New York, NY. BusinessWeek (1998). Speech Technology is the Next Big Thing in Computing, Special Report, Feb. 23. McGraw-Hill: New York. Kobsa, A. and Wahlster, W. (1988). User Models in Dialog Systems. Springer-Verlag: Berlin. Maybury, M. and Wahlster, W. (eds.) (1998). Readings in Intelligent User Interfaces. Morgan Kaufmann Publishers: Los Altos, CA.
The SINIX Consultant – Towards a Theoretical Treatment of Plan Recognition MATTHIAS HECKING Marienstrasse 20, D-56112 Lahnstein, Germany (E-mail: [email protected])
Abstract. We have realized the help system SINIX Consultant (SC) for SINIX1 users. The system is capable of answering – in German – natural language questions concerning SINIX commands, objects, and concepts. But not only does this help system react to inquiries – additionally, the system is capable of activating itself. If the user employs a sequence of SINIX commands (a plan) in order to reach a specific goal, the help system proposes a sequence which reaches the same goal, but, with fewer commands. In this paper, a brief survey of the SINIX Consultant and the realized plan recognizer REPLIX is first given. Then, an initial attempt of a theoretical treatment of plan recognition is presented. This is done within the logical framework. We show how we can use an interval-based logic of time to describe actions, atomic plans, non-atomic plans, action execution, and simple plan recognition. We also show that the recognition of inserted sub-plans managed by REPLIX can be handled as well. Then, we present a problem which cannot be treated in the formalism. Thus, in this paper, we don’t present a full developed theory but nevertheless, a step towards it.
1. Introduction Within the SINIX Consultant (SC) project, we have realized a plan recognizer called REPLIX. This integrated component is able to scrutinize the commands the user typed in and can detect the goal persued by these commands. Beside the program code and various descriptions in natural language, we don’t have an exact representation of the semantics in the field of plan recognition. In order to obtain this, an appropriate mathematical apparatus has to be employed. We assume that symbolic logic is such an apparatus (some arguments for such an assumption can be found in [Genesereth and Nilsson 1987, Preface]). Instead, in a first approach, we used attribute grammars (cf. Hecking and Harbusch 1987). These were dropped because logic presents a formal treatment of the meaning (extensional semantics). Furthermore, we assume that, in general, symbolic logic provides a mathematical basis for AI theory. We do not claim that first-order predicate logic is expressive enough for all problems in AI, although it can be used quite
154
MATTHIAS HECKING
successfully (see Allen’s theory of action and time; cf. Allen 1984). There are though two reasons to employ symbolic logic. First, logic is an old research topic with a lot of well established results (e.g. proof theory or model theory). We can use these results. Second, there are other logics (e.g. logic of knowledge and belief) which were invented to overcome the limitations of first-order predicate logic. These logics are already used to develop theories in AI (see the deduction model of belief from K. Konolige; cf. Konolige 1986). Using these assumptions, we describe a step towards such a logical based theory of plan recognition. Thus, in this paper we do not present a fully developed theory. This report is structured as follows: 1. In section 2, the help systems SC is described, especially the difference between its active and passive capabilities. The structure of SC, the interaction of its components, and the state of implementation follow. 2. In section 3, the REPLIX and the Advisor components are described in more detail. 3. In section 4 we try to identify those concepts in command-oriented plan recognition which should be modelled in a more theoretical manner. 4. In section 5, the first step towards a model is presented. The model is formulated within the logical framework. We describe actions, atomic and non-atomic plans, action execution, and simple plan recognition. We show that we can formulate the concept of inserted plans (which can be handled by REPLIX too) and we mention a problem which cannot be handled in this initial step. 2. Survey of the SINIX Consultant At the University of Saarbrücken, the help system SINIX Consultant (SC) has been in the progress of development since summer 1985. One of the main goals of the project is to integrate passive and active capabilities. The demands on the system are: • to answer natural language questions about concepts, commands, and objects of the operating system SINIX in German, • to conduct a cooperative dialogue, • to infer the goal out of closely examined commands in order to give voluntary advice on ‘better’ plans. Next, we explain the differences between active and passive help systems.
THE SINIX CONSULTANT
155
2.1. Passive help systems If the user seeks information about a concept or a command, the usual way in obtaining it is by either consulting the manual (in simple cases) or by asking a qualified colleague, i.e. posing questions in natural language. The user may need information about those concepts and commands which he rarely uses, or he may ask for advice pertaining to those concepts which are not part of the system. In these cases, the system may respond appropriately. The system delivers the information or simply states, that this is not a known concept (precondition: the help system covers every concept and command of the underlying system). If a help system reacts only to inquiries, we define it as being passive. If the inquiries can be posed in natural language (other possibilities are menus, help keys etc.), the natural language interface (NLI) can exhibit a greater or lesser degree of sophistication. The NLI may by only able to handle simple, syntactically correct questions. More elaborate NLIs may process elliptical questions, syntactically incorrect inquires (robustness), or even manage dialogues. There are several other passive help systems: • the Unix Consultant (UC) (cf. Wilensky et al. 1984, 1986), • the AQUA system (cf. Quilici et al. 1986). 2.2. Active help systems Those concepts and commands which are completely unknown to the user also remain unknown in a passive help system. The user can be made aware of these unknown concepts only if the system is able to become active by itself. A precondition for a help system in becoming active is the ability to identify possible applications of these unknown concepts and commands. One attempt to realize this, is that the system must infer the goal of the user’s actions and must find a more ‘appropriate’ way to reach the same goal. For example, in order to infer the underlying goals of the user’s non-verbal activities, we have a system which scrutinizes the commands the user types in. The ones used here are: mv letter1 invoices2 mv letter2 invoices The system infers the possible goal ‘move more than one file to a directory’. In this example, a more ‘appropriate’ way is that of a sequence of actions through which the user can reach the same goal with fewer keystrokes (there are other possible metrics). After detecting a more efficient way to reach the goal, the system becomes active, and proposes to the user:
156
MATTHIAS HECKING
You can reach the same goal with: mv letter1 letter2 invoices In this example, the user is made aware of the fact that the ‘mv’ command can be used with multiple files. Completely new concepts can also be introduced in this manner. The first active help system was WIZARD (cf. Finin 1983); another, the AKTIVIST (cf. Schwab 1984), is a help system for a screen-oriented editor. 2.3. The structure of the system A short introduction into the system structure and its state of implementation follows. For a more detailed description of the implemented system cf. Hecking et al. 1988, Hecking 1987 and Kemke 1987. The natural language input (cf. Figure 1) and the command input is passed on the Filter, which decides which type of input (NL or command) the user typed in. After this decision, the NL Input is transfered to the morphological analyzer MORPHIX (cf. Finkler and Neumann 1986). The deflected NL Input is the input of the Parser. The parser works in a two-level, pattern oriented fashion and is based on ideas of various DYPAR versions (cf. Boggs et al. 1984). The semantic representation of natural language input is based on case frames. The case frame is handed to the Question Evaluator which determines the type of questions, accesses the Domain Knowledge Base in order to get the desired information, and updates the user model SC-UM in order to record the individual level of experience of the user. Depending on the user model, the system delivers a more, or less detailed explanation. The desired information is represented in an answer frame which is passed on to the NL Generator. If the Filter detects that a command was given, the Command Input is passed on to the Plan Recognizer REPLIX. The plan recognizer detects accomplished plans and interactions (inserted sub-plans, overlapping plans, ignore and interrupt commands) between the plans. The output of the plan recognizer is passed on to the Advisor. First, the Advisor tries to find a better plan with which the user can reach the same goal by using fewer commands. Next, it asks the user model to deliver all unknown concepts and commands of this plan. In a third step, the information out of step one and two is represented in an Advice Frame, which is also passed on to the NL Generator. The NL Generator constructs the natural language answer out of the Answer Frame and the Advice Frame. This answer is then presented to the user.
THE SINIX CONSULTANT
Figure 1.
157
158
MATTHIAS HECKING
2.4. The state of implementation To date, a first version of the SC has been implemented. The modules were integrated into the system and successfully tested. The knowledge base describes 90 commands and 300 concepts pertaining to the following sections of SINIX: • information about the system state, • information about directory and file management, • simple process communication. Questions with regard to these commands and concepts can be posed. The plan recognizer works with a library of 40 suboptimal/optimal plan pairs. MORPHIX, the parser, the Question Evaluator, the user model, the plan recognizer REPLIX, and the Advisor have been realized. The global control structure works agenda-based. The lexicon contains about 2000 words. A real natural language generator together with the possibility of conducting dialogues has not yet been implemented. The natural language answers are based on patterns assembled according to the user’s expertise. A case-frame based parser is being developed. The system is implemented on a Siemens APS 5815 (XEROX 1108–105) in INTERLISP-D and LOOPS. The operating system SINIX is simulated through the SINIX Emulator. The next steps in the development of the SC are: • the extension of the system in order to allow cooperative dialogues, • the realization of a natural language generator, • the realization of a plan generator.
3. Plan Recognition in SC 3.1. The plan recognition component REPLIX In this section, the plan recognizer REPLIX3 is described. For a more comprehensive treatment, the reader should refer to Dengler et al. 1987 or Hecking 1987. The goal of this presentation is to illustrate the capabilities of the realized plan recognizer in order to provide justification for the theoretical treatment. Within the theoretical discussion, the capabilities of REPLIX should be modelled as to clarify any problems arising concerning semantics. The plan recognition process is divided into two different phases: 1. the actual plan recognition (which is done by the REPLIX components; see Figure 1), i.e. the detection of the pursued plan and its associated goal and
THE SINIX CONSULTANT
159
2. the phase where the information about the detected plan is used to produce better plans (this is done by the Advisor; see Figure 1). The component REPLIX was designed, implemented, and successfully integrated into the SC in order to be able to infer underlying goals of nonverbal user actions, i.e. the commands he types in. The main demands on the REPLIX component are: • The possibility of adapting the plan recognizer to other operating systems. This is realized though an abstract syntax which differentiates between command words, flags, and objects. In addition, the wildcards and special characters (e.g. the beginning sign of the flags) of the employed operating system can be specified. • The realization of a special plan editor in order to facilitate the construction of the plans. • The realization of collections of plans (plan packages) and interrupt commands for the focussing of the recognition process. • The recognition of not only simply sequences of plans, but also inserted sub-plans and overlapping plans. • The appropriate handling of commands which are not part of any plan (ignore commands). • The realization of a graphical representation of the internal state of the plan recognizer. The plans which should be recognized are formulated through a special syntax (unnecessary details are left out): [(NameOfPlan (GoalString)) (CmdWord1 Flags1 Specification1 ObjectList1 ) (CmdWord2 Flags2 Specification2 ObjectList2 ) ... (CmdWordn Flagsn Specificationn ObjectListn ) (ListOfIgnoreCommands) (ListOfInterruptCommands)] For convenient reference, each plan is given a unique name. Any possible string represents the goal associated to the plan. Through this string, the adaption of the plan recognizer to the surrounding system (in the SC the Advisor) is realized. The command word (e.g. ‘mv’, ‘mkdir’), flags (e.g. ‘-la’, ‘-f’), a specification, used for complex flags, e.g. in order to determine the position of the flags after the command word, and a list of objects (e.g. the name of the directory used by the ‘mv’ command) can be specified with respect to each command word. In the object list, three different types of parameters can be specified:
160
MATTHIAS HECKING
• the name of a fixed object, i.e. the command is successfully recognized only if the actual name is used, • the name of a schematic variable (any name preceded by an ‘;’), i.e. the command can be used with any parameter. This construction is needed to allow for the possibility of different commands to use the same parameter. • the name of a set variable (any name preceded by an ‘!’) in order to specify the possibility of using a list of parameters. The use of REPLIX is explained in the following example. Example 1: The following plan containing four commands pursued the goal delete a directory together with its content: 1. cd ;dir 2. rm * 3. cd . . 4. rmdir ;dir The first command is used to change the current directory into the specified directory. The corresponding name given to this command is stored in the schematic variable ;dir. With the second command, all files in this directory are removed. The third command changes the current directory back to the parent directory (‘. .’ is the fixed parameter for the parent directory of the current directory). The last command deletes the empty directory. In command 1 and 4, the same schematic variable is used. This mechanism allows the plan only to recognized if, in both commands, the same actual parameter is used. The schematic variables are means in order to realize context-sensitivity within plans. If the user types in the following sequence of commands: 1. cd letters 2. rm * 3. cd . . 4. rmdir letters REPLIX recognizes plan no. 1 and delivers the following information to the Advisor: (*4* (DeleteDir (delete a directory and its content)) ((cd letters) (;dir (letters))) ((rm *) NIL) ((cd . .) NIL) ((rmdir letters) (;dir (letters)))) After the fourth command (*4* is an absolute numeration of the user input), the completion of the plan named DeleteDir was recognized and reported to the Advisor. The name of the plan, the goal, the commands, the
THE SINIX CONSULTANT
161
employed parameters (eg. letters), and the mapping between the schematic variable and the actual parameter (eg. (;dir (letters))) are also passed on to the Advisor. REPLIX is able to detect the completion of two plans in sequence. In order to insert another plan, a pursued plan is often interrupted after a command. The first plan is resumed after completing the inserted plan. Now, suppose a second plan exists together with the goal print two files: 1. lpr ;file1 2. lpr ;file2 Suppose, the employed sequence of commands is as follows: 1. cd letters 2. lpr juliane 3. lpr sabine 4. rm * 5. cd . . 6. rmdir letters then, plan no. 1 is interrupted. After the first command, plan no. 2 is inserted (command no. 2 and 3) and using command no. 4, plan no. 1 is resumed. REPLIX detects this insertion. Besides the completion of the two plans, the insertion of PrintFiles in DeleteDir – after the initial command of DeleteDir – is reported as well. The branch counter allows for the possibility of specifying the number of levels of insertion. If the last commands of a plan are identical to the first commands of the following plan, REPLIX is able to recognize such overlapping. Note that REPLIX is able to recognize each combination of insertions and overlappings interspersed with ignore commands. As already mentioned, the plans which should be recognized are grouped together to form a plan package. A maximum of 20 different plans are contained within a plan package. The use of a specific plan package depends on which section of the operating system the user works in, e.g. the user can work in the section of file and directory management, or in the section of information pertaining to the state of the system. 3.2. The advisor At the moment, we only use a part of the information that REPLIX delivers. The plan packages we use specify employed sub-optimal plans. If e.g. plan no. 1 was recognized, the described information is passed on to the Advisor. The Advisor tries to determine a more optimal plan in the plan library which will also reach the goal of delete a directory together with its content. If this attempt is successful, the Advisor determines those commands pertaining
162
MATTHIAS HECKING
to the optimal plan which, through the user model, are likely to be unknown to the user. The Advisor then formulates an answer which describes the unknown commands within the optimal plan as well as the optimal plan itself along with its appropriate inserted actual parameters. Then, the SC becomes active and presents this information to the user. In the end, the recognized plan, the newly learned commands of the optimal plan, and the optimal plan itself are all notified within the user model.
4. Towards a General Deductive Model of Plan Recognition The experiences gained from designing and implementing the plan recognizer REPLIX build the foundation for the development of a theory of plan recognition. Beside various descriptions in natural language we do not have an exact presentation of the semantics in plan recognition. As mentioned in chapter 1, we use symbolic logic as the framework to reach such an exact description. In the next chapter we first try to identify certain goals which should be reached through the formal model of plan recognition. Then, we give a survey of how plan recognition proceeds, which different levels of plan recognition processes and objects must be modelled. 4.1. The goals of plan recognition If we try to realize a theory of plan recognition we are faced with the question of, apart from domain dependant goals, identifying the general goals of plan recognition. The following questions at least must be answered: • Which goals are persued by the user? • Which sequences of actions (plans) are used to reach each goal? • Which parameters are used in each action? • Which temporal relationships exists between different goals? • Can we order the set of goals, so that the most likely goal forms the top in a goal hierarchy? • Which means can be used to resolve goal conflicts? 4.2. How plan recognition proceeds Figure 2 shows all the necessary steps involved in plan/goal recognition for command-driven systems. In the world model, the relevant aspects of our domain are represented. Because in most domains time plays a crucial role, the representation must
THE SINIX CONSULTANT
Figure 2.
163
164
MATTHIAS HECKING
be chronologically oriented. If actions (e.g. commands) are used, the preconditions and the effects of those actions must be described. This is given in the description of actions. If an action occurs, the world model is changed. The commands used are the foundation for plan recognition. Therefore, we receive a first set of inferred goals. Since these goals are grounded solely on the commands used, we only get a rather restricted set of goals. Therefore, we must take into account other goals, e.g. goals stored in the user model, or long-term goals of the user, or if possible, the goals which could be inferred from natural language utterances. If we take these goals into consideration, we attain a second set of goals. Because there are competing goals conflict resolution is necessary. Thus, we reach a third set of goals in which one goal or a small set of goals are assumed to be the primary goal of the user. 4.3. What should be modelled? From different steps in the process of plan recognition and from experiences with REPLIX, we come up with the following objects and processes of plan recognition which must be modelled if we want to realize the formal model: • time dependent properties of the domain, • the time structure, • there are commands, • the command has various parameters and flags, • there are preconditions for using a command, • the command has effects on the world model, • the application of a command, • the recognition of a used command (simple action recognition), • there are goals, • a goal can be reached with different plans, • how commands and a goal are put together to form an atomic plan, • the recognition of the goal of an atomic plan, • how commands, sub-goals, and a goal are put together to form an nonatomic plan, • the recognition of the goal of an non-atomic plan, • which temporal relationships are possible between the application of commands and used sub-goals, the recognition of these commands and sub-goals, and the recognized goal of an atomic or non-atomic plan. 4.4. Interval-based logic of time Since we are mainly interested in modelling inserted sub-plans, a time logic must be used in our deductive model. A very common approach in AI is the time logic of J. Allen (cf. Allen 1984, Allen 1983). In his general theory of
THE SINIX CONSULTANT
165
Figure 3.
action and time, an interval-based time logic is used. Actually, it is not a time logic because there are no specific temporal operators, like e.g. in the theory of A. Prior (cf. Prior 1957). Still, we adhere to this definition. In this interval-based logic, 13 relations exists between two intervals, e.g. interval T1 can be BEFORE interval T2 or T1 can be DURING T2. The relations and their abbreviations are depicted in Figure 3. If we use these relations in our model, we change freely between the abbreviations and the regular names. We use (T1 < = m T2) to express (T1 < T2) ∧ (T1 = T2) ∧ (T1 m T2).
5. The Deductive Model In this chapter, we try to partially formalize the world model of our domain, the actions, the plans, and the plan recognition which is performed by REPLIX (see section 3.1). Especially the problem of inserted sub-plans is modelled. Our main goal is to identify the limits of our deductive model. Starting with the interval-based time logic, we describe how properties in
166
MATTHIAS HECKING
our domain are represented in the world model. Then we describe several commands, atomic plans, and a non-atomic plan. Starting with a given time structure, we show how the use of the commands change the world model and how plans are used for plan recognition. Finally, we mention what can not be modelled and compare our approach to another formal theory of plan recognition. 5.1. The model of the world Properties of the modelled plan recognition domain are time-dependent. They are valid in an interval. The fact that a property p holds in an interval t is expressed through the predicate HOLDS (p,t). For example, with HOLDS (CurrentDir = /usr/matt/, T1) the fact is expressed that the current directory is /usr/matt/ during the interval T1. Properties hold in each subinterval of the specified interval as well. With HOLDS (LoggedIn (Matthias, Yes), T1) the fact is expressed that the user Matthias is logged in during T1. 5.2. Description of actions Commands, parameters, the fact that a command was used, and the recognition of used commands are all represented as terms. 5.2.1. The ‘cd’ command With ∀x, y, z, t1 , ∃t2 HOLDS(CurrentDir = x, t1 ) ∧ HOLDS(W asU sed(cd(y)), t1 ) ∧ HOLDS(P arameter(cd(y), z), t1 ) ∧ MEETS(t1 , t2 ) ⇒ HOLDS(CurrentDir = x o z, t2 ) ∧ HOLDS(Occur(cd(y), t1 ), t2 ) ∧ HOLDS(Occur(P arameter(cd(y), z), t1 ), t2 )
(1)
THE SINIX CONSULTANT
167
the command cd is formalized. In the second line, the precondition “there must be a CurrentDir” is specified. With WasUsed, the fact that a command of type cd(y) was used is described. If a specific cd command was used, the variable y must be instantiated with a unique identifier. In the fourth line, the parameter z which was used by the cd command is specified. If all these formulae are valid in the world model, then the theorems on the right side of the implication can be deduced. With the predicate MEETS, the fact is specified that there are no intervals allowed between t1 and t2 . Given this possibility, then another command execution could destroy the properties which are valid after the execution of the cd command (the CurrentDir might have be changed). In the seventh, line the fact that the cd command has changed the world model is stated. The new current directory is the concatenation of x and z. HOLDS(Occur(cd(y), t1 ), t2 ) specifies that the employment of the cd command (which was used in interval t1 ) was recognized (simple action recognition) in interval t2 . With the last formula the use of the parameter of the cd command is recognized. 5.2.2. The ‘lpr’ command ∀x, y, z, t1 , ∃t2 HOLDS(CurrentDir = x, t1 ) ∧ HOLDS(I sF ileI n(z, x), t1 ) ∧ HOLDS(W asU sed(lpr(y)), t1 ) ∧ HOLDS(P arameter(lpr(y), z), t1 ) ∧ MEETS(t1 , t2 ) ⇒ HOLDS(Occur(lpr(y), t1 ), t2 ) ∧ HOLDS(Occur(P arameter(lpr(y), z), t1 ), t2 )
(2)
With IsFileIn (z, x), it is guaranteed that the lpr command is only used if the specified parameter z (the file which should be printed) defines an existent file in the current directory. 5.2.3. The ‘rm *’ command ∀x, t1 , ∃t2 HOLDS(W asU sed(rm(x)), t1 ) ∧ HOLDS(P arameter(rm(x), ∗), t1 ) ∧ MEETS(t1 , t2 )
(3)
168
MATTHIAS HECKING
⇒ HOLDS(Occur(rm(x), t1 ), t2 ) ∧ HOLDS(Occur(P arameter(rm(x), ∗), t1 ), t2 ) ∧ “All f iles in the current directory are deleted ” With All files in the current directory are deleted, the effects of the rm * command on the world model are described. We omit the exact formalization because we are mainly interested in the process of plan recognition. 5.2.4. The ‘cd . .’ command ∀x, y, z, t1 , ∃t2 HOLDS(CurrentDir = x|y, t1 ) ∧ HOLDS(W asU sed(cd(z)), t1 ) ∧ HOLDS(P arameter(cd(z), . . .), t1 ) ∧ MEETS(t1 , t2 ) ⇒ HOLDS(CurrentDir = x, t2 ) ∧ HOLDS(Occur(cd(z), t1 ), t2 ) ∧ HOLDS(Occur(P arameter(cd(z), . . .), t1 ), t2 )
(4)
With x|y, the name of the CurrentDir is divided into two parts. x is the name of the parent directory and y is the name of one of its subdirectories. 5.2.5. The ‘rmdir’ command ∀x, y, z, t1 , ∃t2 HOLDS(CurrentDir = x, t1 ) ∧ HOLDS(I sDirI n(y, x), t1 ) ∧ HOLDS(W asU sed(rmdir(z)), t1 ) ∧ HOLDS(P arameter(rmdir(z), y), t1 ) ∧ MEETS(t1 , t2 ) ⇒ HOLDS(Occur(rmdir(z), t1 ), t2 ) ∧ HOLDS(Occur(P arameter(rmdir(z), y), t1 ), t2 ) ∧ “T he directory with name y was deleted 00
(5)
With IsDirIn(y, x), it is guaranteed that the rmdir command is only used if the specified parameter of the command y is a directory in the current
THE SINIX CONSULTANT
169
directory. With The directory with name y was deleted, the effect of the rmdir command on the world model is expressed. 5.3. Description of atomic plans A plan is composed of one or more commands or sub-goals which form the body of the plan and the goal, which is persued by the commands and the sub-goals. The body of an atomic plan consists only of commands. In the body of a non-atomic plan, sub-goals and commands are mixed. In the following, we formalise those atomic plans needed to model the insertion of plans. 5.3.1. The ‘cd’ plan The following example represents an atomic plan: ∀x, t1 , t2 , t3 HOLDS(Occur(cd(x), t1 ), t2 ) ∧ HOLDS(Occur(P arameter(cd(x), ..), t1 ), t2 ) ∧ (MEETS(t2 , t3 ) ∨ BEF ORE(t2 , t3 )) ⇒ HOLDS(Goal(ChangeDir([cd(x)]), t1 ), t3 )
(6)
The second line states the fact that the cd command occured in time interval t1 . This was recognized within the interval t2 (simple action recognition). In the third line, it is specified that the parameter of the cd command must not be . . (the parent directory). In the fourth line, the relationships between the intervals is specified in which the used command was recognized and in which the plan recognition with this atomic plan should be performed: • MEETS(t2 , t3 ) states that the recognition of the goal of the atomic plan can immediately follow the recognition of the use of the command. • BEFORE(t2, t3 ) specifies that the recognition of the goal must not immediately follow the recognition of the employment of the command. There can be intervals between t2 and t3 . So, it is possible to separate goal recognition from that of its employment or from the use of the commands. In the first line following the right arrow, the goal of this atomic plan is specified. HOLDS(Goal(ChangeDir([cd(x)]), t1 ), t3 ) states that the goal ChangeDir was persued in t1 with the command list [cd(x)] and the goal of this atomic plan was recognized in t3 .
170
MATTHIAS HECKING
5.3.2. The ‘lpr’ plan In the following atomic plan two commands are used: ∀x, y, z, u, t1 , t2 , t3 , t4 , t5 HOLDS(Occur(lpr(x), t1 ), t2 ) ∧ HOLDS(Occur(P arameter(lpr(x), y), t1 ), t2 ) ∧ HOLDS(Occur(lpr(z), t3 ), t4 ) ∧ HOLDS(Occur(P arameter(lpr(z), u), t3 ), t4 ) ∧ (t1 < m t3 ) ∧ (t2 < t6 ) ∧ (t4 < m t6 ) ∧ STARTS(t1 , t5 ) ∧ FINISHES(t3 , t5 ) ⇒ HOLDS(Goal(P rintT woF iles([lpr(x), lpr(z)]), t5 ), t6 )
(7)
We assume that if commands occured and if they form a plan, then the interval in which the goal of the plan is persued must contain the intervals in which the commands occured, so, t1 and t3 must be IN t5 . Also, because the commands form the plan, the interval of the first command STARTS the interval of the goal and the interval of the last command FINISHES the interval of the goal. The use of the second command can immediately follows the use of the first command, but intervals between (t1 < m t3 ) are possible. These intervals represent the basis for modelling inserted sub-plans. 5.3.3. The ‘rm *’ plan ∀x, t1 , t2 , t3 HOLDS(Occur(rm(x), t1 ), t2 ) ∧ HOLDS(Occur(P arameter(rm(x), ∗), t1 ), t2 ) ∧ (MEETS(t2 , t3 ∨ BEFORE(t2 , t3 )) ⇒ HOLDS(Goal(DeleteAllF iles([rm(x)]), t1 ), t3 )
(8)
5.3.4. The ‘cd . . . ’ plan ∀x, t1 , t2 , t3 HOLDS(Occur(cd(x), t1 ), t2 ) ∧ HOLDS(Occur(P arameter(cd(x), ..), t1 ), t2 ) ∧
(9)
THE SINIX CONSULTANT
171
(MEETS(t2 , t3 ) ∨ BEFORE(t2 , t3 )) ⇒ HOLDS(Goal(ChangeT oP arentDir([cd(x)]), t1 ), t3 ) 5.3.5. The ‘rmdir’ plan ∀x, t1 , t2 , t3 HOLDS(Occur(rmdir(x), t1 ), t2 ) ∧ HOLDS(Occur(F lag(rmdir(x), −r), t1 ), t2 ) ∧ (MEETS(t2 , t3 ) ∨ BEFORE(t2 , t3 )) ⇒ HOLDS(Goal(DeleteADir([rmdir(x)]), t1 ), t3 )
(10)
5.4. Description of non-atomic plans In a non-atomic plan, the body can contain not only commands but sub-goals as well. For example, in the following formula, the plan to reach the goal delete a directory together with its content is formalized: ∀x, y, z, u, w, t1 , t2 , t3 , t4 , t5 , t6 , t7 , t8 (11) HOLDS(Goal(ChangeDir(x), t1 ), t2 ) ∧ HOLDS(Goal(DeleteAllF iles(y), t3 ), t4 ) ∧ HOLDS(Goal(ChangeT oP arentDir(z), t5 ), t6 ) ∧ HOLDS(Goal(DeleteADir(u), t7 ), t8 )) ∧ HOLDS(Occur(P arameter(GetCommandT oken(x), w), t1 ), t2 ) ∧ HOLDS(Occur(P arameter(GetCommandT oken(u), w), t7 ), t8 ) ∧ (t1 < m t3 ) ∧ (t3 < m t5 ) ∧ (t5 < m t7 ) ∧ (t2 < m t10 ) ∧ (t4 < m t10 ) ∧ (t6 < m t10 ) ∧ (t8 < m t10 ) ∧ IN(t1 , t9 ) ∧ IN(t3 , t9 ) ∧ IN(t5 , t9 ) ∧ IN(t7 , t9 ) ∧ STARTS(t1 , t9 ) ∧ FINISHES(t7 , t9 ) ⇒ HOLDS(Goal(DeleteADirAndAllF iles([x, y, z, u]), t9 ), t10 ) The body consists of the four sub-goals ChangeDir, DeleteAllFiles, ChangeToParentDir, and DeleteADir. In line six and seven, the parameters
172
MATTHIAS HECKING
used in the first and fourth plan in reaching the appropriate sub-goal must be the same. In line eight, the temporal relationships between the sub-goals are given. in the last two lines before the implication sign, the relationships between the intervals in which the sub-goals are persued and the interval in which the overall goal is persued are described. Note that the interval of the goal does not begin before the first sub-goal interval and does not end after the last sub-goal interval. 5.5. Axiom for the detection of inserted plans In order to detect the inserted sub-plans, we must have several goals. If one of these goals are persued in a time interval which is a sub-interval of a second goal, then we say the plan used to reach the first goal is inserted into the plan employed to reach the second goal. We formalize this in: ∀x, t1 , t2 , t3 , t4 , t5 HOLDS(Goal(x, t1 ), t2 ) ∧ HOLDS(Goal(y, t3 ), t4 ) ∧ IN(t1 , t3 ) ∧ (t2 < m t5 ∧ (t4 < m t5 ) ⇒ HOLDS(I nsertedSubP lan(x, t1 , y, t3 ), t5
(12)
We must extend the above formula to exclude those deductions which simply state that x is inserted in y in the case that x is a sub-goal of y. We omit this extension here. 5.6. How to handle insertion in the deductive model We have formalized different commands, several atomic plans, and a nonatomic plan. Now, we show how the world model changes if the commands are executed and how plan recognition proceeds. The example used here is the same as in section 3.1. In this example, the time structure used is given by the following predicates: (T1 m T2) ∧ (T3 m T4) ∧ (T5 m T6) ∧ (T7 m T8) ∧ (T9 m T10) ∧ (T11 m T12) ∧ (T1 <m T3) ∧ (T3 <m T5) ∧ (T5 <m T7) ∧ (T7 <m T9) ∧ (T9 <m Tll) ∧ (T2 <m = T13) ∧ (T4 <m T15) ∧ (T6 <m T15) ∧ IN(T3, T14) ∧ IN(T5, T14) ∧
One representation of the time structure is given in Figure 4. Note that other representations are possible. First command: After the employment of the command cd letters, the subsequent formulae are valid in the deductive model:
174
MATTHIAS HECKING
HOLDS(WasUsed(cd(K1)), T1) ∧ HOLDS(Parameter(cd(K1), letters), T1) ∧ HOLDS(CurrentDir = /usr/matt/, T1) In the first two lines, the command used is specified. The constant K1 names this specific use of the cd command. T1 is the name given to the time interval in which the command occured. Because there is an intervall T2 with MEETS(T1, T2) in the time structure, the description of the cd command (s. formula 1) can be used. The following theorems are deduced: HOLDS(Occur(cd(K1), T1), T2) ∧ HOLDS(Occur(Parameter(cd(K1), letters), T1), T2) ∧ HOLDS(CurrentDir=/usr/matt/letters/, T2) Second command: After the employment of the command lpr juliane, the subsequent formulae are valid in the deductive model: HOLDS(WasUsed(lpr(K2)), T3) ∧ HOLDS(Parameter(lpr(K2), juliane), T3) ∧ HOLDS(CurrentDir = /usr/matt/letters/, T3) ∧ HOLDS(IsFileIn(juliane, /usr/matt/letters/), T3) Here we are faced with the frame problem. We must specify through frame axioms what happens with those formulae valid in an interval with respect to subsequent intervals. A frame axiom must guarantee that in T3, the CurrentDir must be the same as in T2. We assume that appropriate frame axioms are available. For a discussion of the frame problem cf. Brown 1987. With the theorem MEETS(T3, T4) and the command description in formula 2, we can deduce the subsequent formulae: HOLDS(Occur(lpr(K2), T3), T4) ∧ HOLDS(Occur(Parameter(lpr(K2), juliane), T3), T4) Third command: After the employment of the command lpr sabine, the subsequent formulae are valid in the deductive model: HOLDS(WasUsed(lpr(K3)), T5) ∧ HOLDS(Parameter(lpr(K3), sabine), T5) ∧ HOLDS(CurrentDir = /usr/matt/letters/, T5) ∧ HOLDS(IsFileIn(juliane, /usr/matt/letters/), T5)
THE SINIX CONSULTANT
175
With the theorem MEETS(T5, T6) and the command description in formula 2, we can deduce the subsequent formulae: HOLDS(Occur(lpr(K3), T5), T6) ∧ HOLDS(Occur(Parameter(lpr(K3), sabine), T5), T6) Forth command: After the employment of the command rm *, the subsequent formulae are valid in the deductive model: HOLDS(WasUsed(rm(K4)), T7) ∧ HOLDS(Parameter(rm(K4), *), T7) With the theorem MEETS(T7, T8) and the command description in formula 3, we can deduce the subsequent formulae: HOLDS(Occur(rm(K4), T7), T8) ∧ HOLDS(Occur(Parameter(rm(K4), *), T7), T8) ∧ “All files in the current directory are deleted” Fifth command: After the employment of the command cd . . , the subsequent formulae are valid in the deductive model: HOLDS(WasUsed(cd(K5)), T9) ∧ HOLDS(Parameter(cd(K5), . . . ), T9) ∧ HOLDS(CurrentDir = /usr/matt/letters/, T9) With the theorem MEETS(T9, T10) and the command description in formula 4, we can deduce the subsequent formulae: HOLDS(Occur(cd(K5), T9), T10) ∧ HOLDS(Occur(Parameter(cd(K5), . . . ), T9), T10) ∧ HOLDS(CurrentDir = /usr/matt/, T10) Sixth command: After the employment of the command rmdir letters, the subsequent formulae are valid in the deductive model: HOLDS(WasUsed(rmdir(K6)), T11) ∧ HOLDS(Parameter(rmdir(K6), letters), T11) ∧ HOLDS(IsDirIn(letters, /usr/matt/), T11) ∧ HOLDS(CurrentDir = /usr/matt/), T11)
176
MATTHIAS HECKING
With the theorem MEETS(T11, T12) and the command description in formula 5, we can deduce the subsequent formulae: HOLDS(Occur(rmdir(K6), T11), T12) ∧ HOLDS(Occur(Parameter(rmdir(K6), letters), T11), T12) ∧ “The directory with name y was deleted” Use of atomic plans: The employment of the commands are recognized. Now we use the atomic plans to see which goals are being persued. Since the preconditions of plan no. 6 are satisfied (the recognition of the used command was in interval T1 which is before T13), the following theorem is infered: HOLDS(Goal(ChangeDir([cd(K1)]), T1), T13) With plan no. 7 we get: HOLDS(Goal(PrintTwoFiles([lpr(K2), lpr(K3)]), T14), T15) With plan no. 8 we get: HOLDS(Goal(DeleteAllFiles([rm(K4)]), T7), T16) With plan no. 9 we get: HOLDS(Goal(ChangeToParentDir([cd(K5)]), T9), T17) With plan no. 10 we get: HOLDS(Goal(DeleteADir([rmdir(K6)]), T11), T18) Use of the non-atomic plan: Because the preconditions for the non-atomic plan (s. formula 11) are satisfied, we can deduce: HOLDS(Goal(DeletADirAndAllFiles([[cd(K1)], [rmdir(K6)]]), T20), T19)
[rm(K4)],
[cd(K5)],
Deduce the insertion: with formula no. 12, we now can deduce that an insertion exists. The antecedences are: HOLDS(Goal(PrintTwoFiles([lpr(K2), lpr(K3)]), T14), T15) ∧ HOLDS(Goal(DeletADirAndAllFiles([[cd(K1)], [rm(K4)], [cd(K5)], [rmdir(K6)]]), T20), T19)
THE SINIX CONSULTANT
177
Because T24 is IN T20, T15 is BEFORE T21, and T19 is BEFORE T21, we receive: HOLDS(InsertedSubPlans(PrintTwoFiles([lpr(K2), lpr(K3)]), T14, DeletADirAndAllFiles([[cd(K1)], [rm(K4)], [cd(K5)], [rmdir(K6)]]), T20), T21)
5.7. Limits of this approach We have seen that the inserted sub-plans can be modelled quite simply, if we use an interval-based time logic. One deficiency, which is actually a deficiency of the used interval-based time logic, is that it is impossible to specify the maximum amount of time between the first command before and after an inserted sub-plan. For example, if you don’t have the possibility to specify this maximum of interruption time, it might be possible that commands interrupted for several hours are treated as belonging to the same plan. This deficiency is actually due to the lack of not being able to specify the duration time of an interval. This concept is unknown in the logic of time used by Allen. The most elaborated formal theory of plan recognition was developed by H. Kautz (cf. Kautz 1987, Kautz and Allen 1986). Because his time logic is also taken from Allen, his theory is also unable to handle the insertion appropriately. But, with respect to other related topics of plan recognition, the theory of H. Kautz is far more elaborated.
6. Possible Extensions of the Model Because the model presented is only a first step towards an overall theory of plan recognition, we briefly mention those topics which might be handled in such a theory: 1. The time logic used should be altered so that we can model the duration of time intervals. Another interesting question is the necessity of a timepoint based logic of time. 2. As already pointed out, the frame problem occurs. 3. Non-monotonic plan recognition should be handled. 4. The question arises if plan recognition should be modelled with respect to different agents4 . 5. How can incremental plan recognition be handled?
178
MATTHIAS HECKING
7. Summary We have briefly described the realized active help system SINIX Consultant. A more detailed description of the plan recognition component REPLIX was given. The experience gained in realizing the plan recognition component was taken as a first-step basis towards a formal theory of plan recognition. We have shown how we can use an interval-based time logic in order to formulate the theory. The insertion of sub-plans was modelled, but at once, we are faced with a problem which cannot be appropriately handled in this initial approach.
Notes 1 The SINIX Consultant project is partially funded by the Siemens AG as part of the III
(Innovative Information infrastructures) project, a cooperation between the University of Saarbrücken and the Siemens AG. SINIX is the UNIX derivate of the Siemens AG 2 move file ‘letter1’ to the directory ‘invoices’ 3 REPLIX was designed and implemented by D. Dengler, M. Gutmann, G. Hector and M. Hecking. 4 For a treatment of this topic see Hecking 1988.
References Allen, J. (1983). Maintaining knowledge about temporal intervals. Communications of the ACM 26: 832–843. Allen, J. (1984). Towards a general theory of action and time. Artificial Intelligence 23: 123–154. Boggs, W. M., Carbonell, J. G. & Monarch, J. G. (1984). DYPAR – I: Tutorial and Reference Manual. Technical Report, Dept. of Computer Science, Carnegie-Mellon University, Pittsburgh (PA). Brown, F. M. (1987). The Frame Problem in Artificial Intelligence, Morgan Kaufmann Publishers, Los Altos, California. Dengler, D., Gutmann, M. & Hector, G. (1987). Der Planerkenner REPLIX. Memo No. 16, Dept. of Computer Science, University of Saarbrücken, W. Germany. Finin, T. W. (1983). Providing help and advice in task oriented systems. In Proceedings of the 8th International Joint Conference on Artificial Intelligence, 176–178. Finkler, W. & Neumann, G. (1986). MORPHIX – Ein hochportables Lemmatisierungsmodul für das Deutsche. Memo No. 8, Dept. of Computer Science, University of Saarbrücken, W. Germany. Genesereth, M. R. & Nilsson, N. J. (1987). Logical Foundations of Artificial Intelligence. Morgan Kaufmann Publishers, Los Altos, California.
THE SINIX CONSULTANT
179
Hecking, M. (1987). How to Use Plan Recognition to Improve the Abilities of the Intelligent Help System SINIX Consultant. Proceedings of the Second IFIP Conference on HumanComputer Interaction, held at the University of Stuttgart, Federal Republic of Germany, 1–4 September, 1987, 657–662. Hecking, M. (1988). Towards a Belief-Oriented Theory of Plan Recognition. Report No. 50, Dept. of Computer Science, University of Saarbrücken, W. Germany. Hecking, M. & Harbusch, K. (1987). Plan Recognition through Attribute Grammars. Memo No. 17, Dept. of Computer Science, University of Saarbrücken, W. Germany. Hecking, M., Kemke, C., Nessen, E., Dengler, D., Gutmann, M. & Hector, G. (1988). The SINIX Consultant – A Progress Report. Memo No. 28, Dept. of Computer Science, University of Saarbrücken, W. Germany. Kautz, H. A. & Allen, J. F. (1986). Generalized plan recognition. Proceedings of the 5th National Conference of the American Association on Artificial Intelligence, Philadelphia, Pennsylvania, 32–37. Kautz, H. A. (1987). A Formal Theory of Plan Recognition. Report No. TR 215, University of Rochester, Department of Computer Science, 5. Kemke, C. (1987). Representation of Domain Knowledge in an Intelligent Help System. Proceedings of the Second IFIP Conference on Human-Computer Interaction, held at the University of Stuttgart, Federal Republic of Germany, 1–4 September, 1987, 215–220. Konolige, K. (1986). A Deduction Model of Belief. Research Notes in Artificial Intelligence, Morgan Kaufmann Publishers, Los Altos, California. Prior, A. (1957). Time and Modality. Clarendon Press, Oxford. Quilici, A., Dyer, M. & Flowers, M. (1986). AQUA: AN INTELLIGENT UNIX ADVISOR. Proceedings of the 7th European Conference on Artificial Intelligence, 33–38. Th. Schwab. (1984). AKTIVIST – Ein aktives Hilfesystem für den bildschirmorientierten Editor BISY. Diplomarbeit Nr. 232, Institut für Informatik, Universität Stuttgart. Wilensky, R., Arens, Y. & Chin, D. (1984). Talking to UNIX in English: An Overview of UC. Communications of the ACM 27: 574–593, 6. Wilensky, R., Myfield, J., Albert, A., Chin, D., Cox, C., Luria, M., Martin, J. & Wu, D. (1986). UC – A Progress Report. Report No. UCB/CSD 87/303, Computer Science Division (EECS), University of California, Berkeley, 7.
Plan Realization for Complex Command Interaction in the UNIX Help Domain STEPHEN J. HEGNER Umeå University, Department of Computing Science, S-901 87 Umeå, Sweden (E-mail: [email protected]) Abstract. Yucca-* is a consultation system which is designed to provide the UNIX user, through a friendly interface, with detailed expert advice on the use of the UNIX command language. One of the principal design goals of this system is the ability to provide correct responses to technically complex queries whose solution may involve the interconnection of several commands, each with multiple options. The realization of such a goal requires two things. First, representation of dynamic knowledge about command behavior at a sufficient level of detail to support solution of the query is needed. Second, a planning mechanism capable of interconnecting such knowledge into a cohesive solution must be provided. This paper first develops the command dynamics representation techniques employed in Yucca-*. It then examines in detail the plan generation mechanism which is used to solve complex dynamic queries. Particular emphasis is placed upon those aspects of the problem which are unique to this particular domain. Keywords: consultation system, operating system, planning
1. Introduction 1.1. Overall goals Like the other systems described in this collection, Yucca-* is an intelligent help system for the UNIX1 operating system. However, not all operating system consultants have identical goals; there are many points of emphasis upon which effort may be focused. Therefore, it appropriate to begin our description of Yucca-* with a statement of just what it is intended to do, before paying any attention to how. The user community The targeted users of Yucca-* are those persons who possess some familiarity with computer systems, but who lack the detailed knowledge necessary to accomplish a desired task. There are at least two common (overlapping) categories of such users. 1. The first category consists of users who are migrating from another system. That system may be another shared system, such as TOPS-202
182
STEPHEN J. HEGNER
or VM/CMS,3 or it may be an operating system for dedicated microcomputers, such as MS-DOS.4 Such users will be aware, for example, that most operating systems have a command to print the contents of a file, but may be unaware that lpr is the specific command for accomplishing that task in UNIX, or that a combination of the lpr command and the cat command is necessary to print a file with lines numbered. 2. The second category contains occasional but not naïve users. These persons may use UNIX from time to time in the course of their work, but are not intense users, and thus find it easy to forget the details and intricacies of command syntax and semantics. Key characteristic of this user community 1. Type of information sought. Perhaps the most succinct characterization of the members of this user community is that they possess the conceptual knowledge necessary to identify the information which they want, but lack the implementation knowledge of those concepts within the extant domain. They do not need to be told that information is stored in files, or that text editors are used to change the contents of files, but they may not understand the subtle but important differences between UNIX files and TOPS-20 files, nor may they know which editors are available under UNIX, and what their relative features are. 2. Complexity of knowledge. Because the targeted class of users is more sophisticated than true computing novices, so too will be their queries. Rather than simply wanting to know how to print a file, such a user may want to know how to print a file on the laser printer with pagination and with page headers. Form of the utility Yucca-* is designed to be an external utility, rather than an integral system component such as are Cousin (Haye 1982a), (Haye 1982b), (Hayes and Szekely 1983), SARA (Fenschel and Estrin 1982), and CONSUL (Mark et al. 1980). This means that it must run as a separate program on the existing system, to be invoked only when needed. Its installation will not alter the standard user interface in any way. Historical influence on design goals Yucca-* is the outgrowth of three earlier efforts, UCC (Douglass and Hegner 1982), Yucca (Hegner and Douglass 1984), each developed in Franz Lisp, and Yucca-II (Hegner 1988), a proposal which never reached fruition. Each of these earlier systems featured a natural-language interface to the user. It was our feeling in designing those systems that the user interface must mimic the human consultant insofar as possible, particularly requiring little or no
PLAN REALIZATION FOR COMPLEX COMMAND INTERACTION
183
learning on the part of the user, and that natural language was the best way to realize this end. While this may still represent the ideal, experience has lead us to the more pragmatic conclusion that a pure natural-language interface may not be optimal. The number of queries which could not be understood, or which were understood incorrectly, was too high to be acceptable in a working system. Due to the highly structured nature of knowledge about operating system behavior, it seems that, at least in some cases, alternate approaches to user interfaces, such as intelligent menus, may form a viable and even superior alternative, not only in functionality, but in terms of reduced design and implementation effort as well. It is clear that further work on the form of the user interface needs to be done. Regardless of the form of the user interface, once the general class of information which the system must provide has been identified, the knowledge about UNIX which is necessary may be firmly identified. Yucca-* is in fact a design of a knowledge base about UNIX, to which a user interface must be attached. In this paper, we focus on design of that knowledge base, emphasizing in particular planning issues. 1.2. System overview Unlike most domains of knowledge to which artificial intelligence is applied, the behavior of an operating system command language is formally specified and completely understood. In the terminology of Hart (Hart 1982), the source code of the UNIX operating system, together with an understanding of its semantics, forms a deep model of that system. Our main thesis, and the cornerstone of the design of Yucca-*, is that this formality and depth of available domain knowledge should be retained as a distinguishing feature of the knowledge representation scheme within an operating system consultation system, rather than discarded in favor of more general but less formal techniques. Nonetheless, while observing that knowledge about the behavior of UNIX lends itself well to formality, we readily acknowledge that understanding a user’s query is not a completely formal process; the inherent imprecision of the man-machine interface must be addressed in any design. To achieve this end, Yucca-* is designed to be consistent with an underlying philosophy that it is possible and indeed desirable to separate understanding of a user’s query from solving it. As a manifestation of this philosophy, the overall architecture of Yucca-* is sharply divided into two components, identified in Figure 1. The understanding phase The function of the understander is to interface to the user and generate a precise formulation of his questions into formal queries in a specially
184
STEPHEN J. HEGNER
-
User’s Query User-oriented Response
- Formal Query
Understander
-
Formal Response
Formal Knowledge Base and Query Solver
Figure 1. Overall architecture of Yucca-*.
designed formal query language termed OSquel, and to translate solutions of such queries into user-understandable responses. As noted previously, the exact nature of this module is not specified in the top-level design of Yucca-*, but is open to a variety of alternatives, including natural language. However, in any realization, this part of the system contains knowledge relevant to the understanding of queries, but not for solution. For example, posed a query asking how to print a file with pageheaders on the laser printer, the understander knows about printing, laser printers, and pageheaders. However, it does not know that the UNIX command pr is the one to use to put pageheaders on a file, that the lpr command, with the appropriate option, is the means to enqueue a file for printing on the laser printer, nor that the two should be interconnected with a UNIX pipe to accomplish the task. In short, it knows how to communicate with the user regarding conceptualizations of operating system concepts and express them as formal conceptualizations in OSquel, but it knows nothing of their implementation. The solving phase The function of the formal knowledge base and query solver is to find solutions to the formal queries posed by the understander. It does know about the semantics of pr and lpr, as well as how they can be interconnected via a pipe. On the other hand, it has no special knowledge about communicating with the user, such as user misconceptions and beliefs, or defaults implied by natural language. In short, it knows nothing of formulating conceptualization of command language, but everything about translating formal conceptualizations into implementations. Essentially, this module is a formal database of knowledge about the UNIX command language; everything about knowledge representation and inference in this component is completely formal. Thus, it is in this component that we exploit completely the formal aspects of domain knowledge about UNIX.
PLAN REALIZATION FOR COMPLEX COMMAND INTERACTION
185
Coupling of the two components Despite this separation of understanding and solving into two separate components, it is clear that the two modules must share enough information so that each is capable of understanding and expressing knowledge in the formal query language OSquel. To this end, the formal static knowledge base is accessible by both components. This knowledge base contains information about objects and their attributes in the extant domain, but not information about how commands alter their attributes. This information is sufficient to formulate and understand queries, but not to solve them. Major thesis It is a major thesis of this work that a division of the consultation system into two components, as described above, is not only possible but practical and advantageous. Specifically, we propose that detailed procedural domain knowledge can be effectively separated from that knowledge necessary to interface with and understand the user. In particular, we propose that a single “back-end” knowledge base about UNIX can be be used to support a variety of user interfaces of varying form and sophistication. We note that this philosophy is completely opposite to that employed in the UC natural-language consultation system (Wilensky et al. 1988), in which linguistic knowledge and domain knowledge about UNIX are contained in a common knowledge base. Comparison to friendly interfaces to database systems It is difficult to avoid noticing the similarity between the architecture of Yucca-* and that of natural-language front ends to database systems, such as PLANES (Waltz 1978), TEAM (Grosz 1983), and IRUS (Bates et al. 1986), and other “user-friendly” interfaces, such as QBE (Zloof 1977), and BAROQUE (Motro 1986). However, these systems all deal with the problem of attaching a front end to an existing database system, while Yucca-* involves the design and implementation of both components of the system from scratch. In addition, traditional database systems deal only with static knowledge, and so queries involving dynamics need not be supported. However, it is the conceptualization of actions which is the central component of the understander. We do note, however, that Salveter (Salveter 1984) has examined the problem of developing a natural-language front end to database systems which deal with updates as well as queries, and so must support concepts closely related to dynamic queries.
186
STEPHEN J. HEGNER
1.3. Overview of plan realization issues Forms of assistance It is important to distinguish our use of plans and planning from that of other help facilities. There are at least two distinct ways in which assistance may rendered by an operating system help facility. Yucca-* is a consultation system, meaning that it provides immediate real-time help to users regarding specific queries. Genereseth’s MACSYMA Advisor (Genesereth 1979) is perhaps the prototypical example of such a utility. In this regard Yucca-* is similar to the UC UNIX consultation system (Wilensky et al. 1984), (Wilensky et al. 1988). Because such facilities typically provide help only when it is explicitly requested, they are sometimes termed passive assistants. Plan realization is the major planning-related activity of such systems, since the system takes a specification of a desired result and attempts to assemble a plan (qua command sequence) which realizes that result. This is in marked contrast to critiquing systems, which take a user’s attempt at a problem and attempt to suggest corrections and/or improvements. This approach has been strongly promoted in the domain of providing expert medical advice (Miller 1984). However it is also central to several operating system help systems, including Wizard (Finin 1983) and AQUA (Quilici et al. 1986). In such systems, a user’s attempt at solving a simple problem (such as deleting a group of files) is observed, and a “better” approach is suggested by the system. Plan recognition and correction is the key plan-related activity in this sort of system. Since such consultants often operate as observers of the user’s actual attempts at problem solution, they are often termed active assistants. The current design of Yucca-* does not provide such help. We note that some operating system help facilities, such as SC (Hecking et al. 1988) and USCSH (Neelkandan et al. 1987), incorporate both active and passive assistants. Types of queries There are two major flavors of queries which users will pose, and which a consultation system should be prepared to service. 1. Process queries. The terminology process query is borrowed from (Scragg 1975), and refers to queries which deal with the dynamics of UNIX. The most important class of such queries is characterized by the required command being unknown, such as asking how to print a file with pageheaders on the laser printer, although their are other possibilities, such as asking who can read a certain class of files, in which the actor is unknown. 2. Conceptual queries. Occasionally, users will need clarification of the static structure of the underlying system, as characterized by the queries
PLAN REALIZATION FOR COMPLEX COMMAND INTERACTION
187
asking for the definition of a UNIX pipe, or for information on how directories are organized in UNIX. Planning and unknown action queries Although Yucca-* has the capability of answering a wide variety of queries, its strength lies in its ability to answer detailed process queries in which the action is unknown. Consider, as a specific example, the query asking how to print a file with pageheaders and lines numbered on the laser printer. Even once this query is understood, its solution is quite nontrivial. Indeed, the natural solution, which is represented by the command cat -n filename | pr | lpr -Plaser, requires the interconnection of three basic commands, two of which contain explicitly identified options. While it would perhaps be possible to retain explicitly the solution for many such queries, such a knowledge base would at best be inflexible and nonrobust. Rather, Yucca-* is designed to assemble such an interconnection from primitive components, utilizing knowledge bases of basic command and interconnector descriptions. The assembly of such an interconnection is clearly a planning process. The query statement represents the goal, and the interconnection the plan. The overall approach of how such planning is performed, as well as how it is supported with appropriate knowledge bases, is the central topic of this article. Scope of the plan domain Even within the scope of passive consultation systems, there are different directions which may be taken. The emphasis in Yucca-* is on the solution of relatively complex domain-specific queries which involve the interconnection of several commands and options. Thus, our primary concern is how to view commands, command options, and command interconnectors as basic building blocks for complex command behavior, and how to realize an interconnection of such blocks affording a particular behavior. This involves domain specific planning. In the UC system, on the other hand, a major goal is to utilize the UNIX domain as a testbed for the evaluation of general frameworks for knowledge representation and planning. As such, UC emphasizes general planning to a much greater degree. As a result, Yucca-* is capable of answering much more technically complex queries. See (Chin 1988) for a detailed discussion of the rôle of planning in UC. The SC system (Hecking et al. 1988), like Yucca-*, employs domain-specific knowledge representation, but its emphasis is not on plan generation, but rather upon direct representation of command behavior. Yucca-* appears to be unique amongst UNIX consultation systems in its emphasis upon the solution of complex technical queries via plan generation.
188
STEPHEN J. HEGNER
Conceptual
* I/O
-
Conceptual Query Solver
- Descriptions Knowledge Base
HH Y HH HH HH HH j Static Knowledge Base
Query Differentiator
HH Y HH HH HH HH j
* Dynamics Process - Knowledge Query Solver Base
* Clich´e Knowledge Base
HH Y HH j
*
Item Template Knowledge Base
Figure 2. Overall architecture of back end.
2. Overview of the back end In this section, we overview the architecture of the back end, and identify the places in this paper in which the individual components are elaborated. The overall architecture of the back end is depicted in Figure 2. Input queries are classified as either conceptual or process; the difference is easily determined by syntactic means. The sole rôle of the query differentiator is to decide of which type a query is, and to route it to the appropriate solver. This is a trivial task which is not further elaborated in this paper. Each query solver has access to particular knowledge bases tailored to its needs. 2.1. Overview of query solution modules The conceptual query solver The task of the conceptual query solver is to solve conceptual queries. It is not discussed further in this paper.
PLAN REALIZATION FOR COMPLEX COMMAND INTERACTION
189
The process query solver The process query solver actually consists of two distinct units. The command synthesizer is a complex goal identification and planning unit which is used to solve queries in which the action to be performed is the primary unknown. The ultimate goal of this paper is to describe the operation of this query solution unit, as it is “the” planner of the back end. This unit is described in Section 6. The simulator is used to solve all other dynamics queries; for reasons of space limitations, it is not discussed further in this paper. 2.2. Overview of the knowledge bases Although this paper is ultimately about plan realization, such a topic cannot be considered in isolation. Rather, successful planning must be supported by appropriate knowledge representation and inference mechanisms. Therefore, we also provide a description of those aspects of knowledge representation which are relevant to plan generation. The static knowledge base The static knowledge base is central to the back end; all other knowledge bases depend critically upon concepts formulated in this unit. It consists of formal descriptions of the objects classes present in an operating system enivronment, such as file, directory, user, file protection, byte sequence, print queue, and the like. These object classes are organized in both generalization (IS-A) and aggregation (PART-OF) hierarchies. The static knowledge base contains definitions of well over one-hundred classes, and is described in Section 4. The conceptual descriptions knowledge base In Yucca-*, conceptual queries asking for a general description of the concept of pipe or of directory structure of UNIX are solved by providing the user with a short textual response explaining the concept. The conceptual descriptions knowledge base contains the text of those responses. Since this knowledge base is not used in the support of plan realization, it is not discussed further in this paper. The dynamics knowledge base This knowledge base contains formal descriptions of the syntax and semantics of several dozen UNIX commands, as well as definitions of supporting operations such as piping, command sequencing, and redirection. It is discussed in detail in Section 5. The complete representation of the pr command is contained in the Appendix of this paper.
190
STEPHEN J. HEGNER
Figure 3. Syntax of a process query.
The cliché and item template knowledge bases These knowledge bases do not contain any “new” information about the structure or behavior of UNIX. Rather, they play a crucial supporting rôle in the efficient realization of plans. They are described in Section 6.
3. Query formulation All formal queries are expressed in a formal query language termed OSquel. Although the details of query representation cannot be fully understood until the features of the static knowledge base have been elaborated, it is nonetheless useful to provide an overview of query formulation at this point, since the design of the static knowledge base, as well as the other knowledge bases and process query solvers, was strongly motivated by the needs inherent in query formulation as well as solution. 3.1. Process queries Process queries are at the very heart of Yucca-*. The general syntactic format of such a query is shown in Figure 3.5 The underlying idea is that we express dynamics axiomatically, with formulas identifying the precondition necessary for the action to take place, the postcondition asserted as a result of the action, the actor performing the action, as well as the action itself. Each of these four components is represented as a formula in SL, which is a first-order language underlying the static knowledge base. The unknowns, or variables to be bound, are identified in the query-variables: entry. Queries are classified by identifying the location of the major unknown; there are four such classifications.
PLAN REALIZATION FOR COMPLEX COMMAND INTERACTION
191
Unknown action queries In unknown action queries, the primary unknown is the action F. There are three levels of such queries. In the most general case, the unknown action is a general command interconnection. An example would be the formalization of the request to explain how to print a file with pageheaders on the laser printer, which has as a solution pr | lpr -Plaser. Unknown action queries may also designate that the answer be a single command, such as the query asking which single command puts pageheaders on a file or they may designate that the answer be an option of an existing command, such as the query asking which option on the pr command allows specification of the pageheader value. To which of these three classifications a given formal query belongs is determined by the type of the unknown variable in the action slot of the query. Unknown consequence queries In unknown consequence queries, the major unknown is the result of performing an action, which is the postcondition Q of the formal query. Examples include formalizations of asking what happens if one logs out with jobs running in the background or asking what happens if one tries to delete a directory with files in it. Unknown precondition queries These queries ask what is necessary to achieve a given task, and so the primary unknown is the precondition P of the formal query. Examples include formalizations of asking what privileges are necessary to set the system date or asking what properties a directory needs to have so that one may delete it. Unknown actor queries Occasionally, queries arise in which the actor A is the primary unknown. An example is the query which asks who can read another user’s files. It is clear from data gathered on queries submitted to the prototype system UCC that unknown action queries are the dominant form of process query posed by users. Therefore, such queries are given primary emphasis in the design of the back end. This means that the selection of features for the back end is driven by the need to solve such queries as completely and efficiently as possible. The solution of unknown consequence, precondition, and actor queries is given only secondary consideration in Yucca-*, and is not discussed further in this report. Secondary conditions In classifying queries, we have alluded to the “primary unknown”. However, the proper formalization of queries may require more than one
192
STEPHEN J. HEGNER
unknown. For example, in responding to the query asking how to print a file with pageheaders on the laser printer, while the command sequence pr | lpr -Plaser may be the “primary” response, the success of this solution is conditional upon the file identified by being readable by the user. Such a condition would be bound to a variable known as a secondary precondition of the query. Secondary postconditions are also necessary in some instances. 3.2. Example of a process query Let us now take a look at the formalization of a specific unknown action query. The central notion is that of a mutable object; that is, an object whose characteristics may be changed by the execution of some action. In formalizing mutation, all objects are assigned a version number. Two instances of the same object, but with different version numbers, may differ from one another in value. As the specific example, let us formalize the query asking how to print a file with pageheaders and lines numbered on the laser printer. An informal tabulation of how this query is represented in OSquel is presented in Figure 4. Several items are worthy of note. First of all, the version number of the input file #I does not change; this means that the solution is not permitted to change that file in any way. Second, %laser-print-queue (which identifies the laser printer queue) does change in version due to the action; this means that we expect its value to change. Third, the print queue entry #PQE is a mutation of the contents of #I, with first lines numbered and then pageheaders inserted. The decision that pagination should occur after numbering lines is one that is made by the front end in the process of query understanding, based upon what is a reasonable default in the absence of further information. Fourth, the action may be any command sequence, and is not restricted to a single command. Again, this decision is made by the front end, and is the default unless the user explicitly indicates that he is looking for a single command or option. Similarly, the actor defaults to the user asking the query; this selection is also made by default in the front end. Finally, both the preconditions and postconditions have secondary slots for “overflow”; we shall see how these are used when query solution is examined in detail in Section 6. The actual formalization of this query is shown in Figure 5. Of course, we have not yet provided the necessary detail of information on the static knowledge base to fully explain the syntax and meaning. It is presented at this point to give the reader an idea of what an actual formal query looks like, as well as to provide a reference example which will be carried throughout this paper.
PLAN REALIZATION FOR COMPLEX COMMAND INTERACTION
193
Preconditions: − Version 0 of the mutable object file #I is owned by the current user. − The contents of this file is a visible byte sequence. − The mutable object %laser-printer-queue currently exists in version 0. − The variable ?SPP is to be bound to any secondary precondition. Action: − The variable ?CS is to be bound to the command sequence answering the query. Postconditions: − The file #I does not change; it stays at version 0. − %laser-print-queue is mutated to version 1. − Version 1 of the mutable object #PQE identifies a new print queue entry which is owned by the current user and which is appended to version 0 of the queue. − The contents of version 1 of #PQE is the same as that of version 0 of #I, save that it first has lines numbered, and then is paginated with standard pageheaders inserted. − The variable ?SPQ is to be bound to any secondary postcondition. Actor: − The actor is the current user. Figure 4. Sketch of formalization of the query asking how to print a file with pageheaders and lines numbered on the laser printer.
4. The static knowledge base 4.1. Overview and syntactic description The static knowledge base is the most ubiquitous entity in the entire Yucca-* system. Not only is every other component of the back end dependent upon it, but the definition of the query language OSquel is as well. Thus, this knowledge base is known even to the natural language front end. The major purpose of the static knowledge base is to provide descriptions of those classes of objects which are either manipulated by commands or else function as actors in the execution of commands. There are currently definitions in the Yucca-* static knowledge base for over one-hundred object classes, broadly partitioned into groups, as indicated below. 1. The master object classes. These object classes contain descriptions of the top-level components of the entire UNIX system. They include definitions for master-system, user-table, login-table, process-table,
194
STEPHEN J. HEGNER
Figure 5. Formalization in OSquel of the query asking how to print a file with pageheaders and lines numbered on the laser printer.
and directory-structure, as well as for supporting concepts such as directory-tree. 2. The file object classes. These object classes provide the structural definitions for all types of files, including directories and devices. It also includes definitions for supporting concepts such as file-node, i-node-number, and various concepts related to file access, including paths and protection.
PLAN REALIZATION FOR COMPLEX COMMAND INTERACTION
195
3. The byte sequence objects. These object classes provide definitions for special byte sequences, such as visible-byte-sequence and formatter-byte-sequence, as well as listings which represent displays of other collections, such as collective-user-listing and collective-process-listing. 4. The user and connection object classes. These object classes provides descriptions of system users, terminal connections by these users, and the special properties of command processors, but not processes themselves. Represented in particular are the details and parameters involved in a terminal connection, and the attributes seen and defined in a session with the system. 5. The process object classes. The central object class in this category is process, which models the notion of a UNIX process. 6. The printer object classes. This category contains those objects relevant to using system printers; in particular, notions related to print queues and their entries are included. 7. The command object classes. Although the semantics of commands are modelled in the dynamics knowledge base, there are object classes in the static knowledge base which identify commands by name. These are used principally in support of conceptual queries. 8. The fundamental object classes. The fundamental object classes deal with primitive notions such as time and date, bytes, characters, natural numbers and the like. They seldom are used on their own, but are ubiquitous in the definitions of more complex objects. 9. The macro meta-object classes. The macro meta-object classes are not really object classes; rather, they are definitions of macros which may be used to construct new classes. Their use will be discussed in 4.5 below. The classification of objects into these groups, while conceptually useful, is quite informal. On the other hand, the object classes are formally classified in a generalization hierarchy, in much the spirit of generalization systems such as KL-ONE (Brachman and Schmolze 1985). This permits us to allow an object class to inherit all of the attributes of its parent. For example, the class directory-file inherits all of the attributes of the class file, since it is formally declared that a directory is a file. With the exception of the macro meta-object classes, all object class definitions share a common syntax, which is given in Figure 6. Here we have used the common syntactic notation that [] denotes at most one occurrence of a , and + denotes the juxtaposition of one or more ’s. The hierarchical-descriptor is a formula which classifies the object class in the hierarchy, while the following three entries all give information
196
STEPHEN J. HEGNER
Figure 6. Syntax of object class definition.
about the attributes of the object class. The structure: entry defines the “traditional” attributes of the class, and is elaborated in 4.3. The filters: and generation-structure: entries are used to recapture special types of attributes which occur in the modelling of command language behavior, and are described in 4.4 and 4.5, respectively. 4.2. Object instances Object class declarations define the structure which actual objects must take. In Yucca-*, there are two distinct flavors of object instances. Mutable objects Operating system commands manipulate instances of object classes, which we term mutable objects. For example, In the query illustrated in Figure 5, #I, #PQE, and %laser-print-queue, are all names identifying mutable objects. By convention, names beginning with “#” denote object instances which are defined for local use, while names beginning with “%” denote canonical predesignated objects which are known in any use of the consultant. Constant objects It is often useful to have as a standard reference a “default” value for an object class, which is known by name to all parties using the static knowledge base. The object instance identified by %paginated-standard-pageheaders employed in the example query of Figure 5 is one such instance; it identifies an object instance of type pagination which defines visible byte sequence pagination in a canonical default way. Although many of the details of object representation depend upon corresponding details of object class definition, and so are described in subsequent parts of this section, it is nonetheless important that we provide an overall idea of object representation at this point. The syntax of declaration of an object instance is shown in Figure 7. Several examples may be found in the query depicted in Figure 5. The may be either a natural number or the constant identifier fixed:. When it is the latter, a constant object instance, not subject to mutation, is declared. When it is the former, the version number identifies
PLAN REALIZATION FOR COMPLEX COMMAND INTERACTION
197
Figure 7. Syntax of object instance definition.
a particular version of the object instance, of which there may be several. Two object instances with the same name but different version numbers may have different values, but identity of both name and version number mandates identical values. Thus, in any statement or query, version 0 of #I always denotes the same object instance with exactly the same value, but version 1 of that same object instance may differ in value from version 0. The is used to declare explicitly the value of the particular version of the instance; examples will appear later. We now turn to elaboration upon the structure of object classes. 4.3. Simple structure Basic notions The structure: field of an object class definition is used to give a standard programming-language style data type to that object class. Currently, there are eight fundamental constructors which are used in this capacity; they are enumerated:, record:, array:, union:, set-of:, sequence-of:, queue-of:, and stack-of:. In addition, there is the external: declaration which is used to identify structures defined externally to Yucca-*, such as the integers. The definitions of three classes associated with print queues are given in Figure 8. Associated with the fundamental constructors are standard operators. The retrieve: operator is used to extract values from specific instances on a variety of types. There are also type-specific operators, such as enqueue: and remove: for type queue-of:. These operations may be used, along with the equality predicate, to specify rather complex constraints on individual states. See Figure 5 for examples. Foreign attributes One of the more important problems in designing an overall consultation system is user modelling. Even though we expect the user to have a reasonable conceptualization of how an operating system is organized and how it operates, the user may have misconceptions as well. To the extent that such misconceptions occur because the user is migrating from another operating
198
STEPHEN J. HEGNER
Figure 8. Some examples of object class definitions.
Figure 9. Example illustrating foreign attributes.
system which has some similarity to UNIX in general structure but which differs in detail, we may include attributes which are not applicable within UNIX, but which do have meaning in the context of other operating systems. Such attributes are termed foreign. An example illustrating the notion of associating a password with a directory is given in Figure 9. In this figure, note that the attribute password: is designated as foreign, and that its applicability to another system (in this case TOPS-20) is asserted as well. A query enquiring as to a method to set a directory password, say, would not be rejected as not understood; rather, the consultant would be able to respond with a statement that associating a password with a directory is not a situation found in UNIX, but rather only in TOPS-20. It would also be possible to attach an English-language explanation of how UNIX deals with directory access without protection, although we have not incorporated this in the present design.
PLAN REALIZATION FOR COMPLEX COMMAND INTERACTION
199
The method is not without limitations, however. It seems to work rather well for operating systems such as TOPS-20, VAX/VMS,6 and MS-DOS. These systems all conceptualize the working domain of the command language in reasonably similar ways, although they differ substantially in detail. On the other hand, operating systems such as VM/CMS conceptualize the entire notion of a user’s working environment very differently than do UNIX and its relatives. At one point, we attempted to incorporate certain aspects of VM/CMS into the Yucca model, but found it to be almost impossible. Clearly, to address the modelling of users migrating from systems such as VM/CMS, much more sophisticated techniques are necessary. Currently, Yucca-* makes no accommodation for such needs within the formal model. 4.4. Filtered attributes It is often necessary to specify constraints on the values of objects. In particular, mutations effected by commands will change the values of objects in particular ways which are most easily conceptualized by a change of constraint. Referring again to our running example query asking how to print a file with pageheaders and lines numbered on the laser printer, the output is to be a constrained version of the input, the constraints being identified by the addition of pageheaders and line numbers. While it is in principle possible to represent such constraints by defining logical conditions which are to hold on printable byte sequences, such formalizations would be enormously complex, and the actual definitions themselves would be of little direct use. In modelling the notion of pagination, for example, we are not really interested in a microscopic definition of the conditions under which a byte sequence is paginated. Rather, it is enough to know that it is indeed paginated, and omit the irrelevant details. To model this sort of constraint, we employ the notion of a filtered attribute in Yucca-*. Figure 10 illustrates the concepts relevant to representing pagination in this manner. Syntactically, filtered attributes appear to be no different than do ordinary attributes in a record field. However, their use is fundamentally applicative rather than assertive. For example, we cannot directly speak of the contents of a file instance #F as paginated; rather, we must say that a particular version of #F is the result of applying a pagination filter applied to another byte sequence (which might be the contents of an earlier version of that same file). Because of this applicative nature of filters, the order in which they are applied is significant. Referring again to Figure 5, it is asserted in the postcondition that the lines-numbered filter is applied to the result of applying the paginated filter to the contents of version 0 of #I to obtain the contents of
200
STEPHEN J. HEGNER
Figure 10. Example illustrating filtered attributes.
version 1 of #PQE. It is clear that such ordering is necessary, as first numbering lines and then inserting pageheaders is quite different than the other way around. Note from Figure 10 that the allowed value of a filter may be another object class. The allowed values for the paginated filter are taken from legal instances of objects of type pagination, which has a record structure. In the example query, the actual instance used is the default instance %paginated-standard-pageheaders, which will be discussed in more detail in 4.6. 4.5. The display-object problem One of the most complex conceptual problems encountered in modelling the command language environment is the display-object problem. This problem arises because of the need to distinguish between an instance of an object and a display of that instance. For example, when posed a query asking how to find out who is logged onto the system, the proper response, technically speaking, is not a collection of users, but rather a file whose contents is a listing of the identifications of those users. Similarly, the proper response to the query asking how to find out the names of the files in a directory is not a list of files, but rather a file whose contents is a listing of those files. We humans make this translation so automatically that we are usually not even aware of it. Yet, this translation cannot be made automatically by a consultation system. The reason is at least twofold. First, the result needs to be displayed to the user, and we cannot display users; rather, we can only
PLAN REALIZATION FOR COMPLEX COMMAND INTERACTION
201
display byte sequences. In the formal model, it makes no sense to set the contents of standard output to a set of users. Second, such displays typically have special attributes not shared by the objects displayed. For example, it makes no sense to speak of placing files in columns, yet it makes perfect sense to display the names of files in several columns. One way to address this problem, which was that taken in the earlier design of Yucca, is to define directly two classes of objects, one for direct representation and one for display. However, this masks the direct and systematic relationship between objects and their displays, and forces a great deal of manual translation of attributes to displays. To address this problem more systematically, in Yucca-* we employ the notion of a macro metaobject. In most general terms, a macro meta-object is a macro which takes as arguments one or more object classes, and generates a new object class from it. The definition of the macro meta-object which generates a display of the attributes of a set of objects, together with an instantiation yielding an object class definition for the display of a set of users, is shown in Figure 11. Every macro meta-object must have at least one source parameter. The source-parameters: entry identifies the object class or classes which must be supplied as parameters for the expansion. In a display macro, it identifies that which we are to display. In the example of Figure 11, the source parameter is a set of objects of type ?IS. The external-parameters: defines the actual parameter types which must be bound in order to generate the instantiation. In effect, they serve as parameters for the source parameters. In the example, this parameter is ?IS, which represents the base type of the set to be displayed. The macro display-set-attributes places no constraints on it, but the instantiation collective-user-listing binds it to be of type user (since we are generating a display of users). The generators: field of the macro identifies how the input parameter will be used to generate the attribute structure in the expanded macro. In the example, another macro, which generates a display for attributes of a record, is invoked. In addition to the attributes generated in this fashion, there are two sources of special-attributes:. First of all, the macro itself provides attributes which apply to any instantiation of it. In the example of Figure 11, there are four such attributes, identifying criteria for sorting the objects as well as certain display formats. Additionally, the instantiation definition may define additional attributes in its generation-structure: component. In this example, there is just one, user-listing-display-format, which in effect decides whether it is a “w-command-like” listing or “f-command-like” listing which is desired. Although this syntax seems quite imposing, the actual expanded instantiation is much more understandable. It is shown (in part) in Figure 12.
202
STEPHEN J. HEGNER
Figure 11. Example of a macro meta-object and an instantiation.
Figure 12. Example of a full expansion of an instantiation.
PLAN REALIZATION FOR COMPLEX COMMAND INTERACTION
203
Figure 13. Some major global variables.
All of the attributes of the object class user are included, but the values: field of each attribute has been changed. For example, in the definition of object class user:, the allowable values for the field id are in the type user-id. But in the display object, the allowable values are in simple-select, which is just a two-valued type indicating whether or not the value of the attribute is to be displayed. By selecting appropriate values for each field in this expansion, we obtain a detailed and complete description of the format of a display of users. 4.6. Global and default instances Global instances There are several global object instances which retain their meaning permanently. Several of the more important such objects are tabulated in Figure 13. There are also several more “local” global variables which apply to a particular installation, but not globally to UNIX systems. The variable %laser-print-queue used in the query of Figure 5 is one such example. Default instances In addition to global instances, there are several default instances which are used to express queries more succinctly. A particular example is %paginated-standard-pageheaders, which is used to express the pagination condition in the query of Figure 5. Its definition is shown in Figure 14. Refer to the definition of pagination in Figure 10. Basically, it represents the default mode of pagination which is understood, via the formal query language, to both the front end and the back end. This instance is not absolutely necessary; rather, it could be constructed from scratch each time that it is needed. But it is far more efficient to define it once, and use it repeatedly. Of course, this default may be modified if so needed. For example, if the formal query called for 60 lines per page, the syntactic modification shown
204
STEPHEN J. HEGNER
Figure 14. Definition of a default instance.
Figure 15. Syntactic modification to locally alter a default instance.
in Figure 15 within the query would accomplish that task. This approach is most appropriate if a large instance has only a few differences from a default instance. Note that embedded within the definition is another default instance, %standard-pr-pageheader.
5. The dynamic knowledge base 5.1. Principles of command semantics representation If the static knowledge base is considered to be the foundation of the back end, the dynamic knowledge base is the heart. It is in this knowledge base that the semantics of the UNIX command language is embodied. The basic philosophy is very simple. The essence of command language behavior is represented by a relatively small collection of primitive descriptions, together with an interconnection calculus for combining these primitives into more complex ones. Primitive components The syntax of a primitive component (also called a simple dynamic-object class or simple module) is not unlike that of a formal query (although it will generally be much less complex), save that primitive components do not contain free variables. Figure 16 provides a more formal description.
PLAN REALIZATION FOR COMPLEX COMMAND INTERACTION
205
Figure 16. Syntax of dynamic object class definition.
P1 σ1
F1 D1 σ1
P2 σ2 Q1 σ1
F2 D2 σ2
P3 σ3 Q2 σ2
F3
- ...
Q3 σ3
Pk σk
D3 σ3
Fk
Qk σk
Dk σk
Figure 17. Serial interconnection of dynamic modules.
Each of the four components is shown to be optional. This is just an indication that they may default to certain values when not specified; this will be elaborated upon in 5.2. For the time being, assume that all components are present. Interconnection calculus At present, only serial interconnection is supported. The basic idea is as follows. Let D1 = hP1 , F1 , Q1 , A1 i and D2 = hP2 , F2 , Q2 , A2 i be simple components, with Pi denoting the precondition, Fi the action, Qi the postcondition, and Ai the actor of Di . Let Di σ denote the module resulting from the application of the unifier σ to each component of the dynamic description.7 Then we may interconnect D1 to D2 sequentially with unifier σ1 applied to D1 and σ2 applied to D2 if and only if Q1 σ1 |= P2 σ2 . In other words, we require that the postcondition of the first module be strong enough to imply the precondition of the second. A simple component interconnection may thus be viewed as a sequential composition, as illustrated in Figure 17. Adequacy of the interconnection calculus The utility of this interconnection scheme arises from the fact that not only are simple commands represented by such modules, but so too are their options. Thus, to realize the UNIX command “pr -h myheader -l60 foo”, we use the interconnection depicted in Figure 18 below. The module pr-command-main contains the main definition of the semantics of the optionless pr command, while the modules pr-change-
206
STEPHEN J. HEGNER
Figure 18. Realization of the command “pr -t myheader -l60 foo”.
pageheaders and pr-change-pagelength contain the definitions for the “-t” and “-l” options respectively. The other three modules are for command support, and will be discussed in 5.2. The important point here is that the semantics of a command may be realized with the use of three types of simple modules. 1. A main module describing the behavior of the basic command. This module is included in any interconnection realizing the associated command. 2. A collection of modules, one for each option, describing the behavior of that option. A given module from this class is included in the interconnection only if the associated option is selected. 3. A fixed set of “support” modules which are always included in any realization of the given command. In addition to modules defining the behavior of individual commands, there are also modules defining the action of the command interconnectors pipe “|” and sequence “;”, command grouping via the brackets “(..)”, and redirection of input and output. Thus, not only can individual commands be realized within this framework, but interconnections of more than one command can be also. Limitations of the interconnection calculus At this point, we are modelling only the serial interconnection of commands via sequencing and piping, and redirection of input and output. More sophisticated shell programming constructs, such as conditionals and loops, are not supported. This decision has been made largely in the interests of practicality. To support arbitrary programming language style constructs would move Yucca-* out of the domain of a simple command language consultation system and into the domain of a sophisticated programming aid, such as the Programmer’s Apprentice (Rich 1981; Waters 1985a; Waters 1985b). Such an extension would add enormously to the complexity of the project.
PLAN REALIZATION FOR COMPLEX COMMAND INTERACTION
207
Figure 19. Overall syntax of a command manager.
5.2. Command managers The problem with a flat organization It is quite possible to regard the dynamics database as a flat collection of simple modules, in much the same way that one would regard a logic database as a flat collection of clauses. Indeed, this was precisely the approach taken in the earlier Yucca system. However, we found that this leads to at least two difficulties. First, it becomes necessary to incorporate into the definition of each command support and option module specific logical “interconnection” preconditions and postconditions which permit the use of those support modules only with the associated command, and only in the proper sequence. It must be remembered that this interconnection scheme is only logical; we cannot realize each simple module separately. For example, since it is physically impossible to realize the pr command with an option from the cat command, the formal representation must prohibit this as well. The incorporation of these interconnection constraints contributes markedly to the complexity of the precondition and postcondition representations. Second, it fails to make use of critical information which can aid in the efficient search for a proper plan of action; namely, that it is possible, on purely syntactic grounds, to eliminate almost all possible interconnections simply because they associated options or support from one command with those of another. Organization by command In Yucca-*, we have addressed the problem indicated in the previous paragraph by organizing the simple modules associated with each command into a single unit known as a command manager. In addition, there are separate command managers for the redirection, command grouping, and sequential interconnection operators. Each command manager contains three distinct components, as identified in the syntactic description of Figure 19. There are three main types of information embodied in a command manager. 1. The object classes entry contains the semantic description of each simple module associated with that command. This includes the main module, the option modules, and the auxiliary modules. 2. The interconnection syntax embodies two main entities:
208
STEPHEN J. HEGNER
Figure 20. Structure of the command manager for the pr command.
− Direct rules for interconnecting the simple modules, without need to resort to the checking of formal logical interconnection rules. − The information necessary to associate module interconnections with command-line syntax, including user-supplied parameters. 3. The interconnection categories define the information necessary to interconnect one entire command to another. In effect, they define the “port” characteristics of the command. They are discussed further in 5.3. We now illustrate the concept by examining in detail the command manager for the pr command. The overall composition of this manager is depicted in Figure 20 below. Due to its length, the complete formal definition of the command manager is contained in the Appendix. There are several points which should be emphasized relative to this formalism.
PLAN REALIZATION FOR COMPLEX COMMAND INTERACTION
209
1. The semantic description of each individual simple module does not contain any formal constraints on the interconnection. Indeed, examination of the formal definitions in the Appendix reveals that many of the dynamic object classes have no preconditions and/or postconditions. Since the interconnection syntax defines precisely the way in which these modules may be interconnected, such conditions are unnecessary. 2. The modules may contain common variable names which do not represent the same thing. This is not possible in a flat organization, since accidental variable name collision might then lead to erroneous conclusions. 3. The action component is not given for any of the modules. This information is obtained entirely from the interconnection syntax. 4. The actor is listed only in the pr-starter and pr-command-main modules. The actor is assumed to be the same for all components of a local interconnection within a command manager, and hence need not be specified repeatedly. Now let us examine the interconnection depicted in Figure 18 in more detail. 1. The rôle of the pr-starter module is twofold. First, it sets up the initial conditions for execution of a pr command. In particular, it establishes the definition of the default pagination conditions of the command via the pagination object #P. Second, it identifies the actor associated with the action. This latter value defaults to %terminal-connection, as identified in the interconnection syntax specification, but may be changed by altering the input parameter $actor. As indicated in the interconnection syntax, inclusion of this module is mandatory in any realization of pr. 2. The rôle of the pr-input-definition module is to identify the source of input to the command. As indicated in the interconnection syntax, this value defaults to %standard-input if not supplied via the parameter $user-supplied-input-file. This module is also mandatory. 3. The rôle of the pr-command-main module is to represent the core of the command. In this case, it asserts in its preconditions the existence of the mutable objects associated with its input and output, and asserts that both must be readable. In its postconditions it asserts that the contents of the input is copied to the output. Note that no pagination or the like has been applied at this point, although the actual conditions are carried through unchanged. 4. The rôle of the pr-change-pageheader module is to change the pageheader to the user supplied value, delivered via the variable $user-supplied-header. It is specified via an updated: operation on the current instance of the pagination object #P to yield a new version.
210
STEPHEN J. HEGNER
Figure 21. Interconnection categories.
It is an optional module, and may be included or excluded in any interconnection. 5. The rôle of the pr-change-pagelength serves a similar function in changing the pagelength. 6. The module pr-finisher must terminate any realization of pr. It actually applies the filter which was assembled by pr-starter and modified by the option filters to obtain the final output object. It is important to observe that conditions which are not affected by a given filter progress through the interconnection. For example, since the option filters do not affect any of the objects except for #P, these other objects are not altered by the option filters. 5.3. Interconnection categories Each command manager contains the information necessary to describe the assembly of its associated command. However, many query solutions will require the interconnection of several commands. To facilitate the characterization of command interconnection, each command module characterizes the results that it produces in terms of interconnection categories. Presently, there are eleven interconnection categories in the Yucca-* formalism. Six categorize input and five output. They are tabulated in Figure 21. The idea is that only commands which match up may be interconnected. For example, to interconnect via a pipe, the first command must be of type stream output and the second of type stream input. This feature is utilized extensively in the realization of solutions to unknown action queries, as described in the next section.
PLAN REALIZATION FOR COMPLEX COMMAND INTERACTION
input query
Item
- Template Matcher
6 ? Item Templates
- Structural
Synthesizer
- Formal
Finisher
iP P P PP 6 P PP PP PP PP PP PP ? q
Dynamic
- Knowledge Base
211
query
-
solution
Cliche´ Library
Figure 22. Architecture of the command synthesizer.
6. Solution of unknown action queries 6.1. The overall architecture As noted earlier, the process of solving an unknown action query may be regarded as one of plan generation. We are given an initial state (the precondition identified in the formal query), a final state (the postcondition), and a set of possible actions (the dynamics knowledge base), and we must find a plan (i.e., an interconnection of simple modules) which realizes the goal. In recent years, there has been a great deal of progress in the general area of planning, as may be found in the recent anthologies (Brown 1987) and (Georgeff and Lansky 1986). However, we believe that it is still the case that special techniques which take advantage of the particular application at hand will yield far more efficient solutions than will the application of general tools. To this end, we have developed a highly specialized architecture, the command synthesizer, customized to the goal of solving unknown action queries in the UNIX domain. Shown in Figure 22 is its architecture. There are three main steps in the solution process. 1. Selection. In this step, the main commands and options which are candidates for participation in the final solution are selected, without particular regard for the way in which they will be interconnected. This function is executed by the item-template matcher, with the support of the item-template database, and is elaborated in 6.2. 2. Interconnection. In this step, a plan for interconnection of the components selected in step 1 is assembled. This function is executed by the
212
STEPHEN J. HEGNER
structural synthesizer, with the support of the cliché library, and is elaborated in 6.3. 3. Refinement and binding. In this step, the interconnections assembled in step 2 are used to identify the actual variable bindings necessary to solve the query. In addition, the solution of necessary secondary subgoals is addressed at this step. All of this is executed by the formal finisher, and is elaborated in 6.4. 6.2. Item templates and the item template matcher Consider once again the example query asking how to print a file with pageheaders and lines numbered on the laser printer, and its formalization of Figure 5. We know, as UNIX experts, that cat -n filename | pr | lpr -Plaser is a solution to this query, assuming that our laser printer is designated by the system name “laser”. In effect, the above solution is a “plan” which the consultant must assemble. In the first step in assembling this plan, we take advantage of a significant feature of the command language domain. Namely, by examining the postconditions of the plan to be constructed, we can determine immediately a “core” set of participating commands. For the example query, the selections are as follows. 1. Because a new print queue entry is being generated, any plan must include the UNIX command which enqueues entries into print queues; namely, the lpr command. 2. Because the particular queue involved is the laser printer queue, the option on lpr used to specify a particular queue (the “-P” option) must be involved also. 3. Because the new print queue entry must have its lines numbered, the command which accomplishes this task (the “-n” option of the cat command) must be involved in the solution. 4. Because the new print queue entry must be paginated, the command which accomplishes pagination (the pr command) must be a participant in the solution. Of course, there are other participants in the final solution, such as the two pipes. Other queries might require redirection or command grouping as well. However, the “core” commands of the solution have been identified. The item templates To implement effectively a process which selects commands and options as outlined above, a method of efficiently identifying those simple modules
PLAN REALIZATION FOR COMPLEX COMMAND INTERACTION
213
Figure 23. Syntax of an item template.
Figure 24. Example of an item template.
which effect a particular mutation is needed. This function is provided by the item templates. Basically, the item templates form an index from attributes of static object classes to those simple dynamic object classes which alter and/or observe their values. The overall syntax of an item template is shown in Figure 23. For a given attribute, the only entry which is mandatory is that used for altering its value. Figure 24 provides the definition of the item template for the pagination object class. Notice that the association between attributes and simple modules is neither injective nor surjective. For example, the pr-drop-header-trailer module, an option of the pr command, is associated with both the header-size and trailer-size attributes of the pagination object. This is because this module is the (only) way to alter the sizes of the margins in a pagination; they must be modified simultaneously. On the other hand, there is no way to define the page at which printing stops, so no command is associated with the stop-page attribute. The item template matcher The item template matcher is basically a peruser of formal queries. Its primary function is to examine the formal query and identify the mutations which are mandated to take place, and then to select the appropriate simple dynamic modules which are necessary to effect such a mutation. It does so in two passes.
214
STEPHEN J. HEGNER
Figure 25. Synopsis of output of the item template matcher for example query.
1. In the first pass, it sets up a table which identifies explicitly all declared objects occurring in either the precondition or postcondition of the query. It also flags them as fixed or potentially mutated. 2. In the second pass, it examines the prescribed mutations of those objects which are potentially altered. This examination will identify simple attributes of those objects whose values are changed, or filters to be applied to those objects. For each of these simple attributes and filters, the appropriate commands are selected, with the aid of the item templates. The collection of simple dynamic modules so selected, together with bindings to the specific clauses in the formal query to which they apply, is the output of the item template matcher. It is important to emphasize that no interconnection or ordering of these modules is performed at this step. In the terminology of Sacerdoti (Sacerdoti 1977), this first stage of planning is totally nonlinear. For the example query, the set of modules returned, together with their query bindings, is shown in Figure 25. For simplicity, we have shown only a fragment of the formal query associated with the command, while the actual algorithm returns the entire query with pointers into it from the simple modules selected. Universality of this approach It is central to the success of this approach that all unknown action queries be representable by mutations on objects. From our empirical investigations, this does seem to be the case, even when at first thought a query might appear to be of a different form. For example, the query asking simply how to delete a file might appear to be one of deleting an object rather than altering an attribute. However, upon further consideration, it becomes clear that deleting a file is equivalent to altering the directory in which it resides. Thus, in this query, the mutated object is not the file to be deleted, but rather its parent directory. Viewed in this fashion, our approach works well. The item template for the entries attribute of the object class directory points to the simple dynamic module rm-command-main (as well as several simple modules, such as mv-command-main and cp-command-main, which is exactly what we want).
PLAN REALIZATION FOR COMPLEX COMMAND INTERACTION
215
Other entries in item templates In Figure 23, observe that there are four possible entries for an individual item template, rather than just one, with the latter three optional. These optional forms are carryovers from the earlier Yucca system, but still may be used in alternative query formulations. For example, consider the query asking how to obtain a directory listing. The “preferred” formalization in Yucca-* would call for the delivery of a byte sequence on the user’s standard output which is a display of the directory contents. However, in the earlier Yucca system, this could have been formalized as a delivery of the directory contents, with a display filter applied. Since there is no direct display object identified in this case, no alter-value: field is applicable. However, the observe-value: field of the attribute entries of a directory points to the simple module ls-command-main, which is the key participant in any solution. We are retaining these alternate fields at this point because Yucca-* is a highly experimental system, and the possibility that query formalization and solution along this alternate path may be a better ultimate choice remains. 6.3. Clichés and the structural synthesizer The item template matcher delivers the identities of several simple dynamic modules, together with associations to clauses in the formal query, to the structural synthesizer. The task of the structural synthesizer is to interconnect these simple dynamic modules in a fashion suitable for ultimate query solution. The actual process consists of two steps, individual command assembly and cliché selection and interconnection assembly. We now elaborate upon these steps. Individual command assembly The item template matcher delivers to this unit a collection of individual simple dynamic modules, together with query clause attachments. The first step is to take these isolated modules and assemble complete interconnections for individual commands. As all of the information necessary to assemble a complete command is contained in the associated command manager, this is usually a completely straightforward process. However, there are two cases in which complications may occur. 1. It may be the case that multiple instances of the same command are necessary. There are two ways in which this may occur. First, different command options may be associated with the alteration of distinct objects. This occurs very rarely, but it is possible. A rather contrived example is the formalization of the query asking how to concatenate two files with the lines of the first numbered and the control characters made visible in the second. A solution using two distinct instances of cat is
216
STEPHEN J. HEGNER
(cat -n file1; cat -v file2) > result. No solution using only one instance is possible. Second, it may be necessary to interlace filters. This also occurs rarely, but is possible. For example, consider the formalization of the query asking how to process a file so as to paginate it, number lines, use formfeeds to separate pages, and make those formfeeds visible. A solution is cat -n filename | pr -f | cat -v > result which requires two distinct instances of the cat command. Because the pagination must be done after the insertion of line numbers but before the marking of the visible characters, no solution with only one instance of cat is possible. 2. It may be the case that incompatible options have been identified. For example, the pr command options “-h” (use user-supplied pageheader) and “-t” (essentially, do not paginate) are logically incompatible, although the command allows both to be specified and simply ignores the pageheader. This incompatibility is known to the pr command manager, and a request to include both in an interconnection would result in an error. In such an occurrence, the maximal compatible commands will be synthesized (the representations of both “pr -h” and “pr -t” in this case), but these results will be flagged as “incomplete”. In any case, that which is returned is a collection of assembled commands. For the example query, the three assemblies shown in Figure 26 are obtained. Cliché selection and interconnection assembly Once the individual commands which may participate in the solution are selected, the next step is to interconnect them properly. There are four low-level interconnection operations. 1. Piping. This is the standard UNIX output-to-input connection. 2. Sequencing. This is simply sequential execution of commands, as in cat -n file; pr. 3. Grouping. This forces commands to be assembled in blocks, and is represented syntactically in UNIX with parentheses. An example, given earlier in this section, is (cat -n file1; cat -v file2) > result. 4. Redirection. The current model includes input redirection “<”, as well as the three forms of output redirection, “>”, “»”, and “>!”. It must be remembered that the applicable interconnection operators are usually not identified by the item template matcher. Therefore, before the
PLAN REALIZATION FOR COMPLEX COMMAND INTERACTION
217
Figure 26. Examples of assembled commands.
identified commands can actually be interconnected, it is necessary to select the appropriate connectives from the above list. While this could be done by directly examining the formal query and the way in which the commands are bound to its clauses, this would prove to be rather inefficient. Rather, we take advantage of the fact that command interconnections almost always occur in ways based upon a small number of simple patterns, known as clichés. Our use of clichés was inspired by a similar technique utilized in the Programmer’s Apprentice system (Rich 1981), (Waters 1985a), (Waters 1985b). In that system, clichés are used to model common control patterns used by programmers. Our clichés are much simpler, representing only sequential interconnection. A few of the more common clichés employed in Yucca-* are given in the table of Figure 27. Clichés have slots, which must be filled with objects of the appropriate interconnection category. As concrete examples, the two clichés to be used in the solution of our example query are given in Figure 28 below. The actual process of cliché selection and command interconnection proceeds by the following algorithm.
218
STEPHEN J. HEGNER
Apply a filter and redirect the output to a file. Use a file for filter input. Redirect both the input and the output of a filter. Filter a file and print the result. Compose two filters. Roff or TEX a source file. Figure 27. Some common clich´es.
Figure 28. Definitions of some common clich´es.
1. Determine temporal order The formal query, together with the bindings of assembled commands to clauses, is examined to determine a temporal partial order on the fully composed commands. C1 < C2 means that execution of command C1 is to take place prior to the execution of C2 . This order is determined by several factors, including nesting in apply: and append:8 operators. The ordering is partial because in many cases it is not possible to determine easily at this point the appropriate order; indeed, there are cases in which order is not relevant. In the example query, the assembled cat command is determined to precede temporally the assembled pr command, because of the order of application of the filters. However the assembled lpr command is not temporally compar-
PLAN REALIZATION FOR COMPLEX COMMAND INTERACTION
219
able to either at this point. This is because the print queue entry and result of the cat-pr filter are asserted to be equal, and there is no implied temporal ordering on the arguments of “=”. 2. Determine equivalence The commands are associated in a quasi-equivalence, relative to specific functions within the query. Two commands will be determined to be quasiequivalent if they were selected as “alternates” for the same purpose by the item template matcher. For example, under certain conditions, the UNIX commands cat, pr, head, tail, and cp can all have identical effects. Thus, it is possible that more than one may be selected as a candidate for a particular application. In our example, there are no quasi-equivalent commands. 3. Determine interconnection category The interconnection category of the entire formal query is determined by examination. In the case of the running example of printing a file on the laser printer with pageheaders and lines numberered, it is (AND: file-input queue-input enqueue-output) because we are taking an input file, processing it, and then placing it into a print queue. This is easily determined because #I and %laser-print-queue are the only objects declared in the precondition, and %laser-print-queue is the only object to be mutated. 4. Send on simple results If a command has been selected which is not comparable temporally to any other command, and it has an interconnection category binding which matches the interconnection category of the entire query, then that command is passed along, together with the formal query, to the next step. In the example query, this does not occur. 5. Select and fill clichés All clichés whose interconnection categories match those of the query are identified. The slots of these clichés are filled, recursively if necessary. In the example query, the only cliché identified at this step is file->out-filter->print-queue, which is defined in Figure 28. Its print-module slot is filled immediately with the assembled lpr command. However, its initial-filter slot cannot be filled immediately because there are two commands, the assembled cat and pr commands, still to be utilized. Rather, a recursive attempt to assemble these two commands in such a way as to fill the initial-filter slot is initiated. This attempt
220
STEPHEN J. HEGNER
Figure 29. The assembled command for query solution.
easily finds that the cliché file->out-filter->in-out-filter is exactly what is needed. The two commands are used to fill the appropriate slots of this filter, and the result is used to instantiate the initial-filter of the file->out-filter->print-queue cliché. Each such full instantiation is sent on to the formal finisher. 6. Expand clichés The instantiated clichés are expanded. In the case of our example, the result is given in Figure 29. 6.4. The formal finisher The interconnected commands determined by the structural synthesizer represent potential solutions to the query. However, the actual solution, in the form of bindings to query variables, must still be determined. It is the rôle of the formal finisher to achieve this task. The algorithm which it obeys is sketched as follows. 1. Expand the interconnections The interconnections, as supplied by the structural synthesizer, are in the form of a list of simple modules to be executed in sequence. In order to ascertain
PLAN REALIZATION FOR COMPLEX COMMAND INTERACTION
221
Figure 30. Full expansion of example command interconnection.
bindings to query variables, this sequence must be expanded and then identified with a single dynamic object, which we call the solution dynamics. The result will look much like a formal query, but there will be no secondary preconditions or postconditions, the command interconnection will be bound rather than a unknown, and external variables will be replaced by command variables. The result of this process is shown in Figure 30.
222
STEPHEN J. HEGNER
2. Bind the variables Once the potential solutions have been expanded into actual dynamic formulas, it is possible to determine variable connections. This is done in two steps. 1. Unify. Variables in the solution dynamics are replaced with the corresponding ones of the formal query. In the example, the variable #Q has the value %laser-print-queue substituted for it. The command manager for lpr contains the information necessary to make the association between printer names and print queues. 2. Bind the primary unknown. The unknown command is bound to the command found in the command interconnection dynamics. In the example, the variable ?CS would is bound to the solution command sequence. 3. Identify secondary conditions. Any mismatches in the preconditions and/or postconditions must be reconciled. This is done by relegating the mismatched components to secondary conditions. In the example query, the solution dynamics contains the precondition (expand: (readable: ((name: #I) (version: 0)))) which does not match any formula in the precondition of the query. Therefore, it is relegated to be a secondary precondition, and the query variable ?SPP is bound to this entire formula. Since there are no such formulas in the postcondition, the variable ?SPQ is bound to the identically true formula. Multiple solutions All interconnections are processed in this fashion. The one which is actually returned is the one with the weakest secondary preconditions and postconditions. Contradictory conditions In relegating a formula to a secondary precondition or postcondition, it is necessary to ensure that it does not contradict any condition asserted by the formula. As long as the precondition (resp. postcondition) of the solution dynamics simply make additions to those of the formal query, no problem can arise. However, if the formal query asserted additional conditions not matched by the solution dynamics, care must be taken. For example, suppose that our example query were further expanded to ask how to print an unreadable file with pageheaders and lines numbered on the laser printer. The condition that #I be unreadable would now be asserted as a precondition of the query, in direct contradiction to the precondition of the solution dynamics. In such a circumstance, Yucca-* regards the contradictory precondition as a
PLAN REALIZATION FOR COMPLEX COMMAND INTERACTION
223
new subgoal, and formulates, within the formal finisher, two new unknown action queries, one to make the file readable, and another to restore it to its unreadable state. These two new queries are then solved, and interconnected to the original query to yield a final solution. 7. Conclusions and Project Status 7.1. Conclusions and Future Directions The power of formal semantics To model details of a system correctly, one needs to employ a deep model of that system. We have presented a detailed design for a UNIX consultation system in which the underlying model of UNIX employs a deep model based upon completely formal semantics. The major advantage of such a model is that it takes full advantage of the inherent structure of the underlying domain of command language behavior. We believe that, compared to similar systems which embed the modelling and planning mechanism into a general framework of knowledge representation and inference, Yucca-* will operate a greater level of detail with a more modest commitment of resources. Formal specification and documentation Although Yucca-* is an external utility which is to run on top of an existing system, important lessons relative to integral utilities have been learned also. Specifically, we have found many times that had the command semantics of UNIX been formally specified in the first place, before any implementation, not only would the design of the consultant been much simpler, but much anomalous behavior could have been avoided, and so many of the commands themselves would have been much more understandable, and the process of documentation would have been much more systematic. This suggests that a parallel investigation directed at the formal specfication, implementation, and documentation of UNIX commands is an important direction, which we are currently initiating. 7.2. Current Status Knowledge bases The implementation of all Yucca-* back end knowledge bases is well along. We have completed definitions for about 150 static objects, as well as about fifteen UNIX commands and five clichés. In the near future, we expect to have constructed models for a total of about 25 commands and ten to fifteen clichés. Several dozen formal queries have been hand-generated in complete detail, and we are confident of the scope and power of OSquel.
224
STEPHEN J. HEGNER
Solvers A protoptype command synthesizer is currently under development. Although some components have been been implemented in a skeletal fashion, the entire unit is not yet functional. Priority at this point is on the command synthesizer; other query solvers will be built only after the command synthesizer is operational.
8. Recent Developments on the Yucca-* Project Since the original version of this report was completed in November of 1989, there has been further work on implementation of the concepts described in the previous sections. The bulk of the completed work is reported in two M.S. theses (Rachlin 1993) and (Sila 1993). In (Rachlin 1993), John Rachlin presents an implementation of the planning mechanism for Yucca-*, written in Common Lisp. His implementation demonstrates rather conclusively that the original design, based upon item templates and command managers, is an effective one. In (Sila 1993), Noreen Sila presents an implementation of a graphical menu-driven front end for Yucca-*. The implementation uses C++-based tools, including the Forms Library and the Silicon Graphics GL Library. A key feature of such front-ends is their simplicity. Recapturing both static and dynamic queries, the key feature of her design is its simplicity. While menudriven user interfaces are not the most sophisticated, they are quite useable for a wide range of queries. She demonstrates that, using the proper tools, it is possible to construct a rather effective front-end with relatively little effort.
Notes 1 2 3 4 5
UNIX is a trademark of X/Open, Inc. TOPS-20 is a trademark of Digital Equipment Corporation. VM/CMS is a trademark of IBM Corporation. MS-DOS is a trademark of Microsoft Corporation. Actually, there is a slightly more general format, which is occasionally necessary, involving a sequence of actions. Such queries are not elaborated in this report. 6 VAX/VMS is a trademark of Digital Equipment Corporation. 7 We assume a basic familiarity with the rôle of unifiers in logical formulation. See, e.g. (Genesereth and Nilsson 1987) for details. 8 The append: operator is used to append the contents of two byte sequences together.
PLAN REALIZATION FOR COMPLEX COMMAND INTERACTION
Appendix. Formal definition of pr command
225
226
STEPHEN J. HEGNER
PLAN REALIZATION FOR COMPLEX COMMAND INTERACTION
227
References Bates, M., Moser, M. G. & Stallard, D. (1986). The IRUS Transportable Natural Language Database Interface. In Kerschberg, L. (ed.) Expert Database System, Proceedings of the First International Workshop, 617–630. Kiawah Island, SC: Benjamin/Cummings. Brachman, R. J. & Schmolze, J. G. (1985). An Overview of the KL-ONE Knowledge Representation System. Cognitive Science 9: 171–216. Brown, F. M., (ed.) (1987). The Frame Problem in Artificial Intelligence, Proceedings of the 1987 Workshop. Morgan Kaufmann. Chin, D. N. (1988). Intelligent Agents as a Basis for Natural Language Interfaces. Technical Report UCB/USD 88/396, Computer Science Division of EECS, University of California at Berkeley. Douglass, R. J. & Hegner, S. J. (1982). An Expert Consultant for the UNIX Operating System: Bridging the Gap between the User and Command Language Semantics. In Proceedings of the Fourth CSCSI/SCEIO Conference, 119–127. Saskatoon: Canadian Society for the Computational Study of Intelligence. Fenschel, R. S. & Estrin, G. (1982). Self-describing Systems Using Integral Help. IEEE Transactions on Systems, Man, and Cybernetics 12(2): 162–167. Finin, T. W. (1983). Providing Help and Advice in Task Oriented Systems. In Proceedings of the Eighth IJCAI, 176–178. Karlsruhe: Morgan-Kaufmann. Genesereth, M. R. (1979). The Role of Plans in Automated Consultation. In Proceedings of the Sixth IJCAI, 311–319. Tokyo: Morgan-Kaufmann. Genesereth, M. R. & Nilsson, N. J. (1987). Logical Foundations of Aritificial Intelligence. Morgan Kaufmann. Georgeff, M. P. & Lansky, A. L., (eds.) (1986). Reasoning about Actions and Plans, Proceedings of the 1986 Workshop. Morgan Kaufmann. Grosz, B. J. (1983). TEAM: A Transportable Natural Language Interface System. In Proceedings of the 1983 Conference on Applied Natural Language Processing, 39–45. Santa Monica: Association for Computational Linguistics. Hart, P. E. (1982). Directions for AI in the Eighties. SIGART Newsletter 79: 11–16. Hayes, P. J. (1982a). Cooperative Command Interaction Through the Cousin System. In Proceedings of the International Conference on Man/Machine Systems. University of Manchester Institute of Science and Technology.
228
STEPHEN J. HEGNER
Hayes, P. J. (1982b). Uniform Help Facilities for a Cooperative User Interface. In Proceedings of the 1982 NCC. Houston. Hayes, P. J. & Szekely, P. A. (1983). Graceful Interaction through the Cousin Command Interface. Technical Report CMU-CS-83-102, Carnegie Mellon University, Pittsburgh, PA. Hecking, M., Kemke, C., Nessen, E., Dengler, D., Gutmann, M. & Hector, G. (1988). The SINIX Consultant – A Progress Report. Technical Report Memo Nr. 28, Universität des Saarlandes, FB 10 Informatik IV. Hegner, S. J. (1988). Representation of Command Language Behavior for an Operating System Consultation Facility. In Proceedings of the Fourth IEEE Conference on Artificial Intelligence Applications, 50–55. San Diego: IEEE Computer Society. Hegner, S. J. & Douglass, R. J. (1984). Knowledge Base Design for an Operating System Expert Consultant. In Proceedings of the Fifth CSCSI/SCEIO Conference, 159–161. London: Canadian Society for the Computational Study of Intelligence. Mark, W., Wilczynski, D., Lingard, R. & Lipkis, T. (1980). Research Plans in the Area of Cooperative Interactive Systems. Technical report, USC/Information Sciences Institute. Miller, P. L. (1984). A Critiquing Approach to Expert Computer Advice: ATTENDING. Pitman: London. Motro, A. (1986). BAROQUE: A Browser for Relational Databases. ACM Transactions on Office Information Systems 4: 164–181. Neelkandan, H., Matthews, M. M. & Biswas, G. (1987). An Intelligent Assistance System in the Unix Domain. In Proceedings of Third Annual Expert Systems in Government Conference. Computer Science Press. Quilici, A., Dyer, M. & Flowers, M. (1988). Recognizing and Responding to Plan-oriented Misconceptions. Computational Linguistics 14(3): 38–51. Rachlin, J. N. (1993). Intelligent Planning in the UNIX Help Domain. Master’s thesis, University of Vermont. Rich, C. (1981). A Formal Representation for Plans in the Programmer’s Apprentice. In Proceedings of the Seventh IJCAI, 1044–1052. Vancouver: Morgan-Kaufmann. Sacerdoti, E. D. (1977). A Structure for Plans and Behavior. Elsevier North-Holland. Salveter, S. (1984). Natural Language Database Update. Technical Report TR# 84/001, Computer Science Department, Boston University. Scragg, G. W. (1975). Answering Process Questions. In Proceedings of the Fourth IJCAI, 435–442. Tbilisi: Morgan-Kaufmann. Sila, N. P. (1993). The Design of A Graphical Interface for an Intelligent UNIX Consultation System. Master’s thesis, University of Vermont. Waltz, D. L. (1978). An English Language Question Answering System for a Large Relational Database. Communicatios of the ACM 21: 526–539. Waters, R. C. (1985a). KBEmacs: Where’s the AI? AI Magazine 7(1): 47–61. Waters, R. C. (1985b). The Programmer’s Apprentice: A Session with KBEmacs. IEEE Transactions on Software Engineering 11: 1296–1320. Wilensky, R., Arens, Y. & Chin, D. (1984). Talking to UNIX in English: An Overview of UC. Communications of the ACM 27: 574–593. Wilensky, R., Chin, D. N., Luria, M., Martin, J., Mayfield, J. & Wu, D. (1988). The Berkeley UNIX Consultant Project. Computational Linguistics 14(4): 35–84. Zloof, M. M. (1977). Query-by-Example: A Data Base Language. IBM Systems Journal 16: 324–343.
An Assumption-based Truth Maintenance System in Active Aid for UNIX Users JOHN JONES1, MARK MILLINGTON2 and MARIA VIRVOU3 1 Department of Computer Science, University of Hull, Hull HU6 7RX, UK; 2 Interactive
Business Systems, Inc. 2625 Butterfield Road, Oak Brook, Illinois 60521, USA (E-mail: [email protected]); 3 Department of Computer Science, University of Piraeus, 80 Karaoli & Dimitriou St., Piraeus 18534, Greece (E-mail: [email protected])
Abstract. This paper deals with the problem of assigning meaning to the interaction of a user with a command-driven system such as UNIX. This research is part of the construction of an active intelligent help system that monitors users in order to offer spontaneous help when they are facing problems. In order to ensure this, the help system must build and maintain a model of the user. We describe a mechanism that is used by the user modelling component of such a help system. This mechanism makes explicit assumptions about the user which account for different hypotheses about what the user is actually thinking at every stage of the interaction. The consistency of these assumptions is managed by an Assumption-based Truth Maintenance System (ATMS). The selection between a number of different user models is based on the information which is extracted from the feedback that UNIX provides in response to user actions. Keywords: Assumption-based Truth Maintenance System, advice generation, Intelligent Help Systems, UNIX commands, user’s errors, user interfaces, user modelling
1. Introduction Intelligent Help Systems for UNIX are meant to provide additional on-line help to users that interact with UNIX. Depending on the kind of response of the help system they are usually classified into passive and active systems. Passive systems generate response on the user’s initiative (i.e. the user explicitly asks for help) whereas active systems on their own initiative (i.e. they spontaneously intervene in order to offer advice) when they think that the user has problems. Passive help systems are supposed to answer user’s questions in natural language such as “How can I remove a file?” or “What does the command “mv” do ?”. Quite rightly the emphasis is on issues of natural language understanding. A prototypical passive help system is the UNIX consultant (UC) (Wilensky et al. 1986). Active help systems are supposed to monitor the users’ actions and reason about them so that they can offer spontaneous help when they think that the user is in trouble. The model for
230
JOHN JONES ET AL.
this form of help system is the expert who helps out after having observed “over your shoulder” that you are having problems. Our overall goal is to construct an active help system for the UNIX operating system, in common with a number of other groups (Kemke 1986; Matthews and Pharr 1999). In a companion paper in this volume (Virvou et al. 1999) we have argued that to provide intelligent help in this domain requires considerable knowledge of the user and a thorough understanding of the interaction between the user and UNIX. The main repository for this understanding will be a user model developed and maintained by a user modelling component of the help system. Our work to date has been to identify what a properly constituted user model is. In section 2 we describe the features and problems of UNIX as a domain for user modelling for an active help system. In section 3 we briefly describe some related work. In section 4 we give the overall approach taken for modelling UNIX users. In section 5 we present our definition of a user model and approach to constructing it. In section 6 we show the representation of UNIX that underlies our prototype implementation. In sections 7 and 8 we describe our prototype implementation of the user modelling component and in section 9 we show how the prototype works on some examples. Finally, in section 10 we give the conclusions and in section 11 we describe the recent developments in this research.
2. UNIX as a Domain for User Modelling In this section we consider the nature of UNIX itself and its users because these have a considerable impact on user modelling for an active intelligent help system. It is the aim of our research to determine in detail what form a user model should take. However, at this stage we shall simply suppose that a user model contains whatever information is necessary in order to make the advice and help offered, individual to the user and her/his current circumstances. UNIX is an operating system with an elegant system design which attracts a lot of users. However, the user interface is not so elegant. Norman (1981) argues that often command names bear no relationship to their functions and have inconsistent syntax. He also argues that the system’s terseness or silence during interaction puts an additional burden on the user’s memory. Actually, the terseness of UNIX does in fact cause difficulties for the user; if an action executes without an error (as perceived by UNIX) the user is not explicitly informed of the actual effect. This can lead to difficulties where the action achieves an effect different to that which the user both intended and believes occurred. However, somewhat paradoxically, this terseness is
AN ASSUMPTION-BASED TRUTH MAINTENANCE SYSTEM
231
also one of the principal features of UNIX which leads us to believe that we can build the help system we envisage. This factor ensures that the user must explicitly seek information from UNIX in such a manner that an on-line help system can observe the query and thereby determine the user’s focus of attention. On the second count, the semantic representation of UNIX is tractable, a particularly simple representation being presented in section 6. There is a significant subdomain, consisting of the file store and commands to manipulate it such as mkdir, rmdir, cd, cp, rm, mv, ls, lpr and vi, which accounts for a significant proportion of users’ time spent using UNIX and the problems they encounter and is representative of the domain as a whole. In addition, the nature of a computer operating system is such that we can easily create an interface which can take the action issued by the user and reason about it before it is executed. This, for example, gives us the scope to avoid a disaster such as the user inadvertently deleting a file. The aim of our research is to build a system that would provide automatic help in situations where a user’s utilisation of UNIX has been found to be deficient. Deficiencies may occur at a number of levels, such as planning, semantic, environmental,1 syntactic, lexical and typographic. However, there appear to be a number of different notions of adequacy for methods of achieving goals. Users often trade off brevity and/or accuracy in favour of familiarity or (perceived) safety. A typical example of this is achieving ‘move’ by ‘copy and remove’, where the incremental nature of the latter approach allows the user to monitor the success of the individual steps in the plan. Modelling UNIX users is particularly difficult as UNIX is used by a range of users with different levels of knowledge of the system and with different goals in mind. In addition, it offers a wide range of commands and command options as well as a rich collection of methods for combining them. This leads us to think that a simple scheme for user modelling, such as one utilising stereotypes (Rich 1979), would be inadequate. Furthermore, users typically represent a moving target to a user modelling component, in that goals and plans may change and features of the system may be learnt or forgotten. Some aspects of the behaviour of users in the UNIX domain will present great difficulties for a user modeller in an active help system, in that they do not leave a trace in the dialogue between UNIX and the user. The range of such behaviour includes: • Users may arbitrarily remake choices concerning goals and plans; these often seem to be inspired by the interaction with UNIX, but in a manner that is very difficult to relate to the task in hand. For example, users may interrupt sequences of actions aimed at achieving some goal in order to see their mail.
232
• • • •
•
JOHN JONES ET AL.
We distinguish these arbitrary changes from modifications of the user’s plans and goals which become necessary when an error is detected through feedback from UNIX. Users learn by unobservable processes. For example, they seek help from other users or the (off line) manual. Users forget. Users can be selective in their attention. For example, the information gathering command ‘ls’ is often employed to acquire only a fragment of the information it actually presents. Users can be inconsistent. For example, we have observed a user2 achieve her/his aim of moving all the proper files in a particular directory into a new sub-directory called ‘ht’, by creating the new subdirectory and then using the action ‘cp ∗ ht’. The inconsistency will be present if s/he believes that the match of ‘∗’ will include ‘ht’ and that ‘cp’ cannot be used to copy directories. Users make mistakes.
3. Related Work The work which most closely resembles the present proposal for a user modelling component is GUMS (Finin and Drager 1986), in which an architecture for a domain-independent user modelling component for an application program is presented. The user model forms the basis of responses by the user modeller to queries about the user from the application program. An initial model of the user is selected from a hierarchy of stereotypes. A stereotype is a generic model of a class of users of the application program and consists of facts and rules. For example, if the application program is interacting with PC users then the stereotypical user model may assert that: typically, the user understands the term ‘rom’ typically, if the user understands the term ‘rom’ then he understands the term ‘ram’ As the interaction of the application program with the user progresses, additional information about the user is passed to the user modeller for incorporation into the user model. The incorporation of these new facts specialises the stereotype model to the individual user. However, this specialisation raises the problem of the consistency of the resulting user model. For example, with the above stereotype model, suppose the application determines that the user does indeed understand the term ‘rom’ but not the term ‘ram’. Finin
AN ASSUMPTION-BASED TRUTH MAINTENANCE SYSTEM
233
and Drager propose that the consistency of the user model is managed by Doyle’s truth maintenance system (Doyle 1981) in the context of a forwardchaining inference mechanism. A prototype help system component, GUMS, is described which employs simple default rules and backward chaining. Since the scheme Finin and Drager describe is domain-independent, they do not consider how relevant information may be extracted from the interaction between the user and the application program. Their proposal is merely concerned with the management issues involved once the information has been obtained.
4. Overall Approach The development of a user model, and its utilisation, will be over a number of natural time scales. The first major time scale, which we shall refer to as an interaction, is that in which the user issues an action to UNIX, observes the response (if any) and formulates the next action. Thus, an interaction begins when the user presses the ‘return’ key to issue an action and ends when ‘return’ is pressed again to issue the next action. We shall reserve the word command for functions provided by UNIX, and action for a particular instance of the command’s use. Thus an action will involve a command and zero or more arguments. We shall further divide an interaction into two phases: the execution phase, in which UNIX attempts to execute the action, and the feedback phase which begins when the user receives the feedback (if any) or a new shell prompt. The feedback stage is the period in which the user formulates the next action in the sequence. The second major time scale, which we shall refer to as a session, is between logging on to UNIX and the corresponding logging off. Thus, a session will consist of a number of interactions between the user and UNIX. In view of the different time scales over which we shall be observing and modelling the user, we anticipate that different parts of a user model will have differing life-spans. Thus, information concerning the user’s knowledge of UNIX, including any misconceptions, information concerning typical tasks attempted and the manner in which s/he attempts them, will be determined over, and persist for, a number of sessions. Other information will concern the current session only, and will be discarded as soon as it becomes redundant. The motivation for our overall approach comes from hand-analysis of example user sessions, some of which are described in a companion paper in this volume (Virvou et al. 1999). Most sessions were understood by means of a relatively rich interaction between a number of different sources of knowledge, such as those listed above, where no one source could by itself provide a complete understanding. Thus, the overall architecture of the help system
234
JOHN JONES ET AL.
must support diverse sources of knowledge and provide a mechanism to support their interaction in a fruitful and opportunistic manner. For these reasons, the most promising architecture is that of a blackboard system (Hayes-Roth 1983). In an intelligent help system with the blackboard architecture, a user model will consist of a collection of hypotheses in the blackboard. While the precise form of this collection of hypotheses will be described later, we shall begin by presenting some general remarks on our approach to user modelling. Modelling users in the context of an active help system means that the modeller has to extract from the interaction information on which to base the user model. We believe that important opportunities for determining elements of a user model are those events when the user receives explicit feedback from UNIX, such as an error message. If we suppose that generating an error message is never intended, then the user’s beliefs must change during the transition from the execution to the feedback phase. For example, consider the following sequence of actions, which is based on the perquish example discussed in (Virvou et al. 1999). The actions are issued in the user’s home directory, which contains the files ‘fred’ and ‘fredl’: 1% mkdir perqish 2% cp fred perqish 3% cp fredl perquish 4% cd perqish 5% mv fredl drconsult.pas mv: Cannot access fredl The most plausible explanation of the error message generated by action 5 is that the user intended action 3 to be ‘cp fredl perqish’. Interestingly, the typing mistake can be explained by either of the observations that the substring ‘qu’ is common in English or that ‘u’ is next to ‘i’ on the keyboard. The user does not realise that s/he has made a mistake, however, until the error message is received as a result of action 5. On receiving this feedback, s/he is forced to review her understanding of the interaction. We believe that this cue can also provide valuable evidence for a user modeller. Such belief revision by the user must also take place when the feedback produced by UNIX in response to an action such as ‘ls’ or ‘pwd’ is inconsistent with the user’s beliefs. For example, the user may believe that a particular file exists in the current working directory, but subsequently discover that ‘ls’ does not list it. Once again the user must revise her/his beliefs. Equally, a user modeller must revise its own user model if it is inconsistent with the feedback the user is receiving. The user modelling component we have designed will attempt to construct a correct model of what the user believes during the execution phase of an
AN ASSUMPTION-BASED TRUTH MAINTENANCE SYSTEM
235
interaction. For the construction of the user modelling component we will make certain assumptions about the user. We shall assume that the user is attentive (to the extent that s/he notes all feedback from UNIX) and that the only learning and choice revision which takes place is driven by the feedback from UNIX. In this way we rule out most of the difficulties we identified in section 2 and we concentrate only on the possibility of users making mistakes. We assume that the user reasons consistently from a monotonically increasing collection of beliefs until the feedback from UNIX is perceived to be inconsistent with that collection of beliefs. At this point, the user is forced to reappraise the situation, and her/his beliefs must undergo a nonmonotonic shift. This model will provide useful evidence for determining what is believed during the feedback phase and provide a context in which an intelligent help system can offer assistance should it be deemed necessary. In the example given above, it would be practical to indicate that the problem is with action 3 which the user “thought created ∼/perqish/fredl”, if it became apparent that the cause of the error message was not understood. Two of our criteria for an acceptable user model will be given in terms of the consistency of the hypotheses forming the user model. In view of this, we need a flexible mechanism for managing consistency. In addition, modelling the user will require belief revision on the part of the modeller and the ability to compare alternative, and incompatible user models. Thus we have chosen to manage the consistency of the blackboard with an assumption-based truth maintenance system (ATMS) (de Kleer 1984, 1986). The monotonicity of the user’s beliefs that we are assuming needs a little explication. In our user modelling, we aim to capture how the user’s beliefs change as the result of errors being uncovered. However, there are other ways in which beliefs can change which we do not mean to discount by our assumption of monotonicity. For example, if a user deletes the file mbox then s/he must once have thought it existed and now no longer believes that. If we simply represented the user’s beliefs as a collection of facts, this loss of belief would have to be represented as a loss of the corresponding fact. This is a non-monotonic change, which we have suggested should only occur when it is driven by the revelation of some form of error or misconception. We can retain the monotonicity of the user’s beliefs under such circumstance by time-stamping assertions of user beliefs. For example, the belief in the existence of the file mbox will be of the form “up to and including the nth action mbox exists”. Lack of belief in the existence of mbox after removing it with the n+1th action will be represented by the lack of an additional assertion of the form “up to and including the n+1th action mbox exists”. In this way, monotonicity of the set of user beliefs is retained while allowing
236
JOHN JONES ET AL.
the non-monotonic shifts we have identified to take place when an error or misconception is revealed. The user modelling scheme we describe and our choice of an ATMS arose out of our experience with initial prototype user modellers constructed using a blackboard shell (Jones et al. 1988). The consistency-maintenance mechanism employed by the blackboard system proved unsatisfactory for the purposes of user modelling. The principal problem was that it could not maintain several mutually inconsistent sets of active hypotheses at the same time. As with justification-based truth maintenance systems, the blackboard system maintained a single consistent set of hypotheses. For user modelling purposes, we must be able to investigate several different and incompatible interpretations of a sequence of actions and be able to swap effortlessly between them.
5. The User Modelling Mechanism In this section we present a mechanism on which to base the user modelling component of the proposed help system. This mechanism is applicable in the execution phase of an interaction when the currently favoured user model is no longer acceptable on the basis of the feedback that UNIX will provide to the user. We shall concentrate on the particular case in which the feedback is in the form of an error message. This mechanism makes critical use of a formal definition of what constitutes a user model. We stated in section 3 that a user model is to be a collection of hypotheses in the blackboard. With respect to the content of the model, we consider it necessary to include a record of how the user currently views each action issued, which constitutes the current model, as well as how they were viewed in the past, should this be different, which constitutes the past model(s). The user’s view of the actions will include the text of the actions, their semantics and how they relate to achieving her/his goal(s). We shall make no attempt to reproduce any deep structure of user’s mental representations of goals, plans and knowledge of UNIX. In the example presented in section 4, the current user model during the execution phase for action 6 (supposing that there is such an action) should record that action 3 was ‘cp fredl perquish’, whereas the past model records that it was ‘cp fredl perqish’. This is intended to correspond to the plausible explanation that until the error message was received, action 3 was believed to have been ‘cp fredl perqish’. The current model corresponds to the moment the user presses ‘return’ to initiate a new execution phase, and is considered to remain fixed throughout the execution phase.
AN ASSUMPTION-BASED TRUTH MAINTENANCE SYSTEM
237
The difference between a current model and a past model will be in the number of interactions of the session that they account for. A current model must account for all the actions issued so far, past models only account for some initial subsequence. Each past model began life as a current model. The change in status from current to past occurs during the feedback phase when the user assimilates the feedback produced by UNIX as a result of the action just executed. If that feedback is inconsistent with the current model then it becomes a past model and the user must acquire a new current model. Thus, current and past models are inconsistent and are intended to model a non-monotonic shift in the user’s view of the interaction. The user modelling component of the help system must identify the correct current user model. We present four criteria which define an acceptable current user model during the execution phase. These criteria will be used by the user modeller to monitor whether what it currently hypothesises is the current model, remains viable in the light of the latest user action. Should it not, then the user modeller must revise its hypothesis. This change of hypothesis by the modeller during the execution phase does not correspond to a supposed change by the user. As described above, such a change by the user must take place during the feedback phase. The change we are defining is where the user modeller discovers that it has not properly identified the current user model. A current user model is acceptable if it satisfies the following criteria: • Criterion 1: It is internally consistent. The user is assumed to be rational, reasoning consistently from her/his beliefs. • Criterion 2: It is externally consistent with the feedback from UNIX. The user is assumed to be attentive. This is modelled by inserting the feedback UNIX provides into the model. • Criterion 3: It is complete. It must address all the relevant issues, from the user’s typing skills to goals. • Criterion 4: It is parsimonious. The fewer non-default assumptions the better. Criteria 1 and 2 result from the assumptions about the user and the interaction detailed in section 4. Criterion 3 assures that no issues are avoided, while criterion 4 is simply desirable. The user modeller will make explicit assumptions about the user and the interaction which will be communicated to the ATMS by the blackboard system. An important function of the ATMS is to maintain a record of sets of assumptions, called nogood sets, from which the blackboard system has derived a datum equivalent to ‘false’. We shall be able to ensure that criteria 1 and 2 are satisfied by the judicious definition of two kinds of nogood set: • A particular user model cannot be based on two different versions of what the user intended to type for a particular action in the sequence.
238
JOHN JONES ET AL.
• We assume that users do not intend to produce error messages and so the assumptions underlying a proof that the user’s actions do produce one will be a nogood set. Initially the user modeller will favour default assumptions about the user and the interaction, whereby everything is assumed to be correct. Modelling proceeds monotonically until we can derive false in the current user model as the result of some inherently false collection of assumptions being made. Criterion 1 states that the resulting collection of hypotheses is unacceptable as a current user model, and so the user modeller will attempt to select an alternative model in which false is not derivable. This alternative user model will be based on non-default assumptions. The default current user model can only be identified as unacceptable on the basis of an error message. It cannot be discounted on the basis of an inconsistency between the user model and the feedback provided by UNIX in response to commands such as ‘ls’ and ‘pwd’, because there cannot be such an inconsistency. However, once a non-default interpretation has been adopted, it can be discounted in this way. Thus, for example, a current user model which is based on the non-default assumption that it is possible to copy a directory using the command ‘cp’3 is invalidated if the user later investigates the status of the copy of the supposed directory using ‘ls’ in such a manner as to discover that it is in fact a file. It is criterion 2 which suggests the modeller should detect such contradictions, and as this feedback is inserted into the user model criterion 1 suggests we have to revise the current user model. In section 7, we shall present a preliminary implementation of a user modelling component utilising this user modelling scheme.
6. Representation of the UNIX Domain It is sufficient to represent the file store and the effect of actions on it. For this purpose we use a simple STRIPS-like representation (Fikes and Nilsson 1971). The fundamental building block is that of a file or directory and a Prolog term of the form: exists(Name,Type,Contents) asserts that Name (a full UNIX path name) has type Type (file or dir; for proper file and directory respectively) and contents Contents. We shall suppose that the user’s home directory is called ‘home’. Full path names are represented internally using the left-associative binary operator ‘/’, so that the path name ‘/home/prolog/utilities’ will be rendered as ‘home/prolog/utilities’ without loss of generality. For the sake of simplicity, this prototype assumes that the user employs simple path names, which is quite reasonable for the examples we are considering. The user modeller, however, performs the bulk
AN ASSUMPTION-BASED TRUTH MAINTENANCE SYSTEM
239
of its reasoning using complete path names. The contents of a proper file will be a term, which is sufficient, for example, to allow us to represent the fact that the contents of a copy of a file are the same as the contents of the original. As yet, we do not use contents of directories in an important manner, so that they are left as unnamed variables (that is, _) in the example presented in this chapter. A state of the file store is a list of files and directories, along with a record of the current working directory of the form: cwd(Directory) where Directory will be a full path name. A configuration is a state of the file store after N actions have been executed: configuration(N,Configuration) For the analysis of a sequence of actions we must define the initial state of the user’s file store. For the perquish example cited earlier, the initial state is: initial_state([ exists(home, dir, _), exists(home/fred, file, fred), exists(home/fredl, file, fredl), cwd(home) ]). The semantics of a UNIX command is defined by giving the semantics of each of its legitimate uses. For example, ‘mv’ can be employed to move a file into a directory, to move a file to a new file in the same directory or to move a file to an existing file in the same directory. For each such use, there is a unit clause defining the preconditions required for correct execution of that instance of the command, and the effects of execution in terms of additions to, and deletions from, the current configuration of the file store. Preconditions may be both positive and negative, the latter being recorded in the list following ‘without’. The instances of ‘mv’ which rename a file (such that the new name is not that of an existing file or directory) and move a file to an existing directory (which does not contain a file of the same name) are represented by the following unit clauses: semantics(mv(A,B), [needs [exists(A,file,Content)] without [exists(B,_,_)], adds [exists(B,file,Content)], deletes [exists(A,file,Content)] ]).
240
JOHN JONES ET AL.
semantics(mv(A/File,B), [needs [exists(A/File,file,Content), exists(B,dir,_)] without [exists(B/File,_,_)], adds [exists(B/File,file,Content)], deletes [exists(A/File,file,Content)] ]). In these clauses, ‘needs’, ‘adds’ and ‘deletes’, are unary prefix operators, and ‘without’ is an infix binary operator. The semantics of instances of a command that produces error messages are determined “on the fly”, by examining the correct forms to see which precondition was not satisfied. In these cases, the ‘adds’ list will be [error(Message)], while the ‘deletes’ list will be empty. In previous prototype user modellers we have built, instances of commands that produced error messages were represented by unit clauses such as those given above. In these cases, the only ‘add’ was the appropriate error message. The present style of representation avoids the additional search problem caused by extra clauses for command semantics.
7. A Prototype Implementation of the User Modelling Component In this section we describe a prototype implementation, in Prolog, of a user modelling component based on the mechanism we described in section 5. It is outside the scope of this paper to describe the logic programming language Prolog in any detail. The interested reader should consult (Stirling and Shapiro 1986). However, a few introductory remarks may help to explain the fragments of code we shall be presenting. As with Lisp, the syntax in Prolog for data and the program is the same, and it is the context in which they are used which distinguishes between them. The fundamental syntactic unit is the term. The basic terms are constants and variables. Terms with structure, which denote relationships, consist of a functor applied to terms. Those fragments of code we present which are terminated with a fullstop are unit clauses in the Prolog program that constitutes the prototype user modeller. Their utilisation in the execution of the program is such that they encode factual relationships between data items. The remainder of a Prolog program consists of non-unit clauses which define how to compute with the unit clauses and data. Computation takes place in order to solve a goal and utilises the clauses in a backward-chaining manner. Variables in Prolog begin with an upper-case letter, the exception being the anonymous variable. The scope of a variable is the clause in which it appears.
AN ASSUMPTION-BASED TRUTH MAINTENANCE SYSTEM
241
One of the most powerful aspects of Prolog as a programming language is the pattern matching used between terms, namely unification, which can be exploited in controlling clause selection and computation equally. The remaining element of syntax to discuss is the representation in Prolog of lists. The list with members a, b and c is denoted (a,b,c). The construction and destruction of lists is achieved by the same syntactic structures to propagate the implications of the re-derivation (under a new set of assumptions) of a previously derived datum, without the inference mechanism ensuring that this is the case. Completeness is maintained in our application by the inference mechanism. The rules which construct user models utilise the following hypotheses, with their respective intended interpretations, in addition to those described above referring to the configuration: • act(I,Action): Action is the actual text of the user’s Ith action. • intended_action(I,Action): Action is the full path name version of what it is believed the user intended to type as the Ith action. This need not be the full path name version of what was actually typed. • instantiated_semantics(I,Act,Semantics): The Ith action Act has the semantics Semantics in the context of the current file store. Semantics will be a structure of the form [adds Adds, deletes Deletes], where Adds and Deletes are lists of terms of the form ‘exists(A,B,C)’ or ‘cwd(A)’ as appropriate for Act in the current configuration. • goal(Goal,Method): Goal is achieved by Method. For example: goal(move(home/fredl,home/perqish/drconsult_pas), [cp(home/fredl,home/perqish/drconsult_pas),rm(home/f redl)]) Thus, the user modeller constructs user models as collections of assertions of the above form. In addition, the following Prolog goals are utilised in the rules that perform the analysis: • expand(Act,Configuration,Expansion): The full path name version of Act in Configuration is Expansion. • corrected_version(Action,Configuration,Correction): A potential correction of the full path name action Action in Configuration is Correction. Corrections are intended to make allowance for the user making typing or spelling mistakes. This predicate makes use of type information for the arguments of UNIX commands and the current configuration of the file store to propose plausible alternatives to what was actually typed. In this context, word2 is a plausible correction of word1 if: − word2 has one more letter than wordl − word2 has one less letter than wordl − word2 is wordl with two adjacent letters interchanged − word2 is wordl with one letter changed
242
JOHN JONES ET AL.
• instantiate(Act,Configuration,Semantics): The full path name action Action has meaning Semantics in Configuration. • effect(Semantics,Configuration,NewConfiguration): NewConfiguration is the result of effecting Semantics on Configuration. • associated_goal(Act,Configuration,Goal): Goal is a likely goal to associate with Act in Configuration. The Configuration is relevant in order to be able to distinguish whether ‘mv home/a home/b’ achieves the goal of moving ‘home/a’ to ‘home/b’ or ‘home/b/a’ (that is, whether ‘b’ is a directory or not). Two kinds of explicit assumption can be made in a rule: • default(Assumption). • non_default(Assumption). with the obvious interpretations. The user modeller only makes assumptions concerning action(I,Action), that is what it is believed the user intended to type. Thus, the hypothesis intended-action(I,Act) is supported by the assumption default(action(I,Act)) or non-default(action(I,Act)), depending on whether Act is what was actually typed or what it is conjectured that the user intended to type. This distinction will be used to ensure that criterion 4 for a user model is satisfied. The ATMS does not itself distinguish between default and non-default assumptions, rather it is up to the programmer to supply the distinction. To analyse a particular sequence of actions, the prototype first loads the appropriate initial state (provided by the experimenter) and then reads UNIX actions one at a time. The effect of the rules presented below is that each action read is recorded as what the user actually typed. Then the semantics of the action issued and of possible corrections, where appropriate, are determined and the current configuration of the file store updated. Finally, a number of hypotheses about possible goals and actions that achieve them are made. The rules that perform the initial analysis are as follows: act(I,Act) & call(J is I-1) & configuration(J,Configuration) & call(expand(Act,Configuration,Expansion)) & assume(default(action(I,Expansion))) ⇒ intended_action(I,Expansion). act(I,Act) & call(J is I-1) & configuration(J,Configuration) & call((expand(Act,Configuration,Expansion), corrected_version(Expansion,Configuration,Correction)))
AN ASSUMPTION-BASED TRUTH MAINTENANCE SYSTEM
243
& assume(non-default(action(I,Correction))) ⇒ intended_action(I,Correction). intended_action(I,Act) & call(J is I-1) & configuration(J,Configuration) & call(instantiate(Act,Configuration,Semantics)) ⇒ instantiated_semantics(I,Act,Semantics). instantiated_semantics(I,Act,Semantics) & call(J is I-1) & configuration(J,Configuration) & call(effect(Semantics,Configuration,NewConfiguration)) ⇒ configuration(I,NewConfiguration). An instance of the second rule in use is as follows. Suppose that for action 2 the user types ‘cp fredl perquish’ and that the state after the first action is: [exists(home/fredl,file,fredl), exists(home/fred2,file,fred2), exists(home/perqish,dir,_), cwd(home) ]. The full path name version of what the user typed is ‘cp home/fredl home/perquish’. Since one instance of ‘cp’ takes a file as first argument and a directory as second argument, there is a directory ‘home/perqish’ in the current state and ‘perquish’ is a possible mis-type of ‘perqish’ then this rule suggests ‘cp home/fredl home/perqish’ as a possible user intention, making a non-default assumption that this is in fact the case. Finally, the current implementation has a number of rules about goals. Goals are associated with a list of actions which we shall call a method.4 We shall assume that the user has a plan designed to achieve a goal, and a method will be our approximation to that plan. An hypothesis of the form: goal(Goal,Method) asserts that Method is intended to achieve Goal. The rules associate goals with methods based initially on the commands ‘cp’, ‘mv’, and ‘rm’ only. Goals may be associated with single actions: goal(copy(A,B),[cp(A,B)]) or with a number of actions: goal(move(A,B),[cp(A,B),rm(A)]) The commands ‘cd’ and ‘mkdir’ are incorporated dynamically into methods. The presence or precise position of actions involving these commands in
244
JOHN JONES ET AL.
a plan is open to considerable variation by the user, and as such they are difficult to incorporate in explicit methods for goals, other than by enumerating all the possibilities. For example, particular instances of ‘cd’ can be altered or added to or removed from the user’s plan by utilising different path names for arguments to other actions in the plan. Actions involving ‘mkdir’ can be moved towards the beginning of an action sequence, provided that ‘mkdir Name’ does not retreat past ‘rm Name’, ‘rmdir Name’, or the creation of its parent. We have chosen to handle this flexibility during the processing of user actions by the modeller, rather than extend the number or complexity of methods. It is achieved by means of a relationship between these particular actions and actions forming the rest of the method. We call this relationship enablement, and there are two kinds. The first type is of a syntactic nature and is exemplified by the use of ‘cd’ to change the current working directory. This kind of enabling action does not contribute to the satisfaction of the precondition of the action it enables, but simplifies the way in which that action can be expressed by the user. For example, to remove a file called ‘cv’ from the subdirectory ‘papers’ the user can do ‘cd papers; rm cv’ or ‘rm papers/cv’. The use of ‘cd’ merely alters how the file ‘cv’ is referred to. The second type of enabling action does contribute to the satisfaction of the precondition of the action it enables, but is secondary to the main goal of the method. This type of enablement is typified by the use of ‘mkdir’ to create directories. For example, directories have to exist in order to allow files to be created in them, but we see the goal as being to create the file. Enablement is recorded in a method for a goal by an assertion of the form: enables(I,J). A selection of the rules in the prototype user modeller about goals are: intended_action(I,Act) & call(J is I-1) & instantiated_semantics(I,_,Semantics) & call(not Semantics = [adds [error(_)],_]) & configuration(J,Configuration) & call(associated-goal(Act,Configuration,Goal)) ⇒ goal(Goal,[act(I,Act)]). goal(Gl,Acts1) & goal(G2,Acts2) & call((not G1 == G2, combine-goals(Gl,Actsl,G2,Acts2,Goal,Acts))) ⇒ goal(Goal,Acts).
AN ASSUMPTION-BASED TRUTH MAINTENANCE SYSTEM
245
intended-action(I,cd(Dir)) & goal(copy(Dir/Name,Something),Method) & call((member(act(J,cp(Dir/Name,Something)),Method), I < J, not member(enables(I,J),Method), sort([act(I,cd(Dir)),enables(I,J)|Method],Acts))) ⇒ goal(copy(Dir/Name,Something),Acts). The first rule determines the goal associated with an act that does not produce an error message. The second combines distinct subgoals into goals. The third rule states that changing directory to the location of a file to be copied is an enabling action for the goal of copying. The Prolog goal sort/2 constructs a canonical list of actions for the hypothesised goal, including an assertion that action I enables action J. The remainder of the rules about goals simply enumerate obvious relationships between goals such as ‘copy’, ‘remove’ and ‘move’, and further enablements.
8. User Models in the Implementation The user modeller identifies the current user model by employing the four criteria we have given in section 5. A user model is a context in the ATMS. The first two criteria, concerning the internal and external consistency of the user model, are easily established for any user model through the underlying ATMS. • The internal consistency of a user model is ensured by rules that produce nogood sets for the ATMS. These rules ensure that competing hypotheses about what the user intended to type for a particular action result in different contexts, and hence different user models: default(action(I,Act)) & non_default(action(I,Act1)) ⇒ false. non_default(action(I,Act)) & non_default(action(I,Actl)) & call(not Act = Actl) ⇒ false. • The external consistency of the user model is ensured by inserting the feedback each action provokes into the user model. The underlying assumption that the user does not intend to generate an error message can then be represented by a rule stating that when an action in the current configuration produces an error message, then the set of assumptions
246
JOHN JONES ET AL.
which underly this derivation is nogood. If an error message is produced by an action then its instantiated semantics will indicate that in the “adds” list of the action there is an error message. Therefore the rule that ensures the external consistency of the user model is the following: instantiated_semantics(I,_,[adds [error(_)],_]) ⇒ false. • The third criterion for user models concerned completeness of the model. In this prototype user modeller, a complete user model is one which contains a number of assertions of goals which between them account for all the actions performed by the user. The rules are such that this definition of completeness ensures that all other appropriate issues are addressed by the user model. • The fourth, and final, criterion was that of parsimony. This is ensured in the prototype implementation by selecting collections of hypotheses which make the least number of non-default assumptions.
9. Analysis of Some Examples We shall now show how the user modeller handles a number of examples, including the perquish example which has been discussed in section 4. After each example has been introduced we show a trace of the user modeller’s behaviour. Each component of a trace consists of a user input, given immediately after the prompt %, followed by the list of assumptions underlying the current user model. In each of the following examples, we shall assume that the initial file store is the same as for the perquish example. Thus, the user has two files in the current working directory, called fred and fredl. For the first example, suppose that the user mistypes the name of the command involved: % vm fredl new_file Current model fragment is: [non_default(action(l,mv(home/fredl,home/new_file)))] We can see that the user’s typing error is detected, not simply because it is a typing error, but because it would be inconsistent to assume it was not a typing error. The current user model is based on a non-default assumption that the user intended to use the ‘mv’ command. This example exploits the knowledge the prototype has about typing errors. However, in the following example, where there is potentially a typing error, in that the user may have intended to move the file ‘fredl’ rather than ‘fred’, we see a different analysis: % mv fred another_file Current model fragment is:
AN ASSUMPTION-BASED TRUTH MAINTENANCE SYSTEM
247
[default ( action (1,mv (home/fred, home/another_file) ) ) ] The default user model is the preferred user model, even though fred is a plausible mis-type of fredl. A user model which suggests the user intended to type fredl is not selected at this stage because it makes more non-default assumptions. For the ‘perquish’ example, the file name drconsult.pas is rendered as drconsult_pas to make it a Prolog atom and the analysis is as follows: 1. % mkdir perqish Current model fragment is: [default ( action (1, mkdir (home/perqish) ) ) ] 2. % cp fred perqish Current model fragment is: [default(action(l,mkdir(home/perqish))), default(action(2,cp(home/fred,home/perqish)))] 3. % cp fredl perquish Current model fragment is: [default(action(l,mkdir(home/perqish))), default(action(2,cp(home/fred,home/perqish))), default(action(3,cp(home/fred1,home/perquish)))] 4. % cd perqish Current model fragment is: [default(action(l,mkdir(home/perqish))), default(action(2,cp(home/fred,home/perqish))), default(action(3,cp(home/fred1,home/perquish))), default(action(4,cd(home/perqish)))] 5. % mv fredl drconsult_pas Current model fragment is: [default(action(l,mkdir(home/perqish))), default(action(2,cp(home/fred,home/perqish))), non_default(action(3,cp(home/fredl,home/perqish))), default(action(4,cd(home/perqish))), default(action(5,mv(home/perqish/fredl, home/perqish/drconsult_pas)))] We can see that until action 5 the preferred interpretation is the default interpretation. Only when this becomes unacceptable as a user model according to the criteria we have given, is an alternative current model chosen. At this point, the acceptable current model is that which assumes the user
248
JOHN JONES ET AL.
intended action 3 to be ‘cp fredl perqish’. A possible alternative interpretation, that actions 3 and 5 were intended to be ‘cp fred perqish’ and ‘mv fred drconsult.pas’ respectively, is inferior to the selected current model according to criterion 4, because it makes two non-default assumptions. Thus we can see that the criteria we have given select the appropriate current user model during the execution phase based on the currently available user actions. As more becomes known of this sequence it is possible that the user modeller revises its hypotheses. The second kind of nogood set arises when having explored some line of inference an unwanted hypothesis results. In this case, the underlying set of assumptions cannot be guaranteed to lead to a contradiction, rather it depends on the particular circumstances. An example of this form of nogood set results from a configuration which records an error message. This happens in the example given above when we make the default assumption that the user intended to type ‘cp fredl perquish’ for action 3. However, the content of the assumption is not at fault here, it is what follows from it in the particular circumstances. An alternative way of considering the two types of nogood set we have identified is in terms of the number of hypothesis tests appearing in the preconditions of the corresponding rules. The first kind of nogood set correspond to rules with more than one hypothesis test, while the second kind correspond to rules with exactly one hypothesis test. The latter kind of rule identify inherently false hypotheses, while the former identify cases where the combination of several different hypotheses creates the contradiction. With respect to choosing what explicit assumptions a rule should make, the following obvious comment is appropriate. A nogood set of assumptions has to have some element(s) that distinguish it from an acceptable set of assumptions. This is particularly pertinent in the case of the second kind of nogood set indicated above where a particular set of assumptions may or may not lead to a derivation of false. This observation tells us when we need additional assumptions, but does not tell us what they should be about. That is up to the rule-writer.
10. Conclusions We have described a prototype user modelling component for an active help system for UNIX. Its underlying mechanism determines the current user model during the execution phase of an interaction. The user modeller employs four criteria to identify an acceptable user model. These criteria make essential use of the consistency of the hypotheses forming the user model. We have shown how the criteria can be utilised in a forward-chaining
AN ASSUMPTION-BASED TRUTH MAINTENANCE SYSTEM
249
rule interpreter in which consistency is managed by an assumption-based truth maintenance system. The concept of a ‘nogood’ set of assumptions proves very important in this implementation. In our discussion of the mechanism we have concentrated on inconsistencies caused by actions that result in error messages. The assumption-based truth maintenance system has provided a good foundation on which to base our approach to user modelling, in that it supports a precise definition of what constitutes an acceptable user model. However, in building an actual application there remains the difficulty of identifying the relevant assumptions and nogood sets. This is equivalent to having selected a generic knowledge representation scheme, such as semantic nets, and then having to use it to represent the required knowledge. The deficiencies of the prototype user modeller that has been described fall into two categories, relating to the general approach taken and to actual details. With respect to the general approach, the modeller constructs all possible user models without its being able to direct its attention to only a few of them. For example, in the “perquish” session it would not concentrate on the third command only, which was problematic, but would have generated all possible corrections for the second command as well. After the third command the system would have already ended up maintaining six hypothetical user models about what the last two commands could have been intended to be. Thus if there were a lot of similar names in the filestore it would run the risk of a combinatorial explosion. The system would maintain all the hypotheses generated until eventually some hypothetical sequences would end up in some error message. In addition, there is not a mechanism for generating hypotheses which would allow the modelling of possible misconceptions of the user about the semantics of UNIX commands. In summary the main limitations of the described prototype implementation, are the following: 1. It does not have a mechanism for generating hypotheses concerning misconceptions of users. 2. It does not have a mechanism for focusing on certain commands that may have been problematic. 3. It maintains all the hypotheses generated until the command-sequences end up in some error message, if they ever do. In addition, it does not address errors of the user with respect to her/his intentions that do not produce an error message.
11. Recent Developments Following the prototype implementation that has been described, there has been another prototype, called RESCUER (Virvou 1992, 1998) which was
250
JOHN JONES ET AL.
originally constructed to address the limitations mentioned above. These are addressed in the following ways: 1. RESCUER addresses errors in semantics and syntax as well as typing errors and deeper misconceptions. It has a hypotheses generation mechanism which is based on the adaptation and application of a cognitive theory about human reasoning, called “Human Plausible Reasoning theory” (Collins and Michalski 1989). This theory was originally constructed to formalise the reasoning that people use to make plausible guesses. However, RESCUER exploits the fact that plausible guesses may be incorrect and therefore it uses the theory for generating hypotheses about “plausible human errors”. 2. RESCUER evaluates each command typed with respect to certain criteria as to whether this command should attract more of its attention or not. For this reason, it has a mechanism for keeping track of users’ intentions. 3. RESCUER does not get alerted only in cases where there has been an error message. It may decide to respond even in cases where there has not been any error message but there has been evidence that something may have been wrong. For example, action 3 of the “perquish” session would indeed be considered problematic by RESCUER as soon as this was issued despite the fact that the action did not produce any error message. The full summary of RESCUER’s behaviour in the particular example is presented in (Virvou et al. 1999). For the purposes of addressing all types of user misconceptions and mistakes RESCUER maintains a more detailed representation of a user model than the system described. For RESCUER a user model consists of hypothesised user beliefs about: 1. The configuration of the filestore. 2. The command intended. 3. The semantics of the command intended. In particular, the command intended is considered by RESCUER as the command that the user meant to type rather than the command that would achieve her/his goal as considered by the previous system. The reason for this is that RESCUER needs a very detailed representation of the user’s beliefs. Therefore it needs to distinguish between the command that the user had in mind to type and the command actually typed because these can be different if a typing error is involved. On the other hand, RESCUER distinguishes between the command that the user had in mind to type and the command that would achieve her/his goal because again these can be different. RESCUER regards the command the user had in mind to type as the command intended although it may not have been a correct planning method with respect to
AN ASSUMPTION-BASED TRUTH MAINTENANCE SYSTEM
251
her/his goals. What RESCUER considers as the command intended bears the information about possible misconceptions involved.
Acknowledgements We thank Peter Norvig, Wolfgang Wahlster and Robert Wilensky for having made several helpful comments on an earlier version of this paper. We are also grateful to Stephen Hegner and Paul Mc Kevitt for their helpful comments on the current version.
Notes 1 An error at the environment level would be failure to anticipate correctly the environment
in which a particular action is executed. 2 This is discussed in the companion paper in this volume as the ht example (Virvou et al.
1999). 3 On some versions of UNIX, such as Berkeley 4.2, using cp on a directory produces a proper
file which is a byte-level copy of the directory. 4 The term ‘plan’ would be inappropriate, in that methods are relatively unstructured.
References Collins, A. & Michalski R. (1989). The Logic of Plausible Reasoning: A Core Theory. Cognitive Science 13: 1–49. de Kleer, J. (1984). Choices Without Backtracking. In Proceedings of AAAI 84, 79–85. Austin, TX. de Kleer, J. (1986). An Assumption-Based TMS. Artificial Intelligence 28: 127–162. Doyle, J. (1981). A Truth Maintenance System. Artificial Intelligence 12: 231–272. Fikes, R. E. & Nilsson, N. J. (1971). STRIPS: A New Approach to the Application of Theorem Proving to Problem Solving. Artificial Intelligence 2: 189–208. Finin, T. & Drager, D. (1986). GUMS: A General User Modelling System. Report MS-CIS86-35, Department of Computer and Information Science, University of Pennsylvannia. Hayes-Roth, B. (1983). The Blackboard Architecture: A General Framework for ProblemSolving. Report no. HPY-83-30, Stanford Computer Science Department. Jones, J., Millington, M. & Ross, P. M. (1988). A Blackboard Shell in Prolog. In Englemore, R. & Morgan, A. J. (eds.) Blackboard Systems, 533–542. Wokingham: Addison-Wesley. Kemke, C. (1986). The SINIX Consultant: Requirements, Design, and Implementation of an Intelligent Help System for a UNIX Derivative. Bericht Nr. 11, FR.10.2 Informatik IV, University of Saarland, Saarbrucken FRG. Matthews, M. & Pharr, W. (1999). Knowledge Acquisition for Active Assistance. In Hegner, S., Mc Kevitt, P., Norvig, P. & Wilensky, R. (eds.) Intelligent Systems for UNIX. Dordrecht, The Netherlands: Kluwer Academic Publishers (this volume). Norman, D. A. (1981). The Trouble With Unix. Datamation 27(12): 139–150. Rich, E. (1979). User Modelling Via Stereotypes. Cognitive Science 3: 319–354.
252
JOHN JONES ET AL.
Stirling, L. & Shapiro, E. (1986). The Art of Prolog. Cambridge: MIT Press. Virvou, M. (1992). User Modelling Using a Human Plausible Reasoning Theory. Ph.D. thesis, CSRP 251, School of Cognitive and Computing Sciences, University of Sussex, Brighton BN19QH, UK. Virvou, M. (1998). RESCUER: Intelligent Help for Plausible User Errors. In Proceedings of ED-MEDIA/ED-TELECOM 98-World Conferences on Educational Multimedia and Educational Telecommunications. Virvou, M., Jones, J. & Millington, M. (1999). Virtues and Problems of an active help system for UNIX. In Hegner, S., Mc Kevitt, P., Norvig, P. & Wilensky, R. (eds.) Intelligent Systems for UNIX. Dordrecht, The Netherlands: Kluwer Academic Publishers (this volume). Wilensky, R., Mayfield, J., Albert, A. Chin, D., Cox, C., Luria, M., Martin, J. & Wu, D. (1986). UC- A Progress Report. Report no. UCB/CSD 87/303, University of California at Berkeley, Computer Science Division (EECS).
What Do You Know about Mail? Knowledge Representation in the SINIX Consultant CHRISTEL KEMKE Universität Bielefeld, Technische Fakultät, AG Neuroinformatik, Postfach 100131, D-33501 Bielefeld (E-mail: [email protected])
Abstract. The SINIX Consultant is an intelligent help system for the SINIX operating system which answers natural language questions about SINIX concepts and commands and gives unsolicited advice to a user as well. In this paper the representation of domain knowledge in the SINIX Consultant will be discussed, i.e. the representation of concepts of the SINIX operating system in the so-called SINIX Knowledge Base. The SINIX Knowledge Base is a taxonomical hierarchy of SINIX concepts which are divided into objects and actions operating on these objects. A single concept in the knowledge base is described by a set of attributes reflecting structural or syntactical features, the use, application and purpose of the command or object, and additional information for explaining the concept to the user. The description of commands in the SINIX Knowledge Base which is mainly used in order to generate tutorial explanations and advice for the user, and the main ideas of describing SINIX objects are outlined in this paper. Keywords: help systems, intelligent user interfaces, natural language processing, tutoring, user modelling, Unix
1. Introduction The SINIX1 Consultant (SC) is an intelligent help system for the SINIX operating system which answers natural language questions about SINIX concepts and commands and occasionally gives unsolicited advice to a user.2 Example user questions are “How do I delete a file?” and “What is a directory?” and respective SC answers could be “You can use the command rm with the syntax rm to delete a file.” and “A directory contains files and/or other directories. It is possible to build a hierarchical structure of directories in SINIX. See also the related commands mkdir and rm -r.” The formulation of answers in SC depends on information from a user model which is built up during the dialog with the actual user, based on prototypical information; unsolicited advice is given in case the user enters ‘sub-optimal’ command sequences, i.e. too many commands to achieve a certain goal. The SINIX Knowledge Base as part of the SINIX Consultant provides the information necessary to produce such answers and advice for the user. The
254
CHRISTEL KEMKE
SINIX Knowledge Base (SINIX-KB) is a taxonomical hierarchy of SINIX concepts which contains descriptions of more or less abstract objects and actions of the SINIX domain. Domain concepts in the SINIX Knowledge Base on the basic level are commands and objects of the SINIX system, for example the generic objects ‘file’ and ‘directory’ and commands like ‘ls’. Higher level concepts represent more general and unspecific actions and objects, like ‘communicate-with-user’; they reflect a general user’s view of the system and correspond to natural language terms or mental model entities. A single concept in the knowledge base is described with respect to its function, structure, use, and relationships to other concepts. The main emphasis of the developed SINIX-KB was on the representation of commands, including characterizations of their application and function used for user-adequate explanations. SINIX actions are described by a set of attributes reflecting their syntactical structure, their function, and various aspects of their use and relations to other commands. These attributes are needed in order to generate tutorial explanations with a varying degree of detail based on the individual user’s state of knowledge regarding the SINIX system. The descriptions of commands given in the SINIX Knowledge Base are mainly used in order to generate tutorial explanations and advice. Besides the description of commands via a set of informal attributes, a formal representation of the semantical structure of commands and its potential use in question evaluation and planning processes will be outlined in this article. We will also describe the main aspects of the representation of objects in the SINIX Consultant.
2. Survey of the SINIX Knowledge Base The construction of the SINIX Knowledge Base – the SINIX-KB – is based on ideas of the KL-ONE representation formalism (see Brachman and Schmolze 1984) but stresses the importance of the definition of semantic primitives, i.e. concepts and their attributes (or roles) used to provide tutorial explanations of domain concepts in response to a user’s question. The current SINIX-KB is organized as a conceptual hierarchy in which the major differentiation is between objects and actions (see Figure 1).3 The main emphasis in developing the SINIX-KB was placed upon the representation of SINIX commands. Actions in the SINIX-KB are comprised of concrete SINIX commands as well as actions which tend to be more abstract and can be realized through different SINIX commands. These concrete and abstract commands are described by attributes characterizing their syntactical structure, their semantics, their use, and relationships to other commands. Objects in the SINIX-KB, which are also arranged hierarchically, serve mainly as
WHAT DO YOU KNOW ABOUT MAIL?
255
Figure 1. Structure of the SINIX knowledge base.
parameters for actions and thus provide the “material basis” for the system’s operations. They are described by attributes comprised of features representing their state, their structure and use.4 The hierarchical arrangement of SINIX concepts is mainly used to determine a command (concrete or abstract) the user had specified in her question (see section 3.1). The outlined description of commands serves mainly to provide tutorial explanations based on the user’s expertise concerning the SINIX system (see section 3.2).
3. The Representation of Commands in the SINIX KB 3.1. The action hierarchy The action part of the current SINIX-KB consists of about 300 concepts. About 180 are completely described by attributes, 90 of them being concrete SINIX commands and 50 abstract commands. In order to reduce the complexity of the representation task, we excluded shell programming as well as system-management commands. Figure 2 shows a part of the action hierarchy which can be divided into several levels. Higher level concepts reflect more general actions or objects which, in turn, correspond to less specific natural language terms and reflect the user’s general mental model of the SINIX system.
256
CHRISTEL KEMKE
Figure 2. A part of the hierarchy of SINIX actions.
Direct subconcepts of the concept SINIX-action (see Figure 2) reflect the division of commands into task areas and are therefore called themes, e.g. the concept communicate-with-user.5 Descriptions of actions at this global level are useful in answering nonspecific user questions. These arise if the user is unfamiliar with the system but at least recognizes its principal features and capabilities. The concepts on the level below correspond to “canonical” natural language terms – mainly verbs and, occasionally, objects and modifiers used for naming the subordinate commands. Concepts at this level are called abstract commands. The leaves of the hierarchy represent concrete SINIX commands determined by a command name and, occasionally, an option and one or more parameter variables. In some cases, one and only one concrete SINIX command exists which realizes an abstract command denoted by a canonical natural language term or which corresponds to a mental model entity, respectively, e.g. the enroll command, which assigns a key for receiving secret mail. In this case, there is only one level of abstract commands. In other cases, different command names exist, e.g. mail and xsend for sending mail. In these instances, a second level of abstract commands had to be introduced. This arrangement of commands is used by the Question Evaluator in determining the most specific concept which is addressed in the user’s question and whose description serves to answer it. The Question Evaluator finds this concept using the (canonical) verb representing the action, the object affected, and, occasionally, attributes and modifiers mentioned in the phrase. The Answer Formulator makes use of the hierarchical arrangement by inferring from the concept’s position in the hierarchy which information is available for explaining the concept.
WHAT DO YOU KNOW ABOUT MAIL?
257
3.2. The description of command concepts Concepts pertaining to the SINIX-KB are frame-like structures with a fixed set of attributes (slots) the values (fillers) of which are • concept names representing relations between concepts, • syntactical structures in the case of descriptions which are constructed using other attributes, e.g. the syntax of a command, and • non-analyzable strings or texts, which are processed by other system components or are directly displayed to the user, for example, a complete description text or a reference to a bitmap. The attributes describing SINIX actions (cf. Figure 3) can be classified according to three categories reflecting • syntactical aspects, • semantical aspects, and • pragmatical/tutorial aspects of the command. This is not a strict subdivision because some attributes may be used for different purposes, though the main reason for introducing them arises out of the classification. Syntactical aspects The attributes pertaining to this class are used to describe the syntactical structure of a command; they are command name, parameters, option, syntax, sample command. The ‘sample command’ is an instantation of a concrete command, i.e. a command with instantiated parameters and, occasionally, an option. Semantical aspects The attributes used for the semantical description include a description text, used in a first version of SC in order to explain the function of the command verbally, and a graphical representation illustrating this function. The graphical representation – as well as the sample command – is selected by the Answer Formulator when explaining a command to a novice user. Visual impressions should help a beginner to construct a rough mental model of the system’s functions. The formal representation of the semantics of commands which is thought to be described by attributes like ‘preconditions’, ‘main effect’, and ‘side effects’ will be outlined in section 4.2 (see also Figure 5). Pragmatical and tutorial aspects We introduced several attributes to describe pragmatical or tutorial aspects of a command. These attributes are used to provide preventive information or mnemonic help, and they are used in focussing a command being sought or learning new commands not directly addressed in a question.
258
CHRISTEL KEMKE
communicate-with-user supers parameter option synopsis description text
