INFORMATION MODELLING AND KNOWLEDGE BASES XIX
Frontiers in Artificial Intelligence and Applications FAIA covers all aspects of theoretical and applied artificial intelligence research in the form of monographs, doctoral dissertations, textbooks, handbooks and proceedings volumes. The FAIA series contains several sub-series, including “Information Modelling and Knowledge Bases” and “Knowledge-Based Intelligent Engineering Systems”. It also includes the biennial ECAI, the European Conference on Artificial Intelligence, proceedings volumes, and other ECCAI – the European Coordinating Committee on Artificial Intelligence – sponsored publications. An editorial panel of internationally well-known scholars is appointed to provide a high quality selection. Series Editors: J. Breuker, R. Dieng-Kuntz, N. Guarino, J.N. Kok, J. Liu, R. López de Mántaras, R. Mizoguchi, M. Musen, S.K. Pal and N. Zhong
Volume 166 Recently published in this series Vol. 165. A.R. Lodder and L. Mommers (Eds.), Legal Knowledge and Information Systems – JURIX 2007: The Twentieth Annual Conference Vol. 164. J.C. Augusto and D. Shapiro (Eds.), Advances in Ambient Intelligence Vol. 163. C. Angulo and L. Godo (Eds.), Artificial Intelligence Research and Development Vol. 162. T. Hirashima et al. (Eds.), Supporting Learning Flow Through Integrative Technologies Vol. 161. H. Fujita and D. Pisanelli (Eds.), New Trends in Software Methodologies, Tools and Techniques – Proceedings of the sixth SoMeT_07 Vol. 160. I. Maglogiannis et al. (Eds.), Emerging Artificial Intelligence Applications in Computer Engineering – Real World AI Systems with Applications in eHealth, HCI, Information Retrieval and Pervasive Technologies Vol. 159. E. Tyugu, Algorithms and Architectures of Artificial Intelligence Vol. 158. R. Luckin et al. (Eds.), Artificial Intelligence in Education – Building Technology Rich Learning Contexts That Work Vol. 157. B. Goertzel and P. Wang (Eds.), Advances in Artificial General Intelligence: Concepts, Architectures and Algorithms – Proceedings of the AGI Workshop 2006 Vol. 156. R.M. Colomb, Ontology and the Semantic Web Vol. 155. O. Vasilecas et al. (Eds.), Databases and Information Systems IV – Selected Papers from the Seventh International Baltic Conference DB&IS’2006 Vol. 154. M. Duží et al. (Eds.), Information Modelling and Knowledge Bases XVIII Vol. 153. Y. Vogiazou, Design for Emergence – Collaborative Social Play with Online and Location-Based Media
ISSN 0922-6389
Information Modelling and Knowledge Bases XIX
Edited by
Hannu Jaakkola Tampere University of Technology, Finland
Yasushi Kiyoki Keio University, Japan
and
Takahiro Tokuda Tokyo Institute of Technology, Japan
Amsterdam • Berlin • Oxford • Tokyo • Washington, DC
© 2008 The authors and IOS Press. All rights reserved. No part of this book may be reproduced, stored in a retrieval system, or transmitted, in any form or by any means, without prior written permission from the publisher. ISBN 978-1-58603-812-0 Library of Congress Control Number: 2007940891 Publisher IOS Press Nieuwe Hemweg 6B 1013 BG Amsterdam Netherlands fax: +31 20 687 0019 e-mail:
[email protected] Distributor in the UK and Ireland Gazelle Books Services Ltd. White Cross Mills Hightown Lancaster LA1 4XS United Kingdom fax: +44 1524 63232 e-mail:
[email protected]
Distributor in the USA and Canada IOS Press, Inc. 4502 Rachael Manor Drive Fairfax, VA 22032 USA fax: +1 703 323 3668 e-mail:
[email protected]
LEGAL NOTICE The publisher is not responsible for the use which might be made of the following information. PRINTED IN THE NETHERLANDS
Information Modelling and Knowledge Bases XIX H. Jaakkola et al. (Eds.) IOS Press, 2008 © 2008 The authors and IOS Press. All rights reserved.
v
Preface In the last decades information modelling and knowledge bases have become hot topics not only in academic communities related to information systems and computer science but also in business areas where information technology is applied. The 17th European-Japanese Conference on Information Modelling and Knowledge Bases, EJC 2007, continues the series of events that originally started as a cooperation between Japan and Finland as far back as the late 1980’s. Later (1991) the geographical scope of these conferences expanded to cover all of Europe as well as countries outside Europe other than Japan. The EJC conferences constitute a world-wide research forum for the exchange of scientific results and experiences achieved in computer science and other related disciplines using innovative methods and progressive approaches. In this way a platform has been established drawing together researches as well as practitioners dealing with information modelling and knowledge bases. Thus the main topics of the EJC conferences target the variety of themes in the domain of information modelling, conceptual analysis, design and specification of information systems, ontologies, software engineering, knowledge and process management, data and knowledge bases. We also aim at applying new progressive theories. To this end much attention is being paid also to theoretical disciplines including cognitive science, artificial intelligence, logic, linguistics and analytical philosophy. In order to achieve the EJC targets, an international programme committee selected 19 full papers, 8 short papers, 4 position papers and 3 poster papers in the course of a rigorous reviewing process including 34 submissions. The selected papers cover many areas of information modelling, namely theory of concepts, database semantics, knowledge representation, software engineering, WWW information management, context-based information retrieval, ontological technology, image databases, temporal and spatial databases, document data management, process management, and many others. The conference would not have been a success without the effort of many people and organizations. In the Programme Committee, 37 reputable researchers devoted a good deal of effort to the review process in order to select the best papers and create the EJC 2007 programme. We are very grateful to them. Professors Yasushi Kiyoki and Takehiro Tokuda were acting as co-chairs of the programme committee. The Tampere University of Technology in Pori, Finland, promoted the conference in its capacity as organizer: Professor Hannu Jaakkola acted as conference leader and Ms. Ulla Nevanranta as conference secretary. They took care of both the various practical aspects necessary for the smooth running of the conference and for arranging the conference proceedings in the form of a book. The conference is sponsored by the City of Pori, Satakunnan Osuuskauppa, Satakunnan Puhelin, Secgo Software, Nokia, Ulla Tuominen Foundation and Japan Scandinavia Sasakawa Foundation. We gratefully appreciate the efforts of everyone who lent a helping hand.
vi
We are convinced that the conference will prove to be productive and fruitful toward advancing the research and application of information modelling and knowledge bases. The Editors Hannu Jaakkola Yasushi Kiyoki Takahiro Tokuda
vii
Programme Committee Co-Chairs Hannu Jaakkola, Tampere University of Technology, Pori, Finland Hannu Kangassalo, University of Tampere, Finland Yasushi Kiyoki, Keio University, Japan Takahiro Tokuda, Tokyo Institute of Technology, Japan Members Akaishi Mina, University of Tokyo, Japan Bielikova Maria, Slovak University of Technology, Slovakia Brumen Boštjan, University of Maribor, Slovenia Carlsson Christer, Åbo Akademi, Finland Charrel Pierre-Jean, Université Toulouse2, France Chen Xing, Kanagawa Institute of Technology, Japan Ďuráková Daniela, VSB – Technical University Ostrava, Czech Republic Duží Marie, VSB – Technical University of Ostrava, Czech Republic Funyu Yutaka, Iwate Prefectural University, Japan Haav Hele-Mai, Institute of Cybernetics, Estonia Heimbürger Anneli, University of Jyväskylä, Finland Henno Jaak,Tallinn Technical University, Estonia Hosokawa Yoshihide, Nagoya Institute of Technology, Japan Iivari Juhani, University of Oulu, Finland Jaakkola Hannu, Tampere University of Technology, Pori, Finland Kalja Ahto, Tallinn Technical University, Estonia Kawaguchi Eiji, Kyushu Institute of Technology, Japan Leppänen Mauri, University of Jyväskylä, Finland Link Sebastian, Massey University, New Zealand Mikkonen Tommi, Tampere University of Technology, Finland Mirbel Isabelle, Université de Nice Sophia Antipolis, France Multisilta Jari, Tampere University of Technology, Pori, Finland Nilsson Jørgen Fischer, Denmark Technical University, Denmark Oinas-Kukkonen Harri, University of Oulu, Finland Palomäki Jari, Tampere University of Technology, Pori, Finland Pokorny Jaroslav, Charles University Prague, Czech Republic Richardsson Ita, University of Limerick, Ireland Roland Hausser, Erlangen University, Germany Sasaki Hideyasu, Ritsumeikan University, Japan Suzuki Tetsuya, Shibaura Institute of Technology, Japan Thalheim Bernhard, Kiel University, Germany Tyrväinen Pasi, University of Jyväskylä, Finland Vojtas Peter, Charles University Prague, Czech Republic Wangler Benkt, Skoevde University, Sweden
viii
Watanabe Yoshimichi, Yamanashi University, Japan Yoshida Naofumi, Komazawa University, Japan Yu Jeffery Xu, Chinese University of Hong Kong, Hong Kong Organizing Committee Professor Hannu Jaakkola, Tampere University of Technology, Pori, Finland Dept. secretary Ulla Nevanranta, Tampere University of Technology, Pori, Finland Professor Eiji Kawaguchi, Kyushu Institute of Technology, Japan Steering Committee Professor Eiji Kawaguchi, Kyushu Institute of Technology, Japan Professor Hannu Kangassalo, University of Tampere, Finland Professor Hannu Jaakkola, Tampere University of Technology, Pori, Finland Professor Setsuo Ohsuga, Japan Professor Marie Duží, VSB – Technical University of Ostrava, Czech Republic
ix
Contents Preface Hannu Jaakkola, Yasushi Kiyoki and Takahiro Tokuda Programme Committee Comparing the Use of Feature Structures in Nativism and in Database Semantics Roland Hausser Multi-Criterion Search from the Semantic Point of View (Comparing TIL and Description Logic) Marie Duží and Peter Vojtáš
v vii 1
21
A Semantic Space Creation Method with an Adaptive Axis Adjustment Mechanism for Media Data Retrieval 40 Xing Chen, Yasushi Kiyoki, Kosuke Takano and Keisuke Masuda Storyboarding Concepts for Edutainment WIS Klaus-Dieter Schewe and Bernhard Thalheim A Model of Database Components and Their Interconnection Based upon Communicating Views Stephen J. Hegner
59
79
Creating Multi-Level Reflective Reasoning Models Based on Observation of Social Problem-Solving in Infants Heikki Ruuska, Naofumi Otani, Shinya Kiriyama and Yoichi Takebayashi
100
CMO – An Ontological Framework for Academic Programs and Examination Regulations Richard Hackelbusch
114
Reusing and Composing Habitual Behavior in Video Browsing Akio Takashima and Yuzuru Tanaka
134
Concept Modeling in Multidisciplinary Research Environment Jukka Aaltonen, Ilkka Tuikkala and Mika Saloheimo
142
Extensional and Intensional Aspects of Conceptual Design Elvira Locuratolo and Jari Palomaki
160
Emergence of Language: Hidden States and Local Environments Jaak Henno
170
Frameworks for Intellectual Property Protection on Multimedia Database Systems Hideyasu Sasaki and Yasushi Kiyoki
181
x
Wavelet and Eigen-Space Feature Extraction for Classification of Metallography Images Pavel Praks, Marcin Grzegorzek, Rudolf Moravec, Ladislav Válek and Ebroul Izquierdo Semantic Knowledge Modeling in Medical Laboratory Environment for Drug Usage: CASE Study Anne Tanttari, Kimmo Salmenjoki and Lorna Uden
190
200
Towards Automatic Construction of News Directory Systems Bin Liu, Pham Van Hai, Tomoya Noro and Takehiro Tokuda
208
A System Architecture for the 7C Knowledge Environment Teppo Räisänen and Harri Oinas-Kukkonen
217
Inquiry Based Learning Environment for Children Marjatta Kangassalo and Eva Tuominen
237
A Perspective Ontology and IS Perspectives Mauri Leppänen
257
The Improvement of Data Quality – A Conceptual Model Tatjana Welzer, Izidor Golob, Boštjan Brumen, Marjan Družovec, Ivan Rozman and Hannu Jaakkola
276
Knowledge Cluster Systems for Knowledge Sharing, Analysis and Delivery Among Remote Sites Koji Zettsu, Takafumi Nakanishi, Michiaki Iwazume, Yutaka Kidawara and Yasushi Kiyoki
282
A Formal Ontology for Business Process Model TAP: Tasks-Agents-Products Souhei Ito, Shigeki Hagihara and Naoki Yonezaki
290
A Proposal for Student Modelling Based on Ontologies Angélica de Antonio, Jaime Ramírez and Julia Clemente
298
Ontology-Based Support of Knowledge Evaluation in Higher Education Andrea Kő, András Gábor, Réka Vas and Ildikó Szabó
306
When Cultures Meet: Modelling Cross-Cultural Knowledge Spaces Anneli Heimbürger
314
Process Dimension of Concepts Vaclav Repa
322
E-Government: On the Way Towards Frameworks for Application Engineering Marie-Noëlle Terrasse, Marinette Savonnet, Eric Leclercq, George Becker, Thierry Grison, Laurence Favier and Carlo Daffara
330
A Personal Web Information/Knowledge Retrieval System Hao Han and Takehiro Tokuda
338
A Personal Information Protection Model for Web Applications by Utilizing Mobile Phones Michiru Tanaka, Jun Sasaki, Yutaka Funyu and Yoshimi Teshigawara
346
xi
Manufacturing Roadmaps as Information Modelling Tools in the Knowledge Economy Augusta Maria Paci
354
Metadata Extraction and Retrieval Methods for Taste-Impressions with Bio-Sensing Technology Hanako Kariya and Yasushi Kiyoki
359
An Ontological Framework for Modeling Complex Cooperation Contexts in Organizations Bendoukha Lahouaria
379
Information Modelling and Knowledge Bases for Interoperability Solution in Security Area Ladislav Buřita and Vojtĕch Ondryhal
384
On the Construction of Ontologies Based on Natural Language Semantic Terje Aaberge
389
Author Index
395
This page intentionally left blank
Information Modelling and Knowledge Bases XIX H. Jaakkola et al. (Eds.) IOS Press, 2008 © 2008 The authors and IOS Press. All rights reserved.
1
Comparing the Use of Feature Structures in Nativism and in Database Semantics Roland Hausser Universität Erlangen-Nürnberg Abteilung Computerlinguistik (CLUE)
[email protected] Abstract Linguistics has always been a field with a great diversity of schools and sub-schools. This has naturally led to the question of whether different grammatical analyses of the same sentence are in fact equivalent or not. With the formalization of grammars as generative rule systems, beginning with the “Chomsky revolution” in the late nineteen fifties, it became possible to answer such questions in those fortunate instances in which the competing analyses were sufficiently formalized. An early example is the comparison of Context-Free Phrase Structure Grammar (CFPSG) and Bidirectional Categorial Grammar (BCG), which were shown to be weakly equivalent by Gaifman 1961. More recently, the question arose with respect to the language classes and the complexity hierarchies of Phrase Structure Grammar (PS-grammar) and of Left-Associative Grammar (LA-grammar), which were shown to be orthogonal to each other (TCS’92). Here we apply the question to the use of feature structures in contemporary schools of Nativism on the one hand, and in Database Semantics (DBS) on the other. The practical purpose is to determine whether or not the grammatical analyses of Nativism based on constituent structure can be used in Database Semantics.
1 Introduction: Constituent Structure in Nativism In contemporary linguistics, most schools are based on constituent structure analysis. Examples are GB (Chomsky 1981), LFG (Bresnan ed. 1982), GPSG (Gazdar et al. 1985), and HPSG (Pollard and Sag 1987, 1994). Related schools are DCG (Pereira and Warren 1980), FUG (Kay 1992), TAG (Vijay-Shanker and Joshi 1988), and CG (Kay 2002). For historical reasons and because of their similar goals and methods, these schools may be jointly referred to as variants of Nativism.1 Constituent structure is defined in terms of phrase structure trees which fulfill the following conditions:
1.1
D EFINITION OF C ONSTITUENT S TRUCTURE
1. Words or constituents which belong together semantically must be dominated directly and exhaustively by a node. 2. The lines of a constituent structure may not cross (non-tangling condition). 1
Nativism is so-called because it aims at characterizing the speaker-hearer’s innate knowledge of language (competence) – excluding the use of language in communication (performance).
2
R. Hausser / Comparing the Use of Feature Structures in Nativism and in Database Semantics
According to this definition, the first of the following two phrase structure trees is a linguistically correct analysis, while the second is not:
1.2
C ORRECT AND INCORRECT CONSTITUENT STRUCTURE ANALYSIS correct
incorrect
S
VP
S
SP
NP
V
NP
NP
V
NP
Julia
knows
John
Julia
knows
John
There is common agreement among Nativists that the words knows and John belong more closely together semantically than the words Julia and knows.2 Therefore, only the tree on the left is accepted as a correct grammatical analysis. Formally, however, both phrase structure trees are equally well-formed. Moreover, the number of possible trees grows exponentially with the length of the sentence.3 The problem is that such a multitude of phrase structure trees for the same sentence would be meaningless linguistically, if they were all equally correct. It is for this reason that constituent structure as defined in 1.1 is crucial for phrase structure grammar (PS-grammar): constituent structure is the only known principle4 for excluding most of the possible trees. Yet it has been known at least since 1960 (cf. Bar-Hillel 1964, p. 102) that there are certain constructions of natural language, called “discontinuous elements,” which do not fulfill the definition of constituent structure. Consider the following examples:
1.3
C ONSTITUENT S TRUCTURE PARADOX : V IOLATING CONDITION 1 S
VP
V
NP
NP
Suzy
looked
DET
N
DE
the
word
up
Here the lines do not cross, satisfying the second condition of Definition 1.1. The analysis violates the first condition, however, because the semantically related expressions looked – up, or rather the nodes V (verb) and DE (discontinuous element) dominating them, are not exhaustively dominated by a node. Instead, the node directly dominating V and DE also dominates the NP the word. 2 To someone not steeped in Nativist linguistics, these intuitions may be difficult to follow. They are related to the substitution tests of Z. Harris, who was Chomsky’s teacher. 3 If loops like A → ... A are permitted in the rewrite rules, the number of different trees over a finite sentence is infinite! 4 Historically, the definition of constituent structure is fairly recent, based on the movement and substitution tests of American Structuralism in the nineteen thirties and forties.
R. Hausser / Comparing the Use of Feature Structures in Nativism and in Database Semantics
1.4
3
C ONSTITUENT S TRUCTURE PARADOX : V IOLATING CONDITION 2 S
VP
VP
V
NP
NP
Suzy
looked
DET
N
DE
the
word
up
Here the semantically related subexpressions looked and up are dominated directly and exhaustively by a node, thus satisfying the first condition of Definition 1.1. The analysis violates the second condition, however, because the lines in the tree cross. Rather than giving up constituent structure as the intuitive basis of their analysis, the different schools of Nativism extended the formalism of context-free phrase structure with additional structures and mechanisms like transformations (Chomsky 1965), f-structures (Bresnan ed. 1982), meta-rules (Gazdar et al. 1985), constraints (Pollard and Sag 1987, 1994), the adjoining of trees (Vijay-Shanker and Joshi 1988), etc. In recent years, these efforts to extend the descriptive power of context-free phrase structure grammar have converged in the widespread use of recursive feature structures with unification. Consider the following example, which emphasizes what is common conceptually to the different variants of Nativism.
1.5
R ECURSIVE FEATURE STRUCTURES AND UNIFICATION S
phrase structure derivation
NP
VP
V Julia
knows
NP John
lexical lookup noun: Julia num: sg gen: fem
unification
result
verb: know noun: John tense: pres num: sg subj: gen: masc obj: verb: know tense: pres subj: obj: noun: John num: sg gen: masc verb: know tense: pres subj: noun: Julia num: sg gen: fem obj: noun: John num: sg gen: masc
4
R. Hausser / Comparing the Use of Feature Structures in Nativism and in Database Semantics
As in 1.2 (correct tree), the analysis begins with the start symbol S, from which the phrase structure tree is derived by substituting NP and VP for S, etc., until the terminal nodes Julia, knows, and John are reached (phrase structure derivation). Next the terminal nodes are replaced by feature structures via lexical lookup. Finally, the lexical feature structures are unified (indicated by the dotted arrows), resulting in one big recursive feature structure (result). The order of unification mirrors the derivation of the phrase structure tree. On the one hand, the use of feature structures provides for many techniques which go beyond the context free phrase structure tree, such as a differentiated lexical analysis, structure sharing (a.k.a. token identity), a truth-conditional semantic interpretation based on lambda calculus, etc. On the other hand, this method greatly increases the mathematical complexity from polynomial to exponential or undecidable. Also, the constituent structure paradox, as a violation of Definition 1.1, remains.
2 Elimination of Constituent Structure in LA-grammar Instead of maintaining constituent structure analysis when it is possible (e.g. 1.2, correct tree) and taking exception to it when it is not (e.g. 1.3), Left-Associative Grammar completely abandoned constituent structure as defined in 1.1 by adopting another, more basic principle. This principle is the time-linear structure of natural language – in accordance with de Saussure’s 1913/1972 second law (principe seconde). Time-linear means linear like time and in the direction of time. Consider the following reanalysis of Example 1.2 within Left-Associative Grammar (LAgrammar) as presented in NEWCAT’86:
2.1
T IME - LINEAR ANALYSIS OF Julia knows John IN LA- GRAMMAR Julia knows John (v)
Julia knows (a’ v)
Julia (nm)
John (nm)
knows (s3’ a’ v)
Given an input sentence or a sequence of input sentences (text), LA-grammar always combines a “sentence start,” e.g. Julia, and a “next word,” e.g. knows, into a new sentence start, e.g. Julia knows. This time-linear procedure starts with the first word and continues until there is no more next word available in the input. In LA-grammar, the intuitions about what “belongs semantically together” (which underlie the definition of constituent structure 1.1) are reinterpreted in terms of functor-argument structure and coded in categories which are defined as lists of one or more category segments. For example, in 2.1 the category segment nm (for name) of Julia cancels the first valency position s3’ (for nominative singular third person) of the category (s3’ a’ v) of knows, whereby Julia serves as the argument and knows as the functor. Then the resulting sentence start Julia knows of the category (a’ v) serves as the functor and John as the argument. The result is a complete sentence, represented as a verb without unfilled valency positions, i.e., as the category (v). Next consider the time-linear reanalysis of the example with a discontinuous element (cf. 1.3 and 1.4):
R. Hausser / Comparing the Use of Feature Structures in Nativism and in Database Semantics
2.2
5
T IME - LINEAR ANALYSIS OF Suzy looked the word up Suzy looked the word up (v)
Suzy looked the word (up’ v)
Suzy looked the (nn’ up’ v)
Suzy looked (a’ up’ v)
Suzy (nm)
up (up)
word (nn)
the (nn’ np)
looked (n’ a’ up’ v)
Here the discontinuous element up is treated like a valency filler for the valency position up’ in the lexical category (n’ a’ up’ v) of looked. Note the strictly time-linear addition of the “constituent” the word: the article the has the category (nn’ np) such that the category segment np cancels the valency position a’ in the category (a’ up’ v) of Suzy looked, while the category segment nn’ is added in the result category (nn’ up’ v) of Suzy looked the. In this way, the obligatory addition of a noun after the addition of a determiner is ensured. The time-linear analysis of LA-grammar is based on formal rules which compute possible continuations. Consider the following example (explanations in italics):
2.3
E XAMPLE OF AN LA- GRAMMAR RULE APPLICATION (i) rule name Nom+Fverb:
(ii) ss (iii) nw (NP) (NP’ X V) ⇒ | | | | (nm) (s3’ a’ v) Julia knows
(iv) ss’ (v) RP (X V) {Fverb+Main, ...} | | matching and binding (a’ v) Julia knows
An LA-grammar rule consists of (i) a rule name, here Nom+Fverb, (ii) a pattern for the sentence start ss, here (NP), (iii) a pattern for the next word nw, here (NP’ X V), (iv) a pattern for the resulting sentence start ss’, here (X V), and (v) a rule package RP, here {Fverb+Main, ...}. The patterns for (ii) ss, (iii) nw, and (iv) ss’ are coded by means of restricted variables, which are matched and vertically bound with corresponding category segments of the language input. For example, in 2.3 the variable NP at the rule level is bound to the category segment nm at the language level, the variable NP’ is bound to the category segment s3’, etc. If the matching of variables fails with respect to an input (because a variable restriction is violated), the rule application fails. If the matching of variables is successful, the categorial operation (represented by (ii) ss, (iii) nw, and (iv) ss’) is performed and a new sentence start is derived. That the categorial operation defined at the rule level can be executed at the language level is due to the vertical binding of the rule level variables to language level constants. After the successful application of an LA-grammar rule, the rules in its (v) rule package RP are applied to the resulting sentence start (iv) ss’ and a new next word. A crucial property of LA-grammar rules is that they have an external interface, defined in terms of the rule level variables and their vertical matching with language level category segments. This is in contradistinction to the rewrite rules of phrase structure grammar: they do not have any external interface because all phrase structure trees are derived from the same initial S node, based on the principle of possible substitutions.
6
R. Hausser / Comparing the Use of Feature Structures in Nativism and in Database Semantics
3 From LA-grammar to Database Semantics The external interfaces of LA-grammar rules, originally introduced for computing the possible continuations of a time-linear derivation, open the transition from a sign-oriented approach to an agent-oriented approach of natural language analysis.5 While a sign-oriented approach analyses sentences in isolation, an agent-oriented approach analyses sentences as a means to transfer information from the mind of the speaker to the mind of the hearer. In Database Semantics, LA-grammar is used for an agent-oriented approach to linguistics which aims at building an artificial cognitive agent (talking robot). This requires the design of (i) interfaces for recognition and action, (ii) a data structure suitable for storing and retrieving content, and (iii) an algorithm for (a) reading content in during recognition, (b) processing content during thought, and (c) reading content out during action. Moreover, the data structure must represent non-verbal cognition at the context level as well as verbal cognition at the language level. Finally, the two levels must interact in such a way as to model the speaker mode (mapping from the context level to the language level) and the hearer mode (mapping from the language level to the context level). Consider the representation of these requirements in the following schema:
3.1
S TRUCTURING CENTRAL COGNITION IN AGENTS WITH LANGUAGE Cognitive Agent central cognition sign recognition sign synthesis
contex recognition context action
External Reality
language component
theory of grammar
pragmatics
theory of language
context component
peripheral cognition
The interfaces of recognition and action are based on pattern matching. At the context level, the patterns are defined as concepts, which are also used for coding and storing content. At the language level, the concepts of the context level are reused as the literal meanings of content words. In this way, the lexical semantics is based on procedurally defined concepts rather than the metalanguage definitions of a truth-conditional semantics (cf. NLC’06, Chapter 2 and Section 6.2). The data structure for coding and storing content at the context level is based on flat (nonrecursive) feature structures called proplets (in analogy to “droplets”). Proplets are so-called because they serve as the basic elements of concatenated propositions. Consider the following example showing the simplified proplets representing the content resulting from an agent perceiving a barking dog (recognition) and running away (action):
3.2
C ONTEXT PROPLETS REPRESENTING dog barks. (I) run. ⎡
⎤
sur: ⎢noun: dog⎥ ⎥ ⎢ ⎥ ⎢ ⎣fnc: bark ⎦ prn: 22 5
⎡
⎤ ⎡
sur: ⎢verb: bark⎥ ⎢ ⎥ ⎢ ⎥ ⎢arg: dog ⎥ ⎢ ⎥ ⎣nc: 23 run ⎦ prn: 22
⎤
sur: ⎢verb: run ⎥ ⎢ ⎥ ⎢ ⎥ ⎢arg: moi ⎥ ⎢ ⎥ ⎣pc: 22 bark⎦ prn: 23
Clark 1996 distinguishes between the language-as-product and the language-as-action traditions.
R. Hausser / Comparing the Use of Feature Structures in Nativism and in Database Semantics
7
The semantic relation between the first two proplets is intrapropositional functor-argument structure, and is coded as follows: The first proplet with the core feature [noun: dog] specifies the associated functor with the intrapropositional continuation feature [fnc: bark], while the second proplet with the core feature [verb: bark] specifies its associated argument with [arg: dog] (bidirectional pointering). That the first and the second proplet belong to the same proposition is indicated by their having the same prn (proposition number) value, namely 22. The semantic relation between the second and the third proplet is extrapropositional coordination. That these two proplets belong to different propositions is indicated by their having different prn values, namely 22 and 23, respectively. Their coordination relation is coded in the second proplet by the extrapropositional continuation feature [nc: 23 run] and in the third proplet by [pc: 22 bark], whereby the attributes nc and pc stand for “next conjunct” and “previous conjunct,” respectively. The values of the nc and pc attributes are the proposition number and the core value of the verb of the coordinated proposition. By coding the semantic relations between proplets solely in terms of attributes and their values, proplets can be stored and retrieved according to the needs of one’s database, without any of the graphical restrictions induced by phrase structure trees. Furthermore, by using similar proplet at the levels of language and context, the matching between the two levels during language interpretation (hearer mode) and language production (speaker mode) is structurally straightforward. Consider the following example in which the context level content of 3.2 is matched with corresponding language proplets containing German surfaces:
3.3
M ATCHING BETWEEN THE LANGUAGE AND THE CONTEXT LEVEL
sur: Hund language level: noun: dog (horizontal relations) fnc: bark prn: 122
sur: bellt verb: bark arg: dog nc: 123 run prn: 122
internal sur: context level: noun: dog (horizontal relations) fnc: bark prn: 22
sur: fliehe verb: run arg: moi pc: 122 bark prn: 123
matching
sur: verb: bark arg: dog nc: 23 run prn: 22
(vertical relations)
sur: verb: run arg: moi pc: 22 bark prn: 23
The proplets at the language and the context level are alike except that the sur (surface) attributes of context proplets have an empty value, while those of the language proplets have a language-dependent surface, e.g. Hund, as value. On both levels, the intra- and extrapropositional relations are coded by means of attribute values (horizontal relations, indicated by dotted lines). The reference relation between corresponding proplets at the two levels, in contrast, is based on matching (vertical relations, indicated by double arrows). Simply speaking, the matching between a language and a context proplet is successful if they have the same attributes and their values are compatible. Even though the vertical matching takes place between individual proplets, the horizontal semantic relations holding between the proplets at each of the two levels are taken into account as well. Assume, for example, that the noun proplet dog at the language level has the fnc value bark, while the corresponding proplet at the context level had the fnc value sleep. In this case, the two proplets would be vertically incompatible – due to their horizontal relations to different verbs, coded as different values of their respective fnc attributes. Having described the data structure of Database Semantics, let us turn next to its algorithm. For natural language communication, the time-linear algorithm of LA-grammar is used in three different variants: (i) in the hearer mode, an LA-hear grammar interprets sentences of natural language as sets of proplets ready to be stored in the database of the cognitive agent,
8
R. Hausser / Comparing the Use of Feature Structures in Nativism and in Database Semantics
(ii) in the think mode, an LA-think grammar navigates along the semantic relations between proplets, and (iii) in the speaker mode an LA-speak grammar verbalizes the proplets traversed in the think mode as surfaces of a natural language. Consider the following LA-hear derivation of Julia knows John in Database Semantics.
3.4
T IME - LINEAR HEARER - MODE ANALYSIS OF Julia knows John Julia
knows
John
lexical lookup noun: Julia fnc: prn:
verb: know arg: prn:
noun: John fnc: prn:
syntactic−semantic parsing: 1
2
noun: Julia fnc: prn: 1
verb: know arg: prn:
noun: Julia fnc: know prn: 1
verb: know arg: Julia prn: 1
noun: John fnc: prn:
result of syntactic−semantic parsing: noun: Julia fnc: know prn: 1
verb: know arg: Julia John prn: 1
noun: John fnc: know prn: 1
This derivation is similar to 2.1 in that it is strictly time-linear. The differences are mostly in the format. While 2.1 must be read bottom up, 3.4 starts with the lookup of lexical proplets and must be read top down. Furthermore, while the ss and nw in 2.1 each consist of a surface and a category defined as a list, the ss and nw in 3.4 consist of proplets. Finally, while the output of 2.1 is the whole derivation (like a tree in a sign-oriented approach), the output of 3.4 is a set of proplets (cf. result) ready to be stored in the database. The rules of an LA-hear grammar have patterns for matching proplets rather than categories (as in 2.3). This is illustrated by the following example (explanations in italics):
3.5
E XAMPLE OF AN LA-hear RULE APPLICATION (i) rule name (ii) ss-pattern
rule level
NOM+FV:
noun: α fnc:
⎡
verb: β arg:
matching and binding
noun: Julia
proplet level
(iii) nw-pattern (iv) operations (v) rule package
⎢ ⎣fnc:
prn: 1
copy α nw.arg {FV+OBJ, ...} copy β ss.fnc
⎤
⎡
⎤
⎥ ⎦
⎢ ⎣arg:
⎥ ⎦
verb: know prn:
This rule resembles the one illustrated in 2.3 in that it consists of (i) a rule name, (ii) a pattern for the ss, (iii) a pattern of the nw, and (v) a rule package. It differs from 2.3, however, in that the resulting sentence start (iv) ss’ is replaced by a set of operations. During matching, the variables, here α and β, of the rule level are vertically bound to corresponding values at the proplet level. This is the basis for executing the rule level operations at the proplet level. In 3.5, the operations code the functor-argument relation between the subject and the verb by copying the core value of the noun into the arg slot of the verb and the core value of the verb into the fnc slot of the noun. In the schematic derivation 3.4, the copying is indicated by the arrows. The result of the rule application 3.5 is as follows:
R. Hausser / Comparing the Use of Feature Structures in Nativism and in Database Semantics
3.6
9
R ESULT OF THE LA-hear RULE APPLICATION SHOWN IN 3.5 ⎡
⎤ ⎡
⎤
noun: Julia verb: know ⎢ ⎥ ⎢ ⎥ ⎣fnc: know ⎦ ⎣arg: Julia ⎦ prn: 1 prn: 1
In the next time-linear combination, the current result serves as the sentence start, while lexical lookup provides the proplet John as the next word (cf. 3.4, line 2). The example with a discontinuous element (cf. 2.2 and 2.3) is reanalyzed in the hearer mode of Database Semantics as follows:
3.7
H EARER MODE ANALYSIS OF Suzy looked the word up looked
Suzy
the
word
up
lexical lookup noun: Suzy fnc: prn:
verb: look a_1 arg: prn:
noun: n_1 fnc: prn:
noun: word fnc: prn:
adj: up mdd: prn:
syntactic−semantic parsing: noun: Suzy fnc: prn: 2
verb: look a_1 arg: prn:
2
noun: Suzy fnc: look a_1 prn: 2
verb: look a_1 arg: Suzy prn: 2
noun: n_1 fnc: prn:
3
noun: Suzy fnc: look a_1 prn: 2
verb: look a_1 arg: Suzy n_1 prn: 2
noun: n_1 fnc: look a_1 prn: 2
4
noun: Suzy fnc: look a_1 prn: 2
verb: look a_1 arg: Suzy word prn: 2
noun: word fnc: look a_1 prn: 2
5
noun: Suzy fnc: look a_1 prn: 2
verb: look a_1 arg: Suzy word prn: 2
noun: word fnc: look a_1 prn: 2
1
noun: word fnc: prn:
adj: up mdd: prn:
result of syntactic−semantic parsing: noun: Suzy fnc: look up prn: 2
verb: look up arg: Suzy word prn: 2
noun: word fnc: look up prn: 2
One difference to the earlier LA-grammar analysis 2.2 is the handling of the determiner the. In its lexical analysis, the core value is the substitution value n_1. In line 2, this value is copied into the arg slot of look and the core value of look is copied into the fnc slot of the. In line 3, the core value of word is used to substitute all occurrences of the substitution value n_1, after which the nw proplet is discarded. This method is called function word absorption. An inverse kind of function word absorption is the treatment of the discontinuous element up. It is lexically analyzed as a standard preposition with the core attribute adj (cf. NLC’06, Chapter 15). In line 5, this preposition is absorbed into the verb, based on a suitable substitution value. Thus, a sentence consisting of five words is represented by only three proplets.
4 The Cycle of Natural Language Communication In Database Semantics, the proplets resulting from an LA-hear derivation are stored in alphabetically ordered token lines, called a word bank. Each token line begins with a concept,
10
R. Hausser / Comparing the Use of Feature Structures in Nativism and in Database Semantics
corresponding to the owner record of a classic network database, followed by all proplets containing this concept as their core value in the order of their occurrence, serving as the member records of a network database (cf. Elmasri and Navathe 1989). Consider the following example.
4.1
T RANSFER OF CONTENT FROM THE SPEAKER TO THE HEARER
sign Julia
John
noun: John fnc: know prn: 1
Julia
noun: Julia fnc: know prn: 1
knows
John
noun: John fnc: know prn: 1
noun: Julia fnc: know prn: 1 verb: know arg: Julia John prn: 1
verb: know arg: Julia John prn: 1
know
hearer: key−word−based storage
speaker: retrieval−based navigation
The word bank of the agent in the hearer mode (left) shows the token lines resulting from the LA-hear derivation 3.4. Due to the alphabetical ordering of the token lines, the sequencing of the proplets resulting from the LA-hear derivation is lost. Nevertheless, the semantic relations between them are maintained, due to their common prn value and the coding of the functor-argument structure in terms of attributes and values. The word bank of the agent in the speaker mode (right) contains the same proplets as the word bank on the left. Here a linear order is reintroduced by means of a navigation along the semantic relations defined between the proplets. This navigation from one proplet to the next serves as a model of thought and as the conceptualization of the speaker, i.e., as the specification of what to say and how to say it. The navigation from one proplet to the next is powered by an LA-think grammar. Consider the following rule application:
4.2
E XAMPLE OF AN LA-think RULE APPLICATION (i) rule name (ii) ss pattern ⎡
rule level
V_N_V:
verb: β
⎤
⎢ ⎥ ⎣arg: X α Y⎦
(iii) nw pattern (iv) operations
⎡
noun: α
⎢ ⎥ ⎣fnc: β ⎦
prn: k prn: k matching and binding ⎡
proplet level
verb: know
⎤
⎢ ⎥ ⎣arg: Julia John⎦
prn: 1
⎤
output position ss mark α ss
(v) rule package {V_N_V, ...}
R. Hausser / Comparing the Use of Feature Structures in Nativism and in Database Semantics
11
By binding the variables β, α, and k to know, Julia, and 1, respectively, the next word pattern is specified at the rule level such that the retrieval mechanism of the database can retrieve (navigate to, traverse, activate, touch) the correct continuation at the proplet level:
4.3
R ESULT OF THE LA-think RULE APPLICATION ⎡
⎤ ⎡
⎤
verb: know noun: Julia ⎢ ⎥ ⎢ ⎥ ⎣arg: !Julia John⎦ ⎣fnc: know ⎦ prn: 1 prn: 1
In order to prevent repeated traversal of the same proplet,6 the arg value currently retrieved is marked with “!” (cf. NLC’06, p. 44). The autonomous navigation through the content of a word bank, powered by the rules of an LA-think grammar, is used not only for conceptualization in the speaker mode, but also for inferencing and reasoning in general. Providing a data structure suitable to (i) support navigation was one of the four main motivations for changing from the NEWCAT’86 notation of LA-grammar illustrated in 2.1, 2.2, and 2.3 to the NLC’06 notation illustrated in 3.4, 3.5, and 3.7. The other three motivations are (ii) the matching between the levels of language and context (cf. 3.3), (iii) a more detailed specification of lexical items, and (iv) a descriptively more powerful and more transparent presentation of the semantic relations, i.e., functor-argument structure, coordination, and coreference. A conceptualization defined as a time-linear navigation through content makes language production relatively straightforward: If the speaker decides to communicate a navigation to the hearer, the core values of the proplets traversed by the navigation are translated into their language-dependent counterparts and realized as external signs. In addition to this languagedependent lexicalization of the universal navigation, the language production system must provide language-dependent 1. word order, 2. function word precipitation (as the inverse of function word absorption), 3. word form selection for proper agreement. These tasks are handled by language-dependent LA-speak grammars in combination with language-dependent word form production. As an example of handling word order consider the production of the sentence Julia knows John from the set of proplets derived in 3.4:
4.4
P ROPLETS UNDERLYING LANGUAGE PRODUCTION ⎡
⎤ ⎡
⎤ ⎡
⎤
noun: Julia noun: John verb: know ⎥ ⎢ ⎥ ⎢ ⎥ ⎢ ⎣arg: Julia John⎦ ⎣fnc: know ⎦ ⎣fnc: know ⎦ prn: 1 prn: 1 prn: 1
Assuming that the navigation traverses the set by going from the verb to the subject noun to the object noun, the resulting sequence may be represented abstractly as VNN. Starting the navigation with the verb rather than the subject is because the connection between propositions is coded by the nc and pc features of the verb (cf. 3.2 and NLC’06, Appendix A2). Assuming that n stands for a name, fv for a finite verb, and p for punctuation, the time-linear derivation of an abstract n fv n p surface from a VNN proplet sequence is based on the following incremental realization: 6
Relapse, see tracking principles, FoCL’99, p. 454.
12
R. Hausser / Comparing the Use of Feature Structures in Nativism and in Database Semantics
4.5
S CHEMATIC PRODUCTION OF Julia knows John. activated sequence
realization
i V i.1 V i.2 fv V i.3 fv V i.4 fv p V
n N n N n n N N n n N N
n n fv n fv n n fv n p
In line i.1, the derivation begins with a navigation from V to N, based on LA-think. Also, the N proplet is realized as the n Julia by LA-speak. In line i.2, the V proplet is realized as the fv knows by LA-speak. In line i.3, LA-think continues the navigation to the second N proplet, which is realized as the n John by LA-speak. In line i.4, finally, LA-speak realizes the p . from the V proplet. This method can be used to realize not only a subject–verb–object surface (SVO) as in the above example, but also an SOV and (trivially) a VSO surface. It is based on the following principles:
4.6
P RINCIPLES FOR REALIZING SURFACES FROM A PROPLET SEQUENCE
• Earlier surfaces may be produced from later proplets. Example: The initial n surface is achieved by realizing the second proplet in the activated VN sequence first (cf. line i.1 in 4.5 above). • Later surfaces may be produced from earlier proplets. Example: The final punctuation p (full stop) is realized from the first proplet in the VNN sequence (cf. line i.4 in 4.5 above). Next consider the derivation of Suzy looked the word up., represented as an abstract n fv d nn de p surface, whereby n stands for a name, fv for a finite verb, d for a determiner, nn for a noun, de for a discontinuous element, and p for punctuation.
4.7
S CHEMATIC PRODUCTION OF Suzy looked the word up. activated sequence
realization
i V i.1 i.2 i.3 i.4 i.5 i.6
V fv V fv V fv V fv de V fv de p V
n N n N n d N N n d nn N N n d nn N N n d nn N N
n n fv n fv d n fv d nn n fv d nn de n fv d nn de p
R. Hausser / Comparing the Use of Feature Structures in Nativism and in Database Semantics
13
This derivation of an abstract n fv d nn de p surface from an underlying VNN navigation shows two7 instances of function word precipitation: (i) of the determiner the from the second N proplet, and (ii) of the discontinuous element up from the initial V proplet.
5 “Constituent Structure” in Database Semantics? The correlation of the activated VNN sequence and the associated surfaces shown in line i.6 (left) of 4.7 may be spelled out more specifically as follows:
5.1
S URFACES REALIZED FROM PROPLETS IN A TRAVERSED SEQUENCE fv
de
p
n
d
nn
look
up
.
Suzy
the
word
verb: look up arg: Suzy word prn: 1
noun: Suzy fnc: look up prn: 1
noun: word fnc: look up prn: 1
This structure is like a constituent structure insofar as what belongs together semantically (cf. 1.1, condition 1) is realized from a single proplet. Like a deep structure in Chomsky 1965, however, the sequence fv de p n d nn of 5.1 does not constitute a well-formed surface. What is needed here is a transition to the well-formed surface sequence n fv d nn de p:
5.2
S URFACE ORDER RESULTING FROM AN INCREMENTAL REALIZATION n
fv
Suzy
look
verb: look up arg: Suzy word prn: 1
noun: Suzy fnc: look up prn: 1
nn
de
p
word
up
.
d the
noun: word fnc: look up prn: 1
Instead of using a direct mapping like a transformation, Database Semantics establishes the correlation between the “deep” fv de p n d nn sequence 5.1 and the “surface” n fv d nn de p sequence 5.2 by means of a time-linear LA-think navigation with an associated incremental LA-speak surface realization, as shown schematically in 4.7 (for the explicit definition of the complete DBS1 and DBS2 systems of Database Semantics see NLC’06, Chapters 11–14). Note, however, that this “rediscovery” of constituent structure in the speaker mode of Database Semantics applies to the intuitions supported by the substitution and movement tests by Bloomfield 1933 and Harris 1951 (cf. FoCL’99, p. 155 f.), but not to the formal Definition 1.1 based on phrase structure trees. Nevertheless, given the extensive linguistic literature within phrase-structure-based Nativism, let us consider the possibility of translating formal constituent structures into proplets of Database Semantics.
6 On Mapping Phrase Structure Trees into Proplets Any context-free phrase structure tree may be translated into a recursive feature structure. A straightforward procedure is to define each node in the tree as a feature structure with the attributes node, up, and down. The value of the attribute node is the name of a node in 7
Actually, there is a third instance, namely the precipitation of the punctuation p from the V proplet.
14
R. Hausser / Comparing the Use of Feature Structures in Nativism and in Database Semantics
the tree, for example node: S. The value of the attribute up specifies the next higher node, while the value of the attribute down specifies the next lower nodes. The linear precedence in the tree is coded over the order of the down values. Furthermore, the root node S is formally characterized by having an empty up value, while the terminal nodes are formally characterized by having empty down values. Consider the following example of systematically recoding the phrase structure tree 1.2 (correct) as a recursive feature structure:
6.1
R ECODING A TREE AS A RECURSIVE FEATURE STRUCTURE ⎤
⎡
node: S
⎢up: ⎢ ⎢ ⎢ ⎢ ⎢ ⎢ ⎢ ⎢ ⎢down: ⎢ ⎢ ⎢ ⎢ ⎣
⎥ ⎤⎥ ⎥ node: VP ⎥ ⎥⎥ ⎢up: S ⎥ ⎥ ⎢ ⎤⎡ ⎤⎥⎥ ⎡ ⎢ ⎥⎥ ⎢ node: V node: NP ⎥ ⎢ ⎥ ⎢up: VP ⎥⎥ ⎢up: VP ⎥⎥ ⎢ ⎢ ⎢ ⎥⎥ ⎢ ⎡ ⎤⎥ ⎡ ⎤⎥ ⎥ ⎢ ⎥ ⎥ ⎢ ⎢down: ⎢ node: knows ⎥ ⎢ node: John ⎥⎥ ⎥ ⎢ ⎥⎢ ⎥⎥ ⎢ ⎥⎥ ⎢ ⎢ ⎥ ⎢ ⎥ ⎣down: ⎣up: V ⎣ ⎦⎦ ⎣down: ⎣up: NP ⎦ ⎦⎦⎦ ⎡
⎡
⎤
node: NP
⎢up: S ⎥ ⎢ ⎡ ⎤⎥ ⎢ ⎥ node: Julia ⎥ ⎢ ⎢ ⎥⎥ ⎣down: ⎢ ⎣up: NP ⎦⎦
down:
down:
down:
The translation of a phrase structure tree into a recursive feature structure leaves ample room for additional attributes, e.g., phon or synsem, as used by the various schools of Nativism. Furthermore, the recursive feature structure may be recoded as a set of non-recursive feature structures, i.e., proplets. The procedure consists in recursively replacing each value consisting of a feature structure by its elementary node value, as shown below:
6.2
R ECODING 6.1 AS A SET OF PROPLETS
non-terminal nodes ⎡ ⎤ ⎡ ⎤ ⎡ ⎤ ⎡ ⎤⎡ ⎤ node: S node: NP node: VP node: V node: NP ⎥ ⎥ ⎢ ⎥ ⎢ ⎥⎢ ⎢ ⎥ ⎢ ⎦ ⎣up: S ⎦ ⎦ ⎣up: VP ⎦ ⎣up: VP ⎣up: ⎦ ⎣up: S down: knows down: John down: NP VP down: Julia down: V NP terminal nodes ⎡ ⎤ ⎡
⎤ ⎡
⎤
node: knows node: John node: Julia ⎥ ⎢ ⎥ ⎢ ⎥ ⎢ ⎦ ⎣up: NP ⎦ ⎣up: V ⎦ ⎣up: NP down: down: down:
Formally, these proplets may be stored and retrieved in a word bank like the ones shown in Example 4.1. The mapping from phrase structure trees to recursive feature structures (e.g., 6.1) to sets of proplets (e.g., 6.2) is not symmetric, however, because there are structures which can be easily coded as a set of proplets, but have no natural representation as a phrase structure tree. This applies, for instance, to a straight line, as in the following example:
6.3
G RAPHICAL REPRESENTATION OF A LINE H
I
J
K
Such a line has no natural representation as a phrase structure tree, but it does as a set of of proplets, as in the following definition:
R. Hausser / Comparing the Use of Feature Structures in Nativism and in Database Semantics
6.4
15
R ECODING THE LINE 6.3 AS A SET OF PROPLETS ⎡
⎤
⎡
⎤ ⎡
line: H ⎢ ⎥ ⎣prev: ⎦ next: I
start
line: I
intermediate
line: J
⎤
⎥ ⎢ ⎥ ⎢ ⎣prev: H⎦ ⎣prev: I ⎦
next: J ⎡
line: K
next: K ⎤
⎥ ⎢ ⎣prev: J ⎦
finish
next:
The beginning of the line is characterized by the unique proplet with an empty prev attribute, while the end is characterized by the unique proplet with an empty next attribute.8 Proplets of this kind are used in Database Semantics for the linguistic analysis of coordination. The asymmetry between the expressive power of phrase structure trees and proplets must be seen in light of the fact that the language and complexity hierarchy of substitution-based phrase structure grammar (also called the Chomsky hierarchy) is orthogonal to the language and complexity hierarchy of time-linear LA-grammar (cf. TCS’92 and FoCL’99, Part II). For example, while the formal languages ak bk and ak bk ck are in different complexity classes in phrase structure grammar, namely polynomial versus exponential, they are in the same class in LA-grammar, namely linear. Conversely, while the formal languages ak bk and HCFL are in the same complexity class in phrase structure grammar, namely polynomial, they are in different classes in LA-grammar, namely linear versus exponential.
7 Possibilities of Constructing Equivalences Regarding the use of feature structures, the most obvious difference between Nativism and Database Semantics are recursive feature structures in Nativism (cf. 1.5) and flat feature structures in Database Semantics (cf. results in 3.4 and 3.7). The recursive feature structures of Nativism are motivated by the constituent structure of the associated phrase structure trees, while the flat feature structures (proplets) of Database Semantics are motivated by the task of providing (i) a well-defined matching procedure between the language and the context level (cf. 3.3) and (ii) a time-linear storage of content in the hearer mode, a time-linear navigation in the think mode, and a time-linear production in the speaker mode (cf. 4.1). 8
Another structure unsuitable for representation as a phrase structure is a circle: K
H
J
I
There is no natural beginning and no natural end, as shown by the following definition as a set of proplets: ⎡ ⎤ ⎡ ⎤ ⎤ ⎡ ⎤ ⎡
arc: H
arc: I
arc: J
arc: K
next: I
next: J
next: K
next: H
⎥ ⎥ ⎢ ⎢ ⎥ ⎢ ⎥ ⎢ ⎣prev: K⎦ ⎣prev: H⎦ ⎣prev: I ⎦ ⎣prev: J ⎦ In this set, none of the proplets has an empty prev or next attribute, thus aptly characterizing the essential nature of a circle as compared to a line (cf. Example 6.4).
16
R. Hausser / Comparing the Use of Feature Structures in Nativism and in Database Semantics
These differences do not in themselves preclude the possibility of equivalences between the two systems, however. Given our purpose to discover common ground, we found that phrase structure trees and the associated recursive feature structures (cf. 6.1) can be systematically translated into equivalent sets of proplets (cf. 6.2), thus providing Nativism with a data structure originally developed for matching, indexing, storage, and retrieval in Database Semantics. Furthermore, we have seen that something like constituent structure is present in Database Semantics, namely the correlation of semantically related surfaces to the proplet from which they are realized in the speaker mode (cf. 5.1). How then should we approach the possible construction of equivalences between the two systems? From a structural point of view, there are two basic possibilities: to either look for an equivalence between corresponding components of the two systems (small solution), or to make the two candidates more equal by adding or subtracting components (large solution). Regarding a possible equivalence of corresponding components (small solution), a comparison is difficult. Relative to which parameters should the equivalence be defined: Complexity? Functionality? Grammatical insight? Data coverage? Language acquisition? Typology? Neurology? Ethology? Robotics? Some of these might be rather difficult to decide, requiring lengthy arguments which would exceed the limits of this paper. So let us see if there are some parts in one system which are missing in the other. This would provide us with the opportunity to add the component in question to the other system, thus moving inadvertently to a large solution for constructing an equivalence. Beginning with Nativism, we find the components of a universal base generated by the rules of a context-free phrase structure grammar, constrained by constituent structure, and mapped by transformations or similar mechanisms into the grammatical surfaces of the natural language in question. These components have taken different forms and are propagated by different linguistic schools. Their absence in Database Semantics raises the question of how to take care of what the components of Nativism have been designed to do. Thereby, two aspects must be distinguished: (i) the characterization of wellformedness and (ii) the characterization of innateness. For Chomsky, these are inseparable because without a characterization of innateness there are too many ways to characterize wellformedness.9 For Database Semantics, in contrast, the job of characterizing syntactical and semantical wellformedness is treated as a side-effect which results naturally from a well-functioning mechanism of interpreting and producing natural language during communication.
8 Can Nativism be Turned into an Agent-oriented Approach? Next let us turn to components which are absent in Nativism.10 Their presence in DBS follows from the purpose of building a talking robot. The components, distinctions, and procedures in question are the external interfaces for recognition and action (cf. 3.1), a data structure with an associated algorithm modeling the hearer mode and the speaker mode (cf. 4.1), a systematic distinction between the language and the context level as well as their correlation in terms of matching (cf. 3.3), inferences at the context level (cf. NLC’06, Chapter 5), turntaking, etc., all of which are necessary in addition to the grammatical component proper. Extending Nativism by adding these components raises two challenges: (i) the technical problem of connecting the historically grown phrase structure system with the new compo9 This problem is reminiscent of selecting the “right” phrase structure tree from a large number of possible trees (cf. 1.2), using the principle of constituent structure. 10 They are also absent in truth-conditional semantics relative to a set-theoretical model defined in a metalanguage, which has been adopted as Nativism’s favorite method of semantic interpretation.
R. Hausser / Comparing the Use of Feature Structures in Nativism and in Database Semantics
17
nents and (ii) finding a meaningful functional interaction between the original system and the new components. Regarding (i), there is the familiar problem of the missing external interfaces: how should a phrase structure system with transformations or the like be integrated into a computational model of the hearer mode and the speaker mode? Regarding (ii), it must be noted that Chomsky and others have emphasized again and again that Nativism is not intended to model the use of language in communication. Nevertheless, an extension of Nativism to an agent-oriented system would have great theoretical and practical advantages. For the theory, it would substantially broaden the empirical base,11 and for the practical applications, it would provide a wide range of much needed new functionalities such as procedures modeling the speaker mode and the hearer mode. Let us therefore return to the possibility of translating phrase structure trees systematically into proplets (cf. 6.1 and 6.2). Is this formal possibility enough to turn Nativism into an agent-oriented system? The answer is simple: while the translation in question is a necessary condition for providing Nativism with an effective method for matching, indexing, storage, and retrieval, it is not a sufficient condition. What is needed in addition is that the connections between the proplets (i) characterize the basic semantic relations of functor-argument structure and coordination as simply and directly as possible and (ii) support the navigation along these semantic relations in a manner which is as language-independent as possible. For these requirements, constituent structure presents two insuperable obstacles, namely (a) the proplets representing non-terminal nodes and (b) the proplets representing function words. Regarding (a), consider the set of proplets shown in 6.2 and the attempt to navigate from the terminal node Julia to the terminal node knows. Because there is no direct relation between these two proplets in 6.2, such a navigation would have to go from the terminal proplet Julia to the non-terminal proplet NP to the non-terminal proplet S to the non-terminal proplet VP to the non-terminal proplet V and finally to the terminal proplet knows. Yet eliminating these non-terminal nodes12 would destroy the essence of constituent structure as defined in 1.1 and thus the intuitive basis of Nativism. The other crucial ingredient of constituent structure, besides the non-terminal nodes, are the function words. They are important insofar as the words belonging together semantically are in large part the determiners with their nouns, the auxiliaries with their non-finite verbs, the prepositions with their noun phrases, and the conjunctions with their associated clauses. Regarding problem (b) raised by proplets representing function words, let us return to the example Suzy looked the word up, analyzed above in 1.3, 1.4, 2.2, 3.7, and 4.7. 11
As empirical proof for the existence of a universal grammar, Nativism offers language structures claimed to be learned error-free. They are explained as belonging to that part of the universal grammar which is independent from language-dependent parameter setting. Structures claimed to involve error-free learning include 1. 2. 3. 4. 5. 6. 7. 8. 9.
structural dependency C-command subjacency negative polarity items that-trace deletion nominal compound formation control auxiliary phrase ordering empty category principle
After careful examination of each, MacWhinney 2004 has shown that there is either not enough evidence to support the claim of error-freeness, or that the evidence shows that the claim is false, or that there are other, better explanations. 12 In order to provide for a more direct navigation, as in Example 2.1 (result).
18
R. Hausser / Comparing the Use of Feature Structures in Nativism and in Database Semantics
Given that this sentence does not have a well-formed constituent structure in accordance with Definition 1.1, let us look for a way to represent it without non-terminal nodes, but with proplets for the function words the and up. Consider the following tentative proposal, which represents each terminal symbol (word) as a proplet and concatenates the proplets using the attributes previous and next, in analogy to 6.4:
8.1
T ENTATIVE REPRESENTATION WITH FUNCTION WORD PROPLETS ⎡
⎤⎡
⎤⎡
⎤ ⎡
⎤⎡
det: the noun: word noun: Suzy verb: look ⎢prev: ⎥ ⎢prev: Suzy⎥ ⎢prev: look ⎥ ⎢prev: the ⎥ ⎥ ⎥⎢ ⎥⎢ ⎢ ⎥⎢ ⎥ ⎥⎢ ⎥⎢ ⎢ ⎥⎢ ⎣next: look ⎦ ⎣next: the ⎦ ⎣next: word⎦ ⎣next: up ⎦ prn: 2 prn: 2 prn: 2 prn: 2
⎤
prep: up ⎢prev: word⎥ ⎢ ⎥ ⎢ ⎥ ⎣next: ⎦ prn: 2
For the purposes of indexing, this analysis allows the storage of the proplets in – and the retrieval from – locations in a database which are not subject to any of the graphical constraints induced by phrase structure trees, and provides for a time-linear navigation, forward and backward, from one proplet to the next.13 For a linguistic analysis within Nativism or Database Semantics, however, the analysis 8.1 is equally unsatisfactory. What is missing for Nativism is a specification of what belongs together semantically. What is missing for Database Semantics is a specification of the functor-argument structure. For constructing an equivalence between Nativism and Database Semantics we would need to modify the attributes and their values in 8.1 as to 1. retain the proplets for the function words, 2. characterize what belongs semantically together in the surface, and 3. specify the functor-argument structure. Of these three desiderata, the third one is the most important: without functor-argument structure the semantic characterization of content in Database Semantics would cease to function and the extension of Nativism to an agent-oriented approach would fail. For specifying functor-argument structure, the proplets for function words are an insuperable obstacle insofar as they introduce the artificial problem of choosing whether the connection between a functor and an argument should be based on the function words (modifiers) or on the content words (heads). For example, should the connection between looked and the word be defined between looked and the, or between looked and word? Then there follows the question of how the connection between word and the should be defined, and how the navigation should proceed. These questions are obviated in Database Semantics by defining the grammatical relations directly between the content words. Consider the following semantic representation of Suzy looked the word up, repeating the result line of 3.7, though with the additional attribute sem to indicate the contribution of the determiner the after function word absorption:
8.2
S EMANTIC REPRESENTATION WITH FUNCTION WORD ABSORPTION ⎡
⎤⎡
⎤⎡
⎤
noun: word verb: look up noun: Suzy ⎥ ⎢sem: def sg ⎥ ⎥ ⎢sem: pres ⎢sem: nm ⎥⎢ ⎥ ⎥⎢ ⎢ ⎥⎢ ⎥ ⎥⎢ ⎢ ⎣fnc: look up⎦ ⎣arg: Suzy word⎦ ⎣fnc: look up⎦ prn:2 prn: 2 prn: 2 13 The navigation would be powered by rules like that illustrated in 4.2, modified to apply to the attributes of 8.1. For a complete DBS-system handling Example 8.1, consisting of an LA-hear grammar and an LAthink/speak-grammar, see NLC’06, Section 3.6.
R. Hausser / Comparing the Use of Feature Structures in Nativism and in Database Semantics
19
Compared to the five proplets of Example 8.1, this analysis consists of only three. The attributes prev and next have been replaced by the attributes sem (for semantics), fnc (for the functor of a noun), and arg (for the argument(s) of a verb). The functor-argument structure of the sentence is coded by the value look up of the fnc slot of the nouns Suzy and word, and the values Suzy word of the arg slot of the verb look up (bidirectional pointering). During the time-linear LA-hear analysis, shown in 3.7, the function words are treated as full-fledged lexical items (proplets). The resulting semantic representation 8.2 provides grammatical relations which support forward as well as backward navigation. These navigations, in turn, are the basis of the production of different language surfaces. For example, while forward navigation would be realized in English as Suzy looked the word up, the corresponding backward navigation would be realized as The word was looked up by Suzy.14 In 8.2, the contribution of the absorbed function word the is the value def of the cat attribute of the proplet word, while the contribution of the absorbed function word up is the corresponding value of the verb attribute of the proplet look up.15 Defining the grammatical relations solely between content words is motivated not only by the need to establish semantic relations suitable for different kinds of navigation, but also by the fact that function words are highly language-dependent, like morphological variation and word order.
9 Conclusion While Nativism and Database Semantics developed originally without feature structures, they were added later for a more detailed grammatical analysis. This paper describes the different functions of feature structures in Nativism and Database Semantics, and investigates the possible establishment of equivalences between the two systems. Establishing equivalences means overcoming apparent differences. The most basic difference between Nativism and Database Semantics is that Nativism is sign-oriented while Database Semantics is agent-oriented. Ultimately, this difference may be traced to the respective algorithms of the two systems: the rewrite rules of PS-grammar (Nativism) do not have an external interface, while the time-linear rules of LA-grammar (Database Semantics) do. It is for this reason that Nativism cannot be extended into an agent-oriented approach, thus blocking the most promising possibility for constructing an equivalence with Database Semantics. This result complements the formal non-equivalence between the complexity hierarchies of PS-grammar and LA-grammar proven in TCS’92. The argument in this paper has been based on only two language examples, namely Julia knows John and Suzy looked the word up. For wider empirical coverage see NLC’06. There, functor-argument structure (including subordinate clauses), coordination (including gapping constructions), and coreference (including ‘donkey’ and ‘Bach-Peters’ sentences) are analyzed in the hearer and the speaker mode, based on more than 100 examples.
Acknowledgments This paper benefited from comments by Airi Salminen, University of Toronto; Kiyong Lee, Korea University; Haitao Liu, Communication University of China; and Emmanuel Giguet, Université de Caen. All remaining mistakes are those of the author. 14 15
For a more detailed analysis see NLC’06, Section 6.5. In analogy to 2.2, the value up could also be stored as a third valency filler in the arg slot of the verb.
20
R. Hausser / Comparing the Use of Feature Structures in Nativism and in Database Semantics
References Bar-Hillel, Y. (1964) Language and Information. Selected Essays on Their Theory and Application. Reading, MA: Addison-Wesley Bloomfield, L. (1933) Language, New York: Holt, Rinehart, and Winston Bresnan, J. (ed.) (1982) The Mental Representation of Grammatical Relation. Cambridge, MA: MIT Press Chomsky, N. (1965) Aspects of a Theory of Syntax, The Hague: Mouton Chomsky, N. (1981) Lectures on Government and Binding, Dordrecht: Foris Clark, H. H. (1996) Using Language. Cambridge: Cambridge Univ. Press Elmasri, R. & S.B. Navathe (1989) Fundamentals of Database Systems, Redwood City, CA: Benjamin-Cummings Gaifman, C. (1961) Dependency Systems and Phrase Structure Systems, P-2315, Santa Monica, CA: Rand Corporation Gazdar, G., E. Klein, G. Pullum, and I. Sag (1985) Generalized Phrase Structure Grammar. Cambridge, MA: Harvard Univ. Press Harris, Z. (1951) Methods in Structural Linguistics, Chicago: Univ. of Chicago Press Hausser, R. (1986) NEWCAT: Parsing Natural Language Using Left-Associative Grammar, LNCS 231, Berlin Heidelberg New York: Springer (NEWCAT’86) Hausser, R. (1992) “Complexity in Left-Associative Grammar,” Theoretical Computer Science, Vol. 106.2:283-308, Amsterdam: Elsevier (TCS’92) Hausser, R. (1999) Foundations of Computational Linguistics, 2nd ed. 2001, Berlin Heidelberg New York: Springer (FoCL’99) Hausser, R. (2001) “Database Semantics for natural language,” Artificial Intelligence, Vol. 130.1:27–74, Amsterdam: Elsevier (AIJ’01) Hausser, R. (2006) A Computational Model of Natural Language Communication, Berlin Heidelberg New York: Springer (NLC’06) Kay, M. (1992) “Unification,” in M. Rosner and R. Johnson (eds) Computational Linguistics and Formal Semantics, p. 1-30, Cambridge: Cambridge Univ. Press Kay, P. (2002) “An informal sketch of a formal architecture for construction grammar,” Grammars, Vol. 5:1–19, Dordrecht: Kluwer MacWhinney, B. (2004) “A multiple process solution to the logical problem of language acquisition,” Journal of Child Language, Vol. 31:883–914, Cambridge: CUP Pereira, F., and D. Warren (1980) “Definite clause grammars for language analysis – a survey of the formalism and a comparison with augmented transition networks,” Artificial Intelligence, Vol. 13:231–278, Amsterdam: Elsevier Pollard, C., and I. Sag (1987) Information-based Syntax and Semantics, Vol. I: Fundamentals, Stanford: CSLI Pollard, C., and I. Sag (1994) Head-Driven Phrase Structure Grammar, Stanford: CSLI Saussure, F. de (1913/1972) Cours de linguistique générale, Édition critique préparée par Tullio de Mauro, Paris: Éditions Payot Shankar, V., and A. Joshi (1988) “Feature-structure based tree adjoining grammar,” in Proceedings of 12th Internation Conference on Computational Linguistics (Coling’88)
21
Information Modelling and Knowledge Bases XIX H. Jaakkola et al. (Eds.) IOS Press, 2008 © 2008 The authors and IOS Press. All rights reserved.
Multi-Criterion Search from the Semantic Point of View (Comparing TIL and Description Logic)
Marie DUŽÍ, VSB-Technical University Ostrava 17.listopadu 15 708 33 Ostrava Czech Republic
[email protected]
Peter VOJTÁŠ Charles University Prague Malostranské námČstí 25 118 00 Praha 1 Czech Republic
[email protected]
Abstract In this paper we discuss two formal models apt for a search and communication in a ‘multi-agent world’, namely TIL and EL@. Specifying their intersection, we are able to translate and switch between them. Using their union, we extend their functionalities. The main asset of using TIL is a fine-grained rigorous analysis and specification close to natural language. The additional contribution of EL@ consists in modelling multi-criterion aspects of user preferences. Using a simple example throughout the paper, we illustrate the aspects of a multi-criterion search and communication by their analysis and specification in both the systems. The paper is an introductory study aiming at a universal logical approach to the ‘multi-agent world’, which at the same time opens new research problems and trends.
1. Introduction and motivation. In this paper we discuss two formal models that are relevant in the area of search and communication in the multi-agent world, namely Transparent Intensional Logic (TIL) and a fuzzy variant EL@ of the existential description logic EL (see [2]). Since TIL has been introduced and discussed in the EJC proceedings and EL is a well-known logical system, we are not going to introduce in details the technicalities of them. Instead, we provide just a minimal necessary introduction to keep the paper self-contained and concentrate on the analytic and specification role of these systems in the area of a semantic web search that takes into account specific user fuzzy criteria. By comparing the two formalisms we aim at providing a clue to their integration. Last but not least we’d like to illustrate the assets of a rigorous logical approach to the problem. The main asset of using TIL is a fine-grained rigorous analysis and specification close to natural language. The additional contribution of EL@ consists in modelling multi-criterion aspects of user preferences. The paper is an introductory study aiming at a universal logical approach to the ‘multi-agent world’, which at the same time opens new research problems and trends. The EL@ logic is a many-valued version of the existential description logic EL (see [2]) where fuzzification concerns only concepts and the logic is enriched with aggregation (see [21]). Specifying the intersection of TIL and EL@, viz. the TIE@L, we are able to translate
22
M. Duží and P. Vojtáš / Multi-Criterion Search from the Semantic Point of View
and switch between the two systems. Using their union, TI+E@L, we extend their functionalities. Throughout the paper we use a simple example in order to illustrate basic principles, common features, as well as differences of the two systems. Example Consider a simple communication between three agents, A, B and C. The agents can be computational, like web services, database engines, query engines, pieces of software, or even human ones. The agent A sends a message to B asking to find a hotel suitable for A (the structure of the message and the meaning of ‘suitable’ will be discussed later). After obtaining an answer the agent A chooses a hotel and sends another message to the agent C asking to seek a suitable parking place close to the chosen hotel. The criteria of A are: hotel price (e.g., as low as possible), hotel distance to a beach (should be as close as possible), hotel year of building (not too old), parking place price and parking place distance (to the hotel). We are going to describe this scenario simultaneously in two formal models: TIL (Transparent Intensional Logic) and DL (Description Logic). Of course, the model can be made more realistic by considering a larger number of agents searching for specific attribute values (this approach is motivated by Fagin in [10]). When needed, we will switch between the levels of granularity in order to go into more details. Using the DL and/or database notation we are thus going to consider agents of the type User, and the attributes Hotel_Price, Hotel_Beach_Distance, Hotel_Year_of_Construction, Parking_Price, Parking_Distance. Let the values of the attributes (results of the search) be:
Particular attribute preferences of a user U can be evaluated by assigning the preference degree, a real number in the interval [0,1], to the attribute values. For instance, cheap_U(150) = 0.75, close_U (300) = 0.6, new_U (1980) = 0.2, and similarly for the other values. In this way we obtain fuzzy subsets cheap_U, close_U, new_U of the attribute domains, which can be recorded in a fuzzy database operation table (see [15]):
Our reasoning and decision making is driven not only by the preferences we assign to the values of attributes, but also by the weight we assign to the very criteria of interest. For instance, when being short of money, the low price of the hotel is much more important than its closeness to the beach. When being rich we may prefer a modern high-tech equipped hotel situated on the beach. On the other hand a hotel close to the beach may become totally unattractive in a tsunami-affected area. The multi-criterion decision is thus seldom based on a simple conjunctive or disjunctive combination of the respective factors, and we need an algorithm to compute global user preferences as a composition of particular weighted fuzzy values of the selection criteria. The algorithm can be rather sophisticated. However, for the sake of simplicity, let it be just a weighted average:
M. Duží and P. Vojtáš / Multi-Criterion Search from the Semantic Point of View
@U (cheapU , close_U , new_U )
23
2 cheap_U 3 close_U new_U 6
Computing the global degree of preferences of the hotel h1 for the user U, we obtain:
@U (0.75, 0.6, 0.2)
2 0.75 3 0.6 0.2 6
3.5 6
0.58...
Since this value is higher than the value of the hotel h2, the user U is going to choose h1. Of course, another user can have different preferences, and also the preferences of one and the same user may dynamically change in time. Besides the fact that in a multi-agent world we work with vague, fuzzy or uncertain information, we have to take into account also the demand on robustness and distribution of the system. The system has to be fully distributive, and we have to deal with value gaps because particular agents may fail to supply the requested data. On the other hand, in critical and emergency situations, which tend to a chaotic behaviour, the need for an adequate data becomes a crucial point. Therefore the classical systems which are based on the Closed World Assumption are not plausible here. We have to work under the Open World Assumption (OWA), and a lack of knowledge must not yield a collapse of the system. For instance, it may happen that we are not able to retrieve the distance of the hotel h1 to the beach, and the available data are as follows:
There are several possibilities of dealing with lacking data. We may use default values (e.g., average, the best or the worst ones), or treat the missing values as value gaps of partial functions. From the formal point of view, TIL is a hyper-intensional partial O-calculus. By ‘hyper-intensional’ we mean the fact that the terms of the ‘language of TIL constructions’ are not interpreted as the denoted functions, but as algorithmically structured procedures, known as TIL constructions, producing the denoted functions as outputs. Thus we can rigorously and naturally handle the terms that are in classical logics ‘non-denoting’, or undefined;1 in TIL each term is denoting a full-right entity, namely a construction. Hence (well-typed) terms never lack semantics. It may just happen (in well defined cases) that the denoted procedure fails to produce an output function. And if it does not fail it may happen that the produced function fails to have a value at an argument. These features of TIL are naturally combined with and completed by the EL@ fuzzy tools, in particular the aggregation algorithms. The paper is organized as follows: Chapter 2 contains brief introductory remarks on TIL. Chapter 3 introduces the EL@ description logic, and Chapter 4 is devoted to the formal description of our motivating examples, which gives us a flavour of the common features of both the models. As a result, in concluding Chapter 5 we outline a possible hybrid system and specify the trends of future research.
1
For the logic of definedness see [11].
24
2
M. Duží and P. Vojtáš / Multi-Criterion Search from the Semantic Point of View
TIL in brief.
In this Chapter we provide just a brief introductory explanation of the main notions of Transparent Intensional Logic (TIL). For exact definitions and details see, e.g., [5], [7], [8], [19], [20]. TIL approach to knowledge representation can be characterised as the ‘top-down approach’. TIL ‘generalises to the hardest case’ and obtains the ‘less hard cases’ by lifting various restrictions that apply only higher up. This way of proceeding is opposite to how semantic theories tend to be built up. The standard approach consists in beginning with atomic sentences, then proceeding to molecular sentences formed by means of truthfunctional connectives or by quantifiers, and from there to sentences containing modal operators and, finally, attitudinal operators. Thus, to use a simple case for illustration, once a vocabulary and rules of formation have been laid down, a semantics gets off the ground by analysing an atomic sentence as follows: (1)
“Charles selected the hotel h”:
S(a,h)
And further upwards: (2)
“Charles selected the hotel h, and Thelma is happy”:
S(a,h) H(b)
(3)
“Somebody selected the hotel h”:
x S(x,h)
(4)
“Possibly, Charles selected the hotel h”:
S(a,h)
(5)
“Thelma believes that Charles selected the hotel h”:
B(b,S(a,h)).
In non-hyperintensional (i.e., non-procedural) theories of formal semantics, attitudinal operators are swallowed by the modal ones. But when they are not, we have three levels of granularity: the coarse level of truth-values, the fine-grained level of truth-conditions (propositions, truth-values-in-intension), and the very fine-grained level of hyperpropositions, i.e., constructions of propositions. TIL operates with these three levels of granularity. We start out by analysing sentences from the uppermost end, furnishing them with a hyperintensional2 semantics, and working our way downwards, furnishing even the lowest-end sentences (and other empirical expressions) with a hyperintensional semantics. That is, the sense of a sentence such as “Charles selected the hotel h” is a hyper-proposition, namely the construction of the denoted proposition (i.e., the instruction how to evaluate the truth-conditions of the sentence in any state of affairs). When assigning a construction to an expression as its meaning, we specify a procedural know-how, which must not be confused with the respective performatory know-how. Distinguishing performatory know-how from procedural know-how, the latter could be characterised “that a knower x knows how A is done in the sense that x can spell out instructions for doing A.”3 For instance, to know what Goldbach Conjecture means is to understand the instruction to find whether ‘all positive even integers 4 can be expressed as the sum of two primes’. It does not include either actually finding out (whether it is true or not by following a procedure or by luck) or possessing the skill to do so.4 Furthermore, the sentence “Charles selected the hotel h” is an ‘intensional context’, in the sense that its logical analysis must involve reference to empirical parameters, in this case both possible worlds and instants of time. Charles only contingently selected the hotel; i.e., he did so only at some worlds and only sometimes. The other reason is because the analysans must 2 3 4
The term ‘hyperintensional’ has been introduced by Max Cresswell, see [4]. See [16, p.6] For details on TIL handling knowledge see [8].
M. Duží and P. Vojtáš / Multi-Criterion Search from the Semantic Point of View
25
be capable of figuring as an argument for functions whose domain are propositions rather than truth-values. Construing ‘S(a,h)’ as a name of a truth-value works only in the case of (1) and (2). It won’t work in (5), since truth-values are not the sort of thing that can be believed. Nor will it work in (4), since truth-values are not the sort of thing that can be possible. Constructions are procedures, or instructions, specifying how to arrive at less-structured entities. Being procedures, constructions are structured from the algorithmic point of view, unlike set-theoretical objects. The TIL ‘language of constructions’ is a modified hyperintensional version of the typed O-calculus, where Montague-like O-terms denote, not the functions constructed, but the constructions themselves. Constructions qua procedures operate on input objects (of any type, even on constructions of any order) and yield as output (or, in well defined cases fail to yield) objects of any type; in this way constructions construct partial functions, and functions, rather than relations, are basic objects of our ontology. The choice of types and of constructions is not given once for ever: it depends on the area to be analyzed. By claiming that constructions are algorithmically structured, we mean the following: a construction Cbeing an instructionconsists of particular steps, i.e., sub-instructions (or, constituents) that have to be executed in order to execute C. The concrete/abstract objects an instruction operates on are not its constituents, they are just mentioned. Hence objects have to be supplied by another (albeit trivial) construction. The constructions themselves may also be only mentioned: therefore one should not conflate using constructions as constituents of composed constructions and mentioning constructions that enter as input into composed constructions, so we have to strictly distinguish between using and mentioning constructions. Just briefly: Mentioning is, in principle, achieved by using atomic constructions. A construction is atomic if it is a procedure that does not contain any other construction as a used subconstruction (a constituent). There are two atomic constructions that supply objects (of any type) on which complex constructions operate: variables and trivializations. Variables are constructions that construct an object dependently on valuation: they vconstruct, where v is the parameter of valuations. When X is an object (including constructions) of any type, the Trivialization of X, denoted 0X, constructs X without the mediation of any other construction. 0X is the atomic concept of X: it is the primitive, nonperspectival mode of presentation of X. There are two compound constructions, which consist of other constructions: Composition and Closure. Composition is the procedure of applying a function f to an argument A, i.e., the instruction to apply f to A to obtain the value (if any) of f at A. Closure is the procedure of constructing a function by abstracting over variables, i.e., the instruction to do so. Finally, higher-order constructions can be used twice over as constituents of composed constructions. This is achieved by a fifth construction called Double Execution. TIL constructions, as well as the entities they construct, all receive a type. The formal ontology of TIL is bi-dimensional. One dimension is made up of constructions, the other dimension encompasses non-constructions. On the ground level of the type-hierarchy, there are entities unstructured from the algorithmic point of view belonging to a type of order 1. Given a so-called epistemic (or ‘objectual’) base of atomic types (R-truth values, Lindividuals, W-time moments / real numbers, Z-possible worlds), mereological complexity is increased by the induction rule for forming partial functions: where D, E1,…,En are types of order 1, the set of partial mappings from E1 u…u En to D, denoted (D E1…En), is a type of order 1 as well.5 5 TIL is an open-ended system. The above epistemic base {R, L, W, Z} was chosen, because it is apt for naturallanguage analysis, but the choice of base depends on the area to be analysed.
26
M. Duží and P. Vojtáš / Multi-Criterion Search from the Semantic Point of View
Constructions that construct entities of order 1 are constructions of order 1. They belong to a type of order 2, denoted by *1. This type *1 together with atomic types of order 1 serves as a base for the induction rule: any collection of partial functions, type (D E1…En), involving *1 in their domain or range is a type of order 2. Constructions belonging to a type *2 that identify entities of order 1 or 2, and partial functions involving such constructions, belong to a type of order 3. And so on ad infinitum. Definition (Constructions) i) Variables x, y, z, …are constructions that construct objects of the respective types dependently on valuations v; they v-construct. ii) Trivialization: Where X is an object whatsoever (an extension, an intension or a construction), 0X is a construction called trivialization. It constructs X without any change. iii) Composition: If X v-constructs a function F of a type (D E1…Em), and Y1,…,Ym v-construct entities B1,…,Bm of types E1,…,Em, respectively, then the composition [X Y1 … Ym] is a construction that v-constructs the value (an entity, if any, of type D) of the (partial) function F on the argument ¢B1, …, Bn². Otherwise the composition [X Y1 … Ym] does not v-construct anything: it is v-improper. iv) Closure: If x1, x2, …,xm are pairwise distinct variables that v-construct entities of types E1, E2, …, Em, respectively, and Y is a construction that v-constructs an entity of type D, then [Ox1…xm Y] is a construction called closure, which v-constructs the following function F of the type (D E1…Em), mapping E1 u…u Em to D: Let B1,…,Bm be entities of types ȕ1,…,ȕm, respectively, and let v(B1/x1,…,Bm/xm) be a valuation differing from v at most in associating the variables x1,…xm with B1,…,Bm, respectively. Then F associates with the m-tuple ¢B1,…,Bm² the value v(B1/x1,…,Bm/xm)-constructed by Y. If Y is v(B1/x1,…,Bm/xm)improper (see iii), then F is undefined on ¢B1,…,Bm². v) Double execution: If X is a construction that v-constructs a construction X’, then 2X is a construction called double execution. It v-constructs the entity (if any) v-constructed by X’. Otherwise the double execution 2X is v-improper. vi) Nothing is a construction, unless it so follows from i) through vi). The notion of construction is a notion that is the most misunderstood notion of those ones used in TIL. Some logicians ask: Are constructions formulae of type-logic? Our answer: No! Another question: Are they denotations of closed formulae? Our answer: No! So a pre-formal, ‘pre-theoretical’ characteristics is needed: constructions are abstract procedures. Question: Procedures are time-consuming, how can they be abstract? Answer: The realization of an algorithm is time-consuming, the algorithm itself is timeless and spaceless. Question: So what about your symbolic language? Why do you not simply say that its expressions are constructions? Answer: These expressions cannot construct anything they serve only to represent (or encode) constructions. Question: But you could do it like Montague6 did: To translate expressions of natural language into the language of intensional logic, and then interpret the result in the standard manner. What you achieve using ‘constructions’ you would get using metalanguage. Answer(s): First, Montague and other intensional logics interpret terms of their language as the respective functions, i.e., set-theoretical mappings. However, these mappings are the outputs of executing the respective procedures. Montague does not make it possible to mention the 6
For details on Montague system see, e.g., [12, pp. 117-220].
M. Duží and P. Vojtáš / Multi-Criterion Search from the Semantic Point of View
27
procedures as objects sui generis, and to make thus a semantic shift to hyperintensions. Yet we do need a hyperintensional semantics. Notoriously well-known are attitudinal sentences which no intensional semantics can properly handle, because its finest individuation is equivalence.7 Second, our logic is universal: we do not need to work as part-time linguisticians. Using the ‘language of constructions’ we directly encode constructions. Definition ((D-)intension, (D-)extension) (D-)intensions are members of a type (DZ), i.e., functions from possible worlds to the arbitrary type D. (D-)extensions are members of the type D, where D is not equal to (EZ) for any E, i.e., extensions are not functions from possible worlds. Remark on notational conventions: An object A of a type D is called an D-object, denoted A/D. That a construction C v-constructs an D-object is denoted C ov D. We will often write ‘x A’, ‘x A’ instead of ‘[0D Ox A]’, ‘[0D Ox A]’, respectively, when no confusion can arise. We also often use an infix notation without trivialisation when using constructions of truthvalue functions (conjunction), (disjunction), (implication), { (equivalence) and negation (), and when using a construction of an identity. Intensions are frequently functions of a type ((DW)Z), i.e., functions from possible worlds to chronologies of the type D (in symbols: DWZ), where a chronology is a function of type (DW). We will use variables w, w1, w2,… as v-constructing elements of type Z (possible worlds), and t, t1, t2, … as v-constructing elements of type W (times). If C o DWZ v-constructs an D-intension, the frequently used composition of a form [[C w] t], v-constructing the intensional descent of the D-intension, will be abbreviated as Cwt. Some important kinds of intensions are: Propositions, type RWZ. They are denoted by empirical (declarative) sentences. Properties of members of a type D, or simply Į-properties, type (RD)WZ.8 General terms (some substantives, intransitive verbs) denote properties, mostly of individuals. Relations-in-intension, type (RE1…Em)WZ. For example transitive empirical verbs, also attitudinal verbs denote these relations. Omitting WZ we get the type (RE1…Em) of relations-inextension (to be met mainly in mathematics). D-roles, offices, type DWZ, where D (RE). Frequently LWZ. Often denoted by concatenation of a superlative and a noun (“the highest mountain”). Individual roles correspond to what Church in [3] called “individual concept”. The role of the above defined constructions in a communication between agents will be illustrated in Chapter 4, in particular in Paragraph 4.5. Just a note to elucidate the role of Trivialisation and empirical parameters w o Z, t o W: The TIL language is not based on a fixed alphabet: the role of formal constants is here played by Trivialisations of nonconstructional entities, i.e., the atomic concepts of them. Each agent has to be equipped with a basic ontology, namely the set of atomic concepts he knows. Thus the upper index ‘0’ serves as a marker of the atomic concept (like a ‘key-word’) that the agent should know. If they do not, they have to learn it. The lower index ‘wt’ can be understood as an instruction to execute an empirical inquiry (search) in order to obtain the actual current value of an intension, for instance by searching agent’s database or by asking the other agents, or even by means of agent’s sense perception.
7
See [12, p.73] Collections, sets, classes of ‘D-objects’ are members of type (RD); TIL handles classes (subsets of a type) as characteristic functions. Similarly relations (-in-extension) are of type(s) (Rȕ1…ȕm).
8
28
3
M. Duží and P. Vojtáš / Multi-Criterion Search from the Semantic Point of View
The EL@ description logic
In a multi-agent world like the semantic web we need to retrieve, process, share or reuse information which is often vague or uncertain. The applications have to work with procedures that deal with the degree of relatedness, similarity or ranking. These motivations lead to the development of the fuzzy description logic (see, [18]). In this chapter we briefly describe a variant of the fuzzy description logic, namely EL@ (see [21]). One of the principal sources of fuzziness is user evaluation (preference) of crisp values of attributes. For instance, the hotel price is crisp but user evaluation may lead to a fuzzy predicate like a cheap, moderate, or expensive hotel. User preferences are modelled by linearly ordered set of degrees T = [0,1] extending classical truth-values. Thus we have: 0 = False = A = the worst T and 1 = True = T = the best T Now when searching a suitable object we have to order the set of available objects according to the user degrees assigned to object-attribute values. Practical experiences have shown that the ordering is seldom based on a conjunctive or disjunctive combination of particular scores. Rather, we need to work with a fuzzy aggregation function that combines generally incomparable sets of values. The EL@ logic is in some aspects a weakening of Straccia fuzzy description logic and in some other aspects a strengthening.9 The restrictions concern using just crisp roles and not using negation. Moreover, quantification is restricted to existential quantifiers. The extension concerns the application of aggregation functions. Thus we loose the ability to describe fuzziness in roles but gain the ability to compute a global user score. The EL@ alphabet consists of (mutually disjoint) sets NC of concept names containing T, NR role names, NI instance names and constructors containing and a finite set C of combination functions with an arity function ar : C Æ {n N : n 2}. Concept descriptions in EL@ are formed according to the following syntax rules (where @C) The interpretation structures of our description logic EL@ are parameterized by an ordered set of truth-values T (the degrees of membership to a domain of a fuzzy concept) and a set of nary aggregation functions over T. An interpretation structure T is thus an algebra T = {T, , {@•T: @ C }}, where (T , ,T ) is an upper complete semilattice with the top element T, and @•T: Tar(@) Æ T is a lattice of totally continuous (order-preserving) aggregation functions. A T –interpretation is then a pair I = ¢ǻI, •I², with a nonempty domain ǻI and the interpretation of language elements aI ǻI, for a NI AI: ǻI Æ T, for A NC (concepts can be fuzzy, like a suitable hotel) rI ǻI × ǻI, for r NR 9
For details on fuzzy description logic see [18].
M. Duží and P. Vojtáš / Multi-Criterion Search from the Semantic Point of View
29
(Roles remain crisp; however, users may interpret these data in a fuzzy way. We assume that fuzziness should not be attached to the data from the very beginning). The extension of the T –interpretation I to the composed EL@ concepts is given by (@(C1, …, Cn))I(x) = @•(CI1 (x), …, CIn (x)) and ( r.C)I(x) = sup{CI(y): (x, y) rI} The EL@ is a surprisingly expressive language with good mathematical properties. It opens a possibility to define declarative as well as procedural semantics of an answer to a user query formulated by means of a fuzzy concept definition. The discussion on the complexity of particular problems, like satisfiability, the instance problem, the problem of deciding subsumption and the proof of soundness and completeness are, however, out of scope of the present paper. For details, see, e.g., [21].
4
TIL and EL@ combined.
Using the example from the outset we are now going to outline the way of integrating the two systems. We illustrate the work with a typed and / or non-typed language, and the role of basic pre-concepts like a type, domain, concept and role. As stated above, TIL is a typed system. The basic types serve as the pre-concepts.
4.1
Pre-concepts
a) Basic types (TIL): The epistemic base is a collection of: R – the set of truth-values {T, F}, L – the universe of discourse (the set of individuals), W – the set of times (temporal factor) and / or real numbers, Z – the set of possible worlds (modal factor) (EL@): Basic pre-concepts are T and ǻI, as specified in Chapter 3. The description logic does not work explicitly with the temporal and modal factor. However, there is a possibility to distinguish between necessary ex definitione (T-boxes) and contingency of attribute values (A-boxes). Moreover, EL@ contributes the means for handling user preference structures – the preference factor. (TIL): The universe of discourse is the (universal) set of individuals. EL@ works with varying domains of interpretation ǻI. b) Functions and relations TIL is a functional system: Composed (functional) types are collections of partial functions; D-sets and (DE)-relations are modelled by their characteristic functions, objects of types (RD), (RDE), respectively. (EL@): Being a variant of description logic, EL@ is based on the first-order predicate logic where n-ary predicates are interpreted as n-ary relations over the universe. However, in EL@ this is true only for n = 2: binary predicates are crisp roles. In the other aspects EL@ is actually functional; it deals with (crisp) n-ary aggregation functions, and unary predicates (concepts) are interpreted as fuzzy sets by their fuzzy characteristic functions ǻI Æ T.
30
4.2
M. Duží and P. Vojtáš / Multi-Criterion Search from the Semantic Point of View
Assortment of the individuals in the universe
(TIL) properties In order to classify individuals into particular sorts, we use properties of individuals. They are intensions, namely functions that depending on the states of affairs (the modal parameter Z) and time (the parameter W) yield a population of individuals (RL) that actually and currently have the property in question. Example: h1, h2, h3 / L are individuals with the property Hotel / (RL)WZ of being a hotel. In the database setting these individuals belong to the domain of the attribute “hotel”, or, these individuals may instantiate the entity set HOTEL. That h1, h2, h3 are hotels is in TIL represented by the constructions of the respective propositions
OwOt [0Hotelwt 0h1], OwOt [0Hotelwt 0h2], OwOt [0Hotelwt 0h3], where the property Hotel / (RL)WZ is first intensionally descended (0Hotelwt) and then ascribed to an individual: [0Hotelwt 0hi]. Finally, to complete the meaning of ‘hi is a hotel’, we have to abstract over the modal and temporal parameter in order to construct a proposition of type RWZ that hi is a hotel: OwOt [0Hotelwt 0hi]. Gloss the construction as an instruction for evaluating the truth-conditions: In any state of affairs of evaluation (OwOt) check whether the individual (0hi) currently belongs to the actual population of hotels ([0Hotelwt 0hi]). (EL@) equivalents. Names of properties correspond to the elements of NC and NR. The above propositions are represented by membership assertions: Hotel(h1), Hotel(h2), Hotel(h3). The example continued. Let A, B, C / L are individuals with the property of being an agent. In the database setting these individuals belong to the domain of the attribute “user”, or, these individuals may instantiate the entity set AGENT. However, in order to be able to represent nary properties of individuals by means of binary ones, we need to identify particular users. Of course, in case of a big and varying set of users it is not in general possible to identify each user, and we often have to consider (a smaller number of) user profiles. (TIL): That A, B, C are agents is represented by the constructions of the respective propositions:
OwOt [0Agentwt 0A], OwOt [0Agentwt 0B], OwOt [0Agentwt 0C], where the property Agent / (RL)WZ is intensionally descended and then ascribed to an individual: [0Agentwt 0Ai]. Finally, in order to construct a proposition, we have to abstract over the parameters w, t: OwOt [0Agentwt 0Ai]. Gloss: In any state of affairs of evaluation check whether the individual A currently belongs to the actual population of agents. (EL@): The above propositions are represented by membership assertions: Agent(A), Agent(B), Agent(C). (TIL): Parking / (RL)WZ; the property of an individual of being a parking place. For instance, the proposition that p1, p2, …, pn / L are individuals with the property of being a parking place, is constructed by
M. Duží and P. Vojtáš / Multi-Criterion Search from the Semantic Point of View
31
OwOt [0Parkingwt 0pi]. (EL@): These individuals belong to the extension of the concept: Parking(pi).
4.3
Attributes – criteria
In general, attributes are empirical functions, i.e. intensions of a type (Įȕ)WZ. For instance, ‘the President of (something)’ denotes a (singular) attribute. Dependently on the modal factor Z and time W the function in question associates the respective country with the unique individual playing the role of its President. But, for instance, George W. Bush might not have been the President of the USA (the modal dependence), and he has not always been and will soon not be10 the President (the temporal dependence). (TIL) Price / (WL)WZ; an empirical function associating an individual (of type L) with a Wnumber (its price); to obtain a price of a hotel hi, we have to execute an empirical procedure:
OwOt [0Pricewt 0hi]. (EL ) the value of the attribute Price can be obtained, e.g., by an SQL query @
SELECT Price FROM Hotel WHERE Hotel.Name=hi or by using a crisp atomic role hotel_price. (TIL) Distance / (WLL)WZ; an empirical function assigning a W-number (the distance) to a pair of individuals, for instance: OwOt [0Distancewt 0hi 0pi]. (TIL) DistE / (WL)WZ; the empirical function assigning to an individual a W-number (its distance to another chosen entity E – a beach, a hotel, …). (EL@) Database point of view: Assuming we have a schema Distance(Source, Target, Value), this is the value of the attribute Distance.Value. It can be obtained, e.g., by the SQL query SELECT Distance.Value FROM Distance, Hotel WHERE Hotel.Name=hi AND Hotel.Address=x AND Distance.Source=x AND Distance.Target=E DL point of view: In DL we meet a problem here, because the relation Distance is of arity 3 and DL is a binary conceptual model. For each individual E we can consider an atomic role hotel_distance_from_E. (Of course, in practical applications we can combine these approaches). (TIL) Year / (WL)WZ; an empirical function assigning to an individual a W-number (its year of building). (EL@) Database and DL points of view similar as above (TIL) Appertain-to / (R LL)WZ; the binary relation between individuals. For example, a parking place pi belonging to a hotel hi:
OwOt [[0Parkingwt 0pi] [0Hotelwt 0hi] [0Appertain-towt 0hi 0pi]]. (EL@) the relation between a particular hotel and a parking; a crisp role
10
Written in January 2007
32
4.4
M. Duží and P. Vojtáš / Multi-Criterion Search from the Semantic Point of View
Evaluation of criteria by combining user preferences
The procedural semantics of TIL makes it possible to easily model the way particular agents can learn by experience. An agent may begin with a small ontology of atomic primitive concepts (trivialisations of entities) and gradually obtain pieces of information on more detailed definitions of the entities. In TIL terminology each composed construction yielding an entity E is an ontological definition of E. For instance, the agent A may specify the property of being a Suitable (for-A) hotel by restricting the property Hotel. To this end the property Suitable hotel is defined by the construction composing the price, distance and year attribute values and yielding the degree greater than 0.5. (TIL): Suitable-for / ((RL)WZ L (RL)WZ)WZ – an empirical (parameters W, Z) function that applied to an individual (of type L) and a property (of type (RL)WZ) returns a property (of type (RL)WZ). For instance, the property of being a suitable hotel for the agent A can be defined by:
OwOt [0Suitable-forwt 0A 0Hotel] = OwOt Ox [[0Hotelwt x] [[0Evaluatewt 0A [0Pricewt x] [0DistEwt x] [0Yearwt x]] t 0.5]]. By way of further refining, we can again define the atomic concept 0Evaluate. To this end we enrich the ontology by 0Aggregate and 0Apt-for, which can again be refined. And so on, theoretically ad infinitum. Evaluate / (W L WWW)WZ - an empirical function that applied to an individual a and a triple of Wparameters (e.g., price, distance, year) returns a W-number [0,1], which is the preference degree of a particular hotel for the agent a. [0Evaluatewt a par1 par2 par3] = [0Aggregate [0Apt-forwt a par1] [0Apt-forwt a par2] [0Apt-forwt a par3]]. Aggregate / (W WWW) – the aggregation function that applied to the triple of W-numbers returns a W-number = the degree of appropriateness. Apt-for / (W LW) – an empirical function that applied to an individual a / L and a Wparameter pari (e.g., price, distance, and so like) returns a preference scale of the respective parameter pari for the user a. The scale is a W-number [0,1]. For instance, [0Evaluatewt 0A [0Pricewt x] [0DistEwt x] [0Yearwt x]] = [0Aggregate [0Apt-forwt A [0Pricewt x]] [0Apt-forwt A [0DistEwt x]] [0Apt-forwt A [0Yearwt x]]]. The empirical function Evaluate is the key function here. Applied to an individual agent (user) and particular criteria it returns the agent’s preference-degree of a particular object. Each agent may dynamically (parameter W) choose (parameter Z) its own function Evaluate. The algorithm computing the preference-degree of an object consists of two independent subprocedures: i) user preference scale Apt-forwt of the TIL-type (W LW), or using the EL@ notation: ¢user, pari² o [0,1], where pari is the value of a particular criterion (for instance price, distance, etc.). Here the additional role of EL@ comes into play. The EL@ logic makes it possible to choose an appropriate scale algorithm. It can be a specific function for a particular user U1, e.g.:
M. Duží and P. Vojtáš / Multi-Criterion Search from the Semantic Point of View
33
ii) the aggregation function Aggregate of the TIL -type (W WWW), or in the EL@ notation (understood as a many valued connective) @: [0,1]3 o [0,1], computing the global preference degree. Here we consider the Aggregate function as not being user-dependent, but rather system-dependent (therefore, in TIL –notation there is no WZ-parameter). In other words, it is a system algorithm of computing the general user preference. Of course, we might let each user specify his/her/its own algorithm but in practice it suffices to consider different user profiles associated with each aggregation function. Thus the system may test several algorithms of aggregation, e.g., those that were used for users with a similar profile, in order to choose the suitable aggregation. It does not seem to be necessary to further refine the specification in TIL. Instead we either call at this point a software module, or make use of the EL@ logic. In the example above we used the weighted average:
4.5
Communication of agents; messages
The communication aspects are not elaborated in EL@ from the semantic point of view. Hence it represents the added value of TIL when integrating with EL@. However, in SQL we have ORDER BY command and when dealing with preferences we work with the notion of the best, top-k, respectively, answers. The EL@ many valued logic setting understood as a comparative tool (numerical values do not matter) is an appropriate tool for evaluating fuzzy predicates. It provides a good semantics for ordering preferences of answers (see [13]). The TIL-philosophy is driven by the fact that natural language is a perfect logical language. Hence the TIL-specification is close to an ordinary human reasoning and natural-language communication. On the other hand, however, the high expressive power of the TIL language of constructions may sometimes be an obstacle to an effective implementation. This problem is dealt with by the step-by-step refinement as discussed above. At the first step we specify just a coarse-grained logical form of a message; the execution is left to particular Java modules. Then a more fine-grained specification makes it possible to increase agent’s “intelligence” by letting him dynamically decide which finer software modules should be called. To this end we combine Java modules, Prolog, fuzzy Prolog Ciao, etc. (TIL): The general scheme of a message is: Message / (R L L RWZ)WZ
OwOt [0Messagewt 0Who 0Whom OwOt [0Typewt 0What]], where
34
M. Duží and P. Vojtáš / Multi-Criterion Search from the Semantic Point of View
Who /L, Whom /L, What (content)/ĮWZ, Type / (RĮWZ)WZ, What – the subject of the message is a specification of an intension (usually a proposition of type RWZ). (EL@): The description logic does not incorporate a specific semantic logical description of messages. It is usually handled by an implementation component (generally in the Software Engineering part) by dealing with exceptions, deadlocks, etc. (TIL): There are three basic types of messages that concern propositions; i.e., these types are properties of propositions, namely Order, Inform, Query/(RRWZ)WZ. In an ordinary communication act we implicitly use the type Inform affirming that the respective proposition is true. But using an interrogative sentence we ask whether the proposition is true (Query), or using an imperative sentence we wish that the proposition were true (Order). The content of a message is then the construction of a proposition, the scheme of which is given by:
OwOt [0Typewt 0What] o RWZ. In what follows we specify in more details possible typical types of messages. Type = {Seek, Query(Yes-No), Answer, Order, Inform, Unrecognised, Refine,…}; where Typei / (RDWZ)WZ or Typei / (R n)WZ. Examples of a content of a message: [0Seekwt 0What]; What / DWZ o send me an answer = the actual D-value of What in a given state of affairs w,t of evaluation. [0Querywt 0What]; What / RWZ o send me an answer = the actual R-truth-value of What in a given state of affairs w,t. [0Orderwt 0What]; What / RWZ o manage What to be actually True (in a state of affairs w,t.) [0Informwt 0What]; What / RWZ o informing that What is actually True [[0Answerwt 0What] = a / D]; where a = [0Whatwt]; the answer to a preceding query or seek. [0Unrecognisedwt 00What]; the atomic concept 0What has not been recognised; a request for refinement. Note that Unrecognised is of type (R n)WZ, the property of a construction (usually an atomic concept). Therefore the content of the message is not the intension What constructed by 0What, but the construction 0What itself. The latter is mentioned here by trivialisation, therefore 00What. [[0Refinewt 00What] = 0C o DWZ]; an answer to the message on unrecognised atomic concept. The construction C is the respective composed specification (definition) of What, i.e., C and 0What are equivalent, they construct the same entity: C = 0What. For instance, the set of prime numbers can be defined as the set of numbers with two factors: [[0Refinewt 00Prime] = 0[Ox [0Card Oy [0Div x y] = 02]]], where x, y o Nat (the type of natural numbers), Div / (R Nat Nat) – the relation of being divisible by, Card / (Nat (R Nat))– the number of elements of a set.
M. Duží and P. Vojtáš / Multi-Criterion Search from the Semantic Point of View
4.6
35
Example of communication
Now we continue the simple example from the outset. We will analyse a part of the dialog of the three agents A, B, C. Sentences will be first written in ordinary English then analysed using TIL, transformed into the standardised message, and if needed provided by a gloss. For the sake of simplicity we will omit the specification of TIL-types of particular objects contained in a message. However, since the TIL-type is an inseparable part of the respective TIL-construction, we do not omit it in a real communication of agents. For instance, when building an agent’s ontology, each concept is inserted with its typing. Message 1 (A to B): ‘I wish B to seek a suitable hotel for me.’ A(TIL):
OwOt [0Wishwt 0A OwOt [0Seekwt 0B [0Suitable-forwt 0A 0Hotel]]], Wish/(RLRWZ)WZ, Seek/(RL(RL)WZ)WZ,
A(TIL m1):
OwOt [0Messagewt 0A 0B OwOt [0Seekwt [0Suitable-forwt 0A 0Hotel]]]
Gloss:
The agent A is sending a message to B asking to seek a suitable hotel for A.
Message 2 (B to A): However, the agent B does not understand the sub-instruction [0Suitable-forwt 0A 0Hotel], because he does not have the atomic concept 0Suitable-for in his ontology. Therefore, he replies a message to A, asking to explain: ‘I did not recognise 0Suitable-for.’ B(TIL m2):
OwOt [0Messagewt 0B 0A OwOt [0Unrecognisedwt 00Suitable-for]
Remark Thus the lower index wt can be understood as an instruction to execute an empirical inquiry (search) in order to obtain the actual current value of an intension, here the property of being a suitable hotel (for instance by searching agent’s database or by asking the other agents, or even by means of agent’s sense perception). The upper index 0 serves as a marker of the primitive (atomic) concept belonging the agent’s ontology. If it does not, i.e., if the agent does not know the concept, he has to ask the others in order to learn by experience. Message 3 (A to B): The agent A replies by specifying the restriction of the property Hotel to those hotels which are evaluated with respect to price, distance and the year of building with the degree higher than 0.5: A(TIL)
0
Suitable-for / ((RL)WZ L (RL)WZ)WZ, a o L, p o (RL)WZ;
0
Suitable-for = OwOt Oap OwOt Ox [[pwt x] [[0Evaluatehwt a [0Pricewt x] [0DistEwt x] [0Yearwt x]] t 00.5]]. Gloss: The A’s answer message should refine the atomic concept 0Suitable-for. Now there is a problem, however. The agent B would have to remember the respective message asking for the refinement in order to apply the property to proper arguments (namely A and Hotel). This would not be plausible in practice, because A is the aid prayer, not B. Therefore the answer message contains the smallest constituent containing the refined concept: A(TIL m3): 0
OwOt [0Messagewt 0A 0B OwOt [0Refinewt 0[0Suitable-forwt 0A 0Hotel] =
[OwOt Ox [[0Hotelwt x] [[0Evaluatehwt 0A [0Pricewt x] [0DistEwt x] [0Yearwt x]] t 0.5]]]]]
36
M. Duží and P. Vojtáš / Multi-Criterion Search from the Semantic Point of View
In this way the agent B obtains a piece of knowledge what should he look for. Another possibility would be A’s sending the original message 1 refined, i.e., the constituent [0Suitable-forwt 0A 0Hotel] replaced by the new specification: A(TIL m3’):
OwOt [0Messagewt 0A 0B OwOt [0Seekwt
OwOt Ox [[0Hotelwt x] [[0Evaluatehwt 0A [0Pricewt x] [0DistEwt x] [0Yearwt x]] t 0.5]]]] However, we prefer the former, because in this way B learned what a suitable hotel for A means. Or rather, he would learn if he understood 0Evaluateh, which may not be the case if he received the request for the first time. Thus if B does not have the concept in his /her ontology, he again sends a message asking for explaining: Message 4 (B to A): B:
I did not recognise 0Evaluateh.
B(TIL m4):
OwOt [0Messagewt 0B 0A OwOt [0Unrecognisedwt 00Evaluateh]
Message 5 (A to B): A(TIL m5):
OwOt [0Messagewt 0A 0B OwOt [0Refinewt 0 0
[ Evaluatewt 0A [0Pricewt x] [0DistEwt x] [0Yearwt x]] =
= 0[0Aggregate [0Apt-forwt A [0Pricewt x]] [0Apt-forwt A [0DistEwt x]] [0Apt-forwt A [0Yearwt x]]] And so on, the refinement may continue and the agents may learn new concepts (from the theoretical point of view ad infinitum). Anyway, finally B fully understands the message and attempts at fulfilling the task; recall that he is to seek a suitable hotel for A. Note that the whole process is dynamic, even agents’ learning by the process of refining particular atomic concepts. B knows now that actually and currently a hotel suitable for A is such a hotel the price, distance from the beach and the year of building of which evaluate with respect to A’s scaling [0Apt-forwt 0A] with the degree higher than 0.5. But he also knows that it might have been otherwise (the modal parameter w / Z) and it will not have to be always so (the temporal parameter t / W). In other words, A and B now share common knowledge of the composed concept defining the property of being a suitable hotel for A. When eventually B accomplishes his search he sends an answer to A: Message 6 (B to A): A(TIL m6):
OwOt [0Messagewt 0B 0A OwOt [0Answerwt [0Suitable-forwt 0A 0Hotel] = {¢h1,0.7²,{¢h5,0.53²}] 11
Gloss: B found out that there are two instances of the property v-constructed by the construction [0Suitable-forwt 0A 0Hotel], namely the hotel h1 that has been evaluated with the degree 0.7 and h5 with the degree 0.53. Since h1 has been evaluated as better than h5, A chooses the former. At this point the communication can continue as a dialogue between A and C in a similar way as above. The aim is now finding a suitable parking close to the chosen hotel h1 and then asking to navigate to the chosen parking place:
OwOt [0Messagewt 0A 0C OwOt [0Seekwt [0Suitablepwt 0A 0Parking]]] 11
Here we use the classical set-theoretic notation without trivialisation, for the sake of simplicity.
M. Duží and P. Vojtáš / Multi-Criterion Search from the Semantic Point of View
37
OwOt [0Messagewt 0C 0A OwOt [0Unrecognisedwt 00Suitablep] OwOt [0Messagewt 0A 0C OwOt [0Refinewt 0[0Suitablepwt 0A 0Parking] = 0
[OwOt Ox [[0Parkingwt x] [[0Evaluatepwt 0A [0Pricewt x] [0DistEwt x]] t 00.5]]]]]
OwOt [0Messagewt 0C 0A OwOt [0Answerwt [0Suitablepwt 0A 0Parking]] = {{p2,0.93},{p1,0.53}] The message closing the dialogue might be sent from A to C:
OwOt [0Messagewt 0A 0C OwOt [0Orderwt OwOt [0Navigate-towt 0p2]]]. At this point the agent C must have 0Navigate-to in his/her ontology (if he/she does not then the learning process described above begins); C thus knows that he/she has to call another agent D which is a GIS-agent that provides navigation facilities (see [6]). Concluding this paragraph we again compare the TIL approach with EL@. An analogy to the above described means of communication can be found in the DL community. There are heuristics for the top-k search (see [13]). However, these facilities lack any formal / logic / semantic specification. The development of description logic and its variants can be considered as a step forward to the development of languages which extend W3C standards. In [9] a step in this direction is described. In particular the EL@ variant of the description logic can be embedded into classical two-valued description logic with concrete domains (see [1]), and thus also into OWL (or a slight extension of it). Using the results described in this paper, especially the added value of TIL, we can expect the extension of W3C based specification of web service languages using the OWL representation.
5.
Conclusion: A hybrid system
In the previous chapters, especially by using the parallel description of our motivating example in Chapter 4, we tried to show that TIL and EL@ have many features in common. Both the systems can share some basic types, functions, concepts and roles; both the systems distinguish extensional and intentional context (the former being modelled by the intensional descent in TIL and A-Boxes in DL, the latter illustrated here by the (user-) definition or specification of a multi-criterion search). These features can form the intersection TIE@L. On the other hand, both the systems can be enhanced by accommodating features of the other system, thus forming a union TI+E@L. The main contribution of EL@ is the method of modelling multi-criterion aspects of user preferences (some heuristics have been tested in separate works), and computing global user preferences by means of the aggregation functions and scaling. TIL contributes to this union the method of a very fine-grained and rigorous knowledge specification closed to natural language, including procedural hyperintensional semantics. We are convinced that these aspects are crucial for a smooth communication and reasoning of agents in the multi-agent world. Artificial Intelligence is sometimes characterised as a ‘struggle for consistency’. To put it slightly metaphorically, reality is consistent. Only our ‘making it explicit’ in language may lead to paradoxes and inconsistencies due to misinterpretations that are caused by a too coarse-grained analysis of assumptions. The specification of the formal model of the hybrid system is however still a subject of further research. Currently we plan to perform experiments and tests on real data using the hints described in Chapter 4.
38
M. Duží and P. Vojtáš / Multi-Criterion Search from the Semantic Point of View
In the team led by M. Duží, working on the project “Logic and Artificial Intelligence for multi-agent systems” (see http://labis.vsb.cz/), we pursue research on multi-agent systems based on TIL. Currently we implemented software modules simulating the behaviour of mobile agents in a traffic system. The agents can choose particular realisations of their predefined processes; moreover, they are able to dynamically adjust their behaviour dependently on changing states of affairs in the environment. They communicate by message-exchange system. To this end the TIL-Script language (see [14]) has been designed and it is currently being implemented. We also plan to test some modules with EL@ features. The project in which P. Vojtas is involved (see [17]) deals with theoretical models compatible with W3C standards and experimental testing of multi-criterion search dependent on user preferences. We believe that the TIL features will enhance the system with a rigorous semantic description and specification of the software / implementation parts. When pursuing the research we soon came to the conclusion that the area of the semantic web and multi-agent world in general is so broad that it is almost impossible to create a universal development method. Instead we decided to develop a methodology comprising and integrating particular existing and/or newly developed methods as well as our fine-grained rigorous logic. The paper is an introductory study aiming at a more universal logical approach to the ‘multi-agent world’, which at the same time opens new research problems and trends. The main challenges are formal measures (soundness and completeness) and implementation measures of the integrated hybrid system. –––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––– ACKNOWLEDGEMENTS This work has been supported by the project No. 1ET101940420 “Logic and Artificial Intelligence for multiagent systems” within the program “Information Society” of the Czech Academy of Sciences, and by the “Semantic Web” project No. 1ET100300419 of the Czech IT agency.
REFERENCES Baader, F., Calvanese, D., McGuinness, D.L., Nardi, D., Patel-Schneider, P.F. eds. (2002): Description Logic Handbook, Cambridge University Press. [2] Brandt, S. (2004): Polynomial Time Reasoning in a Description Logic with Existential Restrictions, GCI Axioms, and What Else? In R. López de Mantáras et al. eds. In Proc. ECAI-2004, pp. 298-302. IOS Press. [3] Church, A. (1956): Introduction to Mathematical Logic I. Princeton. [4] Cresswell, M.J. (1985): Structured meanings. MIT Press, Cambridge, Mass. [5] Duží, M.(2004): Concepts, Language and Ontologies (from the logical point of view). In Information Modelling and Knowledge Bases XV. Ed. Y. Kiyoki, H. Kangassalo, E Kawaguchi, IOS Press Amsterdam, Vol. XV, 193-209. [6] Duží, M., Ćuráková, D., DČrgel, P., Gajdoš, P., Müller, J. (2007): Logic & Artificial Inteligence for MultiAgent Systems. In Information Modelling and Knowledge Bases XVIII. M. Duží, H. Jaakkola, Y. Kyioki, H.Kangassalo (Eds.), IOS Press Amsterdam, 236-244. [7] Duží, M., Heimburger A. (2006): Web Ontology Languages: Theory and practice, will they ever meet?. In Information Modelling and Knowledge Bases XVII. Ed. Y. Kiyoki, J. Henno, H. Jaakkola, H. Kangassalo, IOS Press Amsterdam, Vol. XVII, 20-37. [8] Duží, M., Jespersen B, Müller, J. (2005): Epistemic Closure and Inferable Knowledge. In the Logica Yearbook 2004. Ed. Libor BČhounek, Marta Bílková, Filosofia Praha, Vol. 2004, 1-15. [9] Eckhardt, A., Pokorný, J., Vojtáš, P. (2006): Integrating user and group preferences for top-k search from distributed web resources, technical report 2006 [10] Fagin, R. (1999): Combining fuzzy information from multiple systems, Journal of Comput. System Sci. 58, 1999, 83-99 [11] Feferman, S. (1995): ‘Definedness’. Erkenntnis 43, pp. 295-320. [12] Gamut, L.T.F. (1991): Logic, Language and Meaning. Volume II. Intensional Logic and Logical Grammar. [1]
M. Duží and P. Vojtáš / Multi-Criterion Search from the Semantic Point of View
39
The University of Chicago Press, Chicago, London. [13] Gurský, P., Lencses, R., Vojtáš, P. (2005): Algorithms for user dependent integration of ranked distributed information. In TCGOV 2005 Poster Proceedings, M. Boehlen et al eds. IFIP – Universitaetsverlag Rudolf Trauner, Laxenburg, ISBN 3-85487-787-0, pp. 123-130 [14] The TIL-Script language description is available at the VSB-TIL homepage: http://www.cs.vsb.cz/TIL [15] Pokorný, J., Vojtáš, P. (2001): A data model for flexible querying. In Proc. ADBIS'01, A. Caplinskas and J. Eder eds. Lecture Notes in Computer Science 2151, Springer Verlag, Berlin, 280-293 [16] Rescher, N. (2002): ‘Epistemic logic’, in: A Companion to Philosophical Logic, D. Jacquette (ed.), Oxford: Blackwell, pp. 478-91. [17] ‘Semantic Web’ project of the Czech IT agency 1ET100300419 [18] Stracia, U. (2001): reasoning with Fuzzy description Logics. Journal of Artificial Intelligence and research 14 (2001), pp. 137-166. [19] Tichý, P. (1988): The Foundations of Frege’s Logic. De Gruyter. [20] Tichý, P. (2004): Pavel Tichý’s Collected Papers in Logic and Philosophy. Svoboda, V., Jespersen, B., Cheyne, C. (editors), Filosofia Prague and University of Otago Press. [21] Vojtáš, P. (2006): EL Description Logics with Aggregation of User Preference Concepts. In Information Modelling and Knowledge Bases XVIII. M. Duží, H. Jaakkola, Y. Kyioki, H. Kangassalo (Eds.), IOS Press Amsterdam, 154-165.
40
Information Modelling and Knowledge Bases XIX H. Jaakkola et al. (Eds.) IOS Press, 2008 © 2008 The authors and IOS Press. All rights reserved.
A Semantic Space Creation Method with an Adaptive Axis Adjustment Mechanism for Media Data Retrieval Xing Chen1, Yasushi Kiyoki2, Kosuke Takano3 and Keisuke Masuda4 1,3
Department of Information & Computer Sciences Kanagawa Institute of Technology 1030 Simo-Ogino, Atsugi-shi, Kanagawa 243-0292, Japan
[email protected],
[email protected] 2 Department of Environmental Information Keio University Fujisawa, Kanagawa 252-8520, Japan
[email protected] 4 Graduate Courses of Information & Computer Sciences Kanagawa Institute of Technology 1030 Simo-Ogino, Atsugi-shi, Kanagawa 243-0292, Japan
[email protected] Abstract. This paper presents a new semantic space creation method with an adaptive axis adjustment mechanism for media data retrieval. The semantic space is essentially required to search semantically related and appropriate information resources from media databases. In the method, data in the media databases are mapped as vectorized metadata on the semantic space. The distribution of the metadata on the semantic space is the main factor affecting the accuracy of the retrieval results. In the method, an adaptive axis adjustment mechanism is used to rotate and combine the semantic correlated axes on the semantic space, and remove axes from the semantic space. We demonstrated by experiments that when the semantic space is created and adjusted based on the semantic correlated factors, the metadata are appropriately and sharply distributed on the semantic space.
1. Introduction Large numbers of heterogeneous databases are spreading in wide area computer network environments to meet the increasing needs of homo-sapiens. People have opportunities to obtain significant information from those heterogeneous databases through the wide-area computer network [1], [2], [7]. However, it is still difficult for users to extract appropriate information without knowledge on the contents and structures of those databases. The development of sophisticated retrieval methods for
X. Chen et al. / A Semantic Space Creation Method with an Adaptive Axis Adjustment Mechanism
41
realizing an intelligent multimedia database environment is an important issue in database research field. Semantic information retrieval methods [3], [5], [6] are proposed for realizing intelligent information retrieval. In the semantic information retrieval methods, the semantic relationship computing is the essential function for extracting the semantically related and appropriate information resources from databases [8], [9], [11], [12]. We have proposed two fundamental frameworks for computing semantic relationships between retrieval candidates and queries in multimedia database environments [3], [4], [6], [8]. In these methods, semantic expression vectors [8] are used as metadata of media data (retrieval-candidate media data) to express attributes, contents and impressions of media data. We have also proposed several fundamental method and systems for extracting semantically correlated factors from data resources [3], [4]. One of the important issues of the semantic associative retrieval is to select appropriate media data according to the requirements of queries. The key point to select the appropriate media is that the metadata are appropriately distributed on the semantic space. The learning mechanism is one of the methods to adjust the metadata distributions on the semantic space [3]. In this paper, we present a method to create a well-structured semantic space on which vectorized metadata are appropriately and sharply distributed. In the method, a vector space is created based on the factors of the metadata. The basic idea of the method is to rotate and combine the axes of the space, which correlate to the same semantic factors and to remove the axes from the space, which reduce the precision of the retrieval results. In the method, the adaptive axis adjustment mechanism is used to implement the ‘rotating’, ‘combining’ and the ‘removing’ operations. Sets of objects in the databases are given as the training data for the extracting the semantic correlated factors from the metadata. When the axes of the vector space are adjusted by the adjustment mechanism based on the semantic correlated factors, a well-structured semantic space is created. On the space, metadata are appropriately and sharply distributed. The outline of the related works and issues for creating semantic space are reviewed in Section 2. In Section 3, we present the new semantic space creation method for realizing appropriate and precise semantic associative retrieval. Several experimental results are presented in Section 4 to clarify the feasibility and effectiveness of the proposed
42
X. Chen et al. / A Semantic Space Creation Method with an Adaptive Axis Adjustment Mechanism
method. 2. Related works and issues about the semantic space In this section, we first review a simple model, the vector space model (VSM) [14], [15]. After that, a method, the Latent Semantic Indexing (LSI) [5], which is used to create the semantic space, is reviewed. We will present the issues when the spaces are created based these methods for information retrieval. 2.1 The vector space model In the vector space model (VSM), retrieval candidates and queries are modeled as vectors of a vector space. If an object (a retrieval candidate or a query) is represented by n features, for example, n index terms, it is assumed that each feature is a vector of unit length and each feature (unit vector) is weighted by its importance. All objects on the vector space are represented as vectors which are linear combinations of weighted features. For example, if a document is represent by n index terms, each term is assumed as a unit length vector and the document is represented as a vector of weighted terms. A common weighting scheme for terms within a document is the frequency of occurrence of the terms in the document. Retrieval processing in vector space models is performed by determining similarity between query and retrieval candidates. Associative coefficients based on the inner product of the retrieval candidate vectors and query vector are used to determine the similarity. The most popular similarity measure is the cosine coefficient, which measures the angle between the retrieval candidate vector and the query vector, and the retrieval candidates are ranked in the decreasing order of this measure. 2.2 The issue of the VSM on information retrieval The standard vector space model assumes that features of retrieval candidates are not correlated, that is, they are pair-wise orthogonal. If index terms are used to represent documents to vectors, the index terms are assumed pair-wise orthogonal. The issue of the vector space created by index terms for information retrieval is explained by Example 1.
X. Chen et al. / A Semantic Space Creation Method with an Adaptive Axis Adjustment Mechanism
43
Example 1. Suppose documents are represented by two index terms, t1, t2. A vector space with two dimensions is created and all documents are represented as two dimensional vectors. Each document vector di can be expressed as: di
d i1t 1 d i 2 t 2 , (i
1, ,6) .
In the figure, let t1 and t2 represent the term basis vectors. That is, t1 and t2 are orthogonal. Let t1 and t(r)2 represents term vectors in the case that t1 and t(r)2 are not orthogonal. In the first case that t1 and t2 are orthogonal, a document is represented as a vector d and in the second case, that is, t1 and t(r)2 are not orthogonal, the document is represented as the vector d(r).
t (2r )
t2
G2
d12
d d (r ) d11
t1
G1
Fig. 1. Two-dimensional vector space in the cases that (1) t1and t2 are orthogonal and (2) they are not orthogonal.
As shown in Figure 1, if the index term vectors t1 and t2 are not orthogonal, components of the document vector are different from those when t1 and t2 are orthogonal. The differences can be written as follows: d ( r ) (d11 G 1 )t1 (d12 G 2 )t 2 , where G 1 is the increasing difference of the document vector on t1 vector and G 2 is the decreasing difference of the document vector on t2 vector. Consider simplified situations that when the index terms t1 and t2 represent a same meaning, we define t1=t2 as shown in Figure 2(a). When the index terms t1 and t2 are different in the meanings, we define t1 and t2 to be orthogonal as shown in Figure 2(b). Let document d1 is represented by index term t1 and document d2 is represented by index term t2, we have d1
d11t1 , d 2
d 22 t 2 .
44
X. Chen et al. / A Semantic Space Creation Method with an Adaptive Axis Adjustment Mechanism
In the vector space model, vectors t1 and t2 are supposed to be orthogonal and each of them is normalized. Therefore, a 2-dimentional space is represented by t1 and t2 as the orthonormal basis. When t1 and t2 represent different meanings, as shown in Figure 2(b), this assumption is acceptable. When t1 and t2 represent a same meaning and they must be represented as vectors overlapped to each other, but they are represented as orthogonal vectors. If the index term t1 is used as the keyword in a query, the retrieval candidates with highest relation to a query are not selected. The query can be represented as: q q1t 1 . The cosine measure, which used as the ranking function to measure similarity between the query and retrieval candidate vectors, can be defined as dci u q cosd i , q . dci u d i qc u q When the scalar product of a query vector and a retrieval candidate vector are orthogonal, the cosine value of them is also zero. As shown in Figure 2(b), when t2 is represented orthogonally to t1, the scalar product of q and d2 is zero, therefore, d2 is not selected. It can be further explained in the following formulas: cosq, d1 1, cosq, d 2 0. It is required when t1 and t2 represent a same meaning, they should be represented overlapped as shown in Figure 2(a), because basis vectors are either pair-wise orthogonal or overlapped. t2 d2 d2
t1 , t 2
t1 d1
d1 (a)
(b)
Fig. 2. (a) The situation that index terms t1 and t2 represent a same meaning. (b) The situation that index terms t1 and t2 represent different meanings. Two documents are represented as two vectors d1 and d2. In the situation (a) they overlap with each other and in the situation (b), they are orthogonal.
X. Chen et al. / A Semantic Space Creation Method with an Adaptive Axis Adjustment Mechanism
45
2.3 Creating the semantic space As explained in Figure 1, when a basis vector semantically correlates to the other basis vectors on the vector space, these basis vectors must be rotated in order to get the correct distribution of media data on the space. We refer the vector space on which media data are mapped according to semantic factors to as the semantic vector space, or simply, the semantic space. Methods of creating semantic space are proposed [3], [5], [6]. The basic idea of these methods is illustrate in the next example, Example 2. Example 2. Consider a two-dimensional vector space with five document vectors. Each document is represented by two indexing terms t1 and t2: d i d i1t1 d i 2 t 2 . Let D be a term-document matrix defined as: ª d11 «d « 21 « « ¬d 51
ª d1 º «d » D « 2» «» « » ¬d 5 ¼
d12 º d 22 »» . » » d 52 ¼
Let R be the matrix with characteristic of R u R ' I , where R ' is the transposed matrix of R and I is the identity matrix. When R is a two-row and two-column matrix, a
projection matrix P can be calculated by the following equation: P D u R. Each column of R is a vector:
>r1
R
r2 @.
Each column of P is a vector representing a document on the vector space with the basis of r1 and r2:
P
ªd1( r ) º « (r ) » «d 2 » , « » « (r ) » «¬d 5 »¼
where,
d i( r )
>d i1
d i 2 @u R .
46
X. Chen et al. / A Semantic Space Creation Method with an Adaptive Axis Adjustment Mechanism
It is expected that vectors of documents on the space with the basis of R are in correct locations. Singular Value Description (SVD) is one of the methods to rotate basis vectors. In the following, we use F, R to represent the vector space-F and vector space-R, respectively. We use D(f) to represent metadata on the space-F and D(r) to represent the metadata on the space-R. The axis on the space-R is represented as fi and the axis on the space R is represented as ri. When SVD is applied on the matrix D(f), an orthogonal space R is obtained: D ( f ) LSR c,
D( f ) R
LS D (r ) , where S is a diagonal matrix that contains singular values, L and R are left and right matrices of the matrix S. R ' is the transposed matrix of R. The diagonal matrix S has the characteristics as the follows: S' S , SS 1
I, where I is the identity matrix. Both the matrices L and R have orthonormal columns, that is RR ' I , LL' I . Therefore, D f u R LS R c u R LS
D r , that is, metadata on the space F are projected on to the space R.
2.4 The issue of the space created by SVD for information retrieval As mentioned above, the basis vectors of R are the rotated vectors of the original space F. Because RR ' I , we will demonstrate in the following, if two vectors q and d are orthogonal on the space F, they are also orthogonal on the space R. That is, the required vector d can not be selected only by rotating basis vectors. The distribution of metadata on the vector spaces F and R created by SVD is illustrated in Example 3.
X. Chen et al. / A Semantic Space Creation Method with an Adaptive Axis Adjustment Mechanism
47
Example 3. Give a matrix D(f) with five vectors represents five retrieval candidates and each retrieval candidate is represented with two features, f1 and f2:
D( f )
ª3 «0 « «1 « «3 «¬2
0º 5»» 2» . » 2» 3»¼
By performing SVD on D(f), matrix R is obtained: 0.8836 º ª 0.46824 « 0.8836 0.46824» . ¬ ¼
R
㪍
f2
㪌
㪻㪉
㪋
d 21( r )
㪊
㪻㪌
㪉 (r ) d 22
d11( r )
㪻㪊
㪻㪋
㪈
f1
㪇 㪄㪉
㪇 㪄㪈㫉㪈
r1
㪄㪉
㪻㪈 㫉㪉
㫆
㪉
㪋
d12( r )
r2
㪍
Fig. 3. Vectors of retrieval candidates on two vector spaces. One of the spaces is created with the vector f1 and f2 as the basis and the other is created with the vectors r1 and r2 as the basis.
The matrix D(r) which represents the vectors of the retrieval candidates on the space created with R is
D( r )
D( f ) u R 2.6508 º ª 1.4047 « 4.418 2.3412 » « » « 2.2354 0.05288» . « » « 3.1719 1.7143 » «¬ 3.5873 0.3628 »¼
48
X. Chen et al. / A Semantic Space Creation Method with an Adaptive Axis Adjustment Mechanism
The distribution of the vectors of retrieval candidates on the two vector spaces is shown in Figure 3. One space is created with f1 and f2 as the basis and the other is created with r1 and r2 as the basis. If a query vector q and a retrieval candidate vector di are orthogonal, the scalar product of them is zero:
qc u d i
0.
When the vectors q and di are projected onto a new orthogonal space R ( RR ' I ), their scalar product on the new space is also zero:
q(r )
qc u R c ,
d i( r )
dci u R c ,
c q ( r ) u d i( r )
qc u R s u dci u R c qc u R u R c u dcic qc u I u d i qc u d i 0.
That is, the required retrieval candidate can not be selected on the new space R. This can be illustrated by using Figure 3. As shown in Figure 3, the vector d1 and d2 are orthogonal on the space with f1 and f2 as basis. This orthogonal characteristic is not changed on the new space with r1 and r2 as basis. Latent Semantic Index (LSI) [5] is the method by using SVD to create an orthogonal space. After the orthogonal space is created, a compression operation is needed to compress the space from n dimensions to k dimensions, where, k < n. In LSI, a space R is created by applying SVD on the matrix D(f). Based on the above illustration on SVD for the basis vector rotation, different from that mentioned in [5], in our point of view, the space R is the space which is created by rotating the axes of the space F. The space compression is the operation to remove basis vectors from the space R. In LSI, the vectors correlating to the small singular values are removed. That is, if R is composed with n basis vectors, R >r1 r2 rn @ ,
X. Chen et al. / A Semantic Space Creation Method with an Adaptive Axis Adjustment Mechanism
49
the vector rn is the first candidate to be removed. Next will be rn-1, rn-2, … However, the issue is that it is not clear that the removed basis vectors really represent the semantic factors or not which are represented by the other un-removed vectors. In Example 3, based on LSI, the axis r2 is removed because it relates to the smallest singular value. However, in the case of Example 3, it is better to remove the axis r1. 3. The semantic space creation method with an adaptive axis adjustment mechanism 3.1 The basic idea of the method As we mentioned in the previous section that when two feature vectors fi and fj semantically correlate to each other, they are not pair-wise orthogonal and must be rotated into the correct position. When semantically correlated vectors q and d are mapped onto the vectors fi and fj on the space-F, vectors q and d are orthogonal on the space-F. If q represents a query vector and d represents a retrieval candidate vector, although they semantically correlate to each other, the retrieval candidate can not be found, because the scalar product of q and d is zero. Our idea is to rotate the axes, which represent the same semantic factors, and to combine them into new vectors. When two feature vectors fi and fj represent a same semantic factor, in our method, they are rotated and combined into a new vector ri: 1
ri
f1 f 2
f
i
f j .
fj
ri
d
q
fi
Fig. 4. The query vector q and the retrieval candidate vector d are overlapped on the vector ri which is created by combining the vectors fi and fj.
When the new created vector ri is used as the basis of the vector space instead of the vectors fi and fj, the vector d can be selected by the query because both the query vector
50
X. Chen et al. / A Semantic Space Creation Method with an Adaptive Axis Adjustment Mechanism
q and the retrieval candidate vector d are overlapped on the vector ri as shown in Figure 4.
In general, if a subspace with the basis of vectors fi, fj, …, fk represents a same feature on the space-F, a new semantic vector space, space-R, is created by rotating and combining those basis vectors on the subspace into a new basis vector ri. Let B represents the set of the vectors fi, fj, …, fk, the basis vectors of the new created semantic vector space can be represented as f l , if f l B,
rl ri
§ · ¨ ¦ f i ¸, f i B . ¨ ¦ f i © fiB ¸¹
1
fi B
Fig. 5. When the vectors fi and fj represent a same feature, they are rotated and combined into a new vector on the space-R. The vector rl equals to the vector fl on the space-R. The created space-R is a 2-dimensional space while space-F is a 3-dimensional space.
On the new created space, space-R, vectors rl and ri are pair-wise orthogonal because ri is on the subspace [fi, fj, …, fk] and the vector rl is orthogonal to the subspace as illustrated in Figure 5. In the following, we will present our semantic retrieval space creation method, which is referred to as Optimal Semantic Space Creation Method (OPTSS). The main purpose of the OPTSS method is to create optimal semantic spaces. This method requires that
X. Chen et al. / A Semantic Space Creation Method with an Adaptive Axis Adjustment Mechanism
51
learning data sets exist. An axis adjustment mechanism is used to rotate, combine and remove axes based on the semantic factors obtained from the learning data sets. The rotating, combining and removing operations are referred to as the OPTSS operation. In the following, we present the outline of our method. At first, an n u n unit matrix R is created as the retrieval space, where n is the number of the feature vectors. After that, basis vectors representing the same factors are searched from the learning data sets. When two basis vectors fi and fj are found representing a same semantic factor, the vector fi and fj are rotated and combined into a single basis vector. After the rotating and the combining operations, the distribution of the learning data is check to see if there are ‘noise’ data exist among them or not. If there are ‘noise’ data among the learning data, the basis vectors which represent the ‘noise’ factors are searched and removed from the space. The rotating, combing and removing operations on the basis vectors are the basic OPTSS operations to create the optimal semantic spaces. These operations are illustrated in Figure 6.
fk Removing fk
fj Rotating and Combining fi and fj
fi
Fig. 6. The feature vectors fi and fj which represent a same feature are combined into one vector. The vector fk which represents a ‘noise’ feature is removed.
3.2 Technical details of the semantic space creation method with the adaptive axis adjustment mechanism
In our method, the space creation operations are divided into two processing steps. The first step is to rotate and combine basis vectors. This step is efficient for improving the precision of queries. The second step is to remove the basis vectors representing the ‘noise’ factors. This step is efficient for improving the precision of the queries. The semantic factors are searched from the learning data sets and the basis vectors representing the ‘noise’ factors are searched by checking the distribution of the learning
52
X. Chen et al. / A Semantic Space Creation Method with an Adaptive Axis Adjustment Mechanism
data to see if there are ‘noise’ data among the them or not. In the case that each of the media data is represented by the n-feature vectors as a metadata, di
d1,i f1 d 2,i f 2 d n ,i f n ,
an n u n unit matrix R is created at first. After that an index vector Cr is generated from a learning data set L, which contains the index of all the features in the learning data set L. The elements in the learning data set L are the vectorized metadata of the media data. The indexing set Cr is created through the following steps: Step-1: Set Cr as an n-dimensional vector. Each element of the vector Cr is set to ‘0’. Step-2: For each element dj in the learning data set L, if the feature value dk,j is greater than a threshold e, the k-th element of the vector Cr is set to ‘1’. The non-zero elements of the index vector Cr indicate the feature vectors that should be rotated and combined. After the index vector Cr is created, the rotating and the combining operations are implemented through the following steps: Step-3: If the k-th element is the first non-zero element of the index vector Cr, the k-th column of the matrix R is replaced by the vector Cr. This column is identified as RCr. Step-4: Remove all the columns of the matrix R which are indicated by the non-zero elements of the index vector Cr except the column RCr which is replaced by the vector Cr. After the above four processing steps, the rotating and the combination operations are finished. The removing operation is implemented through the following steps: Step-1: Set q as an n-dimensional zero vector. Step-2: If the k-th element is the first non-zero element of the index vector Cr, set the k-th element of the vector q to ‘1’. Step-3: For each metadata di, calculate the inner product of di with the vector q on the semantic space R: pi d i u R u q'uR ' . This step will be repeated until the inner products of all the metadata are
X. Chen et al. / A Semantic Space Creation Method with an Adaptive Axis Adjustment Mechanism
53
calculated. Step-4: Reversely sort the metadata based on the inner products calculated by Step-3. The ranked position of di is stored into a variable Ranki, where, i is the index used to indicate the metadata di, its inner product and its ranking position value Ranki. If the ranking value of di is one, Ranki = 1, di is ranked at the top position. Step-5: For each combined feature vectors in the semantic space R, remove one of the combined feature vector fk from the space. Store all the ranking values of the metadata in the learning data set L. If di is the element of the set L, Ranki is stored into the variable Prev_Ranki. Excuse Step-3 and Step-4 again. For each element di in the set L, if the new ranking position of di lower than its previous ranking position after the feature vector fk is removed from the space R, that is, Ranki is greater than Prev_Ranki, it means that the feature vector fk can not be removed from the space. Otherwise, the feature vector fk is removed from the space R. This step will be continued until all the combined feature vectors are tested whether they can be removed from the space R or not. After the above removing processing, the optimal semantic retrieval space R is created. 4. Experiments
Experiments are performed and the experimental results are presented which shows that optimal semantic retrieval space can be created by using OPTSS method proposed in this paper. When the semantic space is optimally created, retrieval candidate documents will be appropriately and sharply distributed on the space. Therefore, the recall and precision of queries will be greatly improved compared to that before the OPTSS operations are performed. We will also show the recall and precision based on OPTSS method and those based on VSM and LSI (Latent Semantic Indexing), respectively. 4.1 Evaluation method
In our experiments, 8,557 English documents are randomly extracted as the retrieval candidate documents from the “Test collection 1 (NTCIR-1) 1,” which contains about 160,000 English documents. These documents are summaries of conference papers presented at academic or professional conferences hosted by Japanese academic 1
NTCIR is a project organized by National Center for Science Information Systems in Japan. http://research.nii.ac.jp/ntcir/
54
X. Chen et al. / A Semantic Space Creation Method with an Adaptive Axis Adjustment Mechanism
societies. The 8,557 documents used in the experiments are extracted from ten query categories of “Robot,” (identified as Q-1), “Document image understanding,” (identified as Q-4) and “Feature dimension reduction,” (identified as Q-5),…, etc.. In NTCIR-1, correct document data sets are prepared for the query categories. We randomly extracted 5 correct documents from each of the ten query categories, respectively. That is, for a query, there exist only five correct documents which are distributed in the 8,557 documents. In the experiments, the “stop-words”, like the article, conjunction, etc., are removed as the previously processing and stemming processing is also performed previously. 811 English words are extracted as a term set from the 50 correct documents. A 8557 u 811 document-term matrix M is created, in which, each element is the appearing frequency of a term in a document. Each row of the matrix M represents a document vector di which is an 811 dimensional vector. In the matrix, d1, d2,…, d5 are the vectors representing the correct documents of the query category Q-1; d1006, d1007,…,d1010 are the vectors representing the correct documents of the query category Q-4; and d1978, d1979,…,d1982 are the vectors representing the correct documents of the query category Q-5. Ten queries, q1, q2,…, q10 are used for search the correct documents of Q-1, Q-2,…, Q-10, respectively. Each query contains one keyword. For example, the keywords of q1, q2 and q3 are “robot”, “image” and “dimension”, respectively. Figure 7 shows the precision rate and the recall rate of the retrieval result of q1 before the OPTSS operations are performed. The precision goes down from 0.0625 to 0.00058432 as the increasing of the recall.
㪩㪼㪺㪸㫃㫃㪄㫇㫉㪼㪺㫀㫊㫀㫆㫅
㪈㪅㪉 㪈 㪇㪅㪏 㪩㪼㪺㪸㫃㫃 㪧㫉㪼㪺㫀㫊㫀㫆㫅
㪇㪅㪍 㪇㪅㪋 㪇㪅㪉 㪇 㪈
㪉
㪊
㪋
㪌
㪥㫆㪅㩷㫆㪽㩷㫋㪿㪼㩷㪺㫆㫉㫉㪼㪺㫋㩷㪻㫆㪺㫌㫄㪼㫅㫋㫊
Fig. 7. The precision and the recall before the performing of the OPTSS operations
X. Chen et al. / A Semantic Space Creation Method with an Adaptive Axis Adjustment Mechanism
55
Figure 8 shows the recall and the precision of OPTSS which is created by adding the five correct documents of Q-1 into the learning data set L. Our experimental result shows that 104 feature vectors are rotated and combined, and 42 feature vectors are removed for creating the optimal semantic space. From Figure 8, it is can be found that the precision is improved great much. The precision reaches as high as 1.0 and goes not bellow than 0.8 when the recall increases from 0.2 to 1.0.
㪩㪼㪺㪸㫃㫃㪄㫇㫉㪼㪺㫀㫊㫀㫆㫅
㪈㪅㪉 㪈 㪇㪅㪏 㪩㪼㪺㪸㫃㫃 㪧㫉㪼㪺㫀㫊㫀㫆㫅
㪇㪅㪍 㪇㪅㪋 㪇㪅㪉 㪇 㪈
㪉
㪊
㪋
㪌
㪥㫆㪅㩷㫆㪽㩷㫋㪿㪼㩷㪺㫆㫉㫉㪼㪺㫋㩷㪻㫆㪺㫌㫄㪼㫅㫋㫊
Fig. 8. The precision rate and the recall rate of the retrieval result of q1 on the semantic space created by the proposed method.
Such high precisions support the conclusion that the created semantic retrieval space is optimal one. This conclusion is supported by the experiment results shown in Table 1. Table 1 shows the recall and the precision rates of the three queries, q1, q2 and q3, which are obtained by using OPTSS. It can be found from the table that very high precision is obtained by creating the semantic space based on our OPTSS method. The precision of q2 reaches as high as 1.0 when the recall reaches to 1.0. The lowest precision is 0.625 which is the result of q3. The table also shows that the total correct documents are ranked in the top 6, 5 and 8, respectively. Table 1. The recall and the precision obtained by using OPTSS 㫈㪈
㫈㪉
㫈㪊
㪩㪸㫅㫂 㪛㫆㪺㫌㫄㪼㫅㫋㩷㪥㫆㪅 㪩㪼㪺㪸㫃㫃㩷㫉㪸㫋㪼 㪧㫉㪼㪺㫀㫊㫀㫆㫅㩷㫉㪸㫋㪼 㪩㪸㫅㫂 㪛㫆㪺㫌㫄㪼㫅㫋㩷㪥㫆㪅 㪩㪼㪺㪸㫃㫃㩷㫉㪸㫋㪼 㪧㫉㪼㪺㫀㫊㫀㫆㫅㩷㫉㪸㫋㪼 㪩㪸㫅㫂 㪛㫆㪺㫌㫄㪼㫅㫋㩷㪥㫆㪅 㪩㪼㪺㪸㫃㫃㩷㫉㪸㫋㪼 㪧㫉㪼㪺㫀㫊㫀㫆㫅㩷㫉㪸㫋㪼 㪈 㪋 㪇㪅㪉 㪈 㪈 㪈㪇㪇㪐 㪇㪅㪉 㪈 㪈 㪈㪐㪎㪏 㪇㪅㪉 㪈 㪉 㪊 㪇㪅㪋 㪈 㪉 㪈㪇㪇㪍 㪇㪅㪋 㪈 㪉 㪈㪐㪏㪉 㪇㪅㪋 㪈 㪊 㪈 㪇㪅㪍 㪈 㪊 㪈㪇㪈㪇 㪇㪅㪍 㪈 㪊 㪈㪐㪏㪇 㪇㪅㪍 㪈 㪌 㪌 㪇㪅㪏 㪇㪅㪏 㪋 㪈㪇㪇㪏 㪇㪅㪏 㪈 㪋 㪈㪐㪏㪈 㪇㪅㪏 㪈 㪍 㪉 㪈 㪇㪅㪏㪊㪊㪊㪊 㪌 㪈㪇㪇㪎 㪈 㪈 㪏 㪈㪐㪎㪐 㪈 㪇㪅㪍㪉㪌
56
X. Chen et al. / A Semantic Space Creation Method with an Adaptive Axis Adjustment Mechanism
Table 2. The recall and the precision obtained by using VSM 㫈㪈
㫈㪉
㫈㪊
㪩㪸㫅㫂 㪛㫆㪺㫌㫄㪼㫅㫋㩷㪥㫆㪅 㪩㪼㪺㪸㫃㫃㩷㫉㪸㫋㪼 㪧㫉㪼㪺㫀㫊㫀㫆㫅㩷㫉㪸㫋㪼 㪩㪸㫅㫂 㪛㫆㪺㫌㫄㪼㫅㫋㩷㪥㫆㪅 㪩㪼㪺㪸㫃㫃㩷㫉㪸㫋㪼 㪧㫉㪼㪺㫀㫊㫀㫆㫅㩷㫉㪸㫋㪼
㪩㪸㫅㫂 㪛㫆㪺㫌㫄㪼㫅㫋㩷㪥㫆㪅 㪩㪼㪺㪸㫃㫃㩷㫉㪸㫋㪼 㪧㫉㪼㪺㫀㫊㫀㫆㫅㩷㫉㪸㫋㪼
Table 2 shows the recall and the precision before the performing of the OPTSS Comparing Table 1 with Table 2, it is clarified that the space created by using OPTSS is the optimal one. 㪈㪅㪉 㪈
㪧㫉㪼㪺㫀㫊㫀㫆㫅
㪇㪅㪏 㪇㪅㪍 㪇㪅㪋 㪇㪅㪉 㪇 㪇
㪇㪅㪉
㪧㫉㪼㪺㫀㫊㫀㫆㫅㩷㫆㫅㩷㪦㪧㪫㪪㪪
㪇㪅㪋
㪇㪅㪍 㪩㪼㪺㪸㫃㫃 㪧㫉㪼㪺㫀㫊㫀㫆㫅㩷㫆㫅㩷㪣㪪㪠
㪇㪅㪏
㪈
㪈㪅㪉
㪧㫉㪼㪺㫀㫊㫀㫆㫅㩷㫆㫅㩷㪭㪪㪤
Fig. 9. The recall precision rates obtained by OPTSS, LSI and VSM
Experiments on VSM and LSI are also performed. In the experiments on LSI, forty-two basis vectors are removed from the space. That is, the space is compressed from 811 dimensions to 769. The experimental result is shown in Figure 9. Figure 9 also shows the recall and the precision rates obtained by OPTSS, LSI and VSM, respectively. It is clear that when the semantic space is optimally created, very high precision can be obtained.
X. Chen et al. / A Semantic Space Creation Method with an Adaptive Axis Adjustment Mechanism
57
5. Conclusion
In this paper, we have presented a semantic media data search space creation method with an adaptive axis adjustment mechanism for adjusting the semantic expression vectors, which representing semantic correlated factors, and the vectors which representing the ‘noise’ factors that make the precision of queries down. By using this method, we can create an optimal semantic space for extracting semantically related and appropriate information adapting to the individual query requirements. The basic idea of our method is to rotate and combine the semantic expression vectors representing the semantic correlated factors and to remove the vectors representing the ‘noise’ factors. The mechanism on how to find the semantic correlated vectors from the learning data set is introduced in technical details. The mechanism for finding the ‘noise’ correlated vectors is also introduced. Experimental results are presented for demonstrating the efficiency of the proposed method. The experimental results also clarify that the optimal semantic retrieval spaces are created by using our method. Another important feature of our method is that the keywords (related to the recall) and the ‘noise’ terms (related to the precision) can be found by our method. In the future work, we will further improving the processing quality of the learning mechanism. Reference
[1] Batini, C.,Lenzelini, M. and Nabathe, S.B., “A comparative analysis of methodologies for database schema integration,” ACM Comp. Surveys, Vol. 18, pp.323-364, 1986. [2] Bright, M.W., Hurson, A.R., and Pakzad, S.H., “A Taxonomy and Current Issues in Multidatabase System,” IEEE Computer, Vol.25, No.3, pp.50-59, 1992. [3] Chen, X. and Kiyoki, Y., “A query-meaning Recognition Method with a Learning Mechanism for Document Information Retrieval,” Information Modelling and Knowledge Bases XV (IOS Press), Vol. 105, pp. 37-54, 2004. [4] Chen, X. and Kiyoki, Y., “A Dynamic Retrieval Space Creation Method for Semantic Information Retrieval,” Information Modelling and Knowledge Bases XVI(IOS Press) Vol. 121, 46-63, 2005. [5] Deerwester, S., Dumais, S. T., Landauer, T. K., Furnas, G. W. and Harshman, R. A., “Indexing by latent semantic analysis,” Journal of the Society for Information
58
X. Chen et al. / A Semantic Space Creation Method with an Adaptive Axis Adjustment Mechanism
Science, vol.41, no.6, 391-407, 1990. [6] Kitagawa, T. and Kiyoki, Y., “A mathematical model of meaning and its application to multidatabase systems,” Proceedings of 3rd IEEE International Workshop on Research Issues on Data Engineering: Interoperability in Multidatabase Systems, pp.130-135, April 1993. [7] Kiyoki, Y., Kitagawa, T. and Hitomi, Y., “A fundamental framework for realizing semantic interoperability in a multidatabase environment,” Journal of Integrated Computer-Aided Engineering, Vol.2, No.1(Special Issue on Multidatabase and Interoperable Systems), pp.3-20, John Wiley & Sons, Jan. 1995. [8] Kiyoki, Y., Kitagawa, T. and Hayama, T., “A metadatabase system for semantic image search by a mathematical model of meaning,” ACM SIGMOD Record, Vol.23, No. 4, pp.34-41, Dec. 1994. [9] Kiyoki, Y. and Kitagawa, T., “A semantic associative search method for knowledge acquisition,” Information Modelling and Knowledge Bases (IOS Press), Vol. VI, pp.121-130, 1995. [10]Kolodner, J.L., “Retrieval and organizational strategies in conceptual memory: a computer model,” Lawrence Erlbaum Associates, 1984. [11]Krikelis, A., Weems C.C., “Associative processing and processors,” IEEE Computer, Vol.27, No. 11, pp.12-17, Nov. 1994. [12]Ogden, C.K., “The General Basic English Dictionary,” Evans Brothers Limited, 1940. [13]Potter J.L., “Associative Computing,” Frontiers of Computer Science Series, Plenumn, 1992. [14]Raghavan, V. V. and Wong, S. K. M. , “A critical analysis of vector space model for information retrieval,” Journal of the American Society for Information Science, Vol.37 (5), p. 279-87, 1986. [15]Salton, G., “Introduction to Modern Information Retrieval,” McGraw-Hill, 1983. [16]Sheth, A. and Larson, J.A., “Federated database systems for managing distributed, heterogeneous, and autonomous databases,” ACM Computing Surveys, Vol.22, No.3, pp.183-236, 1990. [17]Williams, Lippincott and Wilkins, “Stedman's Electronic Medical Dictionary VERSION 5.0,” A Wolters Kluwer Company, 2000 [18]“Fifteenth Edition Harrison's Principles of Internal Medicine CD-ROM VERSION 1.0,” McGraw-Hill, 2001 [19]“Longman Dictionary of Contemporary English,” Longman, 1987.
Information Modelling and Knowledge Bases XIX H. Jaakkola et al. (Eds.) IOS Press, 2008 © 2008 The authors and IOS Press. All rights reserved.
59
Storyboarding Concepts for Edutainment WIS Klaus-Dieter Schewe1 and Bernhard Thalheim2 1 Massey University, Department of Information Systems & Information Science Research Centre, Private Bag 11 222, Palmerston North, New Zealand 2 Christian Albrechts University Kiel, Department of Computer Science, D-24098 Kiel, Germany
[email protected]
[email protected]
Abstract. Edutainment web information systems must be adaptable to the user, to the content currently available, to the technical environment of the user, and to the skills, abilities and needs of the learner. This paper provides conceptions that support this kind of adaptivity and sophisticated support.
1 Introduction 1.1 Edutainment Web Information Systems E-learning is now nearly as popular as e-commerce and e-business. The providers range from universities over non-profit organisations to professional training institutions. Furthermore, sites of museums, exhibitions, etc. can be counted as learning web information systems. In general, every data-intensive information system that is realised in a way that users can access it via web browsers will be called a web information system (WIS). The major intention of a learning WIS should be to support learning. In the case of socalled edutainment systems the provided knowledge is usually easy to grasp. The message is that learning can be fun. The usage of a learning system depends on whether the control of the learning process is left to the user or the system. In both cases, however, it is assumed that the users are willing to learn and match the required prerequisites. The content of a learning site depends on the area that is to be taught. Learning sessions are used as structural means, and navigation through these sessions may be organised in linear form or as a directed acyclic graph [ST05]. Systems may be completely passive allowing only material to be read or downloaded. Other systems may involve upload mechanisms for assessment of the learning progress. According to the progress made, a system may even provide feedback to its user. We concentrate our work on active learning. The functionality of learning sites mainly supports the navigation through the site, i.e., the navigation through the learning material. In contrast to other systems, this navigation is a long-term progress with usually many interruptions. More sophisticated systems provide system-driven repetition and feedback. Apparently, the functionality of such systems is still a matter of research. Also, personal information needs can be supported by providing an interface to e-mail.
60
K.-D. Schewe and B. Thalheim / Storyboarding Concepts for Edutainment WIS
1.2 Achievements and Challenges of Edutainment WIS A large number of websites provide learning and edutainment services has already been developed. Examples of technology-supported learning include computer-based training systems, interactive learning environments [DN03], intelligent computer-aided instruction systems [RK04, BGW92], distance learning systems, and collaborative learning environments. Developers have learned a number of lessons. Edutainment should neither be considered to be yet another presentation from for classical instructionistic learning nor a means for presentation of content that has been used for lecturing. Edutainment should not be understood as the enrichment of learning by multimedia facilities but must concentrate on content that is presented in a pleasing form, that is easy to use and to understand, that is enriched by functionality which corresponds to the content, and that allows to control the progress in learning. Edutainment might support everybody to participate in learning activities on any place at any time within any team. Learning cultures will however limit this globalisation claim. Learner have their profile and their portfolio. Main kind of learning stories are complementary learning, self-organised learning and continuing education on demand. Learner must thus be supported in developing their own learning space and in understanding and developing their learning abilities. Controlling and assessment cannot be completely automatised. Each user is different from the other. The context differs. The learning history is different. At the same time, learners often need an immediate feedback. Edutainment can be supported by a number of devices ranging from computers and PDA’s to mobile phones. We thus distinguish between electronic learning (e-learning) systems and mobile learning (m-learning) systems. Mobile phones and mobility in general are changing people’s way of working, communication and learning. The differences between formal learning, informal learning and way of working will diminish. The real added value of m-learning is in the area of informal learning whereas e-learning may cover formal and informal learning. Scenarios and content are multifaceted, depend on the presentation device, and must be adaptable to the learner, to the learning community. The type of learning will be different from classical ones. The style of learning changes to pro-active formal learning with self study content, virtual classroom and trainer based facilitation. Learning is social, arguing, reflecting, articulating, and debating with others. Majority of learning in the web is informal learning. It is based on ad-hoc information sharing and on communications and collaboration. The functionality of edutainment WIS is also based on functions such as participation, contribution, and annotation. 1.3 Scope of this Paper This paper introduces some new conceptions we have already used in our edutainment WIS projects: DaMiT (a data mining tutoring workbench), KoPra (cooperative learning of database programming), and Learning Lusitia (continuing life-long learning for engineers and alumnis). We extend the classical concept of learning objects [LOM00, LTS00] to open learning objects, show how scenario development [For94] leads to sophisticated support for didactics, and provide an insight into development of appropriate functionality. Additionally we show how brands of edutainment WIS, actor specification, learning scenarios, content and functionality can be developed for sophisticated edutainment WIS.
K.-D. Schewe and B. Thalheim / Storyboarding Concepts for Edutainment WIS
61
2 The Five General Characteristics of Edutainment WIS 2.1 The Brand of Edutainment WIS The pattern or brand of the edutainment WIS P W 2U A (Provider P, knowledge W, user U, activities A) generalises the classical who-to-whom pattern (e.g. B2B). It is specialized for edutainment WIS: Provider: Providers are currently mainly educational institutions or educational communities. In future, we will observe commercialisation of education. So, main providers are going to be companies. If a provider is a singleton person then this provider plays the role of the teacher. Product dimension: Since control and assessment of learning progress is a still unsolved issue and appropriate presentation of complex information is often not feasible, edutainment WIS concentrate on easy-to-understand information or easy-to-grasp knowledge. The associated auxiliary scenarios are based on functions such as validate , control and advice . Therefore, the brand can be characterised by P K 2Clearn,validate,control,advice . User dimension: Users of edutainment WIS are mainly private people. They may be pupils or students, people seeking for continuing education, workers in companies with specific portfolio, or just people interested in auxiliary information. Users can be also groups. The main behavior of users is characterised by the role of the learner or of student. Activity dimension: Activities are currently centered around learning, searching for content, collecting content, and solving exercises. Activities also include to ask questions, to act in teams for problem solving, and to discuss issues associated with the learning material. Edutainment (learning) sites (BK 2Clearn , CK 2Clearn ) are then specialized to the brands: Teachercontent chunks 2Studentreceive,respond,solve in teams,raise questions,possibly apply Teachercontent chunks 2Studentrecognise,listen,work on it,solve exercises,ask urgent questions TeacherKnowledge 2Studentdiscuss,get feedback,work on it TeacherKnowledge 2Student Groupdiscuss,get feedback,work on it and TeacherW isdom 2Studentdiscuss,get feedback,work on it . 2.2 Actors in Edutainment Edutainment WIS are currently mainly or exclusively supporting the pupil or student actor. The behaviour of actors might however be more complex: Pupils obtain knowledge through teachers, their schedules, and their abilities. They need guidance, motivation, and control. Collaborating or cooperating students act in a collaboration depending on a cooperation profile and rights and roles. Communication partners exchange content, discuss and resolve questions, seek for hints or for better understanding. Supporting and motivating partners are users with control, motivation and supporting functions.
62
K.-D. Schewe and B. Thalheim / Storyboarding Concepts for Edutainment WIS
Teachers act in various roles, obligations, rights, and in a variety of involvement. The nature of the activities that constitute teaching depends more on the age of the persons being taught than on any other one thing. Users are different in their history. We distinguish between learner that initially seek to increase their knowledge and skills and users that are seeking continuing education. The first group of learners needs some guidance. Therefore, the learning style is based on pedagogic teacher-based learning. The latter are self-organised. We call then andragogic learners. The differences between the two learning styles are given in the following table: andragogic self-organized learning pedagogic teacher-based learning independent learner dependent learner self-regulated learning not necessarily self-regulated learning self-motivated learning need to be motivated reflective needs help for learning arguably needs help for learning analytical needs help for learning situated in real world context structured engaged with knowledge 2.3 Learning Scenarios and Stories Modelling of e-learning scenarios is more difficult due to the variety of didactic approach, to the variety of learners, to the variety of tasks and the context of the site. For this reason, modelling of dialogue scenes and dialogue steps is more generic than modelling of simple interaction steps that usually lead to deterministic runs through the story space. We, thus, distinguish a number of associations among steps and scenes depending on the content of the learning element. Our solution to this challenge is based on generic parameters that are instantiated depending on the learner, the history, the context etc. Each learning unit is specified by a contextfree expression with a set of parameters. These parameters are instantiated depending on the learner profile, the learner task portfolio, the media objects, the learner computational environment, the data to be used in exercises or algorithms, the presentation environment, and the available and accessible learning elements. Learning scenarios (learn ) are classically based on general learning styles: Sequenced learning is based on a curriculum sequencing in active or passive scenarios based on classical pedagogical approaches. Typical established pedagogical approaches are conditioning, operated learning, model learning, cognitive approaches. Sequencing is extensively studied based on classical didactics. Active scenarios model behaviour of active actors. A typical metaphor to be applied is the shopping bag. Passive scenarios did not get the success that has been expected during the 80ies and 90ies. Tutoring systems are applicable for advisory systems, for help desks, or for training physical skills. Interactive learning is either based on self-organized or content-sequenced or habit-regulated or publish-subscribe scenarios. Group learning is based on a cooperative setting (integrated sub-tasks, black boarding) or in a collaborative setting (cooperation, discussion, development of solution by all members).
K.-D. Schewe and B. Thalheim / Storyboarding Concepts for Edutainment WIS
63
Sequenced learning is currently mainly based on didactic approaches, i.e. on instructions, classical content and classical hermeneutics. Due to the sequential way dependencies among learning units are rather simple. Learning scenes use reception techniques, explication of the learning content, context association, deriving understanding and interpretation, and finally integrating the essence of the content into the knowledge of the learner. 2.4 Content in Edutainment Content used in edutainment WIS is mainly easy-to-grasp and easy-to-understand information or knowledge. Most important associated scenarios are validate , control and advice . Content is given in a large variety. We need therefore to consider the kind of content, the activities that can be associated with the content, the characterisation and annotation of content, and finally the quality characteristics of content. Edutainment content delivery, storage, retrieval, and extraction is still an open research issue. Edutainment WIS provide content to learners depending on their learning task, their personality, their working environment, their learning history and the policy of the content provider. This challenge is more complex than the challenge to generate “content”. Content can be represented by media objects. Content for learning must have high adaptivity. We distinguish a variety of content or generally knowledge: • knowledge and abilities for orientation, e.g., explanation, presentation, history, facts, surveys, overview; • knowledge and abilities for application, skills, abilities rules, procedures, principles, strategy, and laws; • knowledge and abilities for explanation, e.g., ‘why’-knowledge (proof, causal) and ‘what’knowledge (definition, description, argument, assumption, reflection); • knowledge and abilities on sources, e.g. archives, documents, citation, reference, and links; • knowledge and abilities for solving problems, e.g., sample solutions, analogs, training solutions, discovery solution, and examination. All media objects can be provided independently of the learner. We need however to consider also differences. Background knowledge leads to different speed and reception by the learner. Work abilities and habits influence current work. The learning style must be considered in many facets. The social environment is based on cultural and psychological differences. The history of the learning process should be considered if we want to avoid annoying repetitions. The learning portfolio influences occasion, intention and motivation. The learning object presentation is or is not acceptable depending on the profile of the learner. The learning environment is modelled by many technical facets. Content change management allows to provide content with or without refresh. The payment profile may result in content reduction. Learning objects are elements of a new type of computer-based instruction influenced by the object-oriented paradigm of computer science. Learning objects are defined here as any media object which can be used, re-used or referenced during technology supported learning. Examples of learning objects include multimedia content, instructional content, learning objectives, instructional software and software tools, and persons, organizations, or events referenced during technology supported learning. The IEEEs Learning Technology Standards
64
K.-D. Schewe and B. Thalheim / Storyboarding Concepts for Edutainment WIS
Committee has developed an internationally recognized definition of “learning object”: “any entity, digital or non-digital, that can be used, re-used, or referenced during technology supported learning”. This definition is extraordinarily broad. A large variety of learning elements is used in edutainment WIS. Course elements are lecture notes and comments. Exercise material may be given in a textual or in an interactive form Illustration material is often based on animations, complex multimedia elements or on links. Algorithms can be provided in an executable form, as virtual machine or via an interface to the server. Input data for algorithms can be provided with the learning elements or through other elements in the network. 2.5 General Functionality Based on Word Fields During requirements capture WIS design and development we use natural language descriptions for analysing the activities of the users: What are these activities? Does the description indicate any sequencing or continuation? What data is needed for or used by the activities? What are the relationships between these data? As user activities can be described by verbs, we suggest analysing the corresponding word fields. A word field [Kun92, SSS90, BZKK+ 05] is a linguistic system in which similar words that describe the “same” basic semen and are used in the same context are combined to a common structure and data set. In contrast to common synonym dictionary, word fields define the possible and necessary actors, the actions and the context. Word fields can be used for verbs, nouns and adjectives. Functionality of edutainment WIS is also determined by the main word fields we observe for learning. Main word fields applied in learning are: Learn: Learning is a very complex activity. It includes to gain knowledge or understanding of or skill in by study, instruction, or experience. Additionally, learning is associated with memorizing, to come to be able to perform some task, and to know this ability. Learning is based on obtaining content and discover the concepts behind. It is also based on facilities for annotation, ordering, and integrating A user obtains the role of a learner or student. Learners are usually supported by other actors who teach and instruct. Learners determine content with certainty, usually by making an inquiry or other effort. They check the content, find out whether it is useful or they need additional content. Know: Learning is based on skills, abilities, and knowledge. It target on their improvement. The improvement should be measurable. The learning success is then examined. To know means to be cognizant or aware of a fact or a specific piece of information and to possess knowledge or information about. It may include also to know how to do or perform something. The learner obtains firsthand knowledge of states, situations, emotions, or sensations. The change in knowledge is acknowledged and recognized by other actors who can accept to be what is claimed. The word field know used in learning is different from one used in identity or information WIS. Master: Learning is usually intending to master problems and to become completely proficient or skilled in an area. This mastership is closely related to practise and experiment with the new knowledge. The learner has a firm understanding or knowledge of and is on top of of a problem.
K.-D. Schewe and B. Thalheim / Storyboarding Concepts for Edutainment WIS
65
Skill: Skills are abilities that have been acquired by training, e.g. abilities to produce solutions in some problem. A user acting as a learner is possibly trained until he/she obtained these skills. Study: The learner is engage in study and undertakes formal study of a subject Studying requires an endeavor and a try. In learning WIS studying is based on read in detail especially with the intention of learning. Therefore the presentation of the material and the storyboard is an essential part. Studying does not only mean to view content but to check over, to check up, to con, to examine, to inspect, and to survey it. This activity is performed attentively and in detail. Learners need to mind, to perpend, to think (out or over), and to weigh. Therefore, users need time and workplaces. Studying can be performed by oneself without any teacher, supporter, or observer. Studying is often based on the existence of a specific workplace and of a specific workspace. Learners may thus have a study place and workplaces which are given to them, which might be rented and which can be exported to the user. We observe that these word fields have a more complex structure. They lead to functions which require support functions. The difference to the word fields discussed for business WIS is their iterative application. This kind of behaviour has already been observed for community services. Additionally, learning can be performed within temporal communities. Other related word fields are discover, ascertain, catch on, determine, teach, educate, judge, evaluate, advise, innovate and discuss. Learning word fields can be combined and thus form a didactic story. Learning word fields are going to be combined with reasoning word fields such as analogise, analyse, cause, classify, conjecture, counter-example, formalise, generalise, sharpen, specialise, unknown, and weaken. General activity word fields that appear in any story are folded into our learning word fields. Typical general activities are characterised by answer, comment, compose, decompose, define, draw, effect, example, extend, fact, inquire, instantiate, and reduce. 3 Novel Concepts of the Edutainment WIS DaMiT, KoPra, and Learning Lausitia The DaMiT system [JMR+ 03] supports sequenced and interactive learning. Interactive learning is still an open research issue. We have developed and extensively used an implementation in the KO P RA project [SBZ04]. Learners act in a collaborative setting depending on their needs and the goals of the learning program in the project Learning Lausitia. 3.1 Supporting Didactics by SiteLang The website description language SiteLang has been introduced in [TD01]. A scenario is an application-oriented view of parts of the storyboard. The storyboard consists of a set of scenarios, one of which is the main scenario, whereas the others define scenes, a plot specified by a SiteLang process, a set of roles, a set of user types, a set of tasks each associated with a goal, and a set of constraints comprising deontic constraints for the rights and obligations of roles, preference rules for user types, and other dependencies on the plot. We may distinguish four main kinds of general scenarios for the three learning styles: Content scenarios are centered around the content chunks or content suites to be delivered to the learner, to be received by the learner, and to be integrated into his/her knowledge.
66
K.-D. Schewe and B. Thalheim / Storyboarding Concepts for Edutainment WIS
Control scenario are based on assessment or control of the success of learning. They involve at least two different actors: learners and controllers. A third actor involved may be the advisor. Workspace scenario are auxiliary scenario that help the learner to organise the learning material, that enhance the learner content space with memos, with excerpts, own solutions, and collaboration notes. Collaboration scenario support the learner to learn in groups, to communicate with their partners and with other actors such as teachers and advisors, to coordinate activities and to cooperate during solution of exercises, content reception and development of solutions. These four general scenarios can be seen as relatively independent scenarios that can be combined with each other depending in the learning style. We develop the story space by the didactic scenario quadruple in Figure 1 on the basis of relatively independent scenarios and in the second step through their combination. workspace scenario collaboration scenario
content scenario
control scenario Figure 1: The didactic scenario quadruple
Edutainment didactics is currently developed for most learning WIS from scratch and by doing. We decided to develop such didactics systematically. Edutainment didactics can be based on general learning storyboards. Since learning is one of the most complex activities we support a number of additional general stories within the story space: Teaching: Teaching is a complex activity that goes beyond instruction, that includes a process of formal training, that is based a body of specialized knowledge, and satisfies a set of standards of performanceintellectual, practical, and ethical. Teaching intends to learner to know something, to know how to accustom to some action or attitude, and to know the consequences of some action. Educating: Education is similar to teaching but more intention based. It includes training by formal instruction and supervised practice especially in a skill, trade, or profession. It targets to persuade or condition to feel, believe, or act in a desired way. Learners are mentally, morally, or aesthetically educated especially by instruction. Judging and evaluating: Edutainment also is often oriented to form an opinion about through some weighing of evidence and testing of premises and to decide a matter as a judge. Learner are trained in determining or fixing the value of a matter, in determining the significance, worth, or condition of usually by some kind of appraisal and study. Advising and discussing: Advising and discussion scenarios are similar to those we have already considered for community WIS. Advising means to be able to use the background
K.-D. Schewe and B. Thalheim / Storyboarding Concepts for Edutainment WIS
67
knowledge of the learner or the knowledge given by the content chunks for generation of advices or information or for consulting. Discussion techniques are applied in edutainment whenever we need to investigate by reasoning or to argument or to discourse about in order to reach conclusions or to convince. Discussion is an exploration technique that implies a sifting of possibilities especially by presenting considerations pro and con. Innovating: Innovation introduces a new idea, new content or a new activity in order to effect a change. These different learning scenarios can be compiled or integrated within the story space. We use for integration, ordering and hierarchical presentation of scenarios the theory of KAT’s. This combination is necessary whenever we need to consider different didactic approaches: Critical-constructive didactics see learning through interaction via goals. Learn-theoretical didactics consider learning through dialogues between actors. Goal-oriented learning is based on a separation of the intention space into subspaces with sub-processes. Cybernetical didactics uses regulated processes for story development. Critical-constructive didactics are based on interactions, repetitions, and obstruction. Curriculum planing extends classical sequenced or blended learning by schedules, goals, and steps. 3.2 Storyboard Pattern Scenario typically proceed in one or (usually) more scenes. Scenario are describing the ways how the work is performed and are based on didactics, goals, user purposes, and content. A storyboard contributes to achieving a purpose or goals. Scenes are composed of other scenes. We assume that scenes can be hierarchically formed based on basic scenes. Typical basic scenes are the information seeking scene, the collaboration scene, the assessment scene, the result integration scene and the problem solution scene. Scenes can be composed with another scene based on sequential, parallel, alternative etc. composition operators. It seems to be obvious that these scenes have a common structure and a common behaviour. We thus can extract general stories or scenarios. These general stories use general scenes or scene pattern. Patterns represent recurring solutions to software development problems within a particular context. These scene pattern are refined to the concrete scenes. Let us consider a typical scene pattern that involves a number of learners. Problem solution scenes are one of the most complex scenes in edutainment WIS. They typically consist of an orientation or review subscene, of a problem solution scene performed in teams, and of a finalisation subscene. Problem solution scenes are composed of a number of subscenes that can be classified into: • Review of the state-of-affairs: The state of the problem solution is reviewed, evaluated, and analysed. Obligations are derived. Open problem tasks can be closed, rephrased or prioritised.
68
K.-D. Schewe and B. Thalheim / Storyboarding Concepts for Edutainment WIS
• Study of documents and resources: Available documents and resources are checked whether they are available, adequate and relevant for the current scene, and form a basis for the successful completion of the scene. • Discussions and elicitation with other partners: Discussions may be informal, interviewbased, or systematic. The result of such discussions is measured by some quality criteria such as trust or confidence. They are spread among the partners with some intention such as asking for revision, confirmation, or extension of the discussion. • Recording and documentation of solutions: The result of the scene is usually recorded in one solution proposal. • Classification of solutions, requirements, results: Each result developed is briefly examined individually and in dependence of other results from which it depends and to which it has an impact. • Review of the problem solution process: Once the result to be achieved is going to be recorded the solution is examined whether is has the necessary and sufficient quality, whether is must be revised, updated or rejected, whether there are conflicts, inconsistencies or incompleteness or whether more may be needed. If the evaluation results in requiring additional scenes or subscenes then the scene or the subscene is going to be extended by them. This classification is based on the general problem solution framework discussed in [PP45]. Problem solution scenes are usually iterative and thus cyclic as displayed in Figure 2. The Problem solution scene j Application area understanding Y K
Evaluation of solutions
U Deployment of solutions
j Phenomenon understanding :
j y
U Solution preparation K zU Solution development
Figure 2: The subscenes in the edutainment problem solution scene
general frame to problem solution is then based on the problem solution or search problem: Problem characterisation with abstracting from non-essential parts; Context injection for simplification of the problem and of the solution;
K.-D. Schewe and B. Thalheim / Storyboarding Concepts for Edutainment WIS
69
Tools and instruments for solution based on constructors, associations, collections, and classification; Specification/prescription/description as the results of the problem solution problem. Analysis and solution formation and problem solution are two specific activities that depend on each other. Development aims in forming solutions as well as discovering inconsistencies and conflicts and resolving incompleteness. Solution formation is based on the mapping of properties, requirements or phenomenons into WIS solutions. Solutions may have alternative solutions. These alternative solutions can be used at a later scenes instead of the given one. Typical techniques for problem solution and formation are exploration and experimentation, skeptical evaluation, conjecturing and refuting. Additional techniques are investigative ones that use resources. In the evaluation phase we show whether the problem solution result is correct, and if so what kind of tests etc. ought to be devised. We do not require completeness since this is a relative issue. Solution validation is the process of inspection of each solution with reference to solutions it is based on. It is based on the application area description and checks whether the right solutions are developed. Validation produces a report that constitutes the correctness and that produces the extensions and updates necessary for the base solutions. Verification analyses solutions in order to ascertain whether what has been developed satisfies a number of obliged properties. It checks whether the solutions developed so far are correct according to some correctness criteria and according to proof or verification techniques that are currently applicable. Techniques may be informal, i.e., based on verbal arguments or tests, qualitative, i.e. based on abstraction and qualitative reasoning, or formal, i.e. based on model checking and proof techniques. Each of the subscenes can be refined depending on the application, the user, the content, collaboration and the control: User refinement: Each user has a personal profile and a task portfolio. The website should only use those dialogue scenes the learner is assigned to. Learning objects refinement: Learning objects are under constant change. Whether this change is shown to the user depends on the user profile. However, in general refinement to available content is generated for the user. Learning objects refinement: Learning objects have prerequisites, support learning goals, and are associated to other learning objects. These associations must be made available by the system. Scenario refinement: The edutainment WIS also supports an adaption of the entire story currently requested by the user depending on the user and the completion of tasks. Usage refinement: Users are annoyed whenever they did not complete a task and they must begin from scratch after resuming. For this reason, the edutainment WIS must support also refinement to the current usage. It is obvious that such refinement cannot be generated by random application of refinement rules. We observe however that this can be layered. The approach to layering used in the
70
K.-D. Schewe and B. Thalheim / Storyboarding Concepts for Edutainment WIS Current usage refinement User refinement Explicit story refinement Learning object space
Learning object correlates
Edutainment WIS Edutainment WIS content context Edutainment WIS story enrichment Edutainment WIS user profile and portfolio enrichment Edutainment WIS on demand enrichment Figure 3: Layering of generation and filtering against learning units
system is displayed in Figure 3. We generate first the learning scene together with its learning objects for each of the dialogue scenes. Next we extend the set of learning objects by all associated objects. This set of learning objects cannot be delivered to the learner in any order, e.g. prerequisites must be shown first. Some of the learning objects may be shown in any order, i.e., in parallel. Next we filter this set against the storyboard and generate now learning object sequences depending on the scenes. Now we are able to take into consideration the user refinement such as the technical equipment, the channel information such as capacity, the web browser currently in use. Based on this information, we can adapt the website to the current refinement. Finally, we may now enhance the scenes to the specific demands. 3.3 Open Learning Units for Content Classically learning objects are composed elements. Comparing the existing standards such as LOM, SCORM we extract the following basic units: Learning elements are basic components providing the content for singleton learning steps. Typical learning elements are definitions, remarks, proofs, lemmata, illustrations, motivational remarks etc. Learning elements may be associated with learning intentions and require basic skills and knowledge from the learner. These requirements form the context of learning elements. Semantics context specifies the prerequisites and the pieces of knowledge that can be learned. Execution context restricts the utilisation of learning elements, e.g., the environment. Learning modules are the main supporting media objects for lectures. They may be enhanced by indexing, annotation or search functionality. Actors may only call an entire module. Modules may consist of learning elements. Typical composition expressions are regular expressions. The utilisation of learning modules may be based on a number of execution styles such as blackboard execution. Learning modules are typically not materialised. This distinction is too rough for practical usage. Learner may stop learning in any module, may resume learning at a later stage and may define their own way of visiting modules. We thus need a sophisticated, flexible and adaptable mechanism for structuring and compositions of learning modules. We distinguish between learning elements that are simply specific media objects with an additional characterisation, and learning units that can be understood
K.-D. Schewe and B. Thalheim / Storyboarding Concepts for Edutainment WIS
71
as combined learning elements, which are dependent on the learning element and thus only updateable through the learning element. Learning elements can be understood as simple media objects such as text elements, video clips, images etc. Learning units are composed of learning elements, cf. Figure 4. NameOfUnit u Identification HeaderContent Associations to Units {u} Meta-data on Unit u ... Contained Elements NameOfElement e1 Identification Content of Element e1 Associations to Elements {e} Meta-data on Element e1 ...
NameOfElement ek Identification Content of Element ek Associations to Elements {e} Meta-data on Element ek ...
Figure 4: Hierarchical composition of learning elements to learning units
For composition of learning units we extend the specification by metadata, additional functionality, specific scenarios, specific representation styles, filters for playout, a characterisation of learning units spaces that can be associated, and context. The onion playout facility based on XSL rules [TD01] supports the generation of the right content, at the right time, in the right representation, by the right costs, and within the right learning history for each learner. Learning elements and learning units are commonly characterised by • a name of the unit, • a general annotation called header content, • metadata that provide additional information on the unit, and • associations among the media objects. Additionally, learning units are characterised by • expression hierarchically combining learning elements or learning units into the given one, • reusability conditions describing the degree of ease with which constituent media objects may be individually accessed and reused, • common function describing the manner in which the unit is generally used, • extra-object dependence describing whether the unit needs information (such as location on the network) about learning units other than itself, • functions of algorithms and procedures within the media object,
72
K.-D. Schewe and B. Thalheim / Storyboarding Concepts for Edutainment WIS
• potential for inter-contextual reuse describing the number of different instructional contexts in which the learning unit may be used, that is, the unit’s potential for reuse in different content areas or domains, and • potential for intra-contextual reuse describing the number of times the unit may be used within the same content area or domain. The above discussed theory of learning elements, learning units and learning modules is now becoming the state-of-the-art in most edutainment WIS. We are additionally interested in support of learning scenarios which are adaptable in most of the forms discussed above. In order to cope with such requirements we introduce the concept of open learning units. These objects are learning units extended by parameters instantiable by information used in general learning modules such as prerequisites, supporting knowledge for better understanding of the unit, and associated units, parameters for links to other content elements, comments, parameters for replacements strategies depending on availability of content objects, parameters for associating units to ontology objects for flexible classification of learning objects and bottom-up collection of learning scenarios, parameters for learner profile integration through which the open learning unit can be replaced by objects that are more appropriate for the current learner type, parameters keeping track on the learning history and which can be used for adaptation of the unit to the learner’s history, and parameters used to keep track on the payment profile of the learner. Open learning units can be specialized to learning units by instantiating all parameters. In this process, user-specific learning units are generated by step-wise instantiation, extension and specialization of the given open learning unit. This procedure is based on rules which can be specified as attribute grammar rules and which can be transformed to XSL rules. These XSL rules are used to transform the given learning unit that is given as XML document to more specific XML documents. We need, however, to clarify whether an arbitrary specialization order is applicable. Furthermore, the representation style can be added. This adaptation approach has been extensively discussed in [TD01]. Since we change step-by-step the learning unit to be transferred to the learner we call this generation approach the onion generation style. The delivery of open learning units is based on container functions that support derived learning units by filters which support • enrichment of each unit by other learning elements or learning units, • contraction of units to essential material, e.g., for repetition of units already visited, and • cut of learning units to new units according to the restrictions that are applicable.
K.-D. Schewe and B. Thalheim / Storyboarding Concepts for Edutainment WIS
73
This approach has already been used in the DaMiT system. Since DaMiT was centered around data mining material and has been mathematically oriented, a number of specific filters have been developed: • Repetition filters contracted the units to essential definitions and to the informal statements of theorems in the unit. • Definitions filters provided the definitions of the together with all associated definitions used in the statements and theorems of the unit. • Examples filter compiled examples in the unit to consecutive examples with references to statements and definitions. • Quick reminder filter are extracting those elements of the units that ar marked as absolutely essential. • Proposition-and-theorems filters summarised all theorems and statement of a unit together with sketches or ideas of their proofs. 3.4 Enhancements of Edutainment Stories Edutainment WIS authoring is considered to be one of the most difficult tasks. This difficulty is increased by the current approach to develop content and stories from scratch. Classroom teaching does not follow this approach. It is mainly based on sequenced or curriculum-ruled learning approaches. These approaches are easier to use due to the high level of reuse, due to the high-level of similarity within the teaching scenario, and due to the homogeneity of teams after the class has been formed. The last advantage cannot be used in edutainment WIS since the auditory is very heterogeneous, is changing over time, does not follow any timed schedules, has a brought variety of preliminary knowledge and is self-organised. The first two advantages may however be incorporated into edutainment as well. Meanwhile it is well acknowledged that general purpose, general content and general storyboard systems are not feasible. Since any area of knowledge has its specific approaches to learn that knowledge we need specific storyboards for edutainment systems. General purpose learning systems are replaced by specific purpose systems, e.g. learning application of certain knowledge. Any content needs its specific representation. Therefore, we must specialize from one hand side and must be very general for any kind of story required. We shall show that our approach supports these requirements. Authors of content for edutainment may base their storyboard on general “pattern” of scenario that might be useful for the given topic. This general scenario can be then refined by the author. We are interested in such scenario that can be composed from given scenarios. So, the author selects first a general scenario or a storyboard. Next he/she uses pattern for refinement of their scenes. Educational material is assigned to scenes based on the storyboard. This material is modelled by media types [ST04]. Finally, collaboration, control and workspace scenario are folded into the developed scenario. Let us demonstrate the integration based on the experience we gained in one of our edutainment WIS project. Data mining is one of the challenging topics. First, users must learn in a rather interactive form. Second, the outcome of the learning process must be evaluated and interpreted. Third, the data used for data mining are often private or secured data. Therefore,
74
K.-D. Schewe and B. Thalheim / Storyboarding Concepts for Edutainment WIS
the user should not transfer the data. Instead the user must understand how to prepare for the data mining process. Additionally the demand for data mining appears as a part of the daily business of users. In a large project integrating research groups in ten German universities and application groups in half-dozen German software companies we developed the edutainment system DaMiT (data mining tutoring) that educates users to such extent that they can evaluate whether the results of decision support systems are generating correct results and whether results will lead to correct conclusions. The system supports decision learning • by learning basic and advanced topics on data mining on demand, • by applying data mining algorithms to test data for training of users, and • by finally applying these data mining algorithms to data of the application engineer. Surprises, data warehousing, and complex applications often require sophisticated data analysis. The most common approach to data analysis is to use data mining software or reasoning systems based on artificial intelligence. These applications allow to analyse data based on the data on hand. At the same time data are often observational or sequenced data, noisy data, null-valued data, incomplete data, of wrong granularity, of wrong precision, of inappropriate type or coding, etc. Therefore, brute-force application of analysis algorithms leads to wrong results, to losses of semantics, to misunderstandings etc. The storyboard in Figure 2 gives only a general description of the main part of the data mining storyboard. It must be enhanced to become a complete story for learning data mining. We thus need general frameworks for data analysis beyond the framework used for data mining. The enhancement procedure may be based on general story descriptions. In our application approaches known from mathematics for the general mathematical problem solving have been used for derivation of a data analysis framework: Elaboration of needs: Modelling of the tasks and problems and their data requirements. Elaboration of opportunities: Selection of possible appropriate analysis algorithms, categorisation of their outcome and pitfalls within the task and problem scope, development of an application frame for application of the chosen algorithms. Extraction, transformation, and loading of data: Categorisation, extraction of macro- and meta-data, adaption of the data to the analysis needs and modelling of data semantics and pragmatics. Problem solution: Extraction, transformation and loading of macro-data for the chosen analysis algorithms, including cleansing and adaption of the data. Refinement of problem solution: Application, stepwise refinement and correction of the analysis algorithms. Interpretation of analysis: Modelling of the obtained analysis results with their semantics and pragmatics.
75
K.-D. Schewe and B. Thalheim / Storyboarding Concepts for Edutainment WIS
The general data analysis framework has been developed for data mining tasks. It considers modelling of data analysis algorithms, description of their requirements to data, meta-data of their functionality for analysis and transformations of the data. This general framework can be now refined in a number of ways. A typical refinement is displayed in Figure 5. Users learn the opportunities of data mining based on case studies. They explore good and bad studies, get an explanation based on the theory background, and become familiar with modelling techniques. Elaboration of opportunities by analogy Disselect case
j
Empty list of cases
Deficits j in the application domain
K U Survey on case studies Y K
j Selected case study Y y K U Hall of fame for solutions
z
j
Related cases
z
:
Theory behind
9 Model survey
Figure 5: The scenes for elaboration of opportunities for active learning in the DaMiT system
The framework is classically enhanced by scenario that support training of users to data mining based on exercises, examples, case studies etc. 3.5 Context Space Enhancements When determining context we already know the edutainment scenarios we would like to support, the intentions associated with the WIS, the user and learner characterisation on the basis of profiles and portfolios, and the technical environment we are going to use. These restrictions enable a more refined understanding of context within a WIS. [MST05] characterises a WIS by six intertwined dimensions: the intentions, the usage, the content, the functionality, the context, and the presentation. We must thus relate context to the other dimensions. As presentation resides on a lower level of abstraction, it does not have an impact on context. Content and functionality will be used for context refinement. The user model, the specified edutainment scenarios, and the intention can be used for a disambiguation of the meaning and an injection of context. In doing so we distinguish the following facets of context: Learner context: The WIS is used by learners for a number of tasks in a variety of involvements and well understood collaboration. These learners impose their quality requirements on the WIS usage as described by their security and privacy profiles. They need additional auxiliary data and auxiliary functions. The variability of use is restricted by the learner’s context, which covers the learner’s specific tasks and specific data and
76
K.-D. Schewe and B. Thalheim / Storyboarding Concepts for Edutainment WIS
function demand, and by chosen involvement, while the profile of learners imposes exceptions. The involvement and collaboration of learners is based on assumptions of social behaviour and restrictions due to organisational decisions. These assumptions and restrictions are components of the learner’s context. Storyboard context: The meaning of content and functionality to users depends on the stories, which are based on scenarios that reflect learning scenarios and the portfolios of users or learners. According to the profile of these users a number of quality requirements such as privacy, security and availability must be satisfied. The learner’s scenario context describes what the learner needs to understand in order to efficiently and effectively solve his/her tasks in the actual portfolio. The learner’s determine the policy for following particular stories. System context: The edutainment WIS is developed to support a number of intentions. The purposes and intents lead to a number of decisions on the WIS architecture, the technical environment, and the implementation. The WIS architecture has an impact on its utilisation, which often is only implicit and thus leads to not understandable systems behaviour. The technical environment restricts the user due to restrictions imposed by server, channel and client properties. Adaptation to the current environment is defined as context adaptation to the current channel, to the client infrastructure and to the server load. At the same time a number of legal decisions based on regulations, laws and business rules have been incorporated into the WIS. Temporal context: The utilisation of a scene by an learner depends on his/her history of utilisation. Learners may interrupt and resume their activities at any moment of time. As they may not be interested in repeating all previous actions they have already successfully completed, the temporal context must be taken into account. Due to availability of content and functionality the current utilisation may lead to a different story within the same scenario. This entire information forms the context space, which brings together the storyboard specification and the contextual information. Typical questions that are answered on the basis of the context space are: • What content is required by the context space? • What functionality is required by the context space? • What has to be changed for the life cases, the storyboard, etc., if context is considered? As outlined above the context space is determined by the learners, the scenarios, the WIS itself, and the time. It leads to a specialisation of the content, structuring and functionality of the scenes. Context is associated with desirable properties of the WIS such as quality criteria and security and privacy requirements. Quality criteria such as suitability for the users or learnability provide obligations for the WIS development process. Though these criteria are rather fuzzy, they lead directly to a number of implementation obligations that must be fulfilled at later stages, i.e. within the development on the implementation layer. For instance, learnability means comprehensibility, i.e. the WIS must be easy to use, remember, capture and forecast. This requires clarity of the visual representation, predictability,
K.-D. Schewe and B. Thalheim / Storyboarding Concepts for Edutainment WIS
77
directness and intuitiveness. These properties allow the user to concentrate on the tasks. The workflows and the discourse structure correspond to the expectations of the users and do not lead to surprising situations. They can be based on metaphors and motives taken from the application domain. In the same way other quality criteria can also be mapped to development obligations. Other properties that may be associated with context refer to the potential utilisation for other tasks outside the scope of the storyboard. In this case we do not integrate the additional tasks into the storyboard, but instead support these tasks, if this in accordance with our intentions. For instance, we might expect further visits targeting at core concerns of the edutainment WIS. 4 Open Issues and Challenges for Edutainment WIS Learning content must be properly handled. It turns out that this task is one of the most difficult tasks. For this reason, any edutainment WIS must be combined with an authoring WIS that supports authors during appropriate development of content. Quality control of learning objects is necessary since low quality data harms the learning success. We may distinguish a number of reasons for low quality and development strategies for improvement of quality: Incomplete content, mixed content, wrong content, complex associations among content, and mutated content. Functionality development for edutainment WIS also includes the development of very sophisticated supporting facilities for the actors, the content, the context, and the presentation. Most edutainment systems are currently based on scenarios of sequenced learning. 3rd generation systems are aiming in providing best-suited content just in time to the right user, place and device with the best pricing. They challenge current technology. Research is sought on didactics, content integration and delivery, storyboarding, adaptation and context integration, and success control. Open learning objects provide a sophisticated facility for content management. The theory of open learning units should be integrated with didactics based on storyboarding, content adaptation and delivery, and content development. In future, they will be extended with context, e.g., story space, actor, user, payment, portfolio, association, history, etc. Control functionality should be provided for open learning units in the same fashion as we know it already for exercises, tests, and exams for self-control or certification. References [BGW92]
R. M. Briggs, L. J. Gagne, and W. W. Wager. Principles of Instructional Design. Thomson Learning, 1992.
[BZKK+ 05] A. Binemann-Zdanowicz, R. Kaschek, T. Kuss, K.-D. Schewe, B. Thalheim, and B. Tschiedel. A conceptual view of electronic learning systems. Education and Information Technologies, 2005. [DN03]
S. Dohi and S. Nakamura. The development of the dynamic syllabus for school of information environment. In ITHET03, pages 505–510, 2003.
[For94]
H. I. Forsha. The Complete Guide to Storyboarding and Problem Solving. ASQ Quality Press, 1994.
[JMR+ 03]
Klaus P. Jantke, M. Memmel, O. Rostanin, B. Thalheim, and B. Tschiedel. Decision support by learning-on-demand. In Proc. of the 15th Conf. on Advanced Information Systems Engineering (CAiSE ’03), Workshops Proc., Information Systems for a Connected Society. CEUR Workshop Proc. 75, pages 317–328. Technical University Aachen (RWTH), 2003.
78
K.-D. Schewe and B. Thalheim / Storyboarding Concepts for Edutainment WIS
[Kun92]
J. Kunze. Generating verb fields. In Proc. KONVENS, Informatik Aktuell, pages 268–277. Springer, 1992. in German.
[LOM00]
LOM. orking draft v4.1. http://ltsc.ieee.org/doc/wg12/LOMv4.1.htm, 2000.
[LTS00]
LTSC. Learning technology standards committee website. http://ltsc.ieee.org/, 2000.
[MST05]
T. Moritz, K.-D. Schewe, and B. Thalheim. Strategic modelling of web information systems. International Journal on Web Information Systems, 1(4):77–94, 2005.
[PP45]
G. Polya and G. Polya. How to solve it: A new aspect of mathematical method. Princeton University Press, Princeton, 1945.
[RK04]
W. J. Rothwell and H. C. Kazanas. Mastering the Instructional Design Process: A Systematic Approach (Third Edition). Pfeiffer, 2004.
[SBZ04]
J. Sonnberger and A. Binemann-Zdanowicz. Kopra - ein adaptives Lehr-Lernsystem f¨ur kooperatives Lernen. In GMW’2004, Graz, Austria, Sept. 2004, pages 274–285, 2004.
[SSS90]
H. Schreiber, K.-E. Sommerfeld, and G. Starke. Deutsche Wortfelder f¨ur den Sprachunterricht: Verbgruppen. VEB Verlag Enzyklop¨adie, Leipzig, 1990.
[ST04]
K.-D. Schewe and B. Thalheim. Web Information Systems, chapter Structural media types in the development of data-intensive web information systems, pages 34–70. IDEA Group, 2004.
[ST05]
K.-D. Schewe and B. Thalheim. Conceptual modelling of web information systems. Data and Knowledge Engineering, 54:147–188, 2005.
[TD01]
B. Thalheim and A. D¨usterh¨oft. Sitelang: Conceptual modeling of internet sites. In Proc. ER’01, volume 2224 of LNCS, pages 179–192. Springer, 2001.
Information Modelling and Knowledge Bases XIX H. Jaakkola et al. (Eds.) IOS Press, 2008 © 2008 The authors and IOS Press. All rights reserved.
79
A Model of Database Components and their Interconnection Based upon Communicating Views Stephen J. HEGNER† Umeå University Department of Computing Science SE-901 87 Umeå, Sweden
Abstract A formalism for constructing database schemata from simple components is presented in which the components are coupled to one another via communicating views. The emphasis is upon identifying the conditions under which such components can be interconnected in a conflict-free fashion, and a characterization of such, based upon the acyclicity of an underlying hypergraph, is obtained. The work is furthermore oriented towards an understanding of how updates can be supported within the component-based framework, and initial ideas of so-called canonical liftings are presented.
1.
Introduction
Large systems typically possess a complex structure; database systems are no exception. Schemata with thousands of relations are not uncommon; the largest have tens of thousands. If not designed properly, such systems are certain to be unmanageable intellectually, leading to redundancy, design errors, and difficulties as the schema evolves over time. In many engineering settings, including in particular computer hardware, this problems associated with complexity are addressed, at least in part, by designing a large system as the interconnection of simpler (often prefabricated) components. In the field of logic design, for example, this approach has become completely standard [16]. For a variety of reasons, it has not seen as much success in software engineering [17], and in database systems in particular. Although the idea that formal objects which describe computations, such as automata, can be the basic modules of an interconnection calculus dates back to relatively early days of theoretical computer science (see, for example, [2, Sec. 3.2]), corresponding ideas have not found great success in application to more concrete problems. To address some of the shortcomings of more classical approaches, Broy [5] [6] has proposed a formal interconnection calculus for components, including software components in particular. His approach has two flavors. The first is based upon input and output streams with otherwise stateless components; the state being effectively recaptured in the stream. The second is based upon a more conventional state-transition approach. † Much of the research leading to this paper was completed while the author was a visitor at the Information Systems Engineering Group, Department of Computer Science, Christian-Albrechts University of Kiel, Germany.
80
S.J. Hegner / A Model of Database Components and Their Interconnection
Thalheim [24] [22] [25] has recently forwarded the idea of basing database design upon components. While his formal calculus is based upon the state-transition model of Broy, the emphasis is solidly upon a less formal approach to the design and synthesis of schemata within the context of the higher-order entity-relationship model (HERM) [23], which is in turn based upon the classical entity-relationship (ER) model [8]. The present work was motivated by the desire to understand the various aspects of updates, in particular their propagation, in the the context of the database components of Thalheim. As the investigation evolved, however, it gradually became apparent that a somewhat different model of database component was necessary. Although database systems are a form of software system, and therefore it is not unreasonable to use similar formalisms for both, database systems also have special characteristics not shared by all software systems. In particular, in many database modelling scenarios, the central notions are sequences or streams of data, but rather database schemata and their views. This is particularly true in the context of updates. Furthermore, while the ER model and its extensions are widely used in database modelling and design, few if any actual systems are based upon it. Rather, ER-based designs are invariably converted to operational models, such as the relational model or an objectrelational model, for final realization. That an understanding of database updates is closely tied to database views has been recognized since the seminal work of Bancilhon and Spyratos [3], and has been elaborated greatly in [12] [13]. Thus, in an attempt to understand updates in the context of database components, it seems most natural to seek a model of components in which views play a central rôle. In this work, an alternative notion of database component, based upon schemata and views, is forwarded. Roughly speaking, the components are schemata which are interconnected via common views, called ports. Communication is then not via sequences or streams, but rather by applying updates to these ports. An approach to components which is based upon the notions of database schema and view must begin with a choice for the representation of these notions. It turns out that there is relatively little to be gained by restricting the investigation to a particular data model, such as the relational model and its relatives. Rather, much more abstract models, in which a database schema is represented by a (possibly structured) set which represents its databases, and a database morphism is a structure preserving morphism, suffice completely. The formalizations surrounding the constant-complement strategy [3] [12] [13] adapt naturally to the component framework. In particular, the notions developed in these works for view updates provide, perhaps somewhat surprisingly, precisely the notions which are necessary for the representation of component interconnection. The organization of this paper is as follows. In Section 2, the fundamental notions of view-based database components and their interconnections are presented. In Section 3, the notion of the hypergraph of a component is presented, and characterizations of “good” components, in terms of properties of their underlying hypergraphs, are presented. In Section 4, the topic of updates to to interconnections of components is explored briefly. Finally, in Section 5, some conclusions and further directions are presented. 2.
The Basic Ideas of View-Based Database Components
The component-based approach is compositional, not decompositional. In other words, instead of beginning with a large main schema and breaking it into its constituent parts, the starting point is a collection of smaller schemata, together with information on how they may be combined. Since the decomposition of relational schemata is a topic which should be familiar to most readers, it is helpful to begin with a simple example of such decomposition,
S.J. Hegner / A Model of Database Components and Their Interconnection
81
and then identify the conditions under which it may be reversed to yield a component-based composition theory.
2.1 Example — Reconstructing simple components from a decomposition Let E0 be the relational schema comprised of the single relation name R 0 [B1 B2 B3 ], constrained by the set F0 = {B1 → B2 , B2 → B3 } of functional dependencies (FDs), and let LDB(E0 ) denote the (finite) legal databases (i.e., relations) of this schema (relative to approE0 0 priately chosen domains). The view ΠE B1 B2 = (E1 , πB1 B2 ) of E0 is the projection of E0 onto the attributes B1 B2 . More precisely, E1 denotes the schema whose single relation is R1 [B1 B2 ] and whose databases, denoted LDB(E1 ), are precisely the projections of members of LDB(E0 ) onto the attributes B1 B2 , with the view mapping πBE10B2 : r → {(b1 , b2 ) | (∃b3 )((b1 , b2 , b3 ) ∈ r}. E0 E0 0 By construction, πBE10B2 is surjective. Define the views ΠE B2 B3 = (E2 , πB2 B3 ) and ΠB1 B3 =
(E2 , πBE10B3 ) similarly, with R2 [B2 B3 ] and R3 [B1 B3 ], their relation schemes, respectively. It follows from one of the earliest and most widely known results in relational database E0 0 theory [10, p. 31] that the decomposition of E0 into the views {ΠE B1 B2 , ΠB2 B3 } is lossless, in the sense that the decomposition mapping Δ(B1 B2,B2 B3 ) : LDB(E0 ) → LDB(E1 ) × LDB(E2 ) defined by r → (πBE10B2 (r), πBE20B3 (r)) is injective. Similarly, the decomposition of E0 into E0 E0 E0 0 {ΠE B1 B2 , ΠB1 B3 } is lossless. However, the decomposition {Π B1 B2 , ΠB2 B3 } enjoys a second
E0 0 property which {ΠE B1 B2 , ΠB1 B3 } lacks. Specifically, define
(∗)
LDB(E1 ) ⊗ LDB(E2 ) = {(r1 , r2) ∈ LDB(E1 ) × LDB(E2 ) | πBE21 (r1 ) = πBE22 (r2 )}
with πBE21 and πBE22 the obvious projections onto attribute B2 . Then Δ(B1 B2 ,B2B3 ) (LDB(E0 )) = LDB(E1 ) ⊗ LDB(E2 ). Viewed another way, there is a bijective correspondence between the legal databases of E0 and those pairs of databases of E1 and E2 which agree on the common column B. Rissanen [21, Sec. 3] calls this property independence, and shows that it holds precisely in the case that the decomposition is lossless and dependency preserving; the latter meaning that a cover of the FDs of the original schema embeds into the views. In this case, the embedding is particularly simple, with B1 → B2 lying in E1 and B2 → B3 lying in E2 . Indeed, LDB(E1 ) (resp. LDB(E2 )) is exactly the set of relations which satisfy B1 → B2 (resp. E0 0 B2 → B3 ). On the other hand, there is no embedded cover of F 0 into {ΠE B1 B2 , ΠB1 B3 }. and so this independence property does not hold for that context. In [12, 2.17], this property is generalized to much more general classes of constraints. What is remarkable about independence is that the quesK1 K2 tion of whether a pair (r1 , r2 ) ∈ LDB(E1 ) × LDB(E2 ) arises R1 [ B1 B2 ] R2 [ B2 B3 ] from a database of the main schema may be answered with reference only to a combination of local constraints on the component schemata and a view-based compatibility condiG1 tion between them; no other knowledge of the main schema is necessary. This leads naturally to the component-based Figure 1: The interconnection of K1 and K2 1 philosophy. Define the components K1 = (E1 , {ΠE B2 }) and E2 E1 2 K2 = (E2 , {ΠB2 }). The set {ΠB2 } identifies the ports of K1 , and {ΠE B2 } likewise for K2 , with Ei ΠB2 , for i ∈ {1, 2}; the obvious views being defined by projection. The underlying schemata of these two ports are identical. Denote it by G1 ; it has T1 [B2 ] as its sole relational symbol. The interconnection of K1 and K2 is depicted graphically in Figure 1 above.
82
S.J. Hegner / A Model of Database Components and Their Interconnection
This property — that the view schemata of two ports are identical — is called star compatibility, and is central to the interconnection of these components. The star interconnection of K1 and K2 is in effect a join of K1 and K2 ; its schema is the “union” of the schemata of K1 and K2 , subject to the constraint that the two relations agree on the views defined by the components. In more detail, define the schema E12 to have the two relation symbols R2 [B1 B2 ] and R3 [B2 B3 ], constrained by the FDs B1 → B2 and B2 → B3 respectively, as well as the port constraint which stipulates that for any (r1 , r2 ) ∈ LDB(E1 ) × LDB(E2 ), πBE21 (r1 ) = πBE22 (r2 ). Formally, this join of K1 and K2 , the compound component defined by the star interconnection E1 E2 E12 1 E2 {ΠE B2 ΠB2 }, is denoted Cpt{K1 , K2 }, {ΠB2 , ΠB2 } , and is given explicitly by (E12 , {ΠB2 }), 12 with ΠE B2 the view whose schema is E12 and whose view mapping projects either of the two relations of E12 onto attribute B2 . Note that E12 is the schema which is obtained by decomposing E0 into E1 and E2 , and since E12 and E0 are isomorphic in a natural way, essentially the same information is represented. However, the component-based approach is compositional – the components K1 and K2 make no reference whatever to any main schema. E0 0 This same construction does not work for {ΠE B1 B2 , ΠB1 B3 }. Upon defining Δ(B1 B2 ,B1 B3 ) : LDB(E0 ) → LDB(E1 ) × LDB(E2 ) by r → (πBE10B2 (r), πBE10B3 (r)), with (∗ )
E
LDB(E1 ) ⊗ LDB(E2 ) = {(r1 , r2) ∈ LDB(E1 ) × LDB(E2 ) | πBE11 (r1 ) = πB12 (r2 )}
it is not the case that Δ(B1 B2 ,B1 B3 ) (LDB(E0 )) = LDB(E1 ) ⊗ LDB(E2 ). Rather, to determine whether a pair (r1 , r2 ) ∈ LDB(E1 ) × LDB(E2 ) arises as the projection of some r ∈ LDB(E0 ), it is necessary first to compute the join of that pair, since the constraint A 2 → A3 cannot be checked within either of the projections alone. In other words, upon defining the component 0 K2 = (E2 , {ΠE B1 B3 }), it is not the case that the interconnection of K1 and K2 will have a schema E0 0 which is isomorphic to E0 . Mathematically, the congruences of the pair {ΠE B1 B2 , Π B2 B3 }
E0 0 commute, while those of {ΠE B1 B2 , ΠB1 B3 } do not. For details of these ideas, as well as their connection to the constant-complement update strategy for views, consult [12].
2.2 Database contexts As noted in the introduction, the the ideas developed here are not limited to a specific data model, such as the relational model or the ER model. Rather, they apply to virtually any database model. Formally, a database context is a pair S = (D, ) in which D is a class of database schemata and their morphisms, and is a function which associates to each schema D of D a set LDB(D) of legal states, and to each database morphism f : D 1 → D2 of D a function LDB( f ) : LDB(D1 ) → LDB(D2 ). The idea is that a database schema D is modelled by its legal states alone, and that a database morphism is modelled by its underlying function. This is precisely the framework which is used in the original work on the constantcomplement update strategy [3]. Suitable examples for D include the context of all relational schemata with morphisms defined by the relational algebra, nested relational models [20, Ch. 7], and the HERM model [23] together with suitably defined morphisms. This modelling assumption must be faithful to the more structured framework which it represents. In particular, joins, as exemplified in equation (∗) of 2.1, must translate back and forth between the full data model and the -based abstraction. These conditions are so natural that it is difficult to envision any reasonable formalization of database schemata and morphisms which would not satisfy them. Therefore, the straightforward but lengthy list of conditions which must be satisfied are not given here. However, for those readers familiar with the basic language of category theory [1] [15], these conditions can be characterized
S.J. Hegner / A Model of Database Components and Their Interconnection
83
succinctly by requiring that the database schemata and morphisms form a concrete category D with finite limits over the category , with the further condition that the grounding functor :D→ both preserve and reflect limits. notation on morphisms can become quite cumAs a notation convenience, since the bersome, for a database morphism f : D1 → D2 in D, f˚ or ( f )˚will often be used as shorthand for LDB( f ).
2.3 Notational convention Throughout the rest of this paper, unless stated specifically to the contrary, take S to be a database context. Unless stated specifically to the contrary (e.g., in examples), all database schemata and morphisms will be assumed to be based in S. Since views are central to the approach to components given here, a precise definition within the context S is essential. The following definition is based in large part upon those found in [12] and [13], to which the reader is referred for more detail.
2.4 Views Let D be a database schema. A view of D is a pair Γ = (V, γ ) with V a database schema and γ : D → V a database morphism with the property that γ˚ : LDB(D) → LDB(V) is surjective. The zero view on D is ZViewD = (ZSchema, ZMorD ), with ZSchema is a database schema with the property that LDB(ZSchema) consists of exactly one element, and (ZMorD )˚: LDB(D) → LDB(ZSchema) the function which sends every M ∈ LDB(D) to the unique element of LDB(ZSchema). In other words, ZSchema is a constant schema with exactly one state, and ZView D is the view of D which maps every M ∈ LDB(D) to that state. Under the conditions identified in 2.2, a zero view exists for every schema D, and is furthermore unique up to isomorphism. Views occur very frequently in this work. To avoid the need to spell out the complete definition every time, the following convention will be followed. If a view is named by the Greek letter Γ, then the view morphism will be denote using γ , and SchemaΓ will be used as an alias for the underlying view schema. This convention furthermore extends to all subscripted and superscripted variants. For example, for the view Γ defined above, SchemaΓ is an alias for V. Similarly, for a view named Γi , the full definition is Γi = (SchemaΓi , γi ). Additionally, when SchemaX appears as the argument of another notation, X will frequently be used in its stead when no confusion can result. In particular, LDB(Γ i ) will be used as an abbreviation for LDB(SchemaΓi ).
2.5 Components The formal definition of a component follows the pattern introduced in the example of 2.1, but is based not upon the relational model but rather upon the more general model of database schemata and views in the context S. Specifically, a component is an ordered pair C = (Schema(C), Ports(C)) which satisfies the following two conditions. (cpt-i) Schema(C) is a database schema.
84
S.J. Hegner / A Model of Database Components and Their Interconnection
(cpt-ii) Ports(C) is a finite set of views of Schema(C), called the ports of C, with the property that none of the ports are zero views. LDB(C) will be used as notational shorthand for LDB(Schema(C)).
2.6 Name-normalized components In describing interconnections of components, it is essential to be able to recover the identity of the embodying component from the name of a port. While an elaborate tagging formalism could be developed, the solution proposed here is much simpler; namely, that for a given set of components, all port names are globally unique. Since this is only a naming convention, there is no loss of generality in such an assumption. Specifically, let X be a finite set of components. (a) X is called name normalized if for distinct C,C ∈ X , Ports(C) ∩ Ports(C ) = ∅.
(b) For Y ⊆ X , define PortsY = {Ports(C) | C ∈ Y }. Thus, PortsY is just the set of all ports which are associated with some component in Y . (c) If X is name normalized and Γ ∈ PortsX , then SrcCpt(Γ) denotes the source component of Γ, which is the unique C ∈ X for which Γ ∈ Ports(C).
2.7 Star interconnections and interconnection families Let X be a finite set of components. (a) For I ⊆ PortsX , Components(I) denotes {SrcCpt(Γ) | Γ ∈ I}. Thus, Components(I) is just the set of all components which are associated with a port in I. (b) A star-compatible set for X is an I ⊆ PortsX with the property that for all Γ, Γ ∈ I, SchemaΓ = SchemaΓ . (c) A star interconnection over X is a star-compatible set I for X with the property that for distinct Γ, Γ ∈ I, SrcCpt(Γ) and SrcCpt(Γ ) are distinct as well. In this case, I is called a star interconnection of Components(I). (d) A interconnection family for X is a finite set of star interconnections over X . The above definitions are critical to the interconnection model. For a set {C1 ,C2 , . . .,Ck } of components to be coupled in a single star configuration using ports Γ 1 , Γ2 , . . ., Γk , respectively, the schemata of the ports must be identical (and not just isomorphic). This is necessary because in the interconnection, the states of these views must always be the same. The resulting star interconnection, as defined formally below, is illustrated for k = 4 in Figure 2. This is somewhat distinct from the notion of a star component, as defined in [24]. Here it is the interconnection, and not the component itself, which has the star property. It is possible to construct a maximal interconnection family, from which all others can be obtained via appropriate subset operations. (e) For Γ ∈ PortsX , define MaxStarX (Γ) = {Γ ∈ PortsX | SchemaΓ = SchemaΓ }, and define MaxStar(X ) = {MaxStarX (Γ) | Γ ∈ PortsX }. Each member of MaxStar(X ) is a maximal star-compatible set for X . Thus, J is an interconnection family for X iff J is the union of disjoint subsets of members of MaxStar(X ).
S.J. Hegner / A Model of Database Components and Their Interconnection
C1
Γ1 V1
Γ2
C2
V2
Γ3 V3
C3
C2 V
→
def
Vi = SchemaΓi
C1
85
V4 Γ4 C4
C3
C4 def
V1 = V2 = V3 = V4 = V = SchemaΓ
Figure 2: The star-compatibility condition and the interconnection of four components 2.8 Notational convention Unless specifically stated to the contrary, for the rest of this paper, take X to be a namenormalized set of components with J an interconnection family for X .
2.9 Annotated examples In 2.12 and 2.13, examples are presented which illustrate many of the ideas which have been developed thus far, as well as those which will be developed in the rest of this section. Rather than distributing fragments of these examples throughout the text, it seems more appropriate to present them in unified fashion. The reader is therefore encouraged to look ahead to these examples for clarification of the concepts which are developed.
2.10 The compound component defined by an interconnection family A central idea surrounding the component philosophy is that simple components may be combined to form more complex ones. While this idea is simple in principle, there are nevertheless some complicating details which mandate that not every possible interconnection family is suitable for the formation of a complex component. These issues are now investigated in more detail. (a) The schema SchemaX , J defined by J on X is given as follows. LDB(SchemaX , J ) ={MB ∈ LDB(B) | B∈X
(∀I ∈ J)(∀{Γ1 , Γ2 } ⊆ I)(γ˚1(MSrcCpt(Γ1 ) ) = γ˚2 (MSrcCpt(Γ2 ) ))} (b) For C ∈ X , define the natural projection morphism NatProj J;X C : SchemaX , J → Schema(C) on elements by MB B∈X → MC . It is clear that SchemaX , J is the natural schema for the interconnection of the elements of X into a single component, using J as the interconnection family. There is, however, a complication. For this complex component, it is necessary to identify the ports. The obvious solution is to define one common port for each star interconnection. In light of the definition of (a) above, the view of this common port will not depend upon which of the components from X is used to define it. Unfortunately, the combined ports may no longer be views, because the underlying mapping is no longer surjective. In general, the constraints on the schemata of the constituents which arise by combining components may limit the values
86
S.J. Hegner / A Model of Database Components and Their Interconnection
which may appear on the ports. It is therefore essential to identify conditions under which these problems do not occur. The following definitions lay the framework for studying this question in more detail. (c) The component C ∈ X is free in X if for any N ∈ LDB(C), there is an MB B∈X ∈ LDB(SchemaX , J ) with the property that MC = N. (d) Let C ∈ Y and let Γ ∈ Ports(C). The port Γ is said to be free in X with respect to J if for any N ∈ LDB(Γ), there is an MB B∈X ∈ LDB(SchemaX , J ) with γ˚(MC ) = N. (e) X is free for ports with respect to J if for every C ∈ X and Γ ∈ Ports(C), Γ is free in X with respect to J. The condition of the component C being free in X is the stronger of the two; it essentially states that the interconnection does not further constrain C. It is very easy to find examples of situations in which constituent components are not free; see 2.12. The condition of a view Γ being free in X with respect to J is strictly weaker, since the entire component C need not be unconstrained, but rather only that each of its ports must be unconstrained individually. In the example of 2.12, all ports are free with respect to MaxStar(L). Nevertheless, it is possible to construct relatively simple relational examples in which this condition is not satisfied, as illustrated in 2.13. The weaker condition of X being free for ports with respect to J is sufficient to admit a consistent definition for the ports of a compound component. By its very nature, it guarantees that the view mapping for the combined port will be surjective — it is both necessary and sufficient. The formal details are as follows. For parts (f)-(i), assume further that X is free for ports with respect to J. (f) Let C ∈ X and let Γ ∈ Ports(C). Define the lifting of Γ to SchemaX , J to be the pair [X ,J] Γ = (SchemaX , J , [X ,J]γ ) with [Y,J] γ : SchemaX , J → SchemaΓ given as the composition illustrated to the right.
SchemaY, J
NatProjJ;X C [X ,J]
γ
Schema(C)
γ Schema(Γ)
(g) Let I ⊆ J, and let Γ, Γ ∈ PortsX . Define the equivalence relation ≡X,J on liftings of such views by [X ,J]Γ ≡X,J [X ,J]Γ iff Γ = Γ or else there is a I ∈ J with {Γ, Γ } ⊆ X . Let [[X ,J]Γ] denote the equivalence class of [X ,J]Γ under this equivalence relation. (h) Define PortsX , J = {[[X ,J]Γ] | Γ ∈ PortsX }. (i) The compound component defined by X , J is given as follows. CptX , J = (SchemaX , J , PortsX , J ) It is worth repeating that the above notion of compound component is well defined only in the case that X is free for ports with respect to J. It should perhaps be noted that it is not always necessary (or desirable) to include all ports from the constituents in the compound component. However, the choice of which ones to include and which ones to exclude cannot be made on a formal level; rather, it must be a modelling decision. For example, consider forming a compound component from {L 1 , L2 } of 2.12. If one decides to exclude from the compound component those ports which have already been matched, then it would be impossible to connect L3 to the compound of L1 and L2 , since the necessary port has been removed. This must be a design decision, not a mathematical one.
S.J. Hegner / A Model of Database Components and Their Interconnection
87
2.11 Subcomponents of compound components Just as simple components may be combined to construct more complex ones, so too may simpler components be extracted from complex ones. For this extraction process to yield well-defined subcomponents, certain conditions must be met, which are now explored in more detail. In the following, let Y ⊆ X . (a) For
J an interconnection family for X , the relativization of J to Y is ReltJ,Y = {I ∩ ( {Ports(C) | C ∈ Y }) | I ∈ J}. Thus, ReltJ,Y is obtained from J by removing all ports which are not associated with components in Y . (b) The relative schema SchemaY, J defined by J on Y is given as follows. LDB(SchemaY, J ) ={MB ∈
LDB(B) |
B∈Y
(∀I ∈ ReltJ,Y )(∀{Γ1, Γ2 } ⊆ I)(γ˚1(MSrcCpt(Γ1 ) ) = γ˚2 (MSrcCpt(Γ2 ) ))} Thus, SchemaY, J = SchemaY, ReltJ,Y . In other words, the relative schema SchemaY, J is precisely the schema of the compound component CptY, ReltJ,Y . On the other hand, one can also consider the schema obtained by projecting from SchemaX , J the constituent schemata which arise from components in Y . (c) The projected schema ProjSchX Y, J is defined as follows. LDB(ProjSchX Y, J ) ={MB ∈
LDB(B) |
B∈Y
(∃NB B∈X ∈ LDB(SchemaX , J ))(∀B ∈ Y )(NB = MB )} (d) Call Y closed in X with respect to J if ProjSchX Y, J = SchemaY, J . The above closure conditions is very important, because it states that CptY, ReltJ,Y is embedded in CptX , J without the latter imposing any additional constraints. Clearly LDB(ProjSchX Y, J ) ⊆ LDB(SchemaY, J ); the reverse inclusion LDB(SchemaY, J ) ⊆ LDB(ProjSchX Y, J ) holds precisely when Y Is closed in X with respect to J. The definition of a subcomponent now proceeds similarly to that of a compound component, as given in 2.10(i). (f) Under the condition that Y is closed in X with respect to J and that Y is free for ports with respect to ReltJ,Y , define the subcomponent of X , J generated by Y as follows. SubCptX,J Y = (ProjSchX Y, J , PortsY, ReltY, J ) An illustration of why the closure condition is essential for this definition is given at the end of 2.12. As noted at the end of 2.10, the choice of which ports to include and which to exclude in a compound component is a design condition. This is equally true for subcomponents. However, in the latter case, there is a classification which is useful — to partition the ports of SubCptX,J Y into those which connect it to other parts of X and those which do not. The formalization is as follows. Again, for this definition to make sense, it must be assumed that Y is free for ports with respect to ReltY, J .
88
S.J. Hegner / A Model of Database Components and Their Interconnection
(g) Define the set of all ports, external ports, the internal ports, of Y with respect to J, respectively, as follows. AllPortsY, J ={[[Y,ReltY,J] Γ] | Γ ∈ PortsY } ExtPortsY, J ={[[Y,ReltY,J] Γ] | (∃I ∈ J)(∃Γ )(({Γ, Γ } ⊆ I) ∧ (Γ ∈ PortsY ) ∧ (Γ ∈ PortsY ))} IntPortsY, J =AllPortsY, J \ ExtPortsY, J Finally, there is a natural projection morphism, whose underlying function is guaranteed to be surjective, defined as follows. (h) For Z ⊆ Y , define the natural projection morphism NatProjX;J;Y Z : ProjSchX Y, J → ProjSchX Z, J on elements by MB B∈Y → MB B∈Z . Define the natural projection view of ProjSchX Y, J to Z to be ProjViewJ;Y Z = (ProjSchX Y, J , NatProjX;J;Y Z ). 2.12 Example — An illustrative set of components The purpose of this example is to provide a setting in which many of the concepts which are introduced in this paper may be illustrated. It is not intended to model a “real” database situation, but rather to illustrate a wide variety of possibilities. All components are based upon the relational model, and all ports are defined via projections, although neither of these limitations is inherent to the model. By using the familiar relational model, the key ideas can be illustrated in a relatively compact fashion, and certain modelling pitfalls can be highlighted. Table 1 summarizes the key information for each atomic component. For the port names, the superscript identifies the component, while the subscript identifies the attributes which are projected. Since each attribute name is used at most once in each component, this convention is unambiguous. For simplicity, it will be assumed that with each attribute A i is associated a countably infinite domain dom(Ai ), while the legal relations themselves must be finite. Table 2 summarizes information about the ports, grouped by those which have identical underlying schemata. This is a natural grouping, since ports with identical schemata are precisely those which may be coupled to one another. Figure 3 shows all possible (star) interconnections of these components. That is, it connects all ports with identical underlying schemata. The components are shown as rectangles, while the port schemata are displayed as circles. Letting L = {L1 , L2 , L3 , L4 , L5 , L6 , L7 }, the associated interconnection family is MaxStar(L) = {{ΠFA11 , ΠFA21 , ΠFA31 }, {ΠFA34A5 , ΠFA44A5 }, {ΠFA47 , ΠFA57 },
{ΠFA48 , ΠFA78 }, {ΠFA50 , ΠFA60 }, {ΠFA69 , ΠFA79 }, {ΠFA34 , ΠFA44 }}
Of course, there is no requirement that when a set of components is interconnected, all possible connections must be included. Any subset of a member of MaxStar(L) is a valid star interconnection. Thus, any set consisting of disjoint subsets of members of MaxStar(L) is a valid star interconnection over L. Observe that the components L4 , L5 , L6 , and L7 are not free in L with respect to MaxStar(L). Indeed, the interconnection forces the additional constraints A 8 → A7 , A7 → A0 , A0 → A9 , and A9 → A8 on L4 , L5 , L6 , and L7 , respectively. On the other hand, L is free for ports with respect to MaxStar(L). Finally, an illustration of the need for the closure condition in the definition of subcomponent is given. Let X567 = {L5 , L6 , L7 }, J567 = {{ΠFA50 , ΠFA60 }, {ΠFA69 , ΠFA79 }}, and Y57 =
89
S.J. Hegner / A Model of Database Components and Their Interconnection
Comp. Name L1 L2 L3 L4 L5 L6 L7
Schema Name F1 F2 F3 F4 F5 F6 F7
Schema Constraints
Relations R1 [A1 A2 ] R2 [A1 A3 ] R3 [A1A4 A5 ] R4a [A4 A5 A6 ] R4b [A7 A8 ] R5 [A7 A0 ] R6 [A9 A0 ] R7 [A8 A9 ]
A4 → A 5 A4 → A 5 A7 → A 8 A0 → A 7 A9 → A 0 A8 → A 9
Ports ΠFA11 ΠFA21 ΠFA31 ΠFA34 A5 F4 ΠA4A5 ΠFA47 ΠFA48 ΠFA57 ΠFA50 ΠFA69 ΠFA60 ΠFA78 ΠFA79
Table 1: The atomic components of the running example Schema Name H1
Schema Relations
Schema Constraints
T2 [A4 A5 ]
H3
T3 [A7 ]
H4
T4 [A8 ]
H5
T5 [A0 ]
H6
T6 [A9 ]
H7
T7 [A4 ]
ΠFA11
πAF11 : F1 → H1
ΠFA31
πAF13 : F3 → H1
ΠFA21
T1 [A1 ]
H2
Associated Ports and View Mappings
A4 → A 5
πAF12 : F2 → H1
ΠFA34 A5
πAF43A5 : F3 → H2
ΠFA47
πAF74 : F4 → H3
ΠFA44 A5 ΠFA57
πAF44A5 : F4 → H2 πAF75 : F5 → H3
ΠFA48
πAF84 : F4 → H4
ΠFA50
πAF05 : F5 → H5
ΠFA78 ΠFA60 ΠFA69 ΠFA79 ΠFA34 ΠFA44
πAF87 : F7 → H4 πAF06 : F6 → H5 πAF96 : F6 → H6 πAF97 : F7 → H6 πAF43 : F4 → H7 πAF44 : F4 → H7
Table 2: The port schemata of the running example
{L5 , L7 }. Then ReltY57 , J567 = {{ΠFA50 }, {ΠFA79 }}. Operationally, ReltY57 , J567 is equivalent to ∅; that is, it imposes no constraints at all. This implies that the subcomponent SubCptX567 ,J57 ProjSchX567 Y57 , J567 is not well defined. To illustrate this directly, let ML5 = {(a7 , a0 )} ∈ LDB(F5 ) and ML7 = {(a8 , a9 ), (a8, a9 )} ∈ LDB(F7 ) with a8 = a8 . In view of the FD A9 → A0 on F6 , there can be no ML6 ∈ LDB(F6 ) such that (ML5 , ML6 , ML7 ) ∈ LDB(ProjSchX567 Y57 , J567 ), since the fact that ML5 contains only one tuple implies that ML7 can consist of only one tuple as well.
90
S.J. Hegner / A Model of Database Components and Their Interconnection
H7 L1
L2
L3
L4
R1 [ A1 A2 ]
R2 [ A1 A3 ]
R3 [ A1 A4 A5 ]
H1
R4a [ A4 A5 A6 ] R4b [ A7 A8 ] H2
H3
H4
L5
L6
L7
R5 [ A7 A0 ]
R6 [ A9 A0 ]
R7 [ A8 A9 ]
H5
H6
Figure 3: Graphical depiction of all possible interconnections 2.13 Example — A pair of components whose interconnection is not free for ports It is useful to show how a simple interconnection of relational components can violate the condition 2.10(e) of being free for ports. To this end, let A 1 and A2 be attributes with the same countably infinite domain; dom(A1 ) = dom(A2 ). There are three components over these domains, as identified in Table 3. Each component has two ports, one for the proF F jection of its relation on A1 , and a second for A2 . For the ports ΠFAα1 , ΠAβ1 , and ΠAδ1 , let F
the port schema have the single relation symbol TA1 [A1 ], and for the ports ΠFAα2 , ΠAβ2 , and F
ΠAδ2 , let the port schema have the single relation symbol TA2 [A2 ]. There are no constraints associated with these port schemata, other than the domain constraints. However, if L α F F and Lβ are interconnected via Jαβ = {{ΠFAα1 , ΠAβ1 }, {ΠFAα2 , ΠAβ2 }}, then it is easy to see that F
F
F
LDB(Schema{Lα , Lβ }, Jαβ ) = ∅. Similarly, letting Jαδ = {{ΠFAα1 , ΠAβ1 }, {ΠAγ2 , ΠAγ2 }}, it follows that LDB(Schema{Lα , Lδ }, Jαδ ) = {∅}. Thus, {Lα , Lβ , Lδ } is not free for ports with respect to either Jαβ or Jαδ . Comp. Name Lα
Schema Name Fα
Rα [A1 A2 ]
Schema Constraints Rα [A1 ] ⊆ Rα [A2 ]
Lβ
Fβ
Rβ [A1 A2 ]
Rβ [A1 ] ⊆ Rβ [A2 ]
Lδ
Fδ
Rδ [A1A2 ]
Rδ [A1 ] ∩ Rδ [A2 ] = ∅
Relations
Ports ΠFAα1 F ΠAβ1 F ΠAδ1
ΠFAα2 F
ΠAβ2 F
ΠAδ2
Table 3: The atomic components for 2.13
3.
Acyclic Interconnections of Components
As observed in 2.10 and 2.11, the property of a set X of components being free for ports with respect to an interconnection family J is critical for the definition of both compound components and subcomponents. Therefore, it is crucial to identify conditions under which this condition is met. Fortunately, by requiring the acyclicity of a hypergraph derived from the interconnection, it is possible to guarantee the even stronger condition that each constituent component of is free in X .
91
S.J. Hegner / A Model of Database Components and Their Interconnection
The reader is perhaps familiar with the use of hypergraphs in characterizing the structure of relational decompositions [9]. However, there are very substantial differences between the hypergraph of a compound component, as defined here, and the hypergraph of a relational decomposition. The use of hypergraphs in this paper is closer to that found in more general characterizations of desirable schemata, as studied in [11]. In any case, it seems appropriate to give a self-contained presentation of the ideas.
3.1 Hypergraphs and acyclicity To begin, a very brief summary of the key notions from the theory of hypergraphs is given. The standard reference on this subject is the monograph of Berge [4] , to which the reader is referred for details. A hypergraph is a pair G = (V, H) in which V is a finite set of vertices and H ⊆ P(V ) (the set of all subsets of V ), with each h ∈ H containing at least two distinct elements. 1 The members of H are called hyperedges. A path from v1 to vn in G is a sequence v1 , h1 , v2 , h2 , .., vn−1, hn−1 , vn in which the following conditions hold: (i) vi ∈ V for 1 ≤ i ≤ n with {vi | 1 ≤ i ≤ n − 1} all distinct, and {vi | 2 ≤ i ≤ n} all distinct. It may be the case that v1 = vn , but this is not necessary. (ii) hi ∈ H for 1 ≤ i ≤ n − 1 with {hi | 1 ≤ i ≤ n − 1} all distinct. (iii) {vi , vi+1 } ∈ hi for 1 ≤ i ≤ n − 1. The number n − 1 is called the length of the path. A (Berge)2 cycle in G is a path of length at least two from a vertex v to itself. G is called (Berge) acyclic if it does not contain any (Berge) cycles. For V ⊆ V , the full subhypergraph of G generated by V is SubHGraphG,V = (V , {h ∩ V | h ∈ H}). V ⊆ V is closed in G if whenever v1 , vn ∈ V and v1 , h1 , v2 , h2 , .., vn−1, hn−1 , vn is a path from v1 to vn in G, then v1 , h1 ∩V , v2 , h2 ∩V , .., vn−1, hn−1 ∩V , vn is a path from v1 to vn in SubHGraphG,V .
3.2 The hypergraph of an interconnection family Interconnection hypergraphs are defined as follows. (a) The interconnection hypergraph of X defined by J, denoted IntGraphX (J), has X as its set of vertices and {Components(I) | I ∈ J} as its hyperedges. The interconnection hypergraph for MaxStar(L) of 2.12 is shown in Figure 4 to the right. Each hyperedge is represented as an ellipse, with its members the component names which it encircles.
L1
L2
L3
L4
L5
L6
L7
Figure 4:
The interconnection hypergraph of MaxStar(L) of 2.12
(b) For Y ⊆ X , the interconnection hypergraph of Y defined by J, denoted IntGraphY (J), is the full subhypergraph of IntGraphX (J) generated by Y . The key result is the following.
1 In [4, Ch. 17, §1], hyperedges (arêtes) are allowed to have only one member, implying that a hyperedge may connect a vertex to itself. Such edges are not allowed in the formalism presented here. 2 In [4], such entities are called simply cycles, but in other contexts, such as that of [9], many different types of cycles for hypergraphs are investigated, so the qualifier “Berge” is appended.
92
S.J. Hegner / A Model of Database Components and Their Interconnection
3.3 Proposition Assume that IntGraphX (J) is acyclic. (a) Every C ∈ X , and hence every Γ ∈ PortsX , is free in X with respect to J. (b) If Y ⊆ X is a closed set of vertices of IntGraphX (J), then Y is closed in X with respect to J. P ROOF OUTLINE : The idea is very simple. Assume that IntGraphX (J) is acyclic, and choose any C ∈ X and MC ∈ LDB(C). For each I ∈ J, Γ ∈ I ∩Ports(C), and Γ ∈ I with Γ = Γ , choose any MSrcCpt(Γ ) ∈ LDB(SrcCpt(Γ )) with γ˚(MC ) = γ˚ (MSrcCpt(Γ ) ). Since IntGraphX (J) is acyclic, it is guaranteed that this construction will not result in any conflicts of other port matchings. Now choose another C ∈ X from those which were included in the previous step, and repeat the process. The formal proof proceeds by induction. This establishes (a). Part (b) is almost identical, except that the starting point is a member of LDB(SchemaY, J ) instead of a member of LDB(C). 2 Thus, whenever IntGraphX (J) is acyclic, the notion of compound component 2.10(i) is well defined. Furthermore, for all Y ⊆ X with the property that Y is a closed set of vertices in IntGraphX (J), the subcomponent SubCptX,J Y , as given in 2.11(f), is well defined as well. 3.4 Attribute hypergraphs Since Berge acyclicity is not viewed as the appropriate one for the characterization of good schema construction in the relational context [9], it is worthwhile to present a short example to show how the notion of hypergraph for a relational schema differs from that of component interconnection. Consider just the components L3 and L4 of 2.12. A1 A4 A5 A6 A7 A8 The attribute hypergraph for this pair, as well as the component hypergraph, are shown in Figure 5 to the L3 L4 right. It is easy to see that the attribute hypergraph is cyclic. Indeed, (A4 , {A1, A4 , A5 }, A5, Figure 5: The attribute hypergraph {A4 , A5 , A6 , A7 , A8 }, A4) is a path from A4 to itself. (above) and interconnection hyperHowever, no such corresponding path occurs in the graph (below) for L3 and L4 of 2.12 component hypergraph. The key distinction is that in the relational representation, as studied in [9], each attribute is a vertex of the hypergraph, while in the model of this paper, each component is a single vertex, regardless of how many attributes the relations of its schemata may have. In the relational representation, any port view with more than one attribute will result in a Berge-cyclic hypergraph. Thus, the fundamental properties of the underlying hypergraphs in the two representations can be completely different. 4.
Updates to components
The initial motivation for developing the ideas reported here was to study how updates to components propagate throughout the interconnection, and in particular to look for canonical ways to extend an update on a subcomponent to the entire network. While a complete presentation of these ideas must be deferred to a separate article, it is nevertheless instructive to illustrate the key ideas.
S.J. Hegner / A Model of Database Components and Their Interconnection
93
4.1 Liftings and feasible environments Let Z ⊆ Y ⊆ X , and let (M1, M2 ) be an update on ProjSchY, J . (a) (M1 , M2 ) is internal to SubCptX,J Z if γ˚(M1 ) = γ˚(M2 ) for all Γ ∈ ExtPortsZ, J . Thus, if (M1 , M2) is internal, the update can be made without involving any components not in Z. If the desired update is not internal, then it must be lifted to a larger subcomponent. (b) An update (M1 , M2 ) on SubCptX,J Y is called an internal lifting of (M1 , M2 ) to SubCptX,J Y if (M1 , M2 ) is internal to SubCptX,J Y and (NatProjX;J;Y Z )˚(Mi ) = Mi for i ∈ {1, 2}. (c) Y is called a uniformly feasible environment for (M1, M2 ) relative to J if for every M1 ∈ LDB(SubCptX,J Y ) with (NatProjX;J;Y Z )˚(M1 ) = M1 , there is an M2 ∈ LDB(SubCptX,J Y ) with the property that (M1 , M2 ) is an internal lifting of (M1 , M2 ) to SubCptX,J Y . Uniform feasibility is crucial because it says that the update (M1 , M2) on SubCptX,J Z has an internal lifting to SubCptX,J Y regardless of the actual state of that subcomponent. 4.2 Order and canonical updates There is one additional issue regarding the lifting of updates which is not recaptured in the formalism of 4.1. In general, there are many possible liftings of an update (M1 , M2) from SubCptX,J Z to SubCptX,J Y , even in the case that Y is a least uniformly feasible environment for it. The further goal is to find the canonical such lifting — characterized by that which adds the least amount of additional information to the database. The problem of recapturing such minimality has received significant attention in the context of view updates, particularly in the context of logic programming [19]; however here the databases do not generally consist of sets of clauses, and so a different approach is necessary. For the component context developed here, the approach which is currently being developed is to regard the best lifting to be those defined by free updates. The details will be reported in a separate article, but the following example provides a glimpse of the main ideas.
4.3 Example — Canonical liftings of updates to components In this example, there is a total of eight components, including the two which were introduced in 2.1. The overall format is similar to that employed in 2.12; therefore, only significant differences will be elaborated. Information about the atomic components and the ports is given in Tables 4 and 5, respectively. The interconnection family I 1 which will be used is shown below; Figure 6 illustrates this family in graphical form. E6 E6 E5 E5 E7 E8 E3 E2 E4 E4 1 I1 = {{ΠE B2 , ΠB2 }, {ΠA2 , ΠA2 }, {ΠA4A5 , ΠA4 A5 }, {ΠA6 , ΠA6 , ΠA6 }, {ΠA7 , ΠA7 }}
For simplicity, this example will be limited to the characterization of canonical least liftings for insertions. For the realization of such liftings to be nontrivial, it must be possible to insert partial information into relations, and to pad out the remainder with nulls. However, this must be done in a systematic way, paying careful attention to constraints which specify where nulls may appear, and how functional dependencies (FDs) behave in their presence.
94
S.J. Hegner / A Model of Database Components and Their Interconnection
Comp. Name
Schema Name
Relations
K1
E1
R1 [B1 B2 ]
K2
E2
R2 [B2 B3 ]
K3
E3
R3 [A1 A2 ]
K4
E4
R4 [A2 A3 A4 A5 ] S4 [B1 B2 ]
K5
E5
R5 [A4 A5 A6 ]
K6
E6
R6 [A6 A7 ]
K7
E7
R7 [A6 A8 ]
K8
E8
R8 [A7 A9 ]
Ports
Constraints B1 → B 2 ForbidNulls(B1 B2 ) B2 → B 3 ForbidNulls(B2 B3 ) A1 → A 2 ReqNullTup(A1 A2 ) NoPartNulls(A1 A2 ) A2 → A 3 A4 A5 R4 [A3 ] ⊆ S4 [B1 ] NoPartNulls(A2 A3 A4 A5 ) ReqNullTup(A2 A3 A4 A5 ) ForbidNulls(B2 ) n A4 A5 → A6 NoPartNulls(A4 A5 ) ReqNullTup(A4 A5 A6 ) n A6 → A7 ReqNullTup(A6 A7 ) n A6 → A8 ReqNullTup(A6 A8 ) n A7 → A9 ReqNullTup(A7 A9 )
1 ΠE B2 2 ΠE B2 3 ΠE A2
4 ΠE A2
4 ΠE A4 A5
5 ΠE A4 A5 6 ΠE A6
4 ΠE B2
5 ΠE A6 6 ΠE A7
7 ΠE A6 8 ΠE A7
8 ΠE A9
Table 4: The atomic components of the example of 4.3 K1
K2
R1 [ B1 B2 ]
R2 [ B2 B3 ] G1
K4 S4 [ B1 B2 ]
K3
K5
K6
R3 [ A1 A2 ] R4 [ A2 A3 A4 A5 ] R5 [ A4 A5 A6 ] R6 [ A6 A7 ] G2
G3
G4
K7
K8
R7 [ A6 A8 ]
R8 [ A7 A9 ]
G5
Figure 6: Graphical representation of the interconnection family I 1 Nulls: There is a distinguished null marker, denoted n, with n ∈ dom(A) for every attribute A. This null marker is similar to the placeholder described in [18, Sec. 12.5.2]. There are three associated constraint types, each of which takes a list of attribute names as its argument. Since attribute names can occur in only one relation name of a schema, the semantics described below are unambiguous. ForbidNulls(−): This constraint specifies that no tuple may have the value n in any of the attribute positions listed. NoPartNulls(−): (No partial nulls) This constraint specifies if a tuple has the value n in
S.J. Hegner / A Model of Database Components and Their Interconnection
Schema Name G1
Schema Relations T1 [B2 ]
G2
T2 [A2 ]
G3
T3 [A4 A5 ]
Schema Constraints ForbidNulls(B2 )
Associated Ports and View Mappings 1 ΠE B2
πBE21 : E1 → G1
4 ΠE B2
πBE24 : E4 → G1
2 ΠE B2
T4 [A6 ]
πAE23 : E3 → G2
NoPartNulls(A4 A5 )
4 ΠE A4 A5
πAE44A5 : E4 → G3
ReqNullTup(A4 A5 )
ΠA45A5
πAE45A5 : E5 → G3
6 ΠE A6
πAE66 : E6 → G4
4 ΠE A2
E
7 ΠE A6
G5
T5 [A7 ]
G6
T6 [A9 ]
πBE22 : E2 → G1
3 ΠE A2
5 ΠE A6
G4
95
6 ΠE A7 8 ΠE A7
8 ΠE A9
πAE24 : E4 → G2
πAE65 : E5 → G4 πAE67 : E7 → G4 πAE66 : E6 → G5 πAE68 : E8 → G5 πAE98 : E8 → G6
Table 5: The port schemata of the example of 4.3 one of the attribute positions listed, then it must have the value n in every position listed. ReqNullTup(−): (Require null tuple) This constraint specifies that a relation must have at least one tuple which has the value n in every position listed in the argument. Functional dependencies and nulls: In addition to the usual semantics for an FD A → B, there are several variations which further specify the special way in which the null marker n is handled. In all descriptions below, assume (without loss of generality) that R[U] is a relation scheme on attribute set U with A, B ⊆ U, and that r is a tuple over R[U]. n n-FDs: The relation r satisfies the n-FD A → B iff the following two conditions are satisfied. Null extension: For every t ∈ r there is a t ∈ r with t [A] = t[A] and t [B] = n for every B ∈ B. Quasi-functionality: If t,t ∈ r with the property that t[A] = t [A], then at least one of the following three conditions must hold: (i) t = t ; (ii) t[B] = n for every B ∈ B; or (iii) t [B] = n for every B ∈ B.
In words, for an n-FD A → B to be satisfied, there may be at most two tuples over associated with each distinct value for A, one with all nulls in B (required) and possibly one other which is not all null on B. There are three extensions of the notion of an n-FD, which are identified below. n B is null preserving if whenever t ∈ r with t[A] = n Null preservation: The n-FD A → n for some A ∈ A, then t[B] = n for every B ∈ B. The notation A → B indicates that
96
S.J. Hegner / A Model of Database Components and Their Interconnection n the n-FD A → B is null preserving. n Null reflection: The n-FD A → B is null reflecting if whenever If t ∈ r with t[B] = n n for every B ∈ B, then t[A] = n for every A ∈ A. The notation A → B indicates that n the n-FD A → B is null reflecting. n Simultaneous preservation and reflection: The notation A → B indicates that the n-FD n A → B is both null preserving and null reflecting.
⎡ ⎢ ⎢ ⎢ ⎢ ⎣
E3 ⎤ a11 a21 a12 a22 ⎥ ⎥ a13 a23 ⎥ ⎥ a14 a23 ⎦ n
n
E4 ⎤ a21 a31 a41 a51 ⎢ a22 a32 a42 a52 ⎥ ⎢ ⎥ ⎣ a23 a33 a43 a53 ⎦ ⎡
R3
n
⎡ ⎢ ⎢ ⎢ ⎢ ⎣
n
n
n
⎤
a31 b21 a32 b22 ⎥ ⎥ a33 b23 ⎥ ⎥ b11 b21 ⎦ n b22 S
4
⎡ ⎢ ⎢ ⎢ ⎢ ⎢ R4⎢ ⎣
E5 ⎤ ⎡ a41 a51 a61 ⎢ a41 a51 n ⎥ ⎥ ⎢ ⎢ a42 a52 a62 ⎥ ⎥ ⎢ ⎣ a42 a52 n ⎥ ⎥ a43 a53 n ⎦ n n n R
E6 ⎤ a61 a71 a61 n ⎥ ⎥ a62 a72 ⎥ ⎥ a62 n ⎦ n n R
⎡ ⎢ ⎢ ⎢ ⎢ ⎣ 6
E7 ⎤ a61 a81 a61 n ⎥ ⎥ a62 a82 ⎥ ⎥ a62 n ⎦ n n R
E8 ⎤ a71 a91 ⎢ a71 n ⎥ ⎥ ⎢ ⎣ a72 n ⎦ n n R ⎡
8
7
5
E1 c11 c12 c11 c12 R
1
E2 c22 c23 R
2
Figure 7: An example database for the interconnection I1 Figure 7 shows an example of a consistent database for the interconnection I 1 . To illustrate the ideas of canonical updates, a few selected examples will now be considered. Suppose that it is desired to insert the tuple (a15 , a25 ) into the relation of R3 [A1 A2 ] of the schema E3 of K3 . The goal is to identify the canonical lifting of this update from K3 to to a compound component under which this update may be supported naturally, without making any arbitrary choices. First of all, it is important to clarify what is meant by an arbitrary choice. Consider a soluE4 3 tion which lifts the update to the compound component Cpt{K3 , K4 }, {ΠE A2 , ΠA2 } by inserting the tuple (a25 , a31 , a41 , a51 ) into R4 [A2 A3 A4 A5 ], leaving the state of all other components unchanged. This lifting makes an arbitrary choice for the values for attributes A 3 A4 A5 ; note that (a25 , a32 , a42 , a52 ) or (a25 , a33 , a43 , a53 ) would work just as well, and there is no reason to prefer one over the other. All involve parts of tuples which already occur in the database, and so make arbitrary semantic choices for the information associated with (a 15 , a25 ). The canonical solution involves using completely new values in these positions, and so is independent of the other values in these relations. More precisely, let a 35 ∈ dom(A3 ) \ {n}, a45 ∈ dom(A4 ) \ {n}, a55 ∈ dom(A5 ) \ {n}, and b25 ∈ dom(B2 ) \ {n} be distinct domain values which have not already used in any relation. First insert (a25 , a35 , a45 , a55 ) into the relation of R4 [A2 A3 A4 A5 ] and (a35 , b25 ) into the relation of S4 [B1 B2 ]. Note that the second tuple is mandated by the inclusion dependency R4 [A3 ] ⊆ S4 [B1 ]. This further mandates an insertion into E5 E4 E4 3 E5 , so the update must be extended to Cpt{K3 , K4 , K5 }, {ΠE A2 , ΠA2 }{ΠA4 A5 , ΠA4 A5 } . Insert (a45 , a55 , n) into the relation of K5 . The use of the null is a recognition that the constraints do not force one to select a specific value for A6 . Note that this insertion does not violate the n n-FD is A4 A5 → A6 , so that the resulting state is legal. Furthermore, it does not make use of any arbitrary values which already occur in other relations and it is least, in the sense that no subset of the specified insertions will do, and there is no smaller set of components which will support such an insertion using new values. One issue still remains; namely, that arbitrary choices for a35 , a45 , a55 , and b25 were made. Formally, this is reconciled by noting that all other such solutions are isomorphic, up
S.J. Hegner / A Model of Database Components and Their Interconnection
97
to a renaming of the values. A solution to the update problem is thus not a single update, but rather an equivalence class of isomorphic updates. Replacing a35 , a45 , a55 , and b25 by different values which do not occur elsewhere, say a 35 , a45 , a55 , and b25 , results in a solution which is structurally indistinguishable. The resulting solution is canonical, and identifies the natural scope for lifting an insertion to K3 as {K3 , K4 , K5 }. A formal justification of the canonicity of this solution is rooted in the construction of free objects [15, §31] [1, 8.22] over a suitable category of updates, and is beyond the scope of this paper; the details will appear in forthcoming report. However, it is possible to give an informal justification. The idea is that every other possible lifting of the desired update to K3 can be obtained from the canonical one via a combination of adding additional tuples and forcing the “free” values in the canonical update to take on specific values. For example, the lifting which inserts the tuple (a25 , a31 , a41 , a51 ) can be obtained from the canonical one identified above which inserts the tuples (a25 , a35 , a45 , a55 ), (a45 , a55 , n), and (a35 , b25 ) by mapping the “free” values a35 , a45 , a55 , and b25 to the existing values a31 , a41 , a51 , and b21 , respectively. The case of deletions is handled similarly, although it turns out to be somewhat simpler, since there is no need to group isomorphic solutions (as no new values need be inserted). 5.
Closing Remarks
5.1 Conclusions The foundations of a component-based model of database schemata, with the interconnection of components realized via communicating views, has been presented. Particular attention has been paid to the question of when such interconnections are well behaved, and a characterization in terms of the acyclicity of an underlying hypergraph has been presented. Furthermore, the way in which updates propagate through such components has been illustrated, although not fully formalized.
5.2 Further directions This paper is only a beginning, and many topics remain to be studied. Among them are the following. Updates in the component-based framework: This work began as a study of updates in the context of components. Consequently, an important future direction is to complete the formalization of canonical liftings, as discussed in 4.3. A related direction is update via cooperation, in which the realization of a proposed update requires that other components be updated as well, not as a canonical update but rather as one chosen by a user who has update rights for that component. First results on this topic, including a formal model for the update process, are reported in [14]. Upon following the communication between ports that update by cooperation entails, it is possible to infer much about the necessary workflow patterns behind such updates. Initial investigations on this latter topic are now being pursued. Component-based HERM to relational design theory: It is a standard design technique to begin by modelling the enterprise using a flavor of ER, such as the HERM model, and then to translate that design to a relational schema [23, Ch. 10]. A future direction of this research is to extend this design theory to components; that is, to develop systematic
98
S.J. Hegner / A Model of Database Components and Their Interconnection
tools for the translation of a HERM design based upon components, such as elaborated in [25], to a relational design which preserves the component structure, using the component model developed here for this final schema. Rapprochement with the behavioral theory: As already noted in the introduction, the work presented here is motivated by the database-component model of Thalheim [25], which is in turn based upon the more general component model of Broy [6]. It is important to pursue an understanding of the degree to which these two models can be unified, and to understand their fundamental differences as well.
5.3 Remarks on the literature Nearly thirty years ago, Weber [26] suggested that modular design techniques could be applied fruitfully to database systems as well, although no detailed formalization was presented. In [7], a software tool for modular database design is presented. In that approach, the emphasis is not so much upon building systems by interconnecting components as it is in refining the design, specifically by combining and even redefining the so-called conceptual modules via subsumption modules. As such, it does not emphasize basic communication between components as does the framework presented here. Rather, it has much more of a software-engineering flavor. As noted in the introduction, the flavor of component-based modelling of database systems upon which this paper has its roots in the approach of Thalheim [24] [22] [25]. 6.
Acknowledgments
Much of this research was completed while the author was a visitor at the Information Systems Engineering Group at the University of Kiel during parts of 2005 and 2006. He is indebted to Bernhard Thalheim for suggesting the idea that his ideas of database components and the author’s work on views and view updates could have a fruitful intersection, as well as for inviting him to work with his group on this problem. He is furthermore indebted to Bernhard, as well as to the other members of the Information Systems Engineering Group, particularly Hans-Joachim Klein and Peggy Schmidt, for many helpful discussions during the course of this work. Peggy Schmidt also read a preliminary draft of this paper and made numerous suggestions to improve the presentation. References [1] A DÁMEK , J., H ERRLICH , H., AND S TRECKER , G. Abstract and Concrete Categories. WileyInterscience, 1990. [2] A RBIB , M. A. Theories of Abstract Automata. Prentice-Hall, 1969. [3] BANCILHON , F., AND S PYRATOS , N. Update semantics of relational views. ACM Trans. Database Systems 6 (1981), 557–575. [4] B ERGE , C. Graphes et Hypergraphes. Dunod, 1970. [5] B ROY, M. A logical basis for modular software and systems engineering. In SOFSEM (1998), B. Rovan, Ed., vol. 1521 of Lecture Notes in Computer Science, Springer, pp. 19–35. [6] B ROY, M. Model-driven architecture-centric engineering of (embedded) software intensive systems: modeling theories and architectural milestones. Innovations Syst. Softw. Eng. 6 (2007), in press.
S.J. Hegner / A Model of Database Components and Their Interconnection
99
[7] C ASANOVA , M. A., F URTADO , A. L., AND T UCHERMAN , L. A software tool for modular database design. ACM Trans. Database Systems 16, 2 (1991), 209–234. [8] C HEN , P. P. The entity-relationship model - toward a unified view of data. ACM Trans. Database Systems 1, 1 (1976), 9–36. [9] FAGIN , R. Degrees of acyclicity for hypergraphs and relational database schemes. J. Assoc. Comp. Mach. 30, 3 (1983), 514–550. [10] H EATH , I. J. Unacceptable file opearations in a relational data base. In Proceedings of the ACM SIGFIDET Workshop on Data Description, Access, and Control (1971), pp. 19–33. [11] H EGNER , S. J. Characterization of desirable properties of general database decompositions. Ann. Math. Art. Intell. 7 (1993), 129–195. [12] H EGNER , S. J. An order-based theory of updates for database views. Ann. Math. Art. Intell. 40 (2004), 63–125. [13] H EGNER , S. J. The complexity of embedded axiomatization for a class of closed database views. Ann. Math. Art. Intell. 46 (2006), 38–97. [14] H EGNER , S. J., AND S CHMIDT, P. Update support for database views via cooperation. In Advances in Databases and Information Systems, 11th East European Conference, ADBIS 2007, Varna, Bulgaria, September 29 - October 3, 2007, Proceedings (2007), Y. Ioannis and B. Novikov, Eds., Lecture Notes in Computer Science, Springer-Verlag. In press. [15] H ERRLICH , H., AND S TRECKER , G. E. Category Theory. Allyn and Bacon, 1973. [16] K ATZ , R. H., AND B ORRIELLO , G. Contemporary Logic Design, second ed. Pearson Education, 2005. [17] K RUEGER , C. W. Software reuse. ACM Comput. Surveys 24, 2 (1992), 131–183. [18] M AIER , D. The Theory of Relational Databases. Computer Science Press, 1983. [19] M AYOL , E., AND T ENIENTE , E. A survey of current methods for integrity constraint maintenance and view updating. In Proc. ER ’99 Workshops, Paris, Nov. 15-18, 1999 (1999), vol. 1727 of Springer LNCS, Springer-Verlag. [20] PAREDAENS , J., D E B RA , P., G YSSENS , M., AND VAN G UCHT , D. The Structure of the Relational Database Model. Springer-Verlag, 1989. [21] R ISSANEN , J. Independent components of relations. ACM Trans. Database Systems 2, 4 (1977), 317–325. [22] S CHMIDT, P., AND T HALHEIM , B. Component-based modeling of huge databases. In Advances in Databases and Information Systems: 8th East European Conference, ADBIS 2004, Budapest, Hungary, September 22-25, 2004, Proceedings (2004), A. Benczúr, J. Demetrovics, and G. Gottlob, Eds., no. 3255 in Lecture Notes in Computer Science, Springer-Verlag, pp. 113–128. [23] T HALHEIM , B. Entity-Relationship Modeling. Springer-Verlag, 2000. [24] T HALHEIM , B. Database component ware. In ADC (2003), K.-D. Schewe and X. Zhou, Eds., vol. 17 of CRPIT, Australian Computer Society, pp. 13–26. [25] T HALHEIM , B. Component development and construction for database design. Data Knowl. Eng. 54, 1 (2005), 77–95. [26] W EBER , H. Modularity in data base system design: A software engineering view of data base systems. In Issues in Data Base Management, Proceedings of the 4th VLDB, September 1315, 1978, West Berlin, Germany (1978), H. Weber and A. I. Wasserman, Eds., North-Holland, pp. 65–91.
100
Information Modelling and Knowledge Bases XIX H. Jaakkola et al. (Eds.) IOS Press, 2008 © 2008 The authors and IOS Press. All rights reserved.
Creating Multi-Level Reflective Reasoning Models Based on Observation of Social Problem-Solving in Infants Heikki RUUSKA, Naofumi OTANI, Shinya KIRIYAMA, Yoichi TAKEBAYASHI Shizuoka University, Johoku 3-5-1, Hamamatsu, Japan
Abstract. We have created an infant learning environment that has capacity for effective behavioral analysis while providing new views on infant learning, and serves as a basis for creating a corpus of infant behavior. We are gathering behavioral data including but not limited to data in social, spatial and temporal domains, while providing inspiring learning experiences to infants. We have used the corpus for constructing and combining novel models on social interaction and problem solving.
1. Introduction Current intelligent systems are usually limited to producing an effective solution for a single problem type and fail when presented a problem outside of its scope. We believe that in order to create more flexible systems – ones that can handle the many kinds of everyday tasks labeled as commonsensical -- we need advanced methods that can select solution methods according to problem types. In order to develop such methods, we are investigating how humans identify and define problems and solve them. Most commonsensical problems are solved without us being aware of them, making tracing the process difficult. It has been proposed that this is because an adult mind is very complex and already equipped with error handlers that switch to a different solution type quickly, without conscious notice [1]. Therefore, we are focusing our research on infants, who often fail at producing solutions, thus giving us valuable data on where processes are going wrong and what is missing when compared to an adult mind. In this paper, we describe creation of a multimodal infant behavior corpus and how we have used it for creating models of social problem-solving that might describe what is actually happening in infants’ minds. Furthermore, we present some ideas on how the corpus data and the problem-solving models can be used for creating larger scale reflective reasoning systems, and developing inspiring learning environments.
H. Ruuska et al. / Creating Multi-Level Reflective Reasoning Models
101
2. Building Multimodal Infant Behavior Corpus 2.1 Learning Environment We have an experimental parent-child learning environment for infants [2 to 5]. It has two purposes: first, to provide the participating parents a good environment for the wholesome growth of their children and inspiring experiences to the infants, and second, to provide us with a setting where we can regularly monitor the infants’ behavior and development.
Figure 1 Two infants, their mothers, and a teacher engaged in a learning task
Three sixty-minute classes are held weekly, each class consisting of three infant-parent pairs, where the infants are of the same age. There is one teacher per class. The first half of a class takes place in a classroom setting where the teacher utilizes various materials, such as clay, crayons, paper, etc. and has the infants complete various tasks, such as building, drawing or identifying things, usually with the parents’ aid. For the second half of a class, the teacher and parents discuss child care and child learning. During this time, the infants are given various toys and are let to play freely. The program also includes reports on what is happening at homes, including parents’ observations on child’s development. The whole sixty-minute sessions are recorded by four cameras placed at different angles and multiple microphones, including rucksack microphones worn by each infant. The positioning of the cameras and an overview of the classroom and studio is shown in Figure 2. At the time of the writing, we have footage of 51 learning sessions over a year and a half’s time.
102
H. Ruuska et al. / Creating Multi-Level Reflective Reasoning Models
Figure 2 Infant learning environment layout
2.2 Annotating Infant Behavior The audiovisual data recorded during the classes described in the previous subchapter is annotated with meta-knowledge after the class by multiple members of the research team. The annotations include descriptions of infants’ activities, as well as descriptions of actual actions, such as speech, movement, and grasping things. Information on obvious and inferred goals is also recorded. These annotations are currently recorded in a natural language (Japanese) and they form the core of the behavioral corpus. Figure 3 shows an annotation tool used for the purpose.
Figure 3 Annotation tool.
H. Ruuska et al. / Creating Multi-Level Reflective Reasoning Models
103
3. Preliminary Modeling Experiments 3.1 Modeling Based on Observations in Corpus Shortly after starting to build the behavioral corpus described in Chapter 2, we noticed many developmental changes in infants. Some infants that used to be by themselves were becoming more cooperative and intrigued by their surroundings. There was perceivable increase in their social awareness. For example, we observed many cases of social collaboration and solving of interest conflicts between infants in the playroom setting where they were left to themselves. These observations led us to wonder how we might construct a simulated setting where these changes, and processes themselves, could be replicated, i.e. how to construct models of social problem-solving in infants. A central requirement was that the models be such where we would have models for separate individuals, instead of having models for separate cases. This is necessary for achieving better independence from particular situations. The models should also have internal models of themselves and of other people. The models thus constructed could then be checked and compared with multiple recorded scenarios in the corpus, and perhaps even applied to older children or adults. Our approach is similar to that of Piaget [6] in that we do long-term in-depth observation of relatively few cases, as opposed to statistically analyzing a large number of cases. But where Piaget was more concerned with describing developmental stages occurring in children, we are aiming at building and testing a variety of computable commonsense models. Coined in computer terms, our learning environment is a development and debugging platform for models constructed by others and ourselves. Some related models we are testing and applying include Territory of Information theory by Kamio [7], logical models such as proposed by McCarthy [8] and Mueller [9], models for creating large knowledge bases systems such as Cyc [10], probabilistic reasoning, rule-based reasoning, and methods for combining all of these, as proposed by Minsky [11].
3.2 Emotion Machine Commonsense Model The behavioral corpus gives us a framework for examining factors behind infant behavior and learning, but is by itself not enough for creating working models. For this, we need to convert the natural language annotations to a computable form, and build models and testing environments for deciding which theories and models are relevant. We took parts of the commonsensical reasoning model presented by Minsky [11] as a starting point for building computational models of infant behavior. In particular, we found ideas proposed by him and Sloman [12,13] on multi-level approach to problem solving useful for describing social problem-solving.
3.3 Script Language for Computable Models In order to create the basis for computational models, we needed to implement a scripting language for describing infant behavior in a computable form. Whilst a natural language is easiest to use for annotation, natural language processing is not yet advanced enough for
104
H. Ruuska et al. / Creating Multi-Level Reflective Reasoning Models
creating usable computational models based on natural language alone. To begin with, we utilized Narrative-L, a simple description language using Common Lisp syntax that was developed by Push Singh for his EM-ONE architecture [14]. This had an advantage of having an already existing testing environment [15]. We then started developing variants more suited to describing infant behavior. Chapter 4 includes some examples of a scripting language we currently use in its listings. In the following subchapter (3.4), we present an example of the early scripting language.
3.4 Early Example: “Which Is the Driver?” This example is based on a corpus entry of a playroom setting. Figure 4 contains still shots of the scene. In the scenario, a 4-year old boy and a girl both want to play the driver. The girl communicates this to the boy. The boy understands this, but the girl doesn’t understand that the boy also wants to be the driver. The boy recognizes that there is conflict of interest and he pushes the girl off. The girl is hurt and starts crying. The boy tries to cheer her up. In the end, they end trying to play the driver as both know there can only be one in a car. ( def narrative play-driver ( sequential ( observes boy ( is-sitting girl block ) [1]) ( expects boy ( play-role boy driver ) [2]) ( causes [1] [2] )) ) ( def narrative move-and-sit ( desires boy ( play-role boy driver ) [1] ) ( sequential ( moves-close-to boy block [2] ) ( sit boy block [3] )) ( causes [1] [2] ) ( follows [2] [3] )) ) … Parts of “Which is the Driver”, an early narrative script.
Figure 4 Still pictures of the “Which is the Driver” scene. From the left: the girl shows she wants to play the driver by pantomime. The boy wants to be the driver and pushes her off. The girl starts crying, and the boy tries to cheer her up.
H. Ruuska et al. / Creating Multi-Level Reflective Reasoning Models
105
4. A Reflective Problem-Solver: Simulating the Behavioral Corpus Data 4.1 Creating Models for Multi-level Reflective Reasoning In this chapter, we present a new model that can be used to analyze behavior such as in the following description recorded by an external observer: A boy and a girl are in a room with toys. Boy is holding a block. Boy is looking at the girl, and it looks like he wants to give the block to the girl. Boy is just about to throw the block to the girl, but then he stops for some reason. Boy walks over to the girl. Boy hands the block to the girl. Girl is now holding the block. Our model tries to answer the following questions: If the boy has a goal of giving the block to the girl, how does he know how to do this? Furthermore, if two ways of accomplishing the goal: throwing the block, or walking to the girl and handing it over, are available, how does he choose which to pick? In Chapter 3.4, we described an early example of how infant social behavior might be described and modeled in narratives. However, to create more robust models, we need to divide the actions and reasoning processes to more atomic units, so that they can be used in more than one scenario. From this viewpoint, we pursue flexible models that can be combined in novel ways, to allow for interpretation and reproduction of many kinds of infant behavior. Our model uses discrete sub-processes, divided into various levels of reasoning, for creating solutions to simple problems such as the ones mentioned above.
Figure 5 A three level model that creates an action plan based on scripted memory data
The generic framework of the model is illustrated in Figure 5. We have divided the reasoning process for the problem to three levels [16]: one where plans for solving the
106
H. Ruuska et al. / Creating Multi-Level Reflective Reasoning Models
problem are created, one where whether the plans are possible to implement is inspected, and one where whether they are socially or morally viable is decided. An error with a plan causes a new process with the goal of fixing it to be created. Each of the levels carries a selection of sub-processes, or programs, which we call “critics” after Minsky 2006 [11]. Each critic is essentially a way to think of its own: a thought process that is triggered under certain conditions and usually has inputs and outputs. Critics can also activate other critics or set new sub-goals. As the “playing the driver” case is too complicated for the scope of this paper, we will instead focus on the above example of passing an object to another person. Here, we utilize only a limited number of critics: one that identifies the problem as a “current-state -> desired-state” type of a problem, and activates another critic that searches the memory for existing solutions for this problem type; one that launches subprograms for calculating the required time for each action, and adds them up to determine how long carrying out each solution takes; one that checks whether carrying out all the actions in a solution is possible in the current situation; and one that checks whether carrying out the actions in a solution is socially acceptable, by searching the memory for socially negative elements followed by the actions.
4.2 First Level of Reasoning: Action-Planners First, we make some basic assumptions. We assume that the boy has knowledge of the following actions: hold, walk, hand over and throw. These assumptions are justified by past data. We also assume that the boy also has experience data of similar cases, as listed in memory scripts below. The natural language version is to the left and a computational script language version to the right. We shall use the same syntax throughout the text. Memory Script 1. Boy has block. Boy walks to girl. Boy hands over block. Boy observes girl has block.
(setnarrativeMemory script1 (have boy1 block1) (walk-to boy1 girl1) (hand-over boy1 girl1 block1) (observes boy1 (have girl1 block1)))
Memory Script 2. Boy has block. Boy throws block to girl. Boy observes girl has block.
(setnarrativeMemory script2 (have boy1 block1) (throw-at boy1 girl1 block1) (observes boy1 (have girl1 block1)) (observes boy1 (start-cry girl1)) (reprimand mother boy1))
Boy observes girl starts to cry. Boy is scolded by mother.
When presented with a problem of the form “make current state into desired state”, the boy’s first action is to search for an existing solution. He does this through searching for analogous cases.
H. Ruuska et al. / Creating Multi-Level Reflective Reasoning Models
Current state: Boy has block. Desired state: Boy observes girl has block. Solve problem (current state to desired state). Check for analogous situations in memory.
107
(setstate current (have boy1 block1)) (setstate desired (observes boy (have boy1 block1)) (createplan current desired) =>(findanalogies current desired)
After searching for scripts where “Girl has block” is preceded by “Boy has block”, the action planner returns scripts 1 and 2, giving Plan 1: “walk to girl and hand over the block” and Plan 2: “throw the block to girl” [17]. Either of these plans will bridge the gap between the current situation and the desired situation. Plan 1. Boy walks to girl. Boy hands over block.
=>(define-plan plan1 (walk-to boy1 girl1) (hand-over boy1 girl1 block1))
Plan 2. Boy throws block to girl.
=>(define-plan plan2 (throw-at boy1 girl1 block1))
However, before taking any action, the boy would need to know if any of the plans are practical. Determining this is handled by the second level of critics.
4.3 Second Level of Reasoning: Physical Viability Analysis For the second, reflective level, we assume two critics: one that scans the action plans for flaws, and one that calculates the time they take. A plan is flawed if it includes an action impossible to implement in the current situation. For example, there might be a wall between the boy and the girl, so the boy couldn’t simply walk over to the girl. Alternatively, the block might be too heavy or the girl too far away for the boy to throw it to the girl. Here we assume that there are no such problems.
Compare total time for Plan 1 and Plan 2. Plan 2 takes less time. Prefer Plan 2. Check for validity of Plan 1. “Boy walks to girl” is possible. “Boy hands over the block” is possible.
(compare-times plan1 plan2) => (plan2 plan1) => (planpref-speed (plan2 plan1)) (actions-possibility plan1) (possibility (walk-to boy1 girl1)) =>true (possibility (hand-over boy1 girl1 block1) after (walk-to boy1 girl1) after (have boy1
108
H. Ruuska et al. / Creating Multi-Level Reflective Reasoning Models
block1)) =>true Check for validity of Plan 2. “Boy throws block to girl” is possible.
(actions-possibility plan2) (possibility (throw-at boy1 girl1 block1) after (have boy1 block1)) =>true
In the end, both plans are deemed all right by second level critics and are passed on to the third level, with the distinction that Plan 2 is preferable because it takes less time.
4.4 Third Level of Reasoning: Social Viability Analysis On the third, self-reflective level, the plans are evaluated only by one critic in this example case: one that examines whether actions in them are socially acceptable. This heavily simplified [18] critic does this by searching for any scripts in memory where socially unfavorable elements are preceded by actions included in the plans. Check for social compliance of Plan 1. “Boy walks to girl” no problems. “Boy hands over the block” no problems. Check for social compliance of Plan 2. “Boy throws block to girl” not good.
(actions-sociality plan1) (sociallyOK (walk-to boy1 girl1)) =>true (sociallyOK (hand-over boy1 girl1 block1)) =>true (actions-sociality plan2) (sociallyOK (throw-at boy1 girl1 block1)) =>false
Plan 2 failed because in Memory Script 2, the action “boy throws block to girl” is later followed by “boy is scolded by mother”, which the critic regards as a sign of social incompliance. Thus, even though Plan 2 was preferred after the second level of critics, it failed at the third level. Therefore, as the result of the reasoning process, the boy carries out the first step of Plan 1. If the situation hasn’t changed once the first step is completed, he carries out the second step and then checks if the desired result was achieved.
4.5 Applying the Constructed Model to a Separate Case In this subchapter, we show how some elements of the model constructed in previous subchapters can be applied to a different social problem. For simplicity, we will skip the detailed analysis of the previous chapters and only discuss the main points for this case. First, as before, we give a brief description of the events. It should be noted here that the blocks used are soft, furry toy blocks. A girl and a boy are in a room with toys. The girl wants for the boy to play with her. The boy is looking other way. The girl grabs a block. The girl throws the block at the boy. The
H. Ruuska et al. / Creating Multi-Level Reflective Reasoning Models
109
boy turns around to face the girl. The girl grabs another block. The girl waves the block around. The boy picks up a block. The boy comes to the girl. The boy and the girl start waving the blocks around. The questions raised here are, for example: Why does the girl throw a block instead of making noise or saying something? How does the girl know that she can make the boy play with her this way? Why is the boy not angry at being thrown at with a block? We will not try to answer all the possible questions here, but try to explain what might have caused the girl’s behavior. For describing the girl’s behavior, we need to add some additional critics to the ones introduced in (4.1). First, as the girl wants for the boy to play with her, she needs a first level critic that recognizes that she needs to have the boy’s agreement to play with her. For gaining agreement, she could use script knowledge to know that she needs to make a request. Strictly speaking, using script knowledge for such a basic communication model as making a request is probably not necessary, as babies seem to know this innately, when asking for nutrition or change of diapers, for example. As the ultimate goal for the girl is to get the boy to agree to play with her, she uses her communication model for requests to create the following sub-goals: Basic communication model
Derived Sub-goal for Situation
Script Expression
1.Hold other’s attention
Boy pays attention to girl.
2.Deliver message / make request 3.Infer whether agreement was gained
Boy sees girl wants to play.
(observe girl1 (look-at boy1 girl1)) (observe boy1 (desire girl1 (play-together girl1 boy1))) (observe girl1 (agree boy1 (play-together girl1 boy1)))
Boy agrees to play.
Girl first runs the second and third level critics on this plan; finding no problems (there are no physical or social constraints that would directly prevent any of the sub-goals from taking place), the girl sets out to determine how to fulfill each sub-goal. The girl first needs to gain the boy’s attention. For this case, we assume she can have a pick from 1) shouting to the boy, 2) throwing something at the boy, or 3) moving to the front of the boy, based on her experience data. Second-level speed analysis would give them in order of preference of 1-2-3. As the girl doesn’t have social inhibitors on throwing things, at least light-weight and soft things, at people in this case (maybe her mother isn’t present, or she doesn’t have an experience where she was scolded for throwing things, though this is unlikely), all the plans are valid for the first sub-goal. Now that the girl has a plan on how to gain the boy’s attention, she considers the next step: how to convey the message that she wants to play. We assume she has a memory where she, after waving a toy to someone, has played with that someone using that toy. Her verbal skills are insufficient to convey the request by speech so she doesn’t have that option. Thus, she comes up with a plan of picking up a block, holding it and waving it around for fulfilling the second sub-goal. Regarding the third step, the girl recognizes that she can’t make any conclusions or plans before seeing how the boy reacts. We can assume that she has a few ways to identify whether the boy agreed or not: for example, if the boy walked to the girl and started doing something with a toy facing the girl, or if the girl chose the plan to throw the block, the boy
110
H. Ruuska et al. / Creating Multi-Level Reflective Reasoning Models
throwing the block back could be taken for agreement; the boy walking away, or turning away and continuing whatever he was doing could be taken for disagreement. The point is, the ways for the boy to respond are too many, so she will have to make her conclusions afterwards. As the result of this reasoning process, the girl has made the following plan. Step 1: Shout to the boy, or throw something at the boy, or walk to the boy. Step 2: Wave a toy around. Step 3: See how the boy reacts. However, we are missing a piece: why does she decide to throw a block in the end, as the order of preference based on our previous model would cause her to shout? We conclude that she probably throws the block because she wants to play with the block. Also, the act of throwing is also an act of play, which the girl asserts as priming the boy for the suggestion. Therefore, when making her plans she reflects on her intentions, concluding that as she wants to play with the block, she should use the strongest way possible for communicating this.
4.6 Recording Mental States of Individuals From our corpus based approach, we can create different models for each individual based on our observations. First, we record their behavioral data and tendencies. We can then use these records to decide which particular critics and memory scripts apply to whom in which cases. For example, the boy not throwing the block in the first example case and the girl throwing the block in the second could be handled by the same social inhibition critic, with different memories (whether there is a memory of having been scolded linked to throwing) or different situations (whether the person that did the scolding is present) causing differences in behavior. In our approach, we can describe the mental status and available resources of each individual separately, allowing us to create models that bend flexibly to a variety of patterns of social interaction.
4.7 Using Scripted Corpus Data in Simulated Environments The previous subchapters illustrate some models which can be used as a base for analyzing infant behavior in different social situations where he or she is handing things over, or throwing them to, someone else, and communicating intentions. However, whereas the behavioral corpus has an abundance of examples where singular actions such as handing things over or throwing them take place, for complex scenarios the number of real-world examples is usually limited. Therefore, we are building simulated environments for analyzing causality in more complex cases, like the case described in Chapter 4.5, where we can combine individual actions in ways that have not come up in corpus data, for further testing of the models. Our current simulated environment is a three dimensional space with robots and other physical objects, where their behavior is governed by Newtonian physics. The robots act according to pre-defined scripts. This simulated environment is realistic enough to allow us to construct a wide variety of believable scenes.
H. Ruuska et al. / Creating Multi-Level Reflective Reasoning Models
111
5. Future Prospects 5.1 Validity Considerations Models such as the simplified ones described in Chapter 4 naturally give rise to questions about their validity. How can we be sure that anything like these processes exists in the human mind for problem-solving? From observations alone, how can we be sure that a boy who threw a block at the girl actually had any goal of passing the block to girl, instead of the boy just deciding to throw the block that way and the girl just happening to be there? This problem is emphasized since the infants’ ability to communicate their goals verbally is limited. Our approach is to use multiple observers who by course create slightly different interpretations on both the goals and the ways of reasoning behind particular actions. The models thus created are then applied to many different scenarios in the corpus to see if and where they contradict. By having long-term data on individual subjects, we can also check what kind of experiences they have had in the past, and how those would seem to affect their reasoning. Ultimately, we plan to integrate the reasoning models into a simulated environment, as the one described in chapter 4.7, and link the sensory inputs of the virtual actors to this virtual world. In effect we would be replacing the predefined scripts described before with a virtual intelligence system. We can do this because all parts of the model described in Chapter 4, the script memory and the layered processes, are computable. By this, we can better debug the reasoning models: decide which parts of the virtual actors’ behavior are humanlike and which parts need improving.
5.2 Extending the Domains for the Models and Other Further Developments We have mainly discussed how a multimodal infant behavior corpus can be used in creating models for social problem-solving. However, it is possible to broaden the field to include models for formation and acquisition of spatial, temporal and structural concepts, for example. We have started with the social domain in interests of simplicity and testability, as the changes are relatively easily observed. It should be noted that the problem solving models created the way presented in this paper are not intended to be an only representation for problem solving. Our multi-layer reflective model using simple scripts as a memory format should be thought of as a “kernel” that can launch up a wider variety of approaches. For example, a second level critic described in section 4.3 could use an advanced route-finding algorithm for determining whether walking to next to the girl is possible; similarly, a physics simulation could be run to calculate whether throwing the block to the girl is possible. For analyzing causality, we could first use simple probabilities and stochastic operators, and eventually semantic nets combined with fuzzy logic could conceivably be used in deciding whether the fact that the boy was scolded by the mother really had anything to do with the boy throwing the block. These all are areas for future development. As a step onwards, upon gathering a representative collection of models, we are considering testing their compatibility with larger knowledge-bases. As the function of the layered processes described in this paper greatly depends on what kinds of scripts they are
112
H. Ruuska et al. / Creating Multi-Level Reflective Reasoning Models
fed, it should be fruitful to investigate how they work when supplied with broader chains of commonsense knowledge, such as those stored by Cyc [19] and Openmind [20] projects. Finally, we hope to use the corpus as a tool for analyzing the processes of learning, helping us to invent more effective education methods for benefit of the later generations.
6. Conclusion In this paper, we have described an experimental infant learning environment, a multimodal infant behavior corpus formed from observations of that environment, and some models for reflective reasoning that are constructed based on the corpus. These models by their atomic and flexible nature can be applied to various problem solving scenarios. The models developed by our method can be used for launching different types of subordinate problem solvers for even greater flexibility, bringing us closer to realization of comprehensive human-like problem solving systems.
Acknowledgements First and foremost we would like to thank Saki Kawaguchi, Yutaka Sakane and Goh Yamamoto for their participation in the project, and their advice and critical comments. Further, we especially thank Marvin Minsky and the late Push Singh for providing a theoretical basis for many of the ideas which we have built our system on.
Footnotes and References [1] This has been discussed in many works. P.166 of [6] contains an illustrating example. [2] Yoichi Takebayashi: Multimodal Knowledge Contents Design From the Viewpoint of Commonsense Reasoning. Proceedings of GSIS International Symposium on Information Sciences of New Era: Brain, Mind and Society. Sendai, Japan 2005. [3] Atsuyo Shirai, Saki Kawaguchi, Shinichi Sakane, Yutaka Sakane, Yoichi Takebayashi: Design of Infant-Parent Learning Environments with Privacy Protection. Proceedings of The 20th Annual Conference of the Japanese Society for Artificial Intelligence, 2G2-6, 2006. [Japanese] [4] Kota Otake, Naofumi Otani, Saki Kawaguchi, Yutaka Sakane, Shinya Kiriyama, Yoichi Takebayashi: A Consideration on Infant Behavior Corpus. E-007, FIT2005, Japan. [Japanese] [5] Saki Kawaguchi, Kota Otake, Goh Yamamoto, Shogo Ishikawa, Shinya Kiriyama, Shinichi Sakane, Yutaka Sakane, Yoichi Takebayashi: Multimodal Knowledge Authoring System for Empowering Parent-Child Co-Education Environments. Proceedings of The 19th Annual Conference of the Japanese Society for Artificial Intelligence, 3D2-03, 2005. [Japanese] [6] Jean Piaget: The Language and Thought of the Child. New York: Routledge, 1923/2001. [7] Akio Kamio: Territory of Information. Philadelphia: John Benjamin, 1998. [8] John McCarthy: Mathematical logic in artificial intelligence. Daedalus, 117(1), 1988. [9] Erik Mueller: Commonsense Reasoning. Morgan Kaufmann, 2006. [10] Douglas Lenat, R.Guha: Building Large Knowledge-based Systems, Addison-Wesley, 1990. [11] Marvin Minsky: The Emotion Machine. Simon & Schuster, 2006. [12] Aaron Sloman: Beyond Shallow Models of Emotion. Cognitive Processing, 1(1), 2001.
H. Ruuska et al. / Creating Multi-Level Reflective Reasoning Models
113
[13] Marvin Minsky, Push Singh, Aaron Sloman: The St. Thomas common sense symposium: designing architectures for human-level intelligence. AI Magazine, summer, 25(2), 2004. [14] Push Singh: EM-ONE: An Architecture for Reflective Commonsense Thinking, PhD thesis, MIT, 2005. [15] The Roboverse virtual world simulator is described in [9]. It is currently developed by Bo Morgan at MIT. [16] In [6] and [8], 6 levels for reasoning are described. Levels described in this paper correspond to them roughly as follows: action-planners 1&2, physical evaluators 3, social evaluators 4-6, respectively. The levels described in this paper are by no means meant to be comprehensive: we have simply left out the parts which we currently have no data for. [17] If there were no ready solutions in memory, there could be another critic that suggests the boy moves closer to the girl, for example, in order to bring the block nearer to the girl, and see what happens. [18] Only existence of actions is checked here: for a better model, we need to include subprograms that can test for whether there are any actual causal relationships. Also, using classes of objects instead of instances, we wouldn’t be limited to specific objects. However, such classification has its own problems and is out of the scope of this paper. [19] Douglas Lenat: CYC: A large-scale investment in knowledge infrastructure. Communications of the ACM, 38, No.11 (1995) [20] Push Singh, Erik T. Mueller, Grace Lim, Travell Perkins, Wan Li Zhu: Open Mind Common Sense: Knowledge Acquisition from the General Public. Proceedings of the First International Conference on Ontologies, Databases, and Applications of Semantics for Large Scale Information Systems. Irvine, California 2002
114
Information Modelling and Knowledge Bases XIX H. Jaakkola et al. (Eds.) IOS Press, 2008 © 2008 The authors and IOS Press. All rights reserved.
CMO – An Ontological Framework for Academic Programs and Examination Regulations Richard HACKELBUSCH Universität Oldenburg, Escherweg 2, D-26121 Oldenburg, Germany Abstract. Academic institutions release teaching and examination regulations in order to form the statutory framework of academic programs. Because of the fact that these regulations are worded using legal terminology and are often very complicated, students often do not know how to satisfy these laid down program requirements. This can lead to needlessly long study times. In addition, academic boards have to supply an amount of courses that fits the students’ actual demand that is a difficult task because there is often only little information available. Frequent changes of those regulations and the existing of parallel valid different regulations of programs leading to the same degrees may aggravate these problems. In order to be able to offer software support to handle these problems, a computer-understandable representation of academic programs and their examination regulations is needed. In this paper, we present and explain our approach, based upon ontologies. It defines a meta-model that allows such semantic representations. Instantiations of the ontology can be used within a framework, e.g., to implement decision support systems that can help students to decide how they can satisfy the corresponding program requirements, or that can help academic boards to forecast the students’ demand on certain courses, or examinations.
Introduction In order to achieve certain degrees, students have to pass educational programs offered by academic institutions. The statutory framework of those academic programs is formed by legally binding teaching and examination regulations. Like law, these examination regulations are worded using legal terminology and are often complicated. That’s one reason why they are often hard to comprehend by students (and often by lecturers, too). A result is that a lot of students even do not try to read them. This can lead to a large demand of course guidance (see [4]). Yet another reason is the prevailing heterogeneity of examination regulations of different programs. Often, examination regulations of single academic institutions already differ a lot. This can aggravate the problem, e.g., if courses of different programs must be integrated in a single curriculum for example in the case of a “minor subject”. This leads to questions that “external” courses of other academic programs can or must be taken in order to satisfy the corresponding program requirements, and if or how grades have to be annualized. In addition, the fact that valid and different examination regulations can parallel exist — forming the statutory framework of different programs leading to the same degrees — is another problem. This can happen, e.g., after an introduction of a new version of examination regulations. That’s why it happens that students of the same academic institution who are aiming at the same degrees, too, have to satisfy different program requirements. To address those problems, often subsidiary documents like study guides are introduced that are intended to describe possible variations of correct curricula. Those documents are intended to be used as a basis for students planning their curricula. But a major disadvantage of those documents is that they often only describe situations in general. Thus, they often cannot comply with the individual situations of all students. In those cases, they do not bring a lot
R. Hackelbusch / CMO – An Ontological Framework
115
of benefits. Another attempt to solve these problems is the offer of individual study guidance (that is mandatory in some countries, e.g., in Germany [13]) but that can be very expensive. Reformations of academic programs that are to be implemented in the course of the so-called Bologna-Process reduce the problems just conditionally. Often, only the required minimum of the Bologna-guidelines (like modularization, see [9]) are implemented. In contrast, a result of the reformations is that these above described problems now become obvious. From the point of academic boards, there are problems concerning examination regulations, too. Information about the current state of the students’ progress is missing. That’s why supply and demand of courses might be balanced adversely. Academic boards normally just reach information about the number of students who have started their studies in a certain term heading certain degrees under certain examination regulations, and — in some cases — how many students already have finished/aborted their studies. Thus, it can only clearly determined how many students are studying within a certain term. But it is not clear, how far those students have reached in order to satisfy their program requirements to get their degrees. From the point of lecturers, it is often not clear how many students who want to take a certain course have to satisfy that program requirements version, too. Mostly, the expected number of students who want to take that course can only be forecasted by stating the number of students studying in certain terms on knowledge of past terms (assuming that certain courses are taken by students studying in certain terms). This can lead to a adverse balance of supply and demand of courses. But these problems do not only concern the task of deciding whether a course should be offered, or not. They also concern the task of deciding how many resources should be provided in conjunction with the offer of certain courses. Until there is no more information available concerning the progress of the students’ studies, the corresponding academic boards are not able to create an adequate supply to fit the students’ demand. In addition, collecting and analyzing of that information have to be compliant with privacy guidelines (see [19]). This paper is the reengineering of our previous published work (see [7]). 1. Approach In order to be able to offer computer-assisted help to solve the above-described problems, one precondition are computer-readable represented examination regulations. For this purpose, the relevant part of their semantics is the description of requirements that have to be fulfilled in order to get a certain degree. These requirements describe processes that represent possible curricula (see [4]). Such processes could be modeled, e.g., using Event Driven Process Chains (EPKs) introduced by ARIS (see [15]), Unified Markup Language (UML, see [12]), or PetriNets/Workflow-Nets (see [17]). Approaches that allow a direct modeling of processes have the advantage that they offer a more human-engineered way to model examination regulations in comparison to approaches that work exclusively on rule basis, like [8]. The way to convert examination regulations into a computer readable model can be done easier and more intuitively. Beyond the semantic representation of processes described by examination regulations, a further semantic representation of them is preferable, too. The definition of concepts like modules, examinations, lessions, etc., with different attributes like workload, fields, etc., that can be very heterogeneous comparing different academic programs, should be modelable. This also would be one precondition to define, e.g., that courses can be used as substitutes for others. Such semantic representations are mostly difficult to model using classic modeling approaches in particular if there are conditions to be modeled that are related to such
116
R. Hackelbusch / CMO – An Ontological Framework
concepts or attributes (like “sum of workload > 8”, or “of special field of technical computing scince”). That’s why our approach uses ontological concepts in order to define concepts for the semantic representation of examination regulations and their possible curricula in a process view. We call this ontology Curricula Mapping Ontology (CMO). G RUBER defines ontology as “an explicit specification of a conceptualization” [3]. A conceptualization is an abstract model in a defined domain including the relevant identifying vocabulary, reaching consent within a certain community. “In such an ontology, definitions associate the names of entities in the universe of discourse (e.g., classes, relations, functions, or other objects) with humanreadable text describing what the names are meant to denote, and formal axioms that constrain the interpretation and well-formed use of these terms”. A framework for the model interpretation is implemented that can be used as a basis to implement, e.g., decision support systems. It is able to interpret models of the CMO allowing nearly freely defined concepts and attributes, without the need of adapting it (see below). 2. The Ontology The main target of our ontological approach is to offer a meta-model in order to be able to represent possible curricula of academic programs that are each originally regulated by examination regulations. The concept is to represent a process view of those academic programs and examination regulations. The meta-model is the conceptual level of the ontology. Examination regulations and academic programs have to be modeled on their instance level. The concrete interpretation of academic programs and examination regulations that are modeled using the ontology’s meta-model can to be done using the framework, and assigning individually the students’ results. 2.1. Main concepts Two main entity types in such processes can be identified: At first there are (process) steps that have to be successfully taken by students in order to get their degrees. A process step can stand for an examination, a course, or a class, or even a thesis. On the other hand, there are conditions that regulate if a certain student is allowed to try to take such a step. has
Process
fulfilled
* 1
Predecessor
*
Process_ Element
0..1
Successor
*
Predecessor
subClassOf subClassOf
Process_Step
0..1
Condition
hasPostcondition
Figure 1. Basic concepts
In order to represent a process that stands for an academic program, the meta-model allows to arrange process steps and conditions using links to their predecessors and successors in the process that each can be process steps or conditions as well. Therefore, the meta-model defines the entity types Process_Step and Condition (abstract) that each are a subclass of an entity representing an abstract general Process_Element (see figure 1 — abstract concepts are intended to be subclassed and not to be instantiated directly; they are visualized using shadows). A Process contains of a set of such specializations of Process_Element. The framework interpreting such a modeled process has to determine
117
R. Hackelbusch / CMO – An Ontological Framework
a boolean value for each element of the process (fulfilled). TRUE represents a successfully taken process step or a fulfilled condition. On the other hand, FALSE stands for a failed or not yet taken process step or a not fulfilled condition. Condition Nand sub ClassOf
subClassOf
sub ClassOf
Xnor
Logical_ Condition
Xor
subClassOf subClassOf
subClass sub Of ClassOf
Nor
And
Or
Figure 2. Logical conditions
Entities of the type Process_Step can have one or no predecessor and an arbitrary number of successors. A Condition can have an arbitrary number of predecessors and successors. The interpretation of processes modeled using this meta-model is that it is allowed to take a process step if its predecessor has a TRUE boolean value or if it has no predecessor. The question if the boolean value of an instance of Process_Step itself has to be interpreted as TRUE of FALSE depends on the result of an individual attempt of the corresponding student in taking the assigned examination, or course, etc., of the Process_Step instance. Of course, the framework interpreting the modeled examination regulations has to examine the boolean value of instances of Condition of the process, too. The value of a condition depends on the boolean values of the predecessors of the condition and the type of the condition itself (see below). Thus, no cycles of conditions are allowed to be modeled. The type of a condition depends on the type of specialization of Condition that is instantiated and used inside of the process. Specializations of Condition have to be interpreted either as simple logical terms (Logical_Condition) or more or less complex conditions based on the comparison of numerical values (Value_Condition). Logical conditions are the specializations And, Or, Xor, Nand, etc. (see figure 2). The interpretation of instances of those conditions in a process is similar logic gates (see [14]): An Andcondition, for example, has to be interpreted as TRUE (and as FALSE elsewhere) if all predecessors (Process_Step-instances as well as Condition-instances) have to be interpreted as TRUE as well. *
*
Value
Successor
Process_ Element
*
Achievement_ Value Predecessor
Value
subClassOf
Equal
subClassOf
Value
1
subClassOf
Value_Condition
Subtraction
Operator subClassOf
Term
Multiplication
Division Value
Greater
1 subClassOf
Smaller
2
subClassOf
subClassOf subClassOf subClassOf
Addition subClassOf
subClassOf
1
2
Condition
1
subClassOf
Greater_Equals
subClassOf
subClassOf
Smaller_Equals
Unequal
Figure 3. Value conditions
Limited to logical conditions, only very simple academic programs and examination regulations could be easily modeled. Therefore, the meta-model defines the abstract concept
118
R. Hackelbusch / CMO – An Ontological Framework
of a Value_Condition. An instance of Value_Condition represents a comparison of values that each can depend on values of process elements of the set of predecessors of the condition itself. The type of comparison (equals, smaller, greater, ...) depends on the type of specialization of Value_Condition that is instantiated (see figure 3). Each instance of a specialization of Value_Condition references two instances of Value. An instance of Value can be interpreted as a numerical value or as a Term of two Value instances. This Term instance combine two Value instances with a mathematical operator like +, -, *, or / representing a calculation of a numerical value by combining the two values with the mathematical operator. A numerical value either can be directly modeled as a constant on instance level of the ontology instantiating Value directly. Using the concept Achievement_Value, it also can be modeled as a value depending on values of the elements of the predecessor set of the condition (like the number of successfully passed process steps) that has to be calculated by the framework interpreting the model (see below). subClassOf
Achievment_ Value
Condition
1
subClassOf
Passed
subClassOf
2 subClassOf
1
Failed
Achievment_ Type
Aggregator Value
subClassOf
Value_Condition
Count
Figure 4. Concepts for aggregated numerical values
The basic idea of the possibility of modeling value conditions is to offer concepts to model conditions concerning values derived from the predecessor elements of the condition. A simple example for that would be a condition that only can be interpreted as TRUE if at least a certain number of its predecessor elements also have to be interpreted as TRUE. Such an examination regulation could be that a student is only allowed to try to take a certain course if he has already successfully passed, e.g., three of four specific courses. Figure 4 shows an extract of concepts to model aggregated values derived from a certain attribute concerning the predecessor elements of a condition. For this purpose, an instance of Achievement_Value has to reference a specialized instance of Achievement_Type and a specialized instance of Aggregator (like Count, Average, Smallest, etc.) that determines the type of aggregation. The type of specialization of the instance of Achievement_Type determines the attribute that will be aggregated. This can be countable attributes like the number of considered Process_Step instances that are to be interpreted as Passed or as Failed or attributes like grade (Grade) that can be aggregated to an Average value. Process_Step:A
Greater_Equals:Min3
Process_Step:E
Process_Step:B Achievement_ Value:PC
Value:3
Process_Step:C Count:C Process_Step:D
Passed:P 3
Figure 5. An example of a Value_Condition instance
119
R. Hackelbusch / CMO – An Ontological Framework
The above mentioned example of three of four courses to pass in order to be able to take other steps is exemplarily modeled in figure 5: The predecessor elements of the condition Min3 are A, B, C, and D. Min3 is an instance of Greater_Equals that is a specialization of Value_Condition (see figure 3). It has to be interpreted as TRUE if the first value PC is greater or equals than the second value 3. The second value (3) is a simple instance of Value that assigns the constant numerical value 3. The instance that represents the first value (PC) is the interesting one: PC is an instance of Achievement_Value referencing P (an instance of Passed, and thus, a specialization of Achievement_Type, see figure 4) and referencing the counting aggregation representation C. It has to be interpreted as the number of elements of A, B, C, and D that have to be interpreted as Passed. That’s why the condition Min3 has to interpreted that way that it is TRUE if the number of elements of A, B, C, and D that have to be interpreted as Passed is greater or equals than three (and else FALSE). Thus, E only can be taken if at least three of A, B, C, and D have successfully been passed. The question if a condition must be interpreted as TRUE or FALSE can easily be answered by interpreting the modeled representation. But to be able to do this, it might be necessary to determine whether certain Process_Step instances must be interpreted as TRUE or FALSE — that in addition is a key question in order to be able to determine the current progress of a student reaching his degree. An instance of Process_Step has to be interpreted as TRUE if the assigned course or examination has been successfully passed. This decision depends on the result of the corresponding course or examination and the assigned grade scale. A result might be ‘A’, ‘B’, ‘C’, ‘D’, or ‘E’ with ‘A’ to ‘D’ to be interpreted as passed and ‘E’ as failed to pass. It is necessary to represent those grade scales because there might be rules that specify how to calculate the final grade of the degree. Or there might be rules that depend on the average grade or allows a certain number of failed examinations if there are a certain number of “good” grades, too. A closer look on grades and results is made in section 2.2. These concepts of Value_Condition in conjunction with Achievement_Value already allow to model very complex conditions. But there are also concepts needed that allow the representation of choosing elements of the set of elements being the predecessors of a Condition instance. An example would be a condition that regulates that the average value of the result of the three best courses of a set of courses must be smaller than 2.5. definesNegativeWildcard
Condition
Type
Boundary_ Type
Achivement_Value
definesWildcard
0..1
0..1
Availabilty
0..1
*
Complement
Intersection
subClassOf
Set_Operator subClassOf
1
subClassOf
Difference
2
subClass Of
B_Unequal subClassOf
subClass Of
1
B_Equal
subClass Of
B_Smaller
Choosing_Term
subClassOf
Boundary
Chooser
0..1 subClassOf
1
1
0..1
B_Greater
0..1
Value limit
0..1
Ordered
Set_Union subClassOf subClassOf
Descending
Ascending
Figure 6. Concepts for dynamically choosing a subset of predecessor elements of a condition
In order to allow the modeling of a dynamic choosing of predecessor elements of a condition, the ontology defines the concept Chooser that is shown in figure 6. An instance of
120
R. Hackelbusch / CMO – An Ontological Framework
Chooser can be referenced by an instance of Condition or Achievement_Value. It stands for a selection of a subset of the set of predecessor elements of the associated instance of Condition (in the case of Achievement_Value), or of the instance of Condition itself. There are a couple of concepts to select the subset: An instance of Chooser can reference a set of instances of Boundary that represent a boundary of a specific (self-defined) dimension (e.g., grade, or workload). By instantiating a specialization of Boundary, the type of selection (equals, smaller, greater, ...) can be set. In addition, a reference to an instance of Value specifies the value of the boundary. The type is specified by referencing an instance of the specified Boundary_Type (see below). The second concept is the possibility to reference up to one instance of Type. Type can — like Process_Step — define a couple of wildcards to delimitate the set of elements to elements that are applicable for that wildcards (for the “wildcard” concept see section 2.2). The third concept allows selecting an ordered extract of the set of predecessor elements of the Condition. This can be modeled by instancing up to one reference to an instance of a specialization of Ordered. This instance (of Descending, or Ascending) references one instance of Boundary_Type to represent the dimension to that the elements should be sorted. And, finally, there can be up to one reference to an instance of Value to delimitate the maximum number of elements of the subset that have to be chosen using the represented sorting. subClassOf
Achievement_ Value
subClassOf
subClassOf
subClassOf
Boundary_ Type
Type
Workload
Passed subClassOf
Rated Dated
sub ClassOf subClassOf
Failed subClassOf
Taken
Figure 7. Specializations of Achievement_Value and Boundary_Type
These three concepts to delimitate the subset of a Condition instance can be combined. The order of those selection operators has to be interpreted that the Boundary concept has the highest priority fallowed by Type and Ordered. In order to change this priority, Chooser instances can be combined with set operators. To do so, instances of Condition, or Achievement_Value can reference up to one instance of Choosing_Term that is a specialization of Chooser, instead of referencing directly an instance of Chooser. An instance of Choosing_Term represents the combination of two subsets that are represented by a Chooser (or even a Choosing_Term) instance using a set operator (union, intersection, complement ...) chosen by instantiating a specialization of Set_Operator that is referenced by an instance of Choosing_Term, too. In general, specializations of Achievement_Type can be distinct into two sets (see figure 7): One set are concepts that represent types that only can be counted (like Type, Taken, Passed, Failed). The other set contains concepts that represent numerical values that can be aggregated (like Workload, Rated, Dated). Elements of this set are specializations of Boundary_Type (see above). Achievement_Type and Boundary_Type are intended to be subclassed in order to define special types that can be used for modelling conditions. These types have to be connected with attribute definitions of, e.g., specializa-
121
R. Hackelbusch / CMO – An Ontological Framework
tions of Availability (see below) in order to define for that type that definition stands for, and to allow the generic model interpreting framework to understand that concept. Condition:Condition01
B_Greater:B01
Value:4
4
Workload:W Chooser:Chooser01 Rated:R
3
Value:3
Ascending:O01
Figure 8. Selection by defining an ordered number of elements and a boundary
Due to a lack of space, only a simple example of a selection of a subset is described in this paper. Figure 8 shows the representation of a selection of all predecessor elements of a condition that have a workload greater than four, and — if there are some — a selection of the three ones that have the smallest grades (see section 2.2 for the representation of grades and results). In order to model that selection, the Condition instance Condition01 references the Chooser instance Chooser01. Chooser01 itself references the Boundary instance B01 that represents a boundary that includes only those predecessors of Condition01 to that a course is assigned that has a workload of more than four credits. Secondly, B01 references the Ordered instance O01 representing the selection of the first three of the predecessors of Condition01 (of the first selection) to that the smallest rated course results are assigned with. Instantiating Condition01 as a specialization of Value_Condition, e.g., a condition that compares the average grade of the selection with a constant value can be represented. 2.2. Results Before the representation of grades and rules (to retry) to pass a course are discussed (see section 2.3), the association of courses or examinations to Process_Step instances is described. Because of the fact that the type and naming of courses or examinations is very heterogeneous at different academic programs they are not intended to be modeled as concrete subclasses of instances of Process_Step. Instead of this, each instance of Process_Step can assign a wildcard that represent possible courses or examinations that can be assigned to the corresponding instance of Process_Step. These wildcards can be descriptions of available courses or examinations including blank values. For example, it is defined that the Process_Step instance CS has to be assigned with a course named “computing science” with a workload of six credits. Then, e.g., the teacher or the term are non relevant values. That’s why those values can be left blank. This representation has to be interpreted that way that all courses named “computing science” with a workload of six credits can be used by a student to “take” the Process_Step instance CS of the process representing the academic program. Figure 9 shows the concepts to model possible assignments of Process_Step instances using a wildcard. Each instance of Process_Step can reference up to one Availability instance. Subclasses of this abstract concept are concepts of real types of courses or examinations of a concrete academic institution or academic program like “module” or “master thesis”. Exemplarily there is a concept for representing modules as a subclass of Availability shown. A Module is a specialization of Availability and has in this example a couple of datatype properties like Field, Name, Workload, and Type.
122
R. Hackelbusch / CMO – An Ontological Framework 0..1 definesNegativeWildcard
Availability
Field
0..1
subClassOf
0..1
Module
0..1 Name
1 1
Process_Step
0..1
Workload
Grade_Scale
0..1 definesWildcard
0..1 Type
uses
Result
0..1 0..1
...
Date
Grade
Figure 9. Wildcard and result concepts
In order to determine the current individual progress of a student running through his academic program by the framework, each course or examination, the student has tried to take/pass, should be associated with a Process_Step instance. Now it can be determined if the condition that represents the requirements to get the degree has to be interpreted as TRUE because each Process_Step instance can be determined as to be interpreted as TRUE or FALSE. If no try to take or pass an examination or course can be associated to a Process_Step instance, this instance must interpreted as FALSE. In order to be able to use these self-defined attributes as, e.g., values of conditions, they have to be connected with specializations of Achievement_Type. By instantiating Availability_Concept_Connector it can be modeled that self-defined concepts can be used referencing that specialization of Achievement_Type or Boundary_Type (see figure 10). The already introduced concept Workload, e.g., is connected by the connector instance C1 with the corresponding attribute of Module (Workload was already used in the example of figure 8). The way to refer these connectors is shown in section 2.5.
Field
Availability_Concecpt_Connector
Availability
0..1
Achievement_Type
nextConcept
instanceOf
subClassOf instanceOf
0..1
subClassOf
0..1
0..1
Availability_Concept_Connector:C1
Name
Module
Workload
Name
Type
subClassOf
Availability_Concept_Connector:C2
0..1
Workload
URI2
...
subClassOf
Boundary_ Type
URI2 URI1
URI1
Figure 10. Concepts for connecting self-defined concepts with Achievement_Type specializations
The concepts to represent grades will be discussed next. Figure 9 shows concepts to represent results of attempts. To represent results, each instance of Process_Step references one Result instance each referencing a Grade_Scale instance (see below). If a student made an attempt to take an examination or a course, the corresponding information can be assigned to a Result instance of an applicable Process_Step instance. An applicable Process_Step instance is an instance to that the corresponding course or examination can be assigned (see above). This information about a result contains the course information that must be applicable for the corresponding desired Process_Step instance. That means that all values that are explicitly defined by a wildcard of that Process_Step instance must be identical. Beside others, there is also information about the time stamp (Date) of the attempt and the result itself (Grade) needed. All of this information has to be acquired by the framework that uses the ontology. Each Process_Step instance can be associated with a different type of grade scale. For example, for some courses, students just get a certification that they have successfully participated in that course. Thus, grade scales for those Process_Step instances just need
R. Hackelbusch / CMO – An Ontological Framework
123
a differentiation between “passed” and “failed”. For other courses, students might get a more differentiated grade. That’s why the ontology offers concepts to represent diverse types of grade scales. fulfilled
inferiorTo
1 subClassOf
Grade_Scale
Ordinal_Scale
*
subClassOf notFulfilled
Nominal_Scale worst subClassOf
Cardinal_Scale
best
1
fulfilled
1
fulfilled
0..1
*
superiorTo
1
allPossible
subClassOf
*
Cardinal_Grade
average_calculable
1
Ordinal_Grade
Nominal_Grade
subClassOf
Cardinal_Scale _Selection
0..1
*
subClassOf
Grade
0..1
1 Value
subClassOf
Labeling label
1
Figure 11. Concepts for representing grade scales
These concepts are shown in figure 11: A Grade_Scale can either be a Nominal_Scale, an Ordinal_Scale, or a Cardinal_Scale. Each of these scales references corresponding specializations of Grade that each can have a Label (like ‘A’). A Nominal_Scale instance references two sets of Nominal_Grade instances. One set represents grades that stands for a successfully pass; the other set stands for a set of grades that represents a failed attempt. An Ordinal_Scale instance references a set of Ordinal_Grade instances each referencing up to one predecessor and one successor. An Ordinal_Scale instance also references one single Ordinal_Grade instance that stands for the worst grade that represents a successfully pass. Finally, there are concepts for two types of cardinal scales: A “normal” Cardinal_Scale instance references three instances of Cardinal_Grade that are mostly different. One stands for the best grade, one for the worst grade, and one for the worst grade that stands for a successfully pass. Each instance references one instance of Value that represents a numerical value (for Value see figure 3). These up to three grades carve out the borders that delimitate possible cardinal grades. The second concept of a cardinal scale is a specialization of Cardinal_Grade named Cardinal_Grade_Selection. An instance of that specialization references in addition to Cardinal_Grade instances a set of Cardinal_Grade instances that stand for all possible grades. Some examination regulations might define that lecturers are only able to assign marks of a set of cardinal grades and that they are not allowed to assign any mark between those grades. An instance of Cardinal_Grade_Selection also references a constant boolean value that represents the possibility of calculating the average grade of a set of cardinal grades. This calculation can lead to a grade that is not part of the set of allowed grades. If the boolean value is set as TRUE calculated average grades do not have to be part of the set of allowed grades. This boolean value has to be modeled on the instance level of the ontology as a constant. A process step also can refer up to one post condition (see figure 1). Then, a result only can be assigned to such a Process_Step instance if this Condtion instance would have to be interpreted as fulfilled after such an assignment. The concepts of grade scales and grades allow the framework that uses the ontology to determine if the boolean value of a Process_Step instance has to be interpreted as TRUE or as FALSE or as Passed or as Failed if a result is assigned to this instance.
124
R. Hackelbusch / CMO – An Ontological Framework
2.3. Internal Processes Examination regulations define different types of rules that regulate the possibilities of a retry of an attempt. There are rules imaginable that define that a retry of an attempt is possible under certain circumstances — independent of the result itself. There are also rules imaginable that define the number of reattempts of failed attempts. Those kinds of rules are mostly applicable for a number of Process_Step instances and not just for one. Thus, it would be very absurd to model those rules inside the main process of the modeled academic program individually for each Process_Step instance. The whole modeled process would needlessly swell and become very inscrutable. In addition, that would be a very redundant model. That’s why the ontology offers a concept to model processes inside a Process_Step. 1
URI
Achievement_ Value
input
URI
1
0..1
URI
input
toType
0..1
Connectable_ Element
*
fromType
*
1
* input
1 isFulfilled
output subClassOf
Connector subClassOf
usabel For uses
subClassOf
toValue
Selector
subClassOf subClassOf
Type_Extractor
Select_Best
Process_ Step
Switcher
*
Extractor Value_Extractor
Process_ Element
inputFalse
1
Internal_Process
ofType
... subClassOf
1
* PatternOf
1
Achievement_ Type
subClassOf
Process
Figure 12. Concepts for representing internal processes
The main concept to model a Process that must be used “instead” of a Process_Step is Internal_Process and shown in figure 12. An Internal_Process instance represents the process that must be used instead of a Process_Step instance. It references one Process_Step instance that represents a pattern, and it references a couple of Connector instances for connecting the Process with the pattern (see below). It has to be interpreted that way that the Process instance can be basically used instead of all Process_Step instances that are equal ore defined more concrete than the pattern itself. Finally, an Internal_Process instance must explicit reference all Process_Step instances that must be replaced by the Internal_Process instance (and that must be equal or defined more concrete). The concreteness of a Process_Step instance is the way its wildcard is represented. For example, if the pattern Process_Step instance references one wildcard that defines only that there must be six credits to achieve, the pattern is basically usable for all Process_Step instances that references wildcards with at least the definition of exactly six credits to achieve. In that case, all other values of the wildcards are not relevant to identify applicable Process_Step instances. The concept Connector is used to represent two aspects: On the one hand, it has to be represented how the results of process steps inside the process that replaces a certain process step have to be mapped to the result of the replaced process step. Concrete: If the Process_Step instance PS1 is the predecessor of the Process_Step instance PS2, and PS1 must be replaced by the Process instance P, it has to be modeled how the result of PS1 would look like if PS1 is replaced by P, in order to determine if an attempt to take PS2 is possible. On the other hand, it might be needed to connect the pattern Process_Step instance with Process_Step instances inside the Process that should replace certain Process_Step instances to be able to define that values of the corresponding replaced Process_Step instance should be used inside the Process in order to be able to model
R. Hackelbusch / CMO – An Ontological Framework
125
generic internal processes that can be used to replace different instances of Process_Step. If, for example, a Process_Step instance M1 that can only be associated with a course named “Mathematics 1” should be replaced by a Process instance P, all Process_Step instances of P should be exclusively associable with courses named “Mathematics 1”, too. The two directions to use connectors are explained next. Process_Step:Attempt01
GREATER:Condition01
Achievement_Value: AV
Failed:F
Value:0
Process_Step:Attempt02
0
Count:C
Figure 13. An (internal) process regulating the possibilities of a retrial of a failed course or examination
A simple rule regulating the possibilities of a retrial of a failed course or examination would be that a failed course or examination could be repeated once. Modeled as a process on instance level, it can be represented as shown in figure 13: Condition01 has to be interpreted that way that Attempt02 only can be taken if the attempt in taking Attempt01 has failed (the number of failed elements must be greater than zero). Next, a pattern has to be defined and connected with elements of the process, if needed. As already mentioned, therefore, the concept Connector is used. Figure 12 shows the Connector concept: A general instance of Connector can connect two entities of Connectable_Element that each references a to be connected instance by its URI. If these two entities are no Connector instances or instances of its specializations, they have to be of the same type (like Result or Process_Step) or compatible type in the case of using an Extractor (see below). It is also possible to concatenate several Connector instances. In this case, all elements that are not a Connector instance or an instance of its specializations, must be of the same type (or compatible type), too. A general Connector simply connects two instances in a directed way. For example, it is possible to make a connection from the pattern Process_Step instance to each Process_Step instance of the replacing Process (the substitute). That means that — besides conditions defined inside the Process — all those instances inside the Process can only be taken if the Process_Step that is replaced by the Process can be taken, too. It also means that for each of those Process_Step instances inside the Process, there are the same wildcards defined as for the pattern (and therefore, the same wildcards as for the replaced Process_Step). If two entities are connected by a Connector instance, all referenced entities of the “source” then have to be treated as (virtually) referenced by the “target”, too, instead of using these references of the “target”. There is one exception: If the attribute definition of connected entities is optional (“0..1”), then the reference of the “target” if it is set has still to be used instead of using the reference of the “source”. Thus, it is possible to model differentiated wildcards within the internal process. For example, all values of the wildcard of the pattern have to be used, and in addition applicable Availability instances have to reference the type “laboratory”. This concept is rather needed for representing substitutions, e.g., for representing minor subjects (see below). To represent the final grade of a process, it is not adequate, to connect just one entity with another. For example, it should be regulated that the best grade or the last grade of certain trials is used as result for the replaced Process_Step instances. A general Connector
126
R. Hackelbusch / CMO – An Ontological Framework
entity only allows an explicit hard coded connection between two entities. To allow the representation of connections that aggregate values or depend on conditions, specializations of the concept Connector are defined. These are the concepts Switcher, Selector, and Extractor (see figure 12). Switcher has — in addition to Connector — a second input (inputFalse) and references an instance of Process_Element. An instance of Switcher has to be interpreted that way that it connects exactly one of the two referenced input entities with the entities of the output set. that one is connected depends on the interpretation of the boolean value of the referenced Process_Element instance. The abstract concept Selector references a set of entities as input and connects a filtered value of that set with the output entities. Specialized instances of Selector can be used to connect the Result instances of the Process_Step instances inside the Process with the Result instance of the pattern Process_Step instance. With that concepts it can be defined how the Result of a replaced Process_Step instance has to be interpreted. Examples for specializations of Selector are concepts that stand for filtering the best (Select_Best), worst (Select_Worst) — each depending on the associated Grade_Scale — or last (Select_Last) Result instance to connect it with the output entities of the connector. Using Value_Extractor in conjunction with Achievement_Value, a representation of a calculation, e.g., the average value of a set of grades can be modeled. Value_Extractor is a specialization of Extractor that references up to one URI of an attribute definition of the source (fromType) and the target (toType) of the connection. That means that instead of the source its attribute should be used as source if fromType is set. The same is defined for the target and toType. If no aggregation or calculation should be used the extraction representation can be used by instantiating the specialization Type_Extractor of Extractor. An example is shown in figure 16 within the next section. Connector:C1 Process_Step:Attempt01
GREATER:Condition01
Achievement_Value: AV
Failed:F
Result:gradeA01
Count:C
Process_Step:Attempt02
Process_Step:Pattern
Result:gradeA02
Result:gradeP
Value:0
0
Select_Best:C2 ofType
Rated:R
Figure 14. Connecting the internal process regulating the possibilities of a retrial of a failed course or examination with the pattern
Figure 14 shows an exemplary connection of the internal process (that has already been introduced in figure 13) with a pattern. In this simple example, the Process_Step instances Attempt01 and Attempt02 have the same structure as the pattern Process_Step instance Pattern. Furthermore, an attempt to take a course or an examination that is associated with Attempt01 or Attempt02 is only possible if the attempt to take Pattern itself is possible. In addition, the attempt to take Attempt02 is only possible if the attempt of taking Attempt01 has failed (Condition01). The result of Pattern is that Result of Attempt01 and Attempt02 that has the best grade. If — for example —
R. Hackelbusch / CMO – An Ontological Framework
127
Pattern is usable for a Process_Step instance Mathematics and all courses named “Mathematics I” that also have six credits can be associated with Mathematics, the internal process with Attempt01 and Attempt02 can be used instead of Mathematics, and Attribute01 and Attribute02 can only be taken with courses named “Mathematics I” that have six credits, too. The result of Mathematics, then, would have to be interpreted that way that it is the result of Attempt01, unless an attempt to take Attempt02 with a better result has happened (in that second case, it would be interpreted as the result of Attempt02). An assignment of and internal process to a process step means that there are connectors between the step and the pattern, and in the other direction between their results implicitly set. As already mentioned, academic programs regulated by examination regulations are represented on the instance level of the ontology. For each academic program/set of examination regulations a different set of instances of the ontology have to be modeled that each represent the main process of an academic program. In addition to the main process, rules like those shown in figure 14 have to be modeled, too. Each of those rules can be associated to a couple of Process_Step instances of the main process or even used recursively. The requirement is that the pattern Process_Step instance of the rule can be used instead of the Process_Step that should be replaced and the possibility of this substitution is explicitly modeled (usableFor). For each student, the framework that uses a model of an academic program has to assign his achievements to the Process_Step instance to calculate his progress. The fact that an internal process can be used for a couple of Process_Step instances implies that the instances of the Process_Step instances inside those processes each might have to be independently assigned to multiple achievements. 2.4. Substitutions For the same reason that the rules for retrials should be modeled outside the main process of the academic program, different courses through the program like different minor subjects should be modeled outside the main process of the academic program, too. Otherwise, the main process of the academic program would become very intransparent. There are two types of substitutions to differ between: One type is the definition of substitutions for single Process_Step instances. These substitutions can be used, e.g., for modeling the possibility to replace a Process_Step instance applicable for courses with six credits by a Process with two Process_Step instances each applicable for courses with three credits (like seminars) or the definition of multiple possibilities in assigning courses with a step (like one choice of three possibilities). The second type is the definition of substitutions for processes having more than one Process_Step instance. This type of substitution can be used, e.g., for modeling processes for minor subjects. The concepts for modeling substitutions are shown in figure 15. These are very similar to the Internal_Process concepts: A Process_Substitution instance references a Process instance that stands for the pattern (substitutes) and a Process instance that stands for the substitution (bySubstitute). The Process instances that stand for possible processes that can be substituted are referenced by usableFor. Finally a set of Connector instances is referenced, too. Element_Substitution is another specialization of Substitution. The difference between these two concepts is that Element_Substitution defines a pattern that is a single Process_Step instance and not a Process instance, and, of course, that it is usable for single Process_Step instances instead of Process instances, too.
128
R. Hackelbusch / CMO – An Ontological Framework
*
Process_ Element
Process
1
*
1
*
1
usableFor
Connector
*
has
bySubstitute substitutes
substitutes
Element_ Substitution
subClassOf
subClassOf uses
usable For
Process_ Substitution
Substitution
Figure 15. Concepts for representing substitutions
The way to model substitutes is the same that is above described for modeling internal processes. But there are two substantial differences between those two concepts Element_Substitution and Internal_Process: The first difference is that an Internal_Process instance can be used for a couple of Process_Step instances, but for each Process_Step instance, there is only up to one Internal_Process instance applicable. An instance of Element_Substitution can be used for a couple of Process_Step instances, too. But — in difference to the concept Internal_Process — for each Process_Step instance, there are more than one Element_Substitution instances applicable. The second difference is the main difference: To use one substitution for a Process_Step instance that is defined by a Element_Substitution instance is only an option. If there is an instance of Internal_Process for a Process_Step instance defined, the internal process must be used. The only exception of this rule is the use of a substitution. Then, for each Process_Step instance of the Process instance that substitutes the original Process_Step instance it has to be checked if there are some Internal_Process instances defined, and so on. Connector:C1 Type_Extractor:C2 fromType toType
6
3
Field
Workload
Workload
Connector:C3
Module:M1
Process_Step:Pattern
Result:RPattern
Module:S
Process_Step:A
Process:Substitute
Process_Step:B
Result:RB
Element_Substitution:S
Figure 16. A simple example of an element substitution definition
An example of a simple element substitution definition is shown in figure 16. The instance of Element_Substitution S references the pattern Process_Step instance Pattern and the Process instance Substitute. The elements of Substitute are the two Process_Step instances A and B. They are not connected with each other. Thus, the only precondition to take one of the two would be the fulfilled precondition to take the Process_Step element that has been substituted by Substitute (Connector C3). Using the usableFor link, for each Process_Step instance with six credits the possibil-
129
R. Hackelbusch / CMO – An Ontological Framework
ity to replace that Process_Step instances with Substitute can be modeled. The pattern process step Pattern can be basically used instead of all Process_Step instances for that is defined a similar or more concrete wildcard. The Process_Step instances A and B each can be associated to courses with three credits. The field is basically irrelevant (C2, an instance of the Connector specialization Type_Extractor connecting the attribute Field of the source M1 with the attribute Field of the target S), unless there is a “Field” defined by a Process_Step instance that is substituted by Substitute (the Process_Step instance is defined “more concrete” than the pattern), then, courses or examinations associated with A and B have to be of the same Field as the wildcard definition of the substituted process. Finally, this Element_Substitution is defined that way that the result RB of B will have to be used as result of the pattern in the opposite direction (Connector C1). Process_Step:A
Condition:C1
Process_Step:B
Process_Step:C
Process_Step:X
Process_Step:D
Process_Step:Y
Process_Step:E
Process_Step:E
Figure 17. Problems while substituting a process
The substitution of two processes is quite more complicated. If a process should be substituted with another, there might be problems because some elements of those that have to be substituted process are predecessors of elements outside this process that must be mapped by the substitution. These problems are exemplarily shown in figure 17: The process on the left side should be replaced by the process on the right side. The problem is the element E outside both processes that originally is a successor of the Process_Step instance B of the left process. To overcome this problem, each element of the pattern Process must be mapped via Connector instances by the substitute Process. The second problem is that all elements of the pattern process must be mapped to elements of the processes that should be replaced. The mapping between the process that should be substituted and the pattern process does not have to be distinct because there can exist more than one Process_Step instance inside the process that is intended to be substituted that is equal or defined more concrete than a specific Process_Step instance of the pattern process. That is why it is defined that each Process_Step instance of the pattern Process instance must reference via a Connector instance one Process_Step instance of each of the Process instances that are intended to be substituted (usableFor). A typical use case of instances of Substitution is the definition of minor subjects. A minor subject is a part of the academic program that is — regarding the content — mostly independent from the rest of the program, and that typically can be chosen from a set of minor subjects. A minor subject of a computing science academic program, e.g., could be economics. Another different use case is the definition of rules that regulate that a number Process_Step instances must be associated with courses or examinations of a set of a greater number of courses or examinations (e.g., two courses of “Mathematics I”, “Physics I”, or “Economics I”). In order to model the possibilities of choosing a minor subject in the course of an academic program, some free choice Process_Step instances can be modeled in the main process. These free choice Process_Step instances each do not reference a wild-
130
R. Hackelbusch / CMO – An Ontological Framework
card Availability instance. A result of this model is that no course or examination is associable to those free choice Process_Step instances. In addition, all free choice Process_Step instances that build a minor subject part of the academic program together are referenced by another Process instance that concerns the process that is intended to be substituted. For each minor subject a Substitution instance has to be defined that references a substitution inside that the concrete Process_Step instances are modeled each referencing a wildcard Availability instance to be associable to real courses or examinations. In addition to the main process of the academic program, the substitution process can define additional conditions that define whether a certain Process_Step instance can be taken. Process_Step:PS1
Module:M1
Module:M2
Process_Step:PS2
And:Condition01 Process_Step: FreeChoice1
Process_Step: FreeChoice2
usableFor
Process: Minor_Subject
Connector:C4
Connector:C3
Process_Step: PFreeChoice1
Process_Step: PFreeChoice2
Result:gradeFC1
Result:gradeFC2
Select_Best:C1
Connector:C2
Result:grade1
Result:grade2
Process_Step: Mathematics1
Process_Step: Mathematics2
Process: Pattern substitutes
Connector:C5 Process_Substitution: SMathematics
ofType
Rated:R Connector:C6 Module:M3 substituteFor
Process: MSMathematics
Module:M4 Process_Step: Mathematics3
Result:grade3
Module:M5
Figure 18. A simple example of a minor subject substitution
A very simple example of a process substitution representing a minor subject is shown in figure 18: An extract of the main process describing the academic program is on top of the figure. It contains the Process_Step instances PS1, PS2, FreeChoice1, and FreeChoice2, and the Condition instance Condition01. Additionally, also the Process instance Minor_Subject contains the Process_Step instances FreeChoice1 and FreeChoice2. Each of these two instances of Minor_Subject have no reference to any wildcard Availability instance (like M1, or M2 in the case of PS1, and PS2). That’s why no course or examination can be associated with these elements. The Process_Substitution instance SMathematics references a pattern Process instance Pattern, and a substitute Process instance MSMathematics that contains the model of the minor subject. In this example, the minor subject process contains more Process_Step instances than the pattern process. Each of those Process_Step instances Mathematics1, Mathematics2, and Mathematics3 of MSMathematics are referencing a wildcard Availability instance defining the associable courses or examinations (M3, M4, and M5). The Process_Step instance Mathematics1 can be taken if PFreeChoice1 can be taken (Connector instance C5). In addition, C5 also would connect the wildcard of PFreeChoice1 with Mathematics1 if there was some. Mathematics2 can be taken after Mathematics1 has been successfully passed. The results of Mathematics1, and Mathematics2 are represented as to use the best value
R. Hackelbusch / CMO – An Ontological Framework
131
and connected to the result PFreeChoice1 by C1. Finally, the Process_Step instance Mathematics3 can be taken if PFreeChoice2 can be taken. In this example, the pattern process elements PFreeChoice1, and PFreeChoice2 are connected with FreeChoice1, and FreeChoice2 by C3, and C4. Thus, if the process substitution SMathematics is used to replace Minor_Subject, Mathematics1 could be taken if FreeChoice1 can be taken, and Mathematics3 could be taken if FreeChoice2 can be taken (that’s the case if FreeChoice1, and PS1 have both successfully been taken, see Condition01). The result of FreeChoice1 would be the average value of the results of Mathematics1 and Mathematics2, the result of FreeChoice2 would be the result of Mathematics3. 2.5. Frame In this final subsection, the way to bring it all together is explained. Unfortunately, there are some additional important concepts and issues that could not be addressed in this paper. These are concepts, e.g., concerning grouping conditions, regulations depending on the time (for example the time span between two attempts, or the standard period of study), or the status of a student (for example “full-time” or “part-time”) by using freely definable annotations (again, without the need of adapting the framework). Other not explained concepts are concepts that allow the representation of rules to allow an unlimited number of attempts, or that represent general rules for processes (like all associated courses of Process_Step instances of a process must differ in/be equal to certain values). Finally, concepts representing learned knowledge (allowed special-rules, exceptions, etc.) could not explained here, too. Academic_ Program
1
Process_Step
1 Rule_Set Availability_Concept_Connector
* *
*
0..1
Internal_Process
Substitution
Figure 19. Concepts for representing the frame of an acedemic program
In order to bring it all together, figure 19 shows the elements on the conceptual level of the ontology to represent an academic program. Therefore, the concept Academic_Program has to be instantiated. An instance of Academic_Program references one instance of Process_Step that represents the step to get the degree. As other instances of Process_Step, too, that instance can have successors. Thus, an academic program can be modeled, e.g., as a two step program like a diploma program with an intermediate examination (“pre-degree”). Each of these directly referenced Process_Step instances should reference one instance of Internal_Process. This Internal_Process instance, then, stands for possible curricula of the academic program. Secondly, an instance of Academic_Program references one instance of Rule_Set. Rule_Set is a concept whose instance represents all examination regulations of an academic program. These regulations again are represented by Process instances that are referenced by Substitution instances, and Internal_Process instances (that each have to be referenced by that Rule_Set instance). Finally, a Rule_Set instance can reference a couple of Availability_Concept_Connector instances. These instances
132
R. Hackelbusch / CMO – An Ontological Framework
allow value conditions to “access” values, e.g., of instances of attributes of specializations of Availability (see section 2.2). 3. Conclusions and future work Our approach is validated using the ontology language OWL-DL1 and representing examination regulations for a couple of academic programs. Using JENA2 , we developed a framework containing a model interpreter that can interpret CMO-models in connection with a set of result-assignments without the need of adapting it after defining new Availability specializations, or attributes. Our approach is applicable but the modeled ontologies themselves representing academic programs and their examination regulations become very large and difficult to handle without explicit software support. Thus, we plan to develop a software tool that allows an abstract modeling of the process view of academic programs without the need of explicitly handling the CMO itself (e.g., via Protégé3 ). Imaginable concepts would be the use of templates and a graphical process editor based upon the ontological concepts. Currently, the decision support system EUSTEL (introduced in [5]) that uses the concepts of the ontology and their instantiated program models is under development. It integrates the individual data of the students and the supply of courses of the corresponding academic institution. The system will be connected to the learning management system Stud.IP4 in order to allow students to plan their curricula at the same place where they already can check their individual results and the university calendar. In particular, using EUSTEL, students will be able to run through different settings of their individual curricula (e.g., choice/changing of certain courses, choice/changing of primary/minor subject). One key element of the support that will be offered by EUSTEL is the possibility of visualizing the individual curricula, and the possibilities in continuing the studies calculating with certain settings of the corresponding curricula. In addition, lecturers will be supported by EUSTEL in retrieving predictions of the demand for their lessons in certain terms — broken down to different examination regulations applied for the corresponding demanding students. EUSTEL itself is intended to be part of the system described in [6]. The aim of the system is to allow a comparison of academic courses and curricula of different academic institutions. 4. Related work Other approaches to offer computer-assisted decision support in questions of examination regulations are for example H ANUS [8] and G UMHOLD /W EBER [4]. H ANUS exclusively uses a rule-based representation of examination regulations and offers no process view on the conceptual level. In contrast, G UMHOLD /W EBER defines a process-based representation. But it has very restricted possibilities to represent special examination regulations. In both approaches, semantic representation of the contents is not provided. A support of academic boards is not supported in both approaches, too. Other formats are, e.g., CDM, XCRI, IMSLD (see [16]), but they are mostly in frame for very specific types of academic programs. Most of the approaches that are intended to support the target group of academic boards aim financial 1
http://www.w3.org/TR/owl-features/ http://jena.sourceforge.net/ 3 http://protege.stanford.edu/ 4 http://www.studip.de/ 2
R. Hackelbusch / CMO – An Ontological Framework
133
aspects. For example, G OEKEN /B URMEISTER [2] provide a business intelligence solution for the controlling of schools. On the other hand, there are different approaches that are intended to represent legal sources — like law — using ontological concepts. These approaches are mostly more generic and detached from a specific legal domain. One of the first ambitious attempts is the Language For Legal Discourse by M C C ARTY [11]. Other related work of that kind is analized in V ISSER /B ENCH -C ARPON [18]. Another ambitious attempt is the attempt of B OER / VAN E NGERS /W INKELS [1] that is intended to offer ontological concepts for comparing and harmonizing legislation. Technically related is also the work of, e.g., M ALONE /C ROWSTON /H ERMAN [10] that contains concepts for an ontological representation of workflows. References [1] A. Boer, T. van Engers, and R. Winkels. Using Ontologies for Comparing and Harmonizing Legislation. Proceedings of the 9th international conference on Artificial intelligence and law, pages 60 – 69, 2003. [2] M. Goeken and L. Burmester. Entwurf und Umsetzung einer Business-Intelligence-Lösung für ein Fakultätscontrolling. Multikonferenz Wirtschaftsinformatik (MKWI), pages 137 – 152, 2004. [3] T. Gruber. A Translation Approach to Portable Ontology Specifications. Knowledge Acquisition, 5(2):199 – 220, 1993. [4] M. Gumhold and M. Weber. Internetbasierte Studienassistenz am Beispiel von SASy. In doIT SoftwareForschungstag, Stuttgart, November 2003. Fraunhofer IRB Verlag. [5] R. Hackelbusch. EUSTEL – Entscheidungsunterstützung im Technology Enhanced Learning. Christian Hochberger, Rüdiger Liskowsky (Hrsg.): INFORMATIK 2006 – Informatik für Menschen - Band 1, Gesellschaft für Informatik, Bonn, pages 65 – 69, 2006. [6] R. Hackelbusch. Handling Heterogeneous Academic Curricula. A Min Tjoa, Roland R. Wagner (Hrsg.): Proceedings of the Seventeenth International Conference on Databases and Expert Systems Applications (DEXA 2006), 4-8 September 2006, Krakow, Poland, IEEE, IEEE Computer Society Press, Los Alamitos, Washington, Tokyo, pages 344 – 348, 2006. [7] R. Hackelbusch. Ontological Representation of Examination Regulations and Academic Programs. In: Hannu Jaakkola, Yasushi Kiyoki, Takahiro Tokuda (Hrsg.): Proceedings of the 17th European-Japanese Conference on Information Modelling and Knowledge Bases EJC 2007, Tampere University of Technology, Pori, Juvenes, Tampere, pages 115 – 134, 2007. [8] M. Hanus. An Open System to Support Web-based Learning. Proceedings of the 12th International Workshop on Functional and (Constraint) Logic Programming (WFLP 2003), 2003. [9] Kultusministerkonferenz. Rahmenvorgaben für die Einführung von Leistungspunktsystemen und die Modularisierung von Studiengängen. 2004. [10] T. W. Malone, K. Crowston, and G. A. Herman. Organizing Business Knowledge: The MIT Process Handbook. MIT Press, Cambridge, MA, 2003. [11] L. T. McCarty. A Language for Legal Discourse – I. Basic Features. Proceedings of the second international conference on Artificial intelligence and law, ACM Press, pages 180 – 189, 1989. [12] Object Management Group. Unified Modeling Language Specification, version 1.5. OMG document formal/03-03-01, 2003. [13] A. Reich. Hochschulrahmengesetz. Bock Verlag, 2005. [14] Z. Salcic and A. Smailagic. Digital Systems Design and Prototyping: Using Field Programmable Logic and Hardware Description Languages. Springer-Verlag, 2000. [15] A.-W. Scheer. ARIS - Modellierungsmethoden, Metamodelle, Anwendungen. Springer-Verlag, 1998. [16] J. Tattersall, J. Janssen, B. van den Berg, and R. Koper. Using IMS Learning Design to Model Curricula. Proceedings of the International Workshop in Learning Networks for Lifelong Competence Development, 2006. [17] W. M. P. van der Aalst and T. Basten. Inheritance of Workflows: An Approach to Taking Problems Related to Change. Theoretical Computer Science, 270:125 – 203, 2002. [18] P. R. Visser and T. J. Bench-Capon. A Comparision of Four Ontologies for the Design of Legal Knowledge Systems. Artificial Intelligence and Law 6, pages 27 – 57, 1998. [19] B. C. Witt. Datenschutz an Hochschulen. LegArtis Verlag, 2004.
134
Information Modelling and Knowledge Bases XIX H. Jaakkola et al. (Eds.) IOS Press, 2008 © 2008 The authors and IOS Press. All rights reserved.
Reusing and Composing Habitual Behavior in Video Browsing Akio TAKASHIMA, Yuzuru TANAKA Meme Media Laboratory, Hokkaido University, N-13, W-8, Sapporo, Hokkaido, 060-8628, Japan Abstract. We have increasingly more opportunities to use video for our knowledge work, such as monitoring events, reflecting on physical performances, learning subject matter, or analyzing scientific experimental phenomena. In such ill-defined situation, users often create their own browsing styles to explore the videos because the domain knowledge of contents is not useful, and then the users interact with videos according to their browsing style. However, such kind of tacit knowledge, which is acquired through user’s experiences, has not been well managed. The goal of our research is to share and reuse tacit knowledge, and then create new knowledge by composing them in video browsing. This paper describes the notion of reusing habitual behavior of video browsing, and presents examples of composing these behaviors to create new video browsing styles.
1.
Introduction
How people interact with text and images in everyday life involves not simple naive information-receiving processes but complex knowledge-construction processes. Videos as knowledge materials are no exception. As technology advances, we have increasingly more opportunities to use video for our knowledge work, such as monitoring events, reflecting on physical performances, learning subject matter, or analyzing scientific experimental phenomena. In such ill-defined situation, users often create their own browsing styles to explore the videos because the domain knowledge of contents is not useful, and then the users interact with videos according to their browsing style [1]. However, such kind of tacit knowledge, which is acquired through user’s experiences [2], has not been well managed. The goal of our research is to share and reuse tacit knowledge, and then create new knowledge by composing them in video browsing. This paper describes the notion of reusing habitual behavior in video browsing, and gives examples of composing these behaviors to create new video browsing styles. In what follows, we first discuss video browsing process for knowledge work. Section 3 describes our approach to associate user’s habitual browsing behavior with video data, and then describes the system to generate the associations to identify video browsing styles. Section 4 presents how to reuse browsing behavior and compose them. 2. 2.1.
Habitual Behavior in Video Browsing Knowledge in Video Browsing Process
We consider that at least two types of knowledge exist in video browsing process; knowledge of content, and knowledge of browsing. Numerous studies which focus on content based analysis for retrieving or summarizing video had been reported [3][4]. These studies are based on knowledge of content which is semantic information of video data, for example, people tend
A. Takashima and Y. Tanaka / Reusing and Composing Habitual Behavior in Video Browsing
135
to pay attention to the goal scene of soccer game, or captions on news include the summary or location of the news topic. Thus, this approaches only work on the specific purposes (e.g. extracting goal scenes of soccer games as important scenes) which are assumed beforehand. In contrast, knowledge of browsing has possibility to be used for identifying various kinds of scenes. In knowledge work, people watch video more actively, then may have his/her browsing styles. 2.2.
Active Watching
By applying Adler’s notion of active reading [5] to the video viewing experience, we have developed the concept of active watching and support tools based on the concept [6]. One of the key points about active watching is that users need to manipulate video to experience video in various ways in knowledge work. Therefore that leads us an assumption which says that people in knowledge work often poses their habitual browsing behavior; in other words, develops their own video browsing styles. 2.3.
Approach
In this research, we assume the situation in which users solve problems thorough active watching process in knowledge work. Our approach to support users is; Identifying associations between user’s habitual browsing behavior and video data Allowing users to manage (reuse and compose) habitual browsing behavior Several studies have been reported that address using users’ behavior to estimate preferences of the users in web browsing process [7]. On the other hand, little is reported in video browsing process. Mahmood et. al. modeled users’ browsing behavior using HMM and developed a system that generate video previews without any knowledge of video content [8], however, the system can not generate new browsing styles. According to our approach, users can browse videos thorough his/her or another users’ browsing styles and create new browsing styles. Details follow next two sections. 3.
Associating Browsing Behavior with Video
3.1.
Association Elements
We assume following characteristics in video browsing for knowledge work: People often browse video in consistent and specific manners User interaction with video can be associated with low-level features of the video To avoid including domain knowledge, we do not deal with semantic video features in identifying browsing styles. While user's manipulation to a video depends on the meanings of the content and on how the user's thought is, it is hard to observe these aspects. In this work, we tried to estimate associations between video features and user manipulations (See [9] for the detail.) We deal with the low-level features (e.g., color distribution, optical flow, and sound level) as what are associated with user manipulation. The user manipulation indicates changing speeds (e.g., Fast-forwarding, Rewinding, and Slow Playing). Identifying associations from these aspects, which can be easily observed, means that the user can grab tacit knowledge without any domain knowledge of the content of the video.
136
A. Takashima and Y. Tanaka / Reusing and Composing Habitual Behavior in Video Browsing
Fig. 1. The User Experience Reproducer and the overview of the process 3.2.
The User Experience Reproducer
To extract associations and reproduce browsing style for other videos, we have developed a system called the User Experience Reproducer. The User Experience Reproducer consists of the Association Extractor and the Behavior Applier (Fig. 1). The Association Extractor for generating a classifier The Association Extractor identifies relationships between low-level features of videos and user manipulation to the videos. The Association Extractor needs several training videos and the browsing logs by a particular user on these videos as input. To record the browsing logs, the user browses training videos using the simple video browser,
Fig. 2. Examples of the low-level video features
A. Takashima and Y. Tanaka / Reusing and Composing Habitual Behavior in Video Browsing
137
which enables user to control playing speed. We categorized the patterns of changing playing speeds into three types based on the patterns frequently used in informal user observation [6]. The three types areskip, re-examine, and others. Skip means video browsing at the speed higher than the normal playing speed (1.0x.) Re-examine indicates the browsing manipulation which is made from forwarding a video at less than normal speed after the rewinding. Any other browsing manipulations are included others. The browsing logs possess the pairs of a video frame number and the three categorized manipulations which the user actually played the frame. As low-level features, the system analyzes more than sixty properties of each frame such as color dispersion, mean of color value, number of moving objects, optical flow, sound frequency, and so on. Fig. 2 shows examples of the low-level features. Fig. 2(a) indicates statistical data of the representative color in one frame. Fig. 2(b) shows optical flow data of moving objects in a video(See [9] for the detail.) The system never recognizes semantics of contents (e.g. out-of-play scenes or shoot scene of a soccer game.) Then using the browsing logs and the low-level features, The Association Extractor generates a classifier that determines the speed at which each frame of the videos should be played. We employ the C4.5 algorithm for generating decision tree to make a classifier [10]and we used WEKA engine that is data mining software [11]. The Behavior Applier for scheduling and playing a target video The Behavior Applier plays the frames of a target video automatically at each speed in accordance with the play schedule. The play schedule is determined by applying the low-level features of a target video to the classifier. The play schedule is represented in the form of a list of associations between each video frame number (i.e. #1, #2 ...) in a target video and behavior (i.e. Skip, Re-examine, and Others.) The Behavior Applier smooth outliers from the sequence of frames, which should be played at a same speed of before/after frames, and also can visualize whole applied behavior to each frame of the video. In the current implementation, the three types of browsing behaviors (skip, re-examine, others) will be played at the pre-defined speed (3.0x, 0.5x, 1.0x) respectively. A preliminary user study for reusing habitual behavior has conducted, and is described in 4.1 4.
Managing Habitual Browsing Behavior
As described before, the managing of users’ browsing styles possesses a lot of possibilities for knowledge work. In this section we describe the evaluation of reusing habitual behavior and the way to compose these behaviors. 4.1.
Browsing Style Reuse
Using the User Experience Reproducer, we conducted a preliminary user study to extract and reuse one’s browsing style. In this study, we used ten 5min. soccer game videos for training a classifier, and two 5min. soccer game videos for applying the browsing style and for playing automatically. We observed two subjects, so that the process described above was conducted twice. In training phase, SubjectA seemed to have been trying to re-examine (rewind then play at less than normal speed) particular scenes, which show players gathering in front of goal post or show a player kicking the ball to the goal. In addition, he skipped in-play scenes which do not display goal post, and out-of-play scenes. SubjectB tended to skip out-of-play scenes of the games.
138
A. Takashima and Y. Tanaka / Reusing and Composing Habitual Behavior in Video Browsing
Qualitative Analysis in Reusing Through the Behavior Applier, each subject saw the two target videos playing automatically in accordance with their own browsing style. The trial of SubjectA played nearly 80% of important scenes (for SubjectA) at a slower speed. The trial of SubjectB skipped nearly 70% of out-of-play scenes. These percentages were calculated by measuring duration of these scenes manually. The results of informal interview tell that both subjects got satisfaction from the target videos, which are automatically played. It is not easy to describe whether the applied browsing behavior by the system constitutes a perfect fit for the user's particular needs. However, through the user study, it seems possible to reuse tacit knowledge in video browsing without any domain knowledge of the contents. 4.2.
Browsing Style Composition
We associated user’s manipulation with video features; in other words, decomposed one’s browsing style into rules. We then tried to compose each rules so that create another browsing styles. The Timing when a composition is executed is after the play schedules were generated (Fig. 3). As described before, a play schedule is a list of associations between each video frame in a video and user’s behavior. One play schedule is generated through one classifier by the Behavior Applier, thus in the case of Fig. 3, two classifiers are made in order to generate two play schedules respectively. To compose browsing styles, we compose these play schedules with several operations. We defined a few simple operations, such as intersection A B : ^x | x A and x B` , complement A \ B : ^x | x A and x B` , and union A B : ^x | x A or x B` where A and B are sets of video frames that associated with
Fig. 3. Composing play schedules
A. Takashima and Y. Tanaka / Reusing and Composing Habitual Behavior in Video Browsing
139
specific behavior such as fast forwarding or re-examining, x is a specific video frame. Some examples which are made by using these operations are as follows: SUSER1 SUSER 2 S YOU
(ex.1)
( S USER1 S USER 2 ) \ S YOU
(ex.2)
( S USER1 S USER 2 ) S YOU
(ex.3)
Fig. 4 shows these examples visually. The upper three belts in Fig. 4 indicate the estimated behavior of a video thorough the User Experience Reproducer based on three persons’ browsing styles. For instance, the first belt shows that User1 may browse the video at a normal speed first, then skip (Fast-Forward) the second part, re-examine the next scenes, skip the fourth part, and then browse the last part normally. The second and third belts are described in the same manner. The lower three belts corresponds to the three examples of composing browsing behavior. The details as follows: ex.1 describes the intersection of the three browsing styles. In this case, the system estimates that these three persons will skip at earlier scenes. This operation detects meaningful manipulations for all users. In other words, the operation works like a social filtering system if the number of users is bigger enough. ex.2 shows the complement of SYOU in the intersection of SUSER1 and SUSER2. This operation finds the habitual behavior of other users which does not tend to be selected by you. This operation can be regarded as an active help system [12]. ex.3 describes the union of SYOU and the intersection of SUSER1 and SUSER2. You can experience your habitual behavior while taking other users' habitual behavior into consideration (Note, this union operation needs to identify a priority to avoid conflicts between behaviors.) These examples shows that simple compositions of associations between a video frame and browsing behavior can create other meaningful browsing style.
Fig. 4. Examples of estimated browsing behaviors and its compositions
140
A. Takashima and Y. Tanaka / Reusing and Composing Habitual Behavior in Video Browsing
Qualitative Analysis in Composing We conducted another user study for composition by using the same classifiers which described in 4.1. SubjectA and B saw two target videos. Each video was automatically played in accordance with the three types of composition described as follows: S SubA S SubB
(cmp.1)
S SubB \ S SubA or S SubA \ S SubB
(cmp.2)
S SubA S SubB
(cmp.3)
As described in 4.1, SubjectA tend to re-examine exciting scenes near goalpostsand skip outof-play scenes. SubjectB tended to skip out-of-play scenes of the games. As a result of applying the cmp.1, the system fast-forwarded the scenes which were expected to be skipped by both subject and played other scenes at the normal speed. This automatic play became almost same as the browsing style of SubjectB. In the interview given after the browsing, SubjectA said that “Although some scenes (in-play scenes which he wanted to skip) are played at the normal speed, the browsing style is acceptable”. SubjectB regarded this composition as almost the same one of his browsing style. In applying the cmp.2, the result of S SubB \ S SubA was shown to SubjectA, and vice versa. As a result, each subject experienced the browsing behaviors which emerged only in the others. Subject B did not like skipping in-play scenes which do not display goalpost. On the other hand, he got interested in the exciting scenes (for SubjectA) which are played at a slower speed. SubjectA did not like skipping the replay scenes that he usually browses at the normal speed. Both subjects were irritated by the out-of-play scenes which had not been skipped by the system. In applying the cmp.3, SubjectA seemed that the browsing style was acceptable because the automatic play by applying the cmp.3 was similar to the browsing style of SubjectA. SubjectB said “It looks like a digest video of a soccer game.” 5.
Discussion / Future Work
This paper describes the notion of reusing habitual behavior of video browsing, and presents examples of composition using these behaviors to create new video browsing styles. Findings from the user studies Although our user studies had only two subjects, it seems possible to reuse tacit knowledge in video browsing without any domain knowledge of the contents. In composition trial, several positive aspects such as accepting the other user’s browsing styles were found. However, there were negative aspects caused by the forced unexpected browsing behaviors. It seems good to reproduce a certain person’s browsing style to the person; on the other hand, reproducing a certain person’s browsing style to others requires some mechanism which reduces cost of applying unknown browsing styles. Mechanisms should allow users to grab the overview of browsing styles when they reuse or compose their styles. In our current implementation, users must determine the browsing style before users encounter a video to play it automatically. This is another reason that the new mechanisms are required. Some specific interaction patterns based on user groups might be found if we conduct user study with more subjects. Conducting more user studies is our future work.
A. Takashima and Y. Tanaka / Reusing and Composing Habitual Behavior in Video Browsing
141
Habitual browsing behavior In contrast with the research works which employ content-based domain knowledge, little has been reported that addresses composing tacit knowledge such as video browsing style in knowledge work. The fact that video data essentially has temporal aspect might make users browse video passively than other media such as text or image. On the other hand, the fact that we have increasingly more opportunities to use video for our knowledge work might make us browse video more actively. We believe that novice will be able to operate video freely and develop their own browsing styles. To support these users, we need to clarify not only semantic understanding of video content, but also habitual behavior of each user. Timing of composing In this paper, although we described to use play schedules for composing, there are other options about when composition will be executed. We plan to compose browsing logs or classifiers of each user as other types of composition. The timing of composition has possibility to give us much better result. Composing tacit knowledge It is said that the social navigation technique for supporting the user's activity by using information on a past are useful [13]. The contributions of our study is not only give the notion of reusing information on a past but also give the example to create new browsing style. We present three types of composing manipulation in this paper, and composing manipulation still has more possibility to generate new and meaningful browsing styles. Refining the composing manipulation is one of feature work. References [1] Y. Yamamoto, K. Nakakoji, A. Takashima, The Landscape of Time-based Visual Presentation Primitives for Richer Video Experience, Human-Computer Interaction: INTERACT 2005, M.F. Costabile, F. Paterno (Eds.), Rome, Italy, Springer, pp.795-808, September, 2005. [2] Michael Polanyi, Tacit Dimension, Peter Smith Pub Inc., 1983. [3] Y.Nakamura, T. Kanade, Semantic analysis for video contents extraction—spotting by association in news video, Proceedings of the fifth ACM international conference on Multimedia, Seattle, pp.393-401, 1997. [4] A. Ekin, A.M. Tekalp, and R. Mehrotra, Automatic soccer video analysis and summarization, IEEE Trans. on Image Processing, vol. 12, no. 7, pp. 796-807, July 2003. [5] Adler, M. J. and Doren, C. V.: How to Read a Book, Simon and Schuster, New York, 1972. [6] A. Takashima, Y. Yamamoto, K. Nakakoji, A Model and a Tool for Active Watching: Knowledge Construction through Interacting with Video, Proceedings of INTERACTION: Systems, Practice and Theory, Sydney, Australia, pp.331-358, 2004. [7] Y. Seo, B. Zhang, Learning user's preferences by analyzing web-browsing behaviors, Proceedings of International Conference on Autonomous Agents, pp.381-387, 2000. [8] Tanveer Syeda-Mahmood, Dulce Ponceleon, Learning video browsing behavior and its application in the generation of video previews, Proceedings of the ninth ACM international conference on Multimedia, pp.119-128, 2001. [9] A. Takashima, Sharing Video Browsing Style by Associating Browsing Behavior with Low-level Features of Videos, Proceedings of the HCI International Conference (HCII), Beijing, July 2007 (in print). [10] Quinlan,J.R. C4.5:Programs for machine learning. Morgan Kaufmann Publishers, CA, 1993. [11] WEKA: http://www.cs.waikato.ac.nz/ml/weka/ [12] Fischer, G., Lemke, A. C., & Schwab, T., Knowledge-Based Help Systems, In L. Borman & B. Curtis (Eds.), Proceedings of CHI'85 Conference on Human Factors in Computing Systems, ACM, New York, pp. 161-167.1985. [13] A. Dieberger, P. Dourish, K. H¨o¨ok, P. Resnick, and A. Wexelblat. Social navigation: Techniques for building more usable systems. interactions, Vol. 7, No. 6, pp. 36–45, 2000.
142
Information Modelling and Knowledge Bases XIX H. Jaakkola et al. (Eds.) IOS Press, 2008 © 2008 The authors and IOS Press. All rights reserved.
" #" $% $# ' < <" ' $# \"^ ^ ^ `" ^| ^"``~| ^"^$#^"" ~ ^ # " ^ ``" \# ^^#^`^^| `^ ~ ``^ ^ `^" ^ ^^ | ^ ^` ` ~" `^" < "" \ # " | ^^' ^ #\\^$`$^"^ \|
\`^""^"~`^ ` ^ ` ``^ ^ ` ^`^"```|`^^^ ^ `^ `# $ ' $^ \^"``^``^\ `^'`\"'`^ `\$`~^$^`^ | ` \$ ^$ `#\`~"^ " ``^ ^ ^ `^" `^" ^ ^````"""\# "|
""`^'" " ` #"$% $#' <<"'$#\"^^^ `"^| ^"`` \^^\'`^$##^ ##`| $#^"\'#^"\^`' #^"$"``^`~'<` | ^ ^ \" ^ " ``\#`#^"$"^^
J. Aaltonen et al. / Concept Modeling in Multidisciplinary Research Environment
143
#^ `| ` <" ^ ~^"`$# \\`""#^"% \^ ` ^ #^" \^ % \' '^`^<`^ ^`|| `^\<"\^"~'`~^ `"^`<`'"`^`# `^"`^"\^'^^" \``<^`^`^"` '^^\^`^$|` $ " \ ' ^ $\' <` ^ ' ^ ` ` ` ^` ` " ^#""^"'`^` ^ ^`$ ` \ $' " "" ^ #^"`"`"| \"<`^`^"' #\"`^` ~\#^^`^^|
`' `^" `^" ^ `^ `"#"^$^` `^`^"% ^`^$^\\^ $^" `"` ` $ | ~ ~$"`^"`^" ^`#^`^"" "" <\\`^^" ^$" #\\^ ` ~" ^ `^ `^" `^" $ \$ $` ^ " ^^ ^ \ ^' ^ `"` ^ ``^\\| ' $ `^ `# ^ $ $ \ <`^ ^ <$\^`"<"'``"^`"" ^ ^ ` "^ \ ^|
#^^`^`^\\$ '#^$\^``^| \^\ "~^^~
%^^^`^\\^ $ | '`^`"~^#" `"`^`^"\' $"^^^`^$|#$'\^ ^^^\^^'`
144
J. Aaltonen et al. / Concept Modeling in Multidisciplinary Research Environment
^^`% ^ `^`""^^`"|` \' <`' # | ``" `" ^ $"`\#$"^'`% ^ ` \| `' \ ^^ ^ ^^\#`#\^^' ^'`\^"``< ^|`#^\<^^^^ <`^"| ^$'#``"` ~'`#^" "~` $^ `^<"\^ ` ^^\#| ^`$\^^ #\$^~'"^ `\#`| ^' `' \ ^" ' <`' ^ ^ $\<``|
! ^#<"^^$^` """$^| <#^"$ ^ ' ` $ \ ^^% ^ ^\^^`^"##^\` |`^""#^"` ^ $ ` ~' ^ ` `^" ^ \ " ~ `| ~"""<#^ \\^'#'\`$" ""$|^^$$$#$" ^`"| ^^<~| `\#^`\^"'<^ ^``$^`|'" ^#<`'\"^ \`^`|<`'`"||<`^\^' # \ \ " | < ""\^\`" '`\\ $ \ " ` ^ \ " '#^^~$ `^\$"| ~\#^"$ ^`^^"<`''` ""^$\^`| ' ^^ "`\\^ \" ^ \ `" #^"$ " | \ \#```|
J. Aaltonen et al. / Concept Modeling in Multidisciplinary Research Environment
145
"`$~"``^` '^<`^"<\# ^^$^^``^| ^`" ""' '^^``^ ^ ^\ `" \# ^$^ | "^`"\\# \"| <`' #" " \ ^% $' `' ^'^`$|
^^^`^"`^"\ #^"$"^"^"^~' ^`^"$\^^| "^ ""% ¡'`^"'" "" ^"¡$| ^^^^ < $ " ` ~ `^"|\"^#$ ¡^ ^^<~^"$#"| ^""¡"'`\" \ ``~^| `' ^^ " "" ^$\\^| $` ^^$"#$'|
"# ^\$'^$` `##`^\ "^``"~^""" \^ \#"| ¡$^\^ #^^`~ ``^^^`^ ~ `^"| ^^ ^ $ <^# ¡ \"$% ¡||$` `#^~| ¡||¡" `^ `^" ^ ``^' "<"`#^""` $`^"| ¡|| <"<"$""~"~ ^ ^ " \^ ' ^ ^| ^'`""' $^ ^`$ `^" ' ^ ``^ ^" `" $'`^ ¡||$$^`#^`" ""| $^``| ¡|| ^ ^ ^ ~ ` \ `~"`<"<#" `^"^"""$| ^^ `'^`|
146
J. Aaltonen et al. / Concept Modeling in Multidisciplinary Research Environment \| ¡$
$
% $``# ¡| `^ $ ¡"^`~" ¡| `#
& $" `^##` ^$# #^ `` " ^ ~" <"^<" ``^ ¡| ~$ `^^^ $ ""^^ ¡| """ ~`^ <"<~ $ ¡| | `^
`# !!"#
\``^^ ~`^" # $%$ # $% #
' ^`| &% #
" #
`^"$`\^`^ ~ `^" ^ | ^^ ^ <^# \"$% || ^``<^` \#"| ^\^^' `\^^`~\$^\`%' ( '#^`^`$^` $$$"~|"'' ) #^^\<^`\$#$ `$""^``^' '^$\`^` ||¡`^"#"\% '^^ ^"\<`^^< "`^"`^"" "<~^^|\" ^\^```^`^ ||\$ " `\^ ^ ^ ^` ^\ `' # ``^^ \ |^$`^| ||~" ^\^ \' "~ ^ """^$"^`^ `` `"| ^^' `" ^$^ ~^ ~ \#<" `^`"`"| || " ^ " ^¢ $ ^$'%$""`^
J. Aaltonen et al. / Concept Modeling in Multidisciplinary Research Environment
147
\%`^"$ $
% "^`~" | ~
"^ | ~|
& ^ "
$ ^` ~^ ^\< `" #^` ^^| &%$# ^` \$ $`| &%# "`\^ `""^ <"`" | ~ $^"^ \#^` ~| ¡"^ "`^ | \#" # \^| "^" `"\# £^$ | ^$##^ `\ `| ~^ `\^^^<`' ^^ #^ # "`^"| #
`'^^"`"| <`'`"!^\\ "^``\| * ! "!# ^\$'^^`^"$ `^ ~ `^"| ' ^^ " "" $ \ \^|#$'^` \$^^"\^| "$$^$^~`^ "| `'^$^`^' ` ^ `# `^" $ ^ ``^ ^ \ ~ ``^^| $^`^# \$`^"| ``$ ¡'^ $^\`^ '^"""$\^^" ``"`^"||'^"^ '$^^^ `^^"`| `\^ $"""' ^$` " ^"#"#^"$"^ ^`^"``|^"^$' $|'#`^^"
148
J. Aaltonen et al. / Concept Modeling in Multidisciplinary Research Environment
\`^`"^""' \#"\^"'<`'$##| ' \ #<' \ <^ \ `^" `^" " ^§ ` ^ ~| $\^<$ ^ ` ` $ ^` #^"$ "| ""^``$#$^$\\ \^#<"~| " \$^'`^^`'\\`"` ^`$"|^^'^" $ ~#\^^<#^"| + $ "$#
^^\$'""¡$ \^^|^"`^"^'` "$'^$'^`#\\^$` \~^``" ~`^"$ \$ $`| ' $\ $ \$` ^#\"'$\^^` `"`^``^\\\"|^^ \`` \^ `"$'# `$\~"^""`^\^| ^"#^"^ ~'"^^ """^`"`$¤¥"'# ¦\ """ ¦\^""^"$`|
* , ^ ~``^^| ^^"` $`^^^^^ $`^`^| ^^'` $\^ \^`^""$| * `\ ~``^" #" `'^' ' ^^'\$ % " # ^$` " ' " ' " '#^`<` " # ^ # ^` \ " ^ \ " `^#| ^^ `^$`^`$\<^'<`' ` \ ^ $ `^ $| ^^' ^\^"^^^#' "``^^\`^`^" |
J. Aaltonen et al. / Concept Modeling in Multidisciplinary Research Environment
149
"% ~``^
" # / ^^ \ "$ ^ ~ " # ^$^\" `# "% | ^^' $ ^ `"^$#~ " ^ ``^| ' ^ ^ $# ~ ` ^$^$ " & "' "$'^` ^ "( ^ ` `^ " <"#^"^| * ^ ^' `^ ` $ \ ~^ ` " `^ `#| ^^ \$ ` ^^ ^$\`"$\#| \^ \"'^^\"$"``$<` \| `^^`$`^^^ ^^\"^``^`\$| <`'` " `$^<^` ^"<`$#^ ""^^^"| ** $ ~`^`^'`` ^^\~^#`^"$'$^"`
150
J. Aaltonen et al. / Concept Modeling in Multidisciplinary Research Environment \| ^` `^`^"^`^
" #
%
0
78 "#
^
¨
7%/ "#
<
` $ \ ``^ <`^ ^^ |
7% "#
<
$ ¡` """` #|
<^ | `` \ ^
'' `'" 7; "9# < ¨ ^# " ^| \"| ¤##^\ 7< "9# ^ ""^<`' ' "" ^"¡ 7% "=9# ¡ < \`\ `^ ^` #< ^\| %©©###|"|^` < 7 "=9# `` "'\ '' ^ ^ #^| ~ 70 "9#
^^^`$ $|'<#\ $' ") `^"\`^ " ) $ # ") ) '"`"""`#| ` | ' " #' ^$ ^``^^``^` "$\ ^" ^ $ \' "))
"^ " ) | *+ 8 " ``^"^`'^ ~ \ ^^ ^" #" ` ^ $^``^% | ^^^^` ^# | | ` " $ ` ^ \ ^""`#^\ ` <` " |
J. Aaltonen et al. / Concept Modeling in Multidisciplinary Research Environment
151
| `^`'#' ^\^"^``^#^\ `<`'"*' & | \^~`^ $ "$$ ' ^ \ # " " ^ " +
| ^^^` \ " + | ^^^` \ " + + % , ^` ~ # `^ < "" \#"ª¨ | `^" ^ " # ^ $ ^` ~ ^ "" ^$^ ~% \ `^ ! ' ` ! ^ ` " !| ` ^ ' `^ # <<``^| ^`$^`^"' ^^^\^^^`~ <~$' ¡|^#`^"$|^|| + % ``^$~ |\# "^`#% '^"#" "`'^^^\^ ^"^^^\'#' "''^`^^| ^' ``^^^"$#"% ` ^ ^ ^^ ^" # \ ^ `# ` "' | `^ ^`^`"^ # ^' `" "' ^\ " ^ " ' < ^ `^"| ^#^` '#`^\ ` " , ~ ^ `^ ` ^ ` # #^ \ < ` $" $' || \ `^' ` ^`"'`\# ^`^\#"|`\$ $^ " $" #^ \ $`' $ ¡#^^``^"" ( | ^$^\^`` ^`~" # `# $ ~`^ " # |
152
J. Aaltonen et al. / Concept Modeling in Multidisciplinary Research Environment
"%¡~
^ \ " ^`$ $%| ^`\#"% "- "*' & " " " | #$' ^`^"^` ^`\^^| ^'\`|^ ~^" `\`~^$^~$^` ^ #| ^ ~ ^ `~" `^ \ ^`^ # ^^ ^`$^\'`#$\^< | + / $ / ,
`$```^`^$ $$" ^` ~| ' \^ `^" ""^``" \| \$^"< ^~\^| #` ^`^"$^^`^ #<"#|`" ~`^"'`^"^ " "" ^" <~ ^\^` ' ^ \| `\#" " ~| '\` \<^^``\$'#^$ \^"`^\#<| ` ^ ^ ^
J. Aaltonen et al. / Concept Modeling in Multidisciplinary Research Environment
153
^^$^\^^^^"`^" |
'#^`" ~|¦^^\^^` $`^$\``$^`<$" ^|^^'\^`^" $` ` #' "" ` `^ `©< < ` ¦\
" "" ¦ \^ " | ^ \#`<^^ `^"| ^ ^^^""` "^`"$^^ | \%<~'~^"^"$|
> ? > " # " #
> " #
|¡ % «# " \^" ~ `^\ | | ^`` #<| |" ¡%! \ ¦ «^« `^ \`^ "^ #^\ \^\ $`¬" #| #
¡%! # ¦^^ #
` $# \^ '^# \^\ \"`¬ #¬ "^< ¡%! #^` "`^ ` " ^ \#" ` "`^ $\ \ `^'\^ #<| `^^` \"
#" ~ <``^^< #\^^<| ^ ``^#$^`^| ^"$ '||`\` # | ^^' ` ^^ ^ ` ^` ~^\$\^| ^\^ ` ~ \'
154
J. Aaltonen et al. / Concept Modeling in Multidisciplinary Research Environment
<`'`"`"`^`^ `^| + @ "! # ~\`^^^\#"| #`^^^"`|^ ^ #^^\$#`| $$| \^\^"" $\#^| \\^"$|
"^| $| ^\#^`^\| `^\`| ^|
"%«`^`
^'"^^"`'"$ ^<« \| + $ "!# `"~` "^^| " ^^^\$~`"^`' ` $ `\ % !" ' !" ' !"- ' !"% # ' !" ^ " |¡$'` ) `^ \ ^^ % !" ./0 " ) .10 ^ !" .20+ ¦ "^ #^ ` # # ^^ ~^'"\"^"#" `"%\^`"`'^' #\^"'^`"|
J. Aaltonen et al. / Concept Modeling in Multidisciplinary Research Environment
155
"%`"`
`"`!"( \^"^`""\ "`^`<!" !"
!"3 | `"`¢^`## \`\^^^^| < `~\\#^\ ^^ $`^'$` ` ^ ^" | ^ ^ `^$ #" $ $" !" ' \ ^\^\""¢^"~$ ~"`^^!" `"| ` `"\"\``\^ \"\"^^^~^`#|^^' ` # !" ' ^ " `| "~ # <' `^ " #^"#`^``` `" !"( ^ ` .!" 0| ^""'^$#~^'\ ^ ^ ` !" ) 0\`^\#"!"4%` ^``| `\`^` "^ $ $ \^$ `^"' ' ^ """| +* ( ! ! ,
"" \ # " ' ~ `^ $ `#' || \' "~' ^ """ '"| '`\^ `^^<\#^$^
156
J. Aaltonen et al. / Concept Modeling in Multidisciplinary Research Environment
`^^`$`^"|^ | $ \ ^| \ ^" '"""$\~^ |'^"^$$ ~^<`| G ``^~| $``^"`^"| ``" ^ `^" ^` \ # " # ^ ` $ ` ^ ` ^^ ^\| #`<`"` ^\^`~"`#'^ ^`"""| / '|'¡'| ' ||^`# $% ¡ «^| |'|'|®| `` ^|'*+/08 09;!!"; <0 9 !!"| `$ "^ $ `| \' | ¡| | ¤\ " | = > '@'| ''¡'' | "^ "| '%"`¤\| '|||^¬ ¬ <`¬ ¯"|K * N'Q!'| ©| \`^`|''| ¦ `¦\«¤^`¦" | N=XN+ Y °%©©###|#|"© ¡©©¦#\"^©±^%$`\ ~"'¡| ^'|| #^ " ""| 'Z [K $\\] Z > + /^ N * + * N || \' |¡| | #^¤" "^#^""| * K _<+ N '&Q'| "' | <' | | ` ""| ' Z *;*Z`Y@%]` q 0q<[ Z | ^`' #| ¨' ¤| ¡| |' ' | |' «' | | | $ | | ¡| |
"`%"$ \| '{>>>*|$\\]NN X /|^$''| ¨'¤|¡||''||'«' |||$||¡||"" \" "`| 'Z * + ; X * N <;X*N}\~| ¤| \' $ ^' ^ | N 0 q* < °%©©###|\|±^%|| ²''''|''¡|''|''|' ^"' |'³`~'|''|$'¡| |` %#^"^"^" $`'| ||²'||$'^ ||~´~' ^|'Z ; * + + | ¤`'|$^^¤µ\^$| \'|'"'¡'¦''||''| ¤" ¦¤"%
$` $``¦\| || '|¤<| ¨|` ^|'[N `<*N+`!!& [* N `+ |`'"| `' |'`' |'^'|'$®®$'|$'|| ` " "$``^ "| || ^|'Z ` q _+ [ _+[!" | ¤|¤\|
J. Aaltonen et al. / Concept Modeling in Multidisciplinary Research Environment
157
0 / , \\# \\^ ~! '!^!| \| `"^$^`^ %/ % 0 % ; < $% $%`% $%# $%#`% $% " $% "`% $%¤ $%¤`% $%« $%«`% $%¡ $%¡`%
@ , 7 "! # \\# \`~ \`^! | $\^^^^^ \^`\|`^ ^"`| \% ~\`^$¶\`%^^ ^
% $<"| `"'"~^`^"| ^\#`^^©`| ¡ ¡^^^$^"^^ ^|¡"\"\\$ \$^| $`"~`"|^`"^`^ '# #^^#^| ¡ «^`^"\$$ `\##`| ¤ ¤"^"^"©\ `©\|¤$``$^ ^^$| $$"#` \^$^| $<"\#| # #^^"|#^\ ^\'\#$#| « «`"""~^`$ ^$#^`"$^^$"' "$|«`<"\##`| ¨
"'`$`"#`^$# `$"|¡<"$$^^ $|
" "`\" \""""| " `\""'^\<#|
#
#<$| # #| ` ``#`$^$^$^ "~|`©"~©$` | ``^|
158
J. Aaltonen et al. / Concept Modeling in Multidisciplinary Research Environment
, 7 / "!# ` " ^\^`\#| \\^`"|#$'\^ ^"^$"\^`| \^# ``^ \ " " ` ` ^ \ | ^"`\'` % !" ' !" ' !" ' !"$ ' !")5 ' !"6 ' !"%' | !"-' ' !" ) ^ !" " ' \ "\^`^#^`""#| \^#|"`^\ #«'#^!")) | ` \"!" "|^!") '^^`^ !" $' # $ ` \ $| $ ` !" \ " ` `` | ¤ ^ !" !" "| ' \ "^+
"% ``
\^!" ) ¡`¡'# \^\#'\·¤^ ' # ``\ !") ) ) !"*' &' # !"«| "\#¡^ ' ^ \` #' # `^ !" \ ' ^ # # !" ' ¡| `\^\#'# ^\!"))| ¡`¡^^!") ) \ #| ¡` ^ !" ' # ^\
J. Aaltonen et al. / Concept Modeling in Multidisciplinary Research Environment
159
!" ¡` $| ^ !") ( ' # !" ) ) !")| \ $ $' \ ^`^ !" ' # ^ !" $| $ \ $ ` $| ¡` #^ !") ' \ !"( ' # !" ¤' `` ¡`| ¦"#'#`"`\%«^ | ""``"`\##|
" ^ $¶% `¶`| «"` «' ' $' # ^ | "` ' ¤' `' $' ¡`' ¡' #' "' ' $¶%`'^ | \#`^"` \ ^\^ " % || " «'^ \ "^«|
160
Information Modelling and Knowledge Bases XIX H. Jaakkola et al. (Eds.) IOS Press, 2008 © 2008 The authors and IOS Press. All rights reserved.
Extensional and Intensional Aspects of Conceptual Design Elvira LOCURATOLO* & Jari PALOMAKI** *ISTI, Consiglio Nazionale delle Ricerche/Pisa, Italy **Tampere University of Technology/Pori, Finland Abstract: A partition method is designed to achieve the flexibility of semantic models in reflecting changes occurring in real life and the efficiency on object systems. The method of partition has been applied only to the extensional aspects of concepts, i.e. set. In this paper a partition method is applied to the intensional aspects of concepts as well. A particular problem concerning the intensional negation of a concept is solved by defining a restricted intensional negation of a concept, which is important in all practical conceptual design situations all of which are confined in specific restricted part of the universe of discourse. Some further development of partitioning is presented as well. Keywords: Semantic models, partitioning, concept, intensional, extensional.
1. Introduction The Partitioning [5] is a method, which links conceptual and logical descriptions of databases through a formal relation to achieve both the flexibility of semantic data models and the efficiency of object systems. Semantic data models are appropriate models for conceptual design [2]; since they allow the representation of database objects close to the real word objects and the ability to reflect the changes occurring in the real life with flexibility; however, they mostly resulted in database tools. As complete database management systems they have never been implemented efficiently [7]. On the contrary, object models which have abstraction mechanisms similar to those of semantic data models have reached a remarkable level of efficiency [1]. The better result in the engineering of object database systems can be explained at the light of ODMG (Object Data Management Group) observing that in semantic data models each object instance can belong to any class of a specialization hierarchy thus enhancing flexibility, whereas in object models each object instance belongs to one and only one class thus enhancing efficiency while limiting flexibility. The Partitioning has been revised and exploited to define a formal methodology of conceptual database design, [6]. In this paper, it will be applied to the intensional aspects of concepts. The Partitioning is based on recursive decompositions of graphs; however, the graphs that the Partitioning decomposes can also be assimilated to concept networks. Classically, there is the distinction between intension and extension of a concept, where by intension we means the information content inherent to the concept and by extension the set of individuals which fall under the concept. The Partitioning has been applied to the extensional aspects of concepts; in this paper, we will apply this method to the intensional aspects of concepts. Our approach is important from a modelling point of view, since we have the possibility to model the boxes containing the objects both in the intensional as well as in the extensional way. It is also important from a concept theory point of view since the problem of the intensional negation which we present in the section 3 can be solved by using a restricted intensional negation of a concept, which is important in all
E. Locuratolo and J. Palomaki / Extensional and Intensional Aspects of Conceptual Design
161
practical conceptual design situations all of which are confined in specific restricted part of the universe of discourse. The paper is organized as follows: section 2 introduces a problem, section 3 describes the notion of conceptual classes, section 4 considers an intensional containment relation between concepts, section 5 describes a partition method and applies it both to the intensional as well as the extensional side of concepts, section 6 describes a further developments of a partition method, and in the section 7 a conclusion is drawn. 2. A Problem The Partitioning is a method designed to achieve the flexibility of semantic data model in reflecting the changes occurring in real life and the efficiency of object systems. This method performs recursive decompositions of classes supported by semantic data models, called conceptual classes, until all and only disjoint classes, called objects classes, are obtained. Figure 1 illustrates the difference between the conceptual classes and the object classes. A class is a mechanism which models both a set of objects in the databases and a set of attributes associated with those objects. Operations on the class objects are allowed on the Structured Database Schema, model which extends the conceptual classes.
The specialization hierarchy of Figure 1.a defines conceptual classes if the following properties hold: 1. classification: each node of the specialization hierarchy is a class (in our example we have the class person, the class employee and the class student). A node linked with a super-class is a specialized class (in our example we have the specialized class student and the specialized class employee); 2. attribute inheritance: each specialized class (in our example the specialized class student, and the specialized class employee) inherits all the attributes from the super-class (in our example the person class) and may have additional attributes; and 3. object inclusion: the objects of a specialized class (the student objects resp. the employee objects) are a subset of the superclass objects (person objects).
162
E. Locuratolo and J. Palomaki / Extensional and Intensional Aspects of Conceptual Design
Figure 1.b provides a representation of the conceptual classes. It shows that the object intersection of any couple of classes may be not empty. In particular, the intersection between the student objects and the employee objects may be not empty. On the contrary, for object classes represented in Figure 1.c and Figure 1.d, there is the restriction that each object instance must belong to one and only one class, that is to say the most specific class of the specialization hierarchy, and it belongs to all the super-classes only by inheritance. In our example, a student employee object must belong to a provided student-employee class. This constraint allows for more efficient system implementations but limits flexibility. In Figure 1.c and in Figure 1.d the object inclusion property is only intended in terms of attribute inheritance, that is to say, the objects of the student class are intended to be enclosed in the objects of the person class since the student class inherits all the attributes from the person class, but the objects of the two classes are really two disjoint sets. In order to show how the flexibility in modelling the changes occurring in the real life is limited in object classes, let us suppose that the student John becomes an employee. In this case, the corresponding object instance must be removed from the student class and must be inserted into a student-employee class and this is not possible if the student-employee class has not been created beforehand. If later on John completes his studies, the corresponding object must be removed from the student-employee class and must be inserted into the employee class. On the contrary, in conceptual classes the object corresponding to John is inserted into the employee class when the student John becomes an employee and it is removed from the student class when John completes his studies. The Partitioning generates automatically all the classes implicitly specified in the starting conceptual classes (in our example it generates the student-employee class). It is an approach applied to the extensional side of concepts. In this paper, the Partitioning will be applied also to the intensional side of concepts. The next section introduces the notion of Conceptual Classes. 3. The Notion of Conceptual Classes The notion of class-hierarchy is based on the notion of class. A class is a pair
, where v is a term denoting a (sub)set of a given set, and V is a finite set of terms denoting functions from the set denoted by v to a given set of values. That is, if v is a term, d is a denotation function, then d(v) = set(v), i.e. a set denoted by a term v, and if V = {ui ¨ i I } is a set of terms, then d(ui) = fi, where fi is a function fi: d(v) o x, where x is a set of values, i.e. V = { ui ¨ d(ui) = fi }. Intuitively, v is a term for a set, V is a set of terms for functions, which are attributes. On the other hand, the term v connotes a concept of which the extension is the set denoted by v. That is, if v is a term, and c is a connotation function, then c(v) = concept(v). Moreover, the extension of the concept(v) is the set(v). (See figure below). Similarly, the set V of terms connote the concepts of which extensions are the functions denoted by those terms of in the set V. That is, if u V, and c is a connotation function, then c(u) = concept(u) the extension of which is an attribute f denoted by u. Thus, we have both an intensional aspect (i.e. concepts) and an extensional aspect (i.e. sets and functions) of terms. Terms belong to the linguistic level.
E. Locuratolo and J. Palomaki / Extensional and Intensional Aspects of Conceptual Design
163
Functions are understood as attributes, and they are inherited from the set to the subset of that set, i.e. if f: A o B and C A, then f~C: C o B, where f~C is a function f, whose domain is restricted to the set C. Thus, function is restricted to the subset of A. From that it follows that if f: A o B, C A, and f~C: C o B, then f~C f. In the rest of the paper a class is presented as in Figure 2,
where i) class is a pair: , where P is a term for a set of persons. ii) the given sets: d(P), the set of ALPHABETS, and the set of natural numbers N. iii) the attributes: d(name): d(P) o ALPHABETS, d(income): d(P) o N. We can denote a generic attribute of P as {f}P, meaning that persons belonging to d(P) have an attribute f. A graph of Conceptual Classes with two direct descendant of the root has been generalized and represented as in Figure 3,
164
E. Locuratolo and J. Palomaki / Extensional and Intensional Aspects of Conceptual Design
where the directed arrow is a “is-a” relation between classes. Now, the attribute inheritances are described as follows: {n,i}P, {n~s,i~s,m}S, {n~u,i~u,m~u,a}U and E P, and either S E = or S E z . In the case S {n~e,i~e,s}E. Also, U S P, E z , we get {n~s,i~s,s~es,m}S and {n~e,i~e,m~se,s}E. And, if U E z , we get {n~u,i~u,m~u,a,s~u}U. The next section describes an intensional containment relation between concepts. 4. An Intensional Containment Relation In conceptual level we have a relation between concepts, and as such we shall take as a primitive the intensional containment relation between concepts, [3,8]. We can say that a concept a contains intensionally a concept b, and denote it as a b. If the concept a contains intensionally the concept b, then the extension of the concept a is a subset of the concept b. For example, the concept of ‘dog’ contains intensionally the concept of ‘quadruped’, and the set of dogs is a subset of the set of quadrupeds. There are several nonidentical concepts, which are co-extensional, and so we can infer from concepts to its extension, but not vice versa. This is an example of the law of reciprocity, i.e. the more intension, the less extension, and vice versa. By means of the intensional containment relation it is possible to define some operations on concepts, for a more formal presentation see [3,8]: Two concepts are compatible, a A b, if there is a concept x, to which them both are intensionally contained. Def. 2. Two concepts are incompatible, a T b, if they are not compatible. Def. 3. Two concepts are comparable, a H b, if there is a concept x, which is intensionally contained to them both. Def. 4. Two concepts are incomparable, a I b, if they are not comparable.
Def.1.
An intensional negation is defined by means of incompatibility as follows: Def. 5.
An intensional negation of a concept a is a concept b, which is intensionally contained to the every concept x, which is incompatible with it.
That is, the intensional negation of a concept is the greatest lower bound of all those concepts which are incompatible with it. An intensional negation of a concept a is denoted below by a. When two concepts a and b are compatible, the least upper bound exists, and it is denoted by a b. On the other hand, when two concepts a and b are comparable, their greatest lower bound exists, and it is denoted by a
b. Without confusion, if we denote the extension of a concept a by set(a) as well, we can see from the law of reciprocity between intension and extension of a concept that the extension of a concept a b is the intersection of extensions of the concepts a and b, i.e. set(a) set(b), and the extension of concept a
b is the union of extensions of the concepts a and b, i.e. set(a) set(b).
E. Locuratolo and J. Palomaki / Extensional and Intensional Aspects of Conceptual Design
165
In general, intensional negation is very problematic, but for ASSO where the most general super-class determines the universe of discourse, we may modify the Def. 5. so, that the quantifier is restricted to that specific universe of discourse. That is, for example, restricting the universe of discourse by the extension of a concept p, i.e. the set(p) is the most general super-class in question. So, the modified definition of a restricted intensional negation is the following, Def. 5*. A restricted intensional negation of a concept a is a concept b, which is intensionally contained to the every concept x, which is incompatible with it, and moreover, there is such a concept p, which is intensionally contained to the concepts a and b. A restricted intensional negation of a concept a is denoted below by ra.
5. A Partition Method The “is-a” hierarchy defines conceptual classes, if the following properties hold: 1. Classification: each node of the hierarchy is a class. 2. Attribute inheritance: the sub-class inherits all the attributes from the superclass, and may have the same additional attributes. 3. Object inclusion: the sub-class object is a subset of the super-class objects. Now a partition method is the step sequence of partitioning decompositions applied to the conceptual classes represented in Figure 3. The aim of the decomposition is to get all possible conceptual classes implicitly included to the “is-a” hierarchy. Thus, e.g. the subclass <E,{s}> of the super-class
is <E,{n~e,i~e,s}>. A partition of a non-empty set A is a collection of non-empty subsets of A such that: i) For all S and T , either S = T or S T = , and ii) A = S S. Accordingly, a partition of a set A is a collection of non-empty and pairwise disjoint subsets of A, which exhaust the set A. An element of a partition is called a block. Because we are working with concepts, which determine the sets, we are not interested on actual members of those sets. For example, we may understand a partition of a set A as a collection of boxes, where boxes are the subsets of set A. Now even the empty boxes can have labels. Moreover, every element can be only in the one box. So, the requirement of non-emptyness in the definition of in this context is possible to drop out. Drawing the three intersecting sets inside the set P as shown in Figure 4,
166
E. Locuratolo and J. Palomaki / Extensional and Intensional Aspects of Conceptual Design
we get the following six different blocks, (note, by “set(A) \ set(B)” we mean a set theoretical difference, i.e. the intersection of the set(A) with the complement of the set(B)): 1. 2. 3. 4. 5. 6.
set(P) \ (set(S) set(E)), (set(P) set(S)) \ (set(E) set(U)), (set(P) set(S) set(U)) \ set(E), (set(P) set(E)) \ set(S), (set(P) set(E) set(S)) \ set(U), set(P) set(E) set(S) set(U).
These six blocks 1.–6. are in an extensional level. In an intensional level we get the following correspondent formulas for concepts: 1’. 2’. 3’. 4’. 5’. 6’.
concept(P) r(concept(S)
concept(E)), (concept(P) concept(S)) r(concept(E)
concept(U)), (concept(P) concept(S) concept(U)) rconcept(E) , (concept(P) concept(E)) rconcept(S), (concept(P) concept(E) concept(S)) rconcept(U), concept(P) concept(E) concept(S) concept(U).
In this example the restricted intensional negation, “ r”, is restricted to a concept(P). Accordingly, from a modelling point of view we have the possibility to model the boxes containing the objects both in the intensional as well as in the extensional way. Intensionally the modelling is done by concepts, whereas extensionally the modelling is done by Partitioning. However, since the relation between a concept and its extension is many-one, a given box can be an extension for many non-identical concepts, but not vice versa. 6. Some Further Developments of Partitioning The Partitioning maps the conceptual classes into the object classes. In [5], the graph nodes representing the conceptual classes have been labelled by numbers, whereas the class attributes have been labelled by double indexed functions and the inherited attributes by double indexed primed functions. As the general transformation from conceptual classes to object classes is a complex task, the solution has first been determined for the elementary
E. Locuratolo and J. Palomaki / Extensional and Intensional Aspects of Conceptual Design
167
single-path tree, that is to say a tree formed by nodes with only one direct descendent, and then it has been determined for more general basic case. For each of them, the relationships between the determined object classes and those of the previous cases have been established. This addressed the process generalization first to a generic tree then and to a generic graph. Each graph node is characterized by both its set of objects and its set of attributes. The set of objects is represented by a subset of a given set, whereas each attribute is represented as a function defined in the subset of the specified class and assuming values in a given set. Operations on graphs permit to perform recursive decompositions of conceptual classes until object classes are obtained. The method is correct, that is the objects of the obtained classes are all and only the objects of the conceptual classes and these have all and only the attributes defined in the conceptual classes. The Partitioning is a difficult method resulting into disjoint classes which are not recomposed into a class hierarchy satisfying to the property that each object instance belongs to one and only one class. The Revised Partitioning is composed by two phases, called representation and decomposition, respectively. The former permits describing the conceptual classes, whereas the latter permits decomposing them. As to the representation phase, a label connoting the class name and denoting the class objects has been associated with each graph node, whereas a list of attribute names has been associated with each label (See Figure 1.a)). This model implicitly specifies that the objects of each class are represented by a subset of a given set, whereas each attribute has been formalized as a function defined in the specified set of objects and assuming values in an implicitly specified set. As to the decomposition phase, this is a stepwise approach satisfying to the following properties: y Root partitioning: the root objects of the conceptual classes are partitioned into the root objects of the conceptual classes resulting from each decomposition step. The root labels represent the partitioning. y Root labeling. The root labels of the conceptual classes resulting from each decomposition are defined by combining the root label before the decomposition with the labels of the root direct descendants. y Root structuring: the root labels can be decomposed into two parts separated by the “-“sign: the former on the left of this sign consists of label intersections, whereas the latter on the right consists of label unions. One of the two parts can be empty. y Consistency: the following implicit information is specified through the root labels: only the attributes of class X are associated with a node labeled by X Y, whereas the attributes of all the classes X…Y are associated with a node labeled X…Y. From root partitioning, it follows that each object of the original conceptual classes belongs step by step to one and only one of the obtained conceptual classes. From root labeling, it follows that all the information required for schema decomposition is enclosed in the root labels and its direct descendants. From root structuring and consistency, it follows that the attributes can be associated with the classes exploiting information implicitly specified in the root labels. Furthermore, each object has all and only the original attributes. The root structuring is also exploited in order to construct the class hierarchies satisfying to the property that each object instance belongs to one and only one class.
168
E. Locuratolo and J. Palomaki / Extensional and Intensional Aspects of Conceptual Design
In Figure 5, the first decomposition step of the conceptual classes shown in Figure. 1.a has been represented: the root objects of the original conceptual classes have been partitioned into the root objects of the conceptual classes after the decomposition; the following implicit information is specified through the root labels: only the attributes of the class person are associated with the class Person-Employee, whereas both the attributes of the class person and of the class employee are associated with the class PersonxEmployee.
With a further step of decomposition, all disjoint classes are obtained; the method recomposes the disjoint classes to define the object classes as in Figure 6. In the object classes, all the classes implicitly specified in the original conceptual classes are determined and thus the object classes represented in Figure 3 enclose one class more than the original conceptual classes. The concepts of multiple inheritance class, i.e. a class linked with higher level classes through two or more is-a relationships, can be employed also for the conceptual classes; this holds only when the classes which can be declared implicitly within other classes have other specific attributes.
E. Locuratolo and J. Palomaki / Extensional and Intensional Aspects of Conceptual Design
169
7. Conclusion A partition method is designed to achieve the flexibility of semantic models in reflecting changes occurring in real life and the efficiency on object systems. The method of partition has been applied only to the extensional aspects of concepts, i.e. set. In this paper a partition method is applied to the intensional aspects of concepts as well. A particular problem concerning the intensional negation of a concept is solved by defining a restricted intensional negation of a concept, which is important in all practical conceptual design situations all of which are confined in specific restricted part of the universe of discourse. Some further development of partitioning is presented as well. Acknowledgments This paper derives from the research undertaken by the authors during the CNR short term mobility program ( Pise – ’99). References: [1] Cardenas, A. F., and McLeod, D., 1990: Research Foundations in Object-Oriented and SemanticDatabase Systems. Prentice Hall, Englewood Cliffs, NJ 07632. [2] Elmasri, R., and Navathe, S.B., 2000: Fundamentals of Database Systems, Addison-Wesley. [3] Kauppi, R., 1967: Einführung in die Theorie der Begriffssysteme. Acta Universitatis Tamperensis. Ser. A. Vol. 15 Tampere: Tampereen yliopisto. [4] Locuratolo, E., 1998: “ASSO: Portability as a Methodological Goal”. Technical Report IEI B4-05-02, 1998. [5] Locuratolo, E., and Rabitti, F., 1998: “Conceptual Classes and System Classes in Object Databases”. Acta Informatica 35(3), 181-210. [6] Locuratolo, E., 2005: “Model Transformations in Designing the ASSO Methodology”. In Idea Group Inc Transformation of Knowledge, Information and Data: Theory and Applications, 283-302. [7] Nixon, B., and Mylopoulos, J., 1990: “Integration Issues in Implementing Semantic Data Models”. Advances in Database Programming Languages. ACM Press, New York and Reading: AddisonWesley, 187-217. [8] Palomäki, J., 1994: From Concepts to Concept Theory. Acta Universitatis Tamperensis. Ser. A. Vol.416. Tampere: Tampereen yliopisto.
170
Information Modelling and Knowledge Bases XIX H. Jaakkola et al. (Eds.) IOS Press, 2008 © 2008 The authors and IOS Press. All rights reserved.
Emergence of language: hidden states and local environments Jaak HENNO Tallinn Technical University, Tallinn 19086, Estonia
Abstract. Here is considered emergence of language in an environment, where agents communicate about issues which are not observable at the moment of communication, e.g. they search food and exits in a labyrinth and communication helps them to achieve their goals (food, exits). In such a situation message's receiver can not observe message's object at the moment when message is received. It is shown, that these non-observable objects can be mapped to observable objects using local environments. Agents do not build representations of the external word, instead of reasoning on representations of the world they access the world directly through perceptions and actions and their perceptions influence their behaviour. Messages from other agents, i.e. emerging language also change their behaviour and increases effectiveness of the whole population. Language is described via two mappings: syntax (i.e. syntactic objects, words) is interpreted in semantics by the meaning mapping; the speech mapping creates for semantic objects their syntactic denotations. Words in agents language gradually become mediated semantic objects, i.e. obtain the same significance (trigger the same actions) as the actual real-word situations which they denote.
1. Introduction In the last 10 years have appeared many papers where language emergence is investigated using computer simulations [1],[2],[3],[4],[5] etc. The topic is still not well understood and "classical"-style language researchers sometimes do not quite believe in the results obtained: "Reactions vary from fascination and incomprehension to scepticism or downright rejection" [6]. However, most of language researches believe in the method: "… emergent area of consensus is the growing interest in using computational modelling to explore issues relevant for understanding the origin and evolution of language"[7]. Especially popular in language emergence studies have become Sony's robot-dog Aibo [8],[9],[10] etc. In these studies robots research their environment and send and receive messages about objects, which they find, i.e. about objects, which both sender and receiver simultaneously can perceive (so called "word games"[11]). It is assumed, that both communication parties (sender and receiver) unambiguously can identify the topic, object of the message (e.g. using pointing [11]). But pointing is ambiguous (this problem has been discussed already e.g. in [12]; more realistic, probabilistic methods for topic determination in multi-topic environment was considered e.g. in [13]). Assumption of simultaneous perception (grounding) of message's object in the moment of communication is also rather unrealistic in real language acquisition. Children generally do receive little, if any, feedback while learning words [14]; most of the words what we know and use are not taught to us pointing to the corresponding object or event, we have learned them from context - picked them up from conversations,
J. Henno / Emergence of Language: Hidden States and Local Environments
171
from texts etc [15], [16]. When human language emerged, tribes were usually living in caves and the most important topics of communication (e.g. where to find food) where not simultaneously observable for communicating parties. How can agents create a common language, when they search exits and food in a labyrinth, i.e. the typical messages are "found food", "the passage was a dead end"; how can they develop a common language using messages about objects which are not directly observable at the moment of communication? We learn meanings of words from context. For every object and event there is a context, where it appears or occurs, and this context makes it possible to disambiguate the meaning of words. Context can be used also to map non-observable (at the moment when message is received) objects and events to real, observable (later) ones. Another feature of "real-word" language acquisition which is usually not considered in papers considering language emergence is pragmatics - the practical value of learned words. Learning colours as combinations of RGB values (without any practical significance) can give new insights to the design of learning (pattern-matching) algorithms and principles of learned perceptually grounded categories([17]). But it may not be very relevant to understanding the emergence of common vocabulary, since features (e.g. speed) of this process depend essentially on practical value of topics which these words denote. For instance, suppose that a community of agents is searching a labyrinth and the (only) perception what agents are able to receive is whether it is possible to continue along a passage or not (the passage turned out to be a dead end), i.e. the only topic of messages would be "dead end". If agents do not have any goal (they are just wandering) then clearly such a message (if they already understand it) does not change receiver's behaviour. But if they want to find an exit from the labyrinth, then such a message would make the receiver to turn around and search for another route. Thus the whole community will be more active (search more rooms); this increased activity will also increase the probability of encounters, i.e. exchanged messages and greater number of messages will increase the speed of emergence of common vocabulary. Here is an illustration to this. On the next time step agents (1,1) and (1,3) will meet in the room (1,2) and agent (3,1) will send to agent (1,1) message "dead end". If this message does not have any significance for agent (1,1), he would continue from the room (1,2) to rooms (1,3), (2,3) and also find himself in a dead end. But if his goal is to find an exit, he will change direction and continue into room (2,2) and meet there agent from the room (3,1), i.e. he got more possibilities to communicate and learn. All messages with practical value have similar effect - increase the speed of emergence of language. If the value is positive (e.g. food), agents tend to move towards their sources; the concentration of agents around (food) sources increases and they have more communication opportunities; if the value is negative (danger), agents try to avoid their sources and move in the opposite direction(s), the concentration of agents increases in nondangerous areas and probability of communication increases there. In the following is shown, how agents can learn and create a common system of denotations, language (actually only vocabulary) in a situation, where 1) objects and events, which are described in their messages can not be perceived at the moment when message is received; 2) agents have goals; messages are significant to agents and make them (when already understood) to change their behaviour (direction of movement); there are messages with
172
J. Henno / Emergence of Language: Hidden States and Local Environments
positive value (make agents to move forward) and negative value (make agents to move back); 3) agents start without any common language; language is created (learned) when agents exchange messages; when guessing agents do not receive any feedback about correctness of their guess. In some papers where word games are used as a tool to investigate emergence of language it is claimed that feedback is essential for language emergence (e.g. in [17]); however, experiments with word games have shown that feedback is not needed for language emergence (e.g. [19],[20],[21]). The set-up was investigated with a computational model. Used in experiments agents had very natural and simple properties: - agents move in 2D labyrinth; they have goals, which they want to achieve – find exit, find food, beware dangers; - agents perceive properties of their environment, which are essential to achieve their goals (semantic objects and events); they can not change their environment (e.g. mark rooms where they have been); - they can send and receive messages about objects and events; when creating a message, they use their speak function, which maps their experiences (inner state) to lexical symbols, words (agents can create sufficiently many different words); meaning function maps received message (words) into semantic objects and events; - agents are event-driven finite-state systems with inputs and outputs; agents do not build "world model", their outputs are functions of their environment and inner states(they implement "intelligence without representations"[18],[10]); - at the beginning agents do not have any understanding of each other's messages, i.e. do not have any common vocabulary, but in the process they create common vocabulary, where words gradually obtain the same meaning as external objects and situations which they denote and trigger the same changes in their behaviour (behavioural equivalence of denotations with external objects which they denote). 2.Objects and contexts The main idea in agent's reasoning (i.e. why it is possible that agent's can create a common system of denotations, vocabulary and later a language) is disambiguation of meanings when objects are presented in different contexts [19]. When an agent first sees two objects and another agent says two words: "õun, pirn" then certainly there is no way to understand, what is what, so in his vocabulary he has to use both words as possible denotations for both objects. But if he then sees one of these objects in different context and again hears one of the words which was also used earlier then he can place all words for corresponding objects: "õun = apple", "maja = house", "pirn = pear". The previous example (and most publications on language emergence) assumes that the message's topic (object) is unambiguously determined by both sender and receiver. But what if the object or event described in a message is not present (can not be pointed at), if speaker is describing something, what he has encountered earlier and what is now stored in his memory, i.e. his inner, directly non-observable (hidden) state?
J. Henno / Emergence of Language: Hidden States and Local Environments
173
To get a cue there should be some (recognizable by both, speaker and hearer) local context in their environment, where the message's object could be found. Agents act in space and time, so this local context what both agents can perceive should be a finite subspace, which is determined by some cues, which all agents can perceive and understand. This perceivable context allows mapping received words to external objects and events and thus disambiguating meanings. 3.Computational model For physical environment in computational modelling was used the well-known game "Hunt the Wumpus". In this game agents have to find an exit from a rectangular 2D-labyrinth. There are several dangers: a mysterious beast Wumpus, who scares agents and pits, where agents can fall. The dangers were somewhat soften: agents are not destroyed, but always could escape and later warn others about the danger; dangers only affect their movements, they try to avoid dangers. Agents have to feed themselves – in some rooms of the labyrinth they can find food (food sources renew themselves and will not be exhausted). Thus the semantic environment has four sorts of objects: - agents; agents are event-driven finite state machines, who live (i.e. move) in discrete time (all in parallel, i.e. in on one clock cycle all agents move to next room); - rooms; the attributes of a room is the list of its connections with other rooms (where one can go from here); - dangers: Wumpus and pits ; - food. Agent's state consists of components, which store different types of information: - current room attributes (in which directions it is open, i.e. where he can continue) and agent's current direction, i.e. in which direction he was moving when he arrived in this room; thus he can always say, whether a room is a dead end (the only opening is in the opposite direction with his movement); - hunger – how many moves have passed when he last time got food; - danger – there are dangers (Wumpus, pit) nearby. Generally agents always try to move forward into next room (default movement - they have to find an exit); they turn back (180o) only when they can not continue (dead end). Agents can exchange messages. When some agent learned something, he would try to inform others with following messages: "there was a dead end" (when the sender is returning from this dead end); "there is food in this passage"; "there is a danger ahead"; "I'm hungry!" - this message indicates, that there was no food in the last ht rooms (see below). These messages can not be analyzed (understood) at the time when they are received – usually the message's object is not perceivable when message is delivered. For instance, if an agent found a dead end and meets on his way back another agent , then he would tell him that
174
J. Henno / Emergence of Language: Hidden States and Local Environments
this is a dead end, but the dead end itself is not perceivable, there is (at the moment) no external information about the actual object (dead end). To recognize that message (word) produced by another agent means something about external environment (dead end, food, danger) and this object is somewhere nearby, there should be some finite area (messages context environment) which both the sender and receiver can recognize and where this particular object is situated. Message's sender describes in his message only these experiences which occurred in this local context and if messages receiver searches message's context he can find the message's object. For messages context could be used agents short-term memory of fixed length, e.g. all agents remember what they have seen in last five rooms. But this is artificial and nonefficient use of memory: if agent moves along an empty corridor (with only one possibility to continue), why should he remember all these empty rooms? More natural is to consider changes, i.e. where something happened or rooms where it was possible to select between more than one rooms to continue. Thus for first three types of messages the context environment is the corridor where sender and receiver currently are; for the last ("hungry!") message the context area is finite time interval - ht moves (the same for all agents). 4.Agent's data structure and algorithm 4.1 Inputs Agent's can perceive their environment. From a room where they currently are they can observe the following (i.e. meanings, the elements of the semantic space): r - how many other rooms are connected to the current room; they also know, which room the (just) come from and this (last visited room) is not counted, thus they can recognize a dead end (r = 0) and end of a dead-end corridor (r>1), these two inputs are denoted as de and de ; f - whether there is food in the room; dan - whether there is danger in the room (Wumpus or pit); ag - whether there is another agent (or several) in the current room; t – time (tick), entering the next room increases time counter t by one; time is used only to correct agents state of hunger: agent's time (state of hunger) is set to 0 if there is food in the room, on entering next room the time counter is increased until t ht ; after that agent becomes hungry, i.e. its state component H=1 (see below) and agent does not increase its time counter any more; m – message from another agent. 4.2 Agent's State Agent's state consists of six components. The first five correspond to one sort of meanings and are activated/deactivated by entering/leaving the corresponding external context environment; the last holds un-analyzed messages, received from other agents in the current context. The components and their state rules are: - DE - (Dead End) is activated ( DE 1 ), when agent finds itself in an dead end and has to move back; in this state agent can turn left or right ( 90D or 270D ), but not back ( 180D , to direction of the dead end); as soon as agent finds that there are more than one
J. Henno / Emergence of Language: Hidden States and Local Environments
175
possibilities to continue (the room where it come from, i.e. the dead end direction is not counted), DE 0 (this is also the initial value), e.g. on the picture (states are described in the order of movement) DE(1,4) = DE(1,3) = DE(1,2) = DE(1,1) = 0; on the next step agent encounters a dead end: DE(2,1) = DE(1,1) = DE(1,2) = 1; DE(1,3) = 0 (an exit there is more than one possibility to continue); - F (Food) – F 1 , when agent finds food in a room, but food is forgotten ( F 0 , this is also the initial value) as soon as agent exits the passage where food was found, i.e. in a room where there are more than 1 possibilities to continue (the same way as DE), e.g. on the picture F(1,3) = 0, F(1,2) = F(1,1) = F(1,2) = 1 (exit from the current context), F(1,3) = 0; - Dan – (Danger) – there is danger (Wumpus or pit) nearby – the behaviour of this state component is similar to DE, F; - T – time (steps) stores the number of steps, but only until the value ht , when T ht , the component H (Hunger) is activated (H=1) and T remains unchanged; T,H are reset (T , H 0 ) when food is found; - M – messages from another agents (un-altered); the messages will be kept until agent leaves the current context (passage with only one possibility to move forward, i.e. r<2); in the room where agent leaves the current context (a room with r>1) agent analyzes received messages using cues from environment what he stored (i.e. dead end, food, danger) when he was moving in the current context. For memory components DE,F,Dan,H there is a buffer for words which (possibly) can denote the corresponding meaning. For every word is stored also its use count – how many times this word (probably) was used to denote this entity. All these buffers have finite length. Since all agents can "invent" new words if they do not yet have any word for a meaning, it is possible that some agent receives # Ag 1 different words from some meaning ( # Ag is the number of agents), thus in maximally unrestricted simulation length of buffers should be # Ag 1 . However, every exchange of messages decreases probability for need to use a new word. In experiments speed of convergence with buffers of length # Ag / 4 was nearly the same as with length # Ag 1 (but with long buffers the model become very slow), thus the buffer length was usually set to 5. When a buffer gets full, the word with smallest use count is dropped. 4.3 State changes Agent's state changes are triggered by their states and external inputs according to following functions: de l DE (input de turns memory state DE to true); f l F (input f turns F to true); dan l Dan ; t l T max(T 1, ht ) ;ag l speak (DE , F , Dan ) - when agent meets another agent, he speaks about what he knows (has stored in corresponding states DE,F,Dan,H) about dead ends, food, hunger and dangers; ag speak (ag ) l (M ) - if agent meets another agent who speaks, the received message is stored in memory M (un-altered, since there is yet possible make any decisions about its meaning);
DE de l (DE 0) - on leaving a dead-end corridor the value of DE becomes false;
176
J. Henno / Emergence of Language: Hidden States and Local Environments
F de l (F 0) - when leaving a corridor with food the value of F becomes false, agent "forgets" about the food; Dan de l (Dan 0) - when leaving a corridor with a danger agent forgets about danger; M de l M 0 - agent "forgets" everything what he encountered in the last context (but before that the content of M is used to update the function meaning).
4.4 Speak and meaning mappings For the function speak agent selects from the corresponding buffer a word with maximum use count; if the buffer is still empty, agent "invents" a new word, different from all other words which are in his buffers (in [20] were considered also some other tactics of word selection, e.g. agents can sometimes “stick” to their “own” word - what they invented earlier - etc.). The input de (exit from the current context) fires update process for the meaning function. Let the content of the message's short-term memory be M {w1,..., wn }, n 0 and let {B1,..., Bm }, B j {de, dan, f , h } be the elements of semantic domain, which correspond to agent's activated state components, i.e. his experiences in the current contexts; agent has to create a mapping from words to elements of the semantic domain (meanings). The update for the mapping meaning consists of following steps: 1. Receiver separates from the set {w1,..., wn } of received words the known words, i.e. words w k , which are already present in the list of denotations of some B j , j=1,…,m . Receiver increases use counts of these words in corresponding lists and removes these elements B j from the set {B1 ,..., Bm } (words for them are already found). Let {v1 ,..., vk } and {C1 ,..., Cl } , k n , l d m be the remaining words and objects. Since there is no additional information available (what means what), every word vi is mapped to every C j , i.e. m2 ({v1 ,..., vk } o {C1 ,..., Cl } {v1 ,..., vk }
{C1 ,..., Cl } (direct product) - all words vi , i=1,…,k are added to word lists of objects C j , j=1,…,l. 2. The "known" words (i.e. words, which were removed in step 1.) are also removed from word lists of all meanings, which do not occur in the message, i.e. B {B1 ,..., Bn } - it is assumed, that all words in the message are about objects B1 ,..., Bn . In [20],[21] it was shown, that with explicit objects (pointing - receiver gets together with message also an indication of the message's object) the algorithm converges (i.e. agents will create a common vocabulary); the modified version which is used here converges also. The version used here can be interpreted as a version of explicit pointing: all messages are received only on exit from the current context and their objects are the meanings which receiver got from the current context. 4.5 Movements Agent's movements are one of their outputs (another is messages). At the beginning, when they do not yet have any information from others they always try to move forward and turn back only if they encounter a dead end or danger. So there are four forces fs , fde , fh , fdan , which affect their selection of the next move: search and hunger force forward, danger and
J. Henno / Emergence of Language: Hidden States and Local Environments
177
dead end – backwards (but hunger can also move backward, if agent decides that he received message where food was not mentioned and the message is reliable). When an agent already has some words in his word buffers and receives a message from another agent, he decides the next move using the information contained in the message. He decodes the message, i.e. evaluates meaning and reliability of every word w i of the message: m(wi ) is this element from {de, f , dan } , which has the word w i in the corresponding words buffer with highest use count; its reliability r (m(w i )) is 1/ N , where N is the number of states {de, f , dan } which have w i in their word buffer. If the current room is not a dead end (there is at least one possibility to continue), agent decides whether to continue or to move back using the following rules: - if H 1 (agent is hungry) and r ( f ) 0 (i.e. he has a message and for some word from the message m(wi ) f ), then agent continues; - in other cases probability to continue is 1 (r (de ) r (dan )) . 5.Results In the experiments m agents were put randomly into a computer-generated n q n labyrinth with random dangers and food sources. At the beginning all agents were "tabula rasa", i.e. did not have any vocabulary. When they all had found exit, they (in their current vocabulary-learning state) were put into another computer-generated labyrinth and so on, until they managed to create some level of mutual understanding, i.e. common language. The length of the experiment was measured as the number of communications they had. Since they all moved in parallel, the number of encounters in a situation, where somebody had to say something to another (had found food or a dead end), was rather small (especially when the society was small). On the picture 5 agents who had developed some understanding (rate of understood one-word messages > 50%) searched exit in a 10 q 10 labyrinth. A situation where one agent warned another about a dead end can be recognized as agent turning back without any obvious reason and these are marked by small circles (in all in this case there were 15 such communications). In the experiments were measured several parameters: speed of development of common words (all agents used the same word for some meaning), understanding of messages (understanding was always better than the rate of common words), influence of the size of society and the size of labyrinth etc. Very essential was the size of society (number of agents). If there were few agents, they met (i.e. had an opportunity to communicate) seldom if ever, so they could not develop common vocabulary. And even if they did (they were successively placed in many labyrinths, so they had to meet and communicate), they nevertheless could not use each other effectively, could not pass the learned information to others. On the next picture are results of series of experiments, where societies of agents of different sizes were searching a series of 30 q 30 labyrinths. Measured was the average number of rooms which agents passed before they found exit, depending on the level of their common vocabulary (how many messages from others they already understood). Small number of agents (societies of 5 and 20 agents) could develop common vocabulary (after many trials), but could not use it effectively, the vocabulary nearly did not have any effect on their efficiency. Societies of
178
J. Henno / Emergence of Language: Hidden States and Local Environments
200 and 300 agents developed common vocabulary nearly in the same time, but for them the common vocabulary made their search more than two times more efficient.
% of visited rooms
140.00 120.00 100.00
5 agents
80.00
20 agents
60.00
200 agents
40.00
300 agents
20.00 0.00 5 15 25 35 45 55 65 75 85 95 % of understood messages
Figure 1. How size of society increased society's effectiveness
The goal - why agents developed common vocabulary - was to decrease the labyrinth search time, therefore the best parameter for estimating the use of communication skills is the total number of moves agents made before they all found the exit. At the beginning, when agents could not understand each other they wandered more or less randomly and the percentage of rooms searched (from all the possible rooms) was always around 110-130%. When they had learned to understand each other and there was sufficiently many of them (number of agents > number of rooms/10), the percentage of passed rooms decreased rapidly. On the following pictures is the total number of passes what 50 agents made in 30 q 30 labyrinth presented as line thickness; the first presents the situation in the beginning of series (agents did not understand each other and visited nearly all rooms), on the second – at the end, when they understood already > 95% of each other messages – most of them moved along the path which took them to the exit.
Figure 2. How common vocabulary increases society's effectiveness
6.Conclusions and ideas for continuation Here was investigated, how a society of agents could develop common vocabulary in a situation, where agents have goals and try to communicate with each other about external situations and topics which are essential for them (help to achieve their goals), but these objects and situations may not be observable to message's receiver at the moment when they meet and communicate. Thus in a message sender has to describe his previous experiences, his inner state, which is not observable to receiver. It was shown, that agents
J. Henno / Emergence of Language: Hidden States and Local Environments
179
can nevertheless develop common language (here actually only vocabulary) when they use local environment (message's context), which they can recognize. Message's sender describes in his message only these experiences which occurred in this local context and receiver searches this local context in order to understand, what the message was about. The emerging common language has practical value and allows the society to act (to achieve their goals) more efficiently. The presented above agent's structure is very elementary and could be improved in several ways. For example, currently they communicate only if they both are in the same local environment (the first image). But it would be natural, that agents could communicate also when sender has just left the local environment (the second image). For that the receiver should get the direction of the last move of the sender – either sender should be able to communicate this (but for this the vocabulary should be enlarged, they should learn new words) or the receiver should be able to perceive this from sender's state. Another issue which should be investigated further is how they use what they have learned. In the described above simulation agents use the emerging language only to improve their output, to find exit. They do not change their inner structure, the "thinking" algorithm. But learning could also improve their decision-making. In the above model the "natural" context for the agent's state component H is not the current passage, but the time interval of length ht and this could be also used to modify the agent's decision-making algorithm. The discussed algorithm, where agents search in parallel and learn to communicate to each other their findings could be applied in many practical situations. For instance, the presented algorithm works on every non-cyclic graph, e.g. a tree. On the picture are presented results of an experiment, where 3 agents who already understood each other's messages (80%) were searching a tree (target was the root); line thickness indicates, how many edges agents passed; their efficiency was ca 10 times greater than at the beginning of the experiment (when they did not understand each other). References [1] Christiansen, M.; Kirby, S. Eds. (2003) Language Evolution. Oxford University Press: New York. [2] L. Steels (2006). Experiments on the emergence of human communication. Trends in Cognitive Sciences 10(8), pp. 347-349 [3] Baronchelli, A., Felici, M., Loreto, V., Caglioti, E. and Steels, L. (2006) Sharp transition towards shared vocabularies in multi-agent systems. Journal of Statistical Mechanics P06014 [4] Galantucci, B. (2005) An experimental study of the emergence of human communication systems. Cogn. Sci. 29, pp 737–767 [5] Cangelosi, A., Riga, T., Giolito, B., and Marocco, D. (2004) Language emergence and grounding in sensorimotor agents and robots. In First International Workshop on the Emergence and Evolution of Linguistic Communication. May 31- Jun 1 2004, Kanazawa, Japan [6] L. Steels (2006) How to do Experiments in Artificial Language Evolution and Why. In Cangelosi, A., Smith A. and Smith K., editor, Proceedings of the 6th International Conference on The Evolution of Language (EVOLANG6), London [7] M.H. Christiansen, S. Kirby (2003). Language evolution: consensus and controversies. Trends in Cognitive Sciences, 2003, 7:7, pp 300-307
180
J. Henno / Emergence of Language: Hidden States and Local Environments
[8] L. Steels, F. Kaplan (2001) AIBO’s first words: The social learning of language and meaning. Evolution of Communication, 4(1) [9] J. Poudade, L. Landwerlin, P. Paroubek (2006). Cognitive Situated Agents Learn to Name Actions. ECAI 2006 pp 51-55 [10] Deb Roy. (2005). Grounding words in perception and action: Computational insights. Trends in Cognitive Sciences, 9(8) [11] L. Steels, P. Vogt (1997). Grounding adaptive language games in robotic agents. In C. Husbands and I. Harvey, editors, Proceedings of the Fourth European Conference on Artificial Life, Cambridge MA and London, 1997. The MIT Press [12] Quine, W. V (1960). Word and Object. MIT Press: Cambridge, MA [13] P. Vogt (1998). The Evolution of a Lexicon and Meaning in robotic agents through self-organization, In : C. Knight and J.R. Hurford (eds.). The evolution of Language (selected papers from the 2nd International Conference on the Evolution of Language, London, April 6-9). [14] Bloom, P. (2000) How children learn the meaning of words. Cambridge, MA: MIT Press. [15] Sternberg, R.J. (1987). Most vocabulary is learned from context. In M.G. McKeown & M.E. Curtis (Eds.) The nature of vocabulary acquisition. Hillsdale, NJ: Erlbaum. [16] Nagy, W.E. & Herman, P.A. (1987). Breadth and depth of vocabulary knowledge: implications for acquisition and instruction. In M.G. McKeown & M.E. Curtis (Eds.) The nature of vocabulary acquisition. Hillsdale, NJ: Erlbaum [17] L. Steels, T. Belpaeme (2005). Coordinating Perceptually Grounded Categories through Language: A Case Study for Colour. Behavioral and Brain Sciences, 28:4, pp 469-89 [18] R. A.Brooks (1991) Intelligence without representation. Artificial Intelligence 47, 139–159. [19] Andrew D. M. Smith (2002) Evolving Communication through the Inference of Meaning. University of Edinburgh, September 2003. pp 1-418 [20] J. Henno (2002). Emergence of communication and creation of common vocabulary in multi-agent environment. Proceedings of the 12th European-Japanese Conference on Information Modelling and Knowledge Bases. Krippen, Swiss Saxony, Germany. May 27-30, 2002, pp 229-233 [21] J. Henno (2006). Mathematical Model of Natural Languages. ICCC 2006 IEEE International Conference on Computational Cybernetics, Tallinn, Estonia August 20-22, 2006, Proceedings, pp 275-281
Information Modelling and Knowledge Bases XIX H. Jaakkola et al. (Eds.) IOS Press, 2008 © 2008 The authors and IOS Press. All rights reserved.
181
Frameworks for Intellectual Property Protection on Multimedia Database Systems Hideyasu Sasaki Attorney-at-Law, New York State Bar, the Third Judicial Department, Albany, N.Y., U.S.A. Ritsumeikan University, 6-4-10, Wakakusa, Kusatsu, Shiga 525-0045, Japan
[email protected] Yasushi Kiyoki Keio University, 5322 Endo Fujisawa, Kanagawa, 252-8520 Japan
[email protected] Abstract. In this paper, we discuss the issues and future trends on patentable parameter setting components implemented in multimedia database systems. Multimedia databases in the applications of parameter setting components consist of copyrightable metadata. The data-processing processes are patentable in the forms of parameter setting components. The current techniques in parameter setting components enclose a variety of numerical parametric information which inventors would like to cover as trade secret. We present the conditions of copyrightability on the multimedia databases and the patentability on the parameter setting components with the directions for protecting numerical parametric information as trade secret.
1 Introduction The principal concern of this paper is to present the conditions of copyrightability on the multimedia databases and the patentability on the parameter setting components with the directions for protecting numerical parametric information as trade secret. Our secondary concern is to provide researchers and practitioners in information modeling and knowledge bases with legal references on the concepts, issues, trends and frameworks of intellectual property protection regarding “multimedia database systems” in engineering manner. A multimedia database system, as an information system, consists of digital contents in databases and retrieval mechanisms. The intellectual property protection of multimedia database systems is a critical issue in the multimedia database community that demands frameworks for recouping their investment in database design and system implementation. Intellectual property law gives incentive to advance appropriate investment in database design and implementation with two conventional types of intellectual property protection: copyright and patent [1, 2]. Multimedia digital contents take a variety of forms including text, images, photos and video streams, which often commingle in multimedia databases. Nevertheless, present legal studies are not satisfactory as the source of technical interpretation of the intellectual properties regarding multimedia databases. The intellectual property protection of the multimedia databases demands clear and concise frameworks.
182
H. Sasaki and Y. Kiyoki / Frameworks for Intellectual Property Protection
2 Background In this section, we discuss two main issues on the intellectual property protection regarding multimedia database systems. The first issue is the copyright protection of databases to which the multimedia digital contents are stored in multimedia databases. The second issue is the patent protection of the retrieval mechanisms of multimedia database systems. 2.1 Copyright on Multimedia Databases U.S. Copyright Act [3] defines that a compilation or assembling of individual contents, i.e., preexisting materials or data, is a copyrightable entity as an original work of authorship. Gorman and Ginsburg [4], and Nimmer, et al. [5] state that a compilation is copyrightable as far as it is an “original work of authorship that is fixed in tangible form”. Multimedia database systems consist of multimedia digital contents which are indexed and stored in databases for appropriate retrieval operations and the retrieval mechanisms which are optimized and applied to object domains of those databases. The entire multimedia database is copyrightable in the form of a component of “contents-plus-indexes” while static indexes or metadata are fixed to multimedia digital contents in a tangible medium of repository, i.e., database. Static indexes or metadata represent a certain kind of categorization of the entire content of each database (See Fig. 1). The originality on the categorization makes each database copyrightable as is different from the mere collection of its individual contents. What kind of categorization should be original to constitute a copyrightable compilation on the database? The court of American Dental Ass’n v. Delta Dental Plan Ass’n [6] determined that minimal creativity in compilation sufficed this requirement of originality on databases. Any standard or framework on the requirement is not clear in the technical or engineering meanings. A uniform framework on the categorization regarding indexes or metadata of databases must be formulated in engineering manner. The European Union has legislated and executed a scheme for protecting a database including its content per se, known as the sui generis right of database protection [7, 8, 9]. That European scheme shares the same issue on the originality regarding the categorization of multimedia digital contents in databases. 2.2 Patent on Multimedia Database Systems U.S. Patent Act [10] defines that a data-processing process or method is patentable subject matter in the form of a computer-related invention, i.e., a computer program. The computer program is patentable as far as the “specific machine . . . . . . produce(s) a useful, concrete, and tangible result . . . for transforming . . . ” physical data (“physical transformation”) [11]. The computer-related inventions often combine means for data-processing, some of which are prior disclosed inventions. A retrieval mechanism in a multimedia database system consists of a number of “processes”, i.e., methods or means for data processing in the form of combination of computer programs. A set of programs focuses on image processing, while another set of programs operates text mining, for example. Meanwhile, the processes in a retrieval mechanism of a multimedia database system comprise means or components for parameter setting which is adjusted to retrieve specific kinds of multimedia digital contents, for example, images in certain domains. The problem is that which process is to realize technical advancement (nonobviousness) on its combination of the prior arts and is to be specific/enable on its parameter setting. These two issues are emerging
H. Sasaki and Y. Kiyoki / Frameworks for Intellectual Property Protection
183
KEYWORD-BASED RETRIEVAL DATABASE DESIGNER
DB CONTENTS
INDEXES/
+ METADATA DICE 4
DICE 6
CREATE
INTEGRATE (STATIC COMPILATION)
CONTENT CREATOR Figure 1: Formulation for copyrighting multimedia databases.
problems in the advent of multimedia database systems. Uniform frameworks on the novel combination and the specific parameter setting must be formulated in engineering manner, respectively. 3 Frameworks for Intellectual Property Protection In this section, we outline the frameworks for intellectual property protection regarding multimedia database systems: copyrightable database and patentable retrieval mechanism. 3.1 Multimedia Database as Copyrightable Entity Our framework for copyrighting the multimedia database determines which type of database should be copyrightable in the form of a component of contents-plus-indexes [12, 13, 14]. The collection of static indexes and individual contents forms a component of contents-plusindexes. That component identifies the entire content of each database, as is a static and copyrightable compilation. Copyrightable compilation is to be of sufficient creativity, i.e., originality in the form of a component of contents-plus-indexes. The set of conditions on the original categorization regarding indexes or metadata is formulated as below [13, 14]: A categorization regarding indexes or metadata is original only when 1. The type of indexes or metadata accepts discretionary selection in the domain of a problem database; otherwise, 2. The type of taxonomy regarding indexes or metadata accepts discretionary selection in the domain of a problem database. A typical case of non-original categorization is a photo film album database which has indexes of consecutive numbers. That case does not accept any discretion in the selection
184
H. Sasaki and Y. Kiyoki / Frameworks for Intellectual Property Protection
I. PATENTABLE SUBJECT MATTER OF PROCESSES FOR CBIR FUNCTIONS + TECHNICAL CONTRIBUTION (E.U.) (ENTRANCE TO PATENT PROTECTION)
DO CLAIMS COMPRISE THE MEANS no FOR PARAMETER SETTING ? yes
PATENTABLE OR NOT AS THE PROCESSES RELATED TO CBIR FUNCTIONS (out)
II. NONOBVIOUSNESS OR INVENTIVE STEPS (TECHNICAL ADVANCEMENT) (1) DO THE PRIOR ARTS PREDICATE A COMBINATION OF THE MEANS FOR PERFORMING A FUNCTIONAL PROCESS OF CBIR ? no (2) DO THE FUNCTION REALIZE QUANTITATIVE AND/OR QUALITATIVE ADVANCEMENT ?
(1-a) DO THE DESCRIPTIONS SPECIFY THE FORMULAS FOR PARAMETER SETTING ?
(1-b)DOES THE DISCUSSED INVENTION HAVE A CO-PENDING APPLICATION THAT SPECIFIES THE ABOVE FORMULAS ?
no (out)
yes DO THE PROCESSES HAVE THE IMPROVED FORMULAS FOR PARAMETER SETTING BASED ON PRIOR DISCLOSED MEANS FOR CBIR ? yes no (2-a) DO THE PROCESSES REALIZE A NEW FUNCTION BY COMBINING THE PRIOR DISCLOSED MEANS ?
yes
OBVIOUS NOT AS no PATENTABLE
no
yes
no (out)
yes (2-b) DO THE DESCRIPTIONS DISCLOSE EXAMPLES OF VALUES ON PARAMETER SETTING ?
no (out)
yes (I) WORKING OR PROPHETIC EXAMPLES OF INITIAL VALUES OR WEIGHTS ON PARAMETER SETTING
yes
III. ENABLEMENT OR CLARITY OF CLAIMS (CLEAR SPECIFICATION) PATENTABLE AS A DOMAINSPECIFIC APPROACH OF CBIR
(ii) WORKING EXAMPLES OF THE RANGE OF VALUES ON PARAMETER SETTING
PATENTABLE AS A DOMAINGENERAL APPROACH OF CBIR
Figure 2: Formulation for patenting the retrieval processes.
of the type of indexes or metadata, or the type of taxonomy. The photo film album database uses its respective film numbers as indexes for its retrieval operations. The taxonomy of the indexes is only based on the consecutive numbering without any discretion in its selection of the type of indexes or taxonomy regarding a multimedia database. Meanwhile, the discretionary selection of the type of indexes or metadata, or taxonomy constitutes copyrightable compilation of minimal creativity, i.e., originality on the categorization regarding indexes or metadata. A typical case of discretionary selection of the type of indexes or metadata is the web document encyclopedia as a multimedia database. Suppose that a database restores pictures of starfish which are manually and numerically numbered by day/hour-chronicle interval that is based on their significant life stages from birth to death. That database is to be an original work of authorship as a copyrightable compilation in the form of a component of contents-plus-indexes. That database of discretionary type of numbering or indexing is an original, i.e., copyrightable database. 3.2 Multimedia Database System as Patentable Mechanism Our framework for patenting the retrieval mechanisms of multimedia database system determines which type of retrieval mechanism should be patentable in the form of a component of novel combination of prior disclosed processes and/or a component of specific parameter setting (See Fig. 2) [15, 16, 17, 14]. The frameworks focus on the following three requirements for patentability: “patentable subject matter” (entrance to patent protection), “nonobviousness” (technical advancement) and “enablement” (specification) [18]. The requirement for nonobviousness on the combination of the processes for data-
H. Sasaki and Y. Kiyoki / Frameworks for Intellectual Property Protection
185
processing as the retrieval mechanism in a multimedia database system is listed as below [17]: 1. The processes for performing a retrieval mechanism must comprise the combination of prior disclosed means to perform certain mechanism which is not predicated from any combination of the prior arts; in addition, 2. The processes for performing a retrieval mechanism must realize quantitative and/or qualitative advancement. Otherwise, the discussed processes are obvious so that they are not patentable as the processes for performing a retrieval mechanism. First, a combination of prior disclosed means should not be “suggested” from any disclosed means “with the reasonable expectation of success” [19]. Second, its asserted function on the discussed mechanism must be superior to the conventional functions which are realized in the prior disclosed or patented means in the field of the retrieval mechanism of multimedia database system. On the latter issue, several solutions for performance evaluation are proposed and applicable. Another general strategy is restriction of the scope of problem claims into a certain narrow field to which no prior arts have been applied. This claiming strategy is known as the local optimization of application scope. A component for parameter setting realizes thresholding operations in the form of a computer program with a set of ranges of parametric values. In retrieval mechanisms, parametric values determine, as thresholds, which candidate image is similar to an exemplary requested image by computation of similarity of visual features [20, 21, 22, 23]. That parameter setting component is to be a computer-related invention in the form of computer program as far as that parameter setting is sufficiently specified to enable a claimed invention or retrieval mechanism [24]. The requirement for enablement on the parameter setting component of the retrieval mechanism in a multimedia database system is listed as below [17]: (1-a) The descriptions of the processes for performing a retrieval mechanism must specify the formulas for parameter setting; otherwise, (1-b) the disclosed invention of the processes should have its co-pending application that describes the formulas in detail; in addition, (2-a) the processes must perform a new mechanism by a combination of the prior disclosed means; otherwise, (2-b) the processes should have improved formulas for parameter setting which is based on the prior disclosed means for performing a retrieval mechanism, and also should give examples of parametric values on parameter setting in descriptions. For 2-b, the processes must specify the means for parameter setting by “giving a specific example of preparing an” application to enable those skilled in the arts to implement their best mode of the processes without undue experiment [25, 26]. U.S. Patent and Trademark Office [24, 27] suggested that the processes comprising the means, i.e., the components for parameter setting must disclose at least one of the following examples of parametric values on parameter setting: (i) Working or prophetic examples of initial values or weights on parameter setting; (ii) Working examples of the ranges of parametric values on parameter setting. The “working examples” are parametric values that are confirmed to work at actual laboratory or as prototype testing results. The “prophetic examples” are given without actual work by one skilled in the art.
186
H. Sasaki and Y. Kiyoki / Frameworks for Intellectual Property Protection
The retrieval mechanisms of multimedia database systems are patentable in the form of components of novel combinations of prior disclosed processes and/or components of specific parameter settings while they are to satisfy the above conditions. 3.3 A Simulation Example for the Formulated Procedural Diagram The proposed formulation in Fig. 2 should be clear with its application to an exemplary multimedia database system. We apply it to “Virage Image Retrieval”(VIR), which was developed in the early 1990s as a typical content-based retrieval of visual objects stored in digital image database systems. VIR is an indexing method for an image search engine with “primitives”, which compute similarity of visual features extracted out of typical visual objects, e.g., color, shape and texture of images. VIR evaluates similarity of images with ad hoc weights, i.e., parametric values, which are given to the parameter setting components for correlation-computation, by user-preference. Its claims consist of “function containers” as means-plus-functions for feature extraction and similarity computation. Its first claim, as described below, constitutes the primitives as the means-plus-functions. Those primitives realize a domain-general approach of CBIR by the formulas on parameter setting. VIR Claim # 1. A search engine, comprising: a function container capable of storing primitive functions; . . . a primitive supplying primitive functions . . . . . . , wherein the primitive functions include an analysis function . . . . . . of extracting features from an object . . . . First in Fig. 2, on its patentable subject matter, its retrieval processes consisting of the formula for parameter setting are to be determined as patentable subject matter in the form of computer programs. Those data-processing processes generate physical transformation on a specific machine, i.e., a computer memory with certain classification results. Second, on its nonobviousness, those data-processing processes are inventive steps that consist of combinations of the prior arts on thresholding functions as implemented in the integration of classification based on similarity computation, visual feature extraction and automatic indexing techniques. Those combinations are not predicated from any conventional keyword-based retrieval technique. Third, on its enablement, VIR’s description of preferred embodiments gives its clear specification on the formulas for parameter setting that realizes a domain-general approach of CBIR that was a brand new technology at the time. VIR Description . . . . . . For primitives having multiple dimensions, . . . . . . , An equation for an exemplary Euclidean metric is as follows. Primitive design. A primitive encompasses a given feature’s representation, extraction, and comparison function. . . . . . . The constraints are as follows: Primitives, in general, map to cognitively relevant image properties of the given domain. The formulation should take advantage of a threshold parameter (when available),. . . . . . . The retrieval mechanisms of multimedia database systems are patentable in the form of components of novel combinations of prior disclosed processes and/or components of specific parameter settings while they are to satisfy the above conditions.
H. Sasaki and Y. Kiyoki / Frameworks for Intellectual Property Protection
187
3.4 Trade Secret in Parameter Settings An emerging problem is discussed on the parameter settings of retrieval mechanisms. Patent application on the parameter setting components demands applicants as developers to make public the detailed know-how on the best range of parametric values in practice. The discovery of those parametric values needs considerable pecuniary investment in research and development. That kind of knowledge should be kept covered in the form of trade secret but not be open in public via patent application. The multimedia database community demands a scheme that determines which parameter setting component should be patentable or kept secret regarding multimedia database systems. 3.5 Embedding Trade Secret in Parameters It is necessary to prepare a scheme that determines how and which part of parameter setting components should take the form of trade secret. The problem is how to interpret the “working examples” of initial values or weights on parameter setting and the ranges of parametric values. The requirement for patenting parameter setting components as computer-related inventions demands inventors to make public their discovered “working examples” on those parameter values: initial values or ranges. The practice in patent application, nonetheless, does not always force applicants to disclose to examiners complete and perfect evidences on those initial values or ranges of parametric values, but those values as should work in their best mode at the present art. In the reality of application practice, inventors have three choices for embedding trade secrets on their know-how of parametric values in the forms of patentable parameter components: 1. On the initial values, their prophetic examples should be disclosed in patent application, instead of working examples; 2. On the ranges of parametric values, those ranges should be widened as possible at the best but not complete mode; 3. Otherwise, the ranges of parametric values should be replaced with several initial values of prophetic examples. The issue is when those patentable parameter setting components should be allowed to embed trade secrets on their parametric values. The framework or set of conditions to realize that problem depends on application cases. 4 Conclusions and Future Works In this article, we have discussed issues on intellectual property protection regarding multimedia database systems which consist of indexed multimedia digital contents in databases and retrieval mechanisms. We have presented the frameworks for copyrighting the database of multimedia database systems in the form of a component of contents-plus-indexes, and for patenting the retrieval mechanism of multimedia database systems in the form of a combination of processes and/or a component of parameter settings. We have also pointed out an emerging problem on the trade secret of parameter setting components and the possible directions for its solution.
188
H. Sasaki and Y. Kiyoki / Frameworks for Intellectual Property Protection
We are working to formulate a framework when certain patentable parameter setting components embed trade secret on their parametric values in several industrial fields of application software: authentication and encryption in the music file share industry and the embedded systems and software in the automobile electronics industry. In the field of visual information retrieval, the multimedia database community faces a variety of problems. Especially, a problem demands urgent solutions for the future progress of multimedia database systems: a framework for protecting a database as a whole. In addition, databases contain multimedia information including images and videos. Portable information devices allow people to easily access to a large amount of downloadable multimedia files stored in distributed databases around the world. Network technology for efficient data transactions often triggers unauthorized misappropriation of those multimedia files that are important intellectual assets. Even the sui generis right of database protection discussed in Europe is not to protect any database as a whole in the present legal system. It is necessary to prepare a framework for protecting entire databases including their contents. That framework should determine how and which type of database is to be protected as a whole. Acknowledgements This study is supported financially in part by the Grant-in-Aid for Scientific Research (“KAKENHI”) of the Japanese Government: No. 18,700,250 (FY 2006-2009). This study is also supported financially in part by the Microsoft Grant on Intellectual Property Research Promotion for the Year of 2005. References [1] J.M. Jakes and E.R. Yoches, Legally Speaking: Basic Principles of Patent Protection for Computer Science, Communications of the ACM 32(8) (1989) 922–924 . [2] C. Junghans and A. Levy, Intellectual Property Management: A Guide for Scientists, Engineers, Financiers, and Managers, Hoboken, NJ: John Wiley & Sons. (2006) . [3] U.S. Copyright Act, 17 U.S.C. Sec. 101, & 103, (2005) . [4] R.A. Gorman and J.C. Ginsburg, Copyright: Cases and Materials (6th ed.), University casebook series. Charlottesville, NC: The Michie Company. (2002) . [5] M.B. Nimmer, P. Marcus, D.A. Myers, and D. Nimmer, Cases and Materials on Copyright & Other Aspects of Entertainment Litigation Including Unfair Competition (7th ed.), Dayton, OH: LexisNexis. (2006) . [6] American Dental Assfn v. Delta Dental Plan Assfn, 126 F.3d 977 (7th Cir. 1997) . [7] J. Reinbothe, The Legal Protection of Non-creative Databases. In: Proc. of the Database Workshop of the International Conference of Electronic Commerce and Intellectual Property, (WIPO/EC/CONF/99/SPK/22-A), WIPO. Geneva, Switzerland, September 14–16, 1999. [8] P. Samuelson, Legally Speaking: Legal Protection for Database Content, Communications of the ACM 39(12) (1996) 17–23. [9] T. Aplin, Copyright Law in the Digital Society: The Challenges of Multimedia, Oxford, U.K.: Hart Publishing. (2005) . [10] U.S. Patent Act, 35 U.S.C. Sec. 101, 103, & 112, (2005) . [11] In re Alappat, 33 F.3d 1526, 31 U.S.P.Q.2d 1545 (Fed. Cir. 1994) (en banc) . [12] H. Sasaki and Y. Kiyoki, A Proposal for Digital Library Protection. In: Proc. of the 3rd ACM/IEEE-CS Joint Conference on Digital Libraries, Los Alamitos, CA: IEEE Computer Society Press. Houston, TX, May 27–31, 2003, p. 392 .
H. Sasaki and Y. Kiyoki / Frameworks for Intellectual Property Protection
189
[13] H. Sasaki and Y. Kiyoki, Copyrighting Digital Libraries from Database Designer Perspective. In: Proc. of the 7th International Conference on Asian Digital Libraries (ICADL), Lecture Notes in Computer Science, 3334. Berlin: Springer-Verlag. Shanghai, China, December 11–14 (2004) pp. 626–629 . [14] H. Sasaki and Y. Kiyoki, Multimedia Digital Library as Intellectual Property. In: Design and Usability of Digital Libraries: Case Studies in the Asia Pacific, Idea Group Press. (2005) 238– 253 . [15] H. Sasaki and Y. Kiyoki, Patenting Advanced Search Engines of Multimedia Databases. In: S. Lesavich (Ed.), Proc. of the 3rd International Conference on Law and Technology , International Society of Law and Technology (ISLAT). Anaheim, Calgary, Zurich: Acta Press. Cambridge, MA, November 6–7 (2002) pp. 34–39 . [16] H. Sasaki and Y. Kiyoki, Patenting the Processes for Content-based Retrieval in Digital Libraries. In: E.-P. Lim, S. Foo, C. Khoo, H. Chen, E. Fox, S. Urs, & T. Costantino (Eds.) Proc. of the 5th International Conference on Asian Digital Libraries (ICADL), Lecture Notes in Computer Science, 2555. Berlin: Springer-Verlag. Singapore, December 11–14 (2002) pp. 471–482 . [17] H. Sasaki and Y. Kiyoki, A Formulation for Patenting Content-based Retrieval Processes in Digital Libraries, Journal of Information Processing and Management 41(1) (2005) 57–74 . [18] R.P. Merges and J.F. Duffy, Patent Law and Policy: Cases and Materials (3rd ed.), Dayton, OH: LexisNexis. (2002) . [19] In re Dow Chemical Co., 837 F.2d 469, 473, 5 U.S.P.Q.2d 1529, 1531 (Fed. Cir. 1988) . [20] Y. Rui, T.S. Huang, and S.F. Chang, Image Retrieval: Current Techniques, Promising Directions and Open Issues, Journal of Visual Communication and Image Representation 10(4) (1999) 39–62. [21] A.W.M. Smeulders, M. Worring, S. Santini, A. Gupta, and R. Jain, Content-based Image Retrieval at the End of the Early Years, IEEE Trans. on Pattern Analysis and Machine Intelligence 22(12) (2000) 1349–1380. [22] A. Yoshitaka and T. Ichikawa, A Survey on Content-based Retrieval for Multimedia Databases, IEEE Trans. on Knowledge and Data Engineering 11(1) (1999) 81–93. [23] S. Deb, Multimedia Systems and Content-based Retrieval, Hershey, PA: Idea Group Inc. (2004) . [24] U.S. Patent and Trademark Office, Examination Guidelines for Computerrelated Inventions, 61 Fed. Reg. 7478 (Feb. 28, 1996) (“Guidelines”). Available: http://www.uspto.gov/web/offices/pac/dapp/oppd/patoc.htm. (1996) . [25] Autogiro Co. of America v. United States, 384 F.2d 391, 155 U.S.P.Q. 697 (Ct. Cl. 1967) . [26] Unique Concepts, Inc. v. Brown, 939 F.2d 1558, 19 U.S.P.Q.2d 1500 (Fed. Cir. 1991) . [27] U.S. Patent and Trademark Office, Examination Guidelines for Computerrelated Inventions Training Materials Directed to Business, Artificial Intelligence, and Mathematical Processing Applications (“Training Materials”). Available: http://www.uspto.gov/web/offices/pac/compexam/examcomp.htm. (1996) .
190
Information Modelling and Knowledge Bases XIX H. Jaakkola et al. (Eds.) IOS Press, 2008 © 2008 The authors and IOS Press. All rights reserved.
Wavelet and Eigen-Space Feature Extraction for Classification of Metallography Images Pavel Praks a,1 , Marcin Grzegorzek b , Rudolf Moravec c , Ladislav Válek d , and Ebroul Izquierdo b a Dept. of Applied Mathematics, Technical University of Ostrava, Czech Republic b Multimedia & Vision Research Group, Queen Mary University of London, UK c Research and Development, Mittal Steel Ostrava plc, Ostrava, Czech Republic d QT - Production Technology, Mittal Steel Ostrava plc, Ostrava, Czech Republic Abstract. In this contribution a comparison of two approaches for classification of metallography images from the steel plant of Mittal Steel Ostrava plc (Ostrava, Czech Republic) is presented. The aim of the classification is to monitor the process quality in the steel plant. The first classifier represents images by feature vectors extracted using the wavelet transformation, while the feature computation in the second approach is based on the eigen-space analysis. Experiments made for real metallography data indicate feasibility of both methods for automatic image classification in hard industry environment. Keywords. Measurement, hard industry, human factors, content-based image retrieval, wavelet transformation, statistical classification, numerical linear algebra, partial symmetric eigenproblem, iterative solvers.
1. Introduction Any meaningful human activity requires perception. Under perception realization, evaluation, and interpretation of sensory impressions is understood. It allows the human to acquire knowledge about the environment, to react to it, and finally to influence it. There is no reason in principle why perception could not be simulated by some other matter, or instance, a digital computer [6]. The aim of the simulation is not the exact modeling of the human brain activities, but the obtainment of similar perception results. Research activities concerned with the mathematical and technical aspects of perception are the field of pattern recognition. One of the most important perceptual abilities is vision. The processing of visual impressions is the task of image analysis. The main problem of image analysis is the recognition, evaluation, and interpretation of known patterns or objects in images. 1 Dept. of Information and Knowledge Engineering, University of Economics, Prague, Czech Republic. Correspondence to: Pavel Praks, Dept. of Applied Mathematics, VŠB – Technical University of Ostrava, 17. listopadu 15, CZ 708 33 Ostrava, Czech Republic. Tel.: +420 59 732 4181; Fax: +420 59 691 9597; E-mail: [email protected].
191
P. Praks et al. / Wavelet and Eigen-Space Feature Extraction
2n × 2n Gray Level Image f κ,ρ ..
xm
.
l
k
s=0
s = −1
bs,0,0 bs,1,0 bs,2,0 bs,3,0 bs,0,0 bs,1,0
d2,s
bs,0,1 bs,1,1 bs,2,1 bs,3,1 bs,0,1 bs,1,1
2|bs|
..
bs,0,2 bs,1,2 bs,2,2 bs,3,2 .
d0,s
s = s = −2 bs
d2,s
d2,s+1
d0,s d1,s d1,s
d0,s+1
d1,s+1
bs,0,3 bs,1,3 bs,2,3 bs,3,3
Figure 1. 2D signal decomposition with the wavelet transformation for a local neighborhood of size 4× 4 pixels. The final coefficients result from gray values b0,k,l and have the following meaning: b−2 : low-pass horizontal and low-pass vertical, d0,−2 : low-pass horizontal and high-pass vertical, d1,−2 : high-pass horizontal and high-pass vertical, d2,−2 : high-pass horizontal and low-pass vertical.
In this paper, the problem of automatic pattern classification in real metallography images from the steel plant of Mittal Steel Ostrava plc (Ostrava, Czech Republic) is addressed. The objective is to monitor the process quality in the steel plant. For this reason two different image classification algorithms are used and compared in this contribution. The first one computes feature vectors with the wavelet transformation, while in the second one the eigen-space analysis is applied. The paper is structured as follows. Section 2 describes the theoretical background of the wavelet-based image classifier. In Section 3, intelligent image retrieval using the partial eigen-problem is presented. Experimental comparison of these two approaches for image classification follows in Section 4, while Section 5 closes the paper with some conclusions.
2. Statistical Wavelet-Based Classification In this section, a statistical wavelet-based approach for image classification is presented. Section 2.1 describes the training of statistical models for different image concepts. These models are then used for image classification, which is presented in Section 2.2. 2.1. Training of Statistical Concept Models Before images can be classified in the recognition phase (Section 2.2), statistical models Mκ for all image concepts Ωκ considered in a particular classification task are learned in the training phase. The concept modeling starts with the collection of training data. In this work real metallography images from a cooking plant are used for this reason. Subsequently, the original training images are converted and resized into gray level images of size 2n × 2n (n ∈ N) pixels. In all these preprocessed training images 2D local feature vectors cκ,m are extracted using the wavelet transformation [4]. Training images are divided into neighborhoods of size 2|bs| × 2|bs| (in Figure 1, 4 × 4 pixels). These neighborhoods are treated as 2D discrete signals b0 and decomposed to low-pass and high-pass coefficients. The resulting coefficients bsb, d0,bs , d1,bs , and d2,bs are then used for feature vector computation
192
P. Praks et al. / Wavelet and Eigen-Space Feature Extraction
cκ,m (xm ) =
ln(2sb|bsb|) ln[2sb(|d0,bs | + |d1,bs | + |d2,bs |)]
.
(1)
The feature vectors cκ,m are modeled by normal density functions pκ,m = p(cκ,m |μκ,m , σ κ,m ) .
(2)
Due to the large number of training images for each concept Ω κ , it is possible to estimate the mean value vector μκ,m and the standard deviation vector σ κ,m for all image locations xm , i. e., all feature vectors cκ,m . Finally, statistical models Mκ for all image concepts Ωκ are created and ready for use in the classification phase (Section 2.2) 2.2. Image Classification Once the concept modeling (Section 2.1) is finished, the system is able to classify images taken from a real world environment. First, a test image f is taken, preprocessed, and local feature vectors cm are computed in it in the same way as in the training phase (Section 2.1). Second, the classification algorithm based on the Maximum Likelihood (ML) Estimation is started. The task of the image classification algorithm is to find the concept Ω κb , (or just its index κ ) of the test image f . In order to do so, the density values for all concepts Ω κ have to be compared to each other. Assuming that the feature vectors c m are statistically independent on each other, the density value for the given test image f and concept Ω κ is computed with pκ =
m=M
p(cm |μκ,m , σ κ,m ) ,
(3)
m=1
where M is the number of all feature vectors in the image f . All data required for computation of the density value pκ with (3) is stored in the statistical concept model Mκ . These density values are then maximized with the Maximum Likelihood (ML) Estimation [10] κ = argmax pκ κ
.
(4)
Having the index κ of the resulting concept the classification problem for the image f is solved.
3. Latent Semantic Indexing In this section, we present the intelligent image retrieval using the partial eigen-problem. The numerical linear algebra is used as a basis for the information retrieval in the retrieval strategy called Latent Semantic Indexing, see for instance [1], [2]. LSI can be viewed as a variant of a vector space model, where the database is represented by the document matrix, and a user’s query of the database is represented by a vector. LSI also contains a low-rank approximation of the original document matrix via the Singular Value Decomposition (SVD) or the other numerical methods. The SVD is used as an automatic tool
P. Praks et al. / Wavelet and Eigen-Space Feature Extraction
193
for identification and removing redundant information and noise from data. The next step of LSI involves the computation of the similarity coefficients between the filtered user’s query and filtered document matrix. The well-known cosine similarity can be used for a similarity modeling. Recently, the methods of numerical linear algebra are also successfully used for the face recognition and reconstruction [5], image retrieval [8,7], as a tool for information extraction from internet data [9] and for iris recognition problem [7]. The "classical" LSI application in information retrieval algorithm has the following basic steps: i) The Singular Value Decomposition of the term matrix using numerical linear algebra. SVD is used to identify and remove redundant noise information from data. ii) The computation of the similarity coefficients between the transformed vectors of data and thus reveal some hidden (latent) structures of data. Numerical experiments pointed out that some kind of dimension reduction, which is applied to the original data, brings to the information retrieval following two main advantages: (i) automatic noise filtering and (ii) natural clustering of data with "similar" semantic. 3.1. Image Coding In our approach [8,7], a raster image is coded as a sequence of pixels. Then the coded image can be understood as a vector of an m-dimensional space, where m denotes the number of pixels (attributes). Let the symbol A denote a m × n term-document matrix related to m keywords (pixels) in n documents (images). Let us remind that the (i, j)element of the term-document matrix A represents the colour of i-th position in the j-th image document. 3.2. Implementation Details of Latent Semantic Indexing In this section we will describe the possible software implementation of the Latent Semantic Indexing method. Let the symbol A denotes the m × n document matrix related to m pixels in n images. The aim of SVD is to compute decomposition A = U SV T ,
(5)
where S ∈ Rm×n is a diagonal matrix with nonnegative diagonal elements called the singular values, U ∈ Rm×m and V ∈ Rn×n are orthogonal matrices1 . The columns of matrices U and V are called the left singular vectors and the right singular vectors respectively. The decomposition can be computed so that the singular values are sorted in decreasing order. The full SVD decomposition (5) is memory and time consuming operation, especially for large problems. Although the document matrix A is often sparse, the matrices U and V have a dense structure. Due these facts, only a few k-largest singular values of A and the corresponding left and right singular vectors are computed and stored in memory. The number of singular values and vectors which are computed and kept in memory 1A
matrix Q ∈ Rn×n is said to be orthogonal if the condition Q−1 = QT holds.
194
P. Praks et al. / Wavelet and Eigen-Space Feature Extraction
can be chosen experimentally as a compromise between the speed/precision ratio of the LSI procedure. We implemented and tested LSI procedure in the Matlab system by Mathworks. Following [2] the Latent Semantic Indexing procedure can be written in Matlab by the following way. Procedure Original LSI [Latent Semantic Indexing] function sim = lsi(A,q,k) % Input: % A ... the m × n matrix % q ... the query vector % k ... Compute k largest singular values and vectors; k ≤ n % Output: % sim ... the vector of similarity coefficients [m,n] = size(A); 1. Compute the co-ordinates of all images in the k-dimensional space by the partial SVD of a document matrix A. [U,S,V] = svds(A,k); % Compute the k largest singular values of A; The rows of V contain the coordinates of images. 2. Compute the co-ordinate of a query vector q qc = q’ * U * pinv(S); % The vector qc includes the co-ordinate of the query vector q; The matrix pinv(S) contains reciprocals of non-negative singular values (an pseudoinverse); The symbol ’ denotes the transpose superscript. 3. Compute the similarity coefficients between the co-ordinates of the query vector and images. for i = 1:n % Loop over all images sim(i)=(qc*V(i,:)’)/(norm(qc)*norm(V(i,:))); end; % Compute the similarity coefficient for i-th image; V (i, :) denotes the i-th row of V . The procedure lsi returns to a user the vector of similarity coefficients sim. The i-th element of the vector sim contains a value which indicate a "measure" of a semantic similarity between the i-th document and the query document. The increasing value of the similarity coefficient indicates the increasing semantic similarity. 3.3. Partial Eigen-problem The image retrieval process can be powered very effectively when the time consuming Singular Value Decomposition of LSI is replaced by the partial symmetric eigenproblem, which can be solved by using fast iterative solvers [7]. Let us assume the following relationship between the singular value decomposition of the matrix A and the symmetric eigenproblem of the symmetric square matrices A T A:
P. Praks et al. / Wavelet and Eigen-Space Feature Extraction
195
Figure 2. An example of results of the wavelet (left) and partial eigen-problem based image retrieval (right). The query image is situated in left up corner and it is related to the query LCT52XP1010229_a.jpg. The well-classified images are at positions 1, 3 and 5 for the wavelet method and at positions 1 – 3 for the partial eigen-problem method.
A = U SV T
(6)
AT = (U SV T )T = V S T U T
(7)
AT A = V S T (U T U )SV T = V S T SV T
(8)
Moreover, let us assume the SVD decomposition (5) again. Because of the fact that the matrix V is orthogonal, the following matrix identity holds: AV = U S.
(9)
Finally, we can express the matrix U in the following way: AV S + ≈ U
(10)
Here the symbol S + denotes the Moore-Penrose pseudoinverse (pinv). Let us accent that the diagonal matrix S contains only non-negative singular values for real cases; The singular values less than tol ≈ 0 are cut off by the Matlab eigs(A’*A, k) command. There is no exact routine for the selection of the optimal number of computed singular values and vectors [3]. For this reason, the number of singular values and associated singular vectors used for the partial symmetric eigenproblem was estimated experimentally, but it seems that k < 10 is suitable for real image databases. For example, we choose k = 8 for the large-scale NIST TRECVID 2006 data [11]. In contrast to the SVD approach, the size of the partial symmetric eigenproblem (the size of AT A matrix) does not depend on the number of pixels (keywords) at all. Since the number of computed singular values k << n for real problems and k is small, the image retrieval using the partial symmetric eigenproblem is more efficient [7] than the "classical" SVD approach [2].
196
P. Praks et al. / Wavelet and Eigen-Space Feature Extraction
Figure 3. An example of results of the wawelet (left) and partial eigen-problem based image retrieval (right). The query image is situated in left up corner and it is related to the query LDT70SP1010023_a.jpg. The well-classified images are at positions 1 – 3 and 7 for the wavelet method and at positions 1 – 4, 6 and 7 for the partial eigen-problem method.
Figure 4. An example of results of the wawelet (left) and partial eigen-problem based image retrieval (right). The query image is situated in left up corner and it is related to the query SCK60U12.jpg. The well-classified images are at positions 1 – 4 and 7 – 9 for the wavelet method and at positions 1 – 3, 5 and 7 for the partial eigen-problem method.
4. Experiments and Results 4.1. Experimental Data We experimented with real metallography images taken from the steel plant of Mittal Steel Ostrava plc, Ostrava, Czech Republic. In fact, we deal with sample images of con-
P. Praks et al. / Wavelet and Eigen-Space Feature Extraction
197
Figure 5. An example of results of the wawelet (left) and partial eigen-problem based image retrieval (right). The query image is situated in left up corner and it is related to the query SDK53M27. All retrieved images (except image no. 2) are well-classified by the wavelet method. All retrieved images (except image no. 4) are well-classified by the the partial eigen-problem method.
tinuously cast steel from billet device for continuous steel casting. This device produces billets of 180 mm square, 160 and 210 mm round. The closer parameters are stated in the Table 1. Steel samples from the cast billets are taken away for the device for continuous steel casting. These are crosscuts of the cast billets. These samples are conveyed into metallography lab where they are mechanically adjusted. In order to stress a sample macrostructure, crosscut etching is done. Consequently, photographs of these etched crosscuts are being taken. Commissioned on:
7 December 1993
Type:
billet, radial, two-point alignment
Heat volume:
205 tons
Casting method:
closed, through submerged nozzles and stoppers
Casting arc radius:
10.5 ; 21m (two-point alignment)
Cooling of semi-product:
water (single component)
Cutting of semis:
torch cutting
Slab marking: punching, 10-character code Table 1. Basic chosen parameters of the device for continuous casting No. 1.
Evaluation time of one image
0.36 secs.
Local feature vectors from neighborhoods
8 × 8 pixels
Type of wavelet transformation 8 TAB Johnshon Wavelet Table 2. Properties of the Statistical Wavelet-Based Classification method.
198
P. Praks et al. / Wavelet and Eigen-Space Feature Extraction
Properties of the document matrix A Number of keywords Number of documents Size in memory
458×480 = 219 840 40 67.089 MB
The SVD-Free LSI processing parameters Dim. of the original space 40 Dim. of the reduced space (k) 6 Time for AT A operation 0.64 secs. Results of the eigensolver 0.219 secs. The total time 0.859 secs. Table 3. Image retrieval using the partial eigen-problem method; Properties of the document matrix (up) and LSI processing parameters (down).
Photographs from the verification of electromagnetic steel mixing in the crystallizer have been used for verifying of the described methods. The total number of images in the image database was 83. The number of images in the training set was 20. 4.2. Experimental Results The results of image retrieval experiments are presented in Fig. 2 - Fig. 5. The resulted images are presented by decreasing order of similarity. The query image is situated in left up corner. The similarity of the query image and the retrieved image is written in parentheses. In order to achieve well arranged results, only 9 most significant images are presented. The presented shape of the crosscuts does not respond to a reality completely (they were slightly deformed at the photograph evaluation). It can be stated that these are the first results for billets 180mm square and 210mm round. The evaluated subject was the whole crosscut of billet samples. 4.3. Conclusions for Experiments Our results indicate that the both methods can automatically recognize the shape and the type of images found in our image database. The behaviour of both methods is close to the classification of a human expert. Moreover, the results of Table 2 and Table 3 indicate a possibility of real-time analysis. The first results point out that the discussed methods can be also used for the evaluation of crosscut macro structure of billet samples. In order to achieve more precise evaluation results, individual areas of a sample crosscut images should be deeply analyzed in the future work. This deeper image analyze is also important for searching metallurgical relations in images, which are hidden in the image database.
5. Conclusion In this paper, a comparison of two approaches for automatic pattern classification in images taken from a real world environment has been presented. The experimental data in the form of metallography images has been provided by the Mittal Steel Ostrava plc (Ostrava, Czech Republic). The objective of this research activity is to monitor the quality
P. Praks et al. / Wavelet and Eigen-Space Feature Extraction
199
process in the steel plant. The first classifier used for this reason (Section 2) represents image patterns by feature vectors extracted with the wavelet transformation, while the second one (Section 3) is based on the eigen-space analysis. Classification results for experiments presented in Section 4 prove a very high performance of both approaches in a real world environment. In the future, the statistical wavelet-based approach (Section 2) will be combined with the eigen-based analysis (Section 3). One can imagine that a fusion of these two methods will bring a significant improvement in terms of classification rates. Acknowledgments The research has been also partially supported by a program "Information Society" of the Academy of Sciences of the Czech Republic, project No. 1ET401940412. The work leading to this contribution has been partially supported by the European Commission under contract FP6-027026-K-SPACE. References [1] W. M. Berry, Z. Drmaˇc, and J. R. Jessup. Matrices, vector spaces, and information retrieval. SIAM Review, 41(2):336–362, 1999. [2] D.A. Grossman and O.Frieder. Information retrieval: Algorithms and heuristics. Kluwer Academic Publishers, Second edition, 2000. [3] Berry W. M., Dumais S. T., and O’Brien G. W. Using linear algebra for intelligent information retrieval. SIAM Review, 37:573–595, 1995. [4] S. Mallat. A theory for multiresolution signal decomposition: The wavelet representation. IEEE Transactions on Pattern Analysis and Machine Intelligence, 11(7):674–693, July 1989. [5] N. Muller, L. Magaia, and B. M. Herbst. Singular value decomposition, eigenfaces, and 3d reconstructions. SIAM Review, 46(3):518–545, 2004. [6] H. Niemann. Pattern Analysis and Understanding. Springer-Verlag, Berlin, Heidelberg, Germany, 1990. [7] Praks P., Machala L., and Snášel V. On SVD-free Latent Semantic Indexing for iris recognition of large databases. Multimedia Data mining and Knowledge Discovery. Ed. V. A. Petrushin and L. Khan, Springer-Verlag London Limited, 2007. [8] P. Praks, J. Dvorský, and V. Snášel. Latent semantic indexing for image retrieval systems. In SIAM Conference on Applied Linear Algebra. The College of William and Mary, Williamsburg, USA, http://www.siam.org/meetings/la03/proceedings/Dvorsky.pdf, July 2003. [9] V. Svátek, M. Labský, P. Praks, and O. Šváb. Information extraction from html product catalogues: coupling quantitative and knowledge-based approaches. In Dagstuhl Seminar on Machine Learning for the Semantic Web. Research Center for Computer Science, Wadern, Germany, http://www.smi.ucd.ie/Dagstuhl-MLSW/proceedings/labskysvatek-praks-svab.pdf, February 2005. [10] A. R. Webb. Statistical Pattern Recognition. John Wiley & Sons Ltd, Chichester, UK, 2002. [11] P. Wilkins, T. Adamek, P. Ferguson, M. Hughes, G. J. F. Jones, G. Keenan, K. McGuinness, J. Malobabic, N. E. O’Connor, D. Sadlier, A. F. Smeaton, R. Benmokhtar, E. Dumont, B. Huet, B. Merialdo, E. Spyrou, G. Koumoulos, Y. Avrithis, R. Moerzinger, P. Schallauer, W. Bailer, Q. Zhang, T. Piatrik, K. Chandramouli, E. Izquierdo, L. Goldmann, M. Haller, T. Sikora, P. Praks, J. Urban, X. Hilaire, and J. M. Jose. K-space at trecvid 2006. In Proceedings of the TRECVid Workshop, Gaithersburg, Maryland, USA, 2006. NIST, http://wwwnlpir.nist.gov/projects/tvpubs/tv6.papers/k-space.pdf.
200
Information Modelling and Knowledge Bases XIX H. Jaakkola et al. (Eds.) IOS Press, 2008 © 2008 The authors and IOS Press. All rights reserved.
Semantic knowledge modeling in medical laboratory environment for drug usage: CASE study Anne TANTTARI, Kimmo SALMENJOKI Department of Computer Science, University of Vaasa Box 700, 65101 Vaasa, Finland Lorna UDEN Department of Computer Science, University of Staffordshire Beaconside Staffordshire, UK Abstract. In this paper we will consider the usage of knowledge modelling and design with conceptual design and analysis in improving the knowledge description of medical data. There exists many XML and information system (IS) based approaches for producing easy to use, reliable, trustworthy and coherent medical systems and services, like the concepts of HL7. In computer applications web, web services and semantic web based technologies and tools are being used in the business and ecommerce scenarios to produce user-focused, (business) process oriented services in future medical IS. Our case is focused on laboratory exams and their knowledge analysis for predicting and analyzing the usage of various drugs by patients. We show with our case the role of semantic knowledge modelling in coordinating various medical services in a real hospital setting, and hope to extend our work in the future towards semantic web based applications that improve and ease the patient treatments in the IS of the future hospitals.
1. Introduction Traditionally medical care is highly technology and information technology (IT) oriented. In Finland the coordination and collaboration between medical organizations have been advocated and increased in the last ten years. In last years, the overall process of patient treatment and its integration with various existing IS has caught the main attention of development. In this paper we will focus on the knowledge modeling and design of laboratory diagnosis. We describe the usage of semantic knowledge design for this case. We will describe more generally the role and possibilities of present knowledge modeling and design, mainly originating from the software development and system theoretical scenarios [6].
2. Satellite model for the Combined Information of the Laboratory Exams and Drugs “A conceptual model is a model of a subject area or area of knowledge, sometimes called a domain, that represents the primary entities (the things of the domain), the relationships among entities, the attribute values (sometimes called properties and property values) of the entities and the relationships, and sometimes rules that associate entities, relationships, and attributes (or all three) in more complicated ways.”[2]. Semantic web with enhanced knowledge modelling provides the generic setting of refining the information granularity for any domain
A. Tanttari et al. / Semantic Knowledge Modeling in Medical Laboratory Environment
201
of applications [1]. Using databases which are traditionally modelled with ERD diagrams as sources of data, we will use information analysis in describing the inherent properties and interrelations of laboratory data with the de facto standard of Finnish medicine descriptions in Pharmaca Fennica [12]. Pharmaca Fennica is an encyclopaedia produced by the Pharmaceutical Information Centre containing all the drugs existing in the Finnish market. For each drug, this contains its ingredients and the related diseases and traumas that can be cured with that specific medicine. The information of this book is reliable and fully covering all materials as well as available online. Ultimately, the improved knowledge awareness will lead towards shared ontologies and unified knowledge sources [9, 11]. In defining the properties of our information entities with their relations, we will form the taxonomy for laboratory test usage. This analysis was done informally and intuitively using the domain expertise of the real world experts. The outcome of this analysis was a formal model of our problem. Although there are various approaches for concept analysis, we believe that the most suitable model to be used is the satellite model, which is based on the inherent structure and consumption models of the laboratory processes and treatment approaches. This is because the satellite model is good in embracing various roles and models into a unified knowledge description. In our case, the knowledge contained in the model will be shared and utilized by various persons and medical process in daily work. There are varying ways of using satellite model in practice. When we look at the medical descriptions of laboratory tests they share several features. By features we mean not only fixed facts but also the connections arising from the domain context. In looking at the documents we found, for example, that in certain laboratory tests the bentsodiatsepin is an element of attention in the test description. However, the present descriptive documents do not contain this concept based either in its universal nature or in its typical property classification when comparing it with other related features. Hence these laboratory tests share the common property that they are a group of tests that are used for finding the usage of bentsodiatsepin as a drug. Of course, these tests also have a multitude of other knowledge and information properties, which will be here left out due to complexity as well as to maintain clarity of our generic approach [3, 4]. Using the medical drug table and therapy group classification of Pharmaca Fennica [12] we have developed the detailed satellite model. In this paper we only give a demonstrative excerpt from the overall model in [16]: Neurological disease
Temesta Wyeth BENTSODIATSEPIN
dizziness
Loratsepam Xanor Pfizer
anxiety
psychopharmaceutical drug depression
Diatsepam
Alprox Orion Pharm
Oksatsepam
Alpratzolam Generics Merck NM
Xanor Depot Pfizer
Figure 1.
Alpratsolam
Opamox Diazepam Desitin Desitin
Oxamin Oxepam
Except from the satellite model, full model available in [16]
In the higher part of the medical hierarchy of Pharmaca Fennica the drug therapy main class and therapy subclasses have been used as a basis for the classificaion. Using the produced satellite model of Figure 1 we have collected a list of all those laboratory tests addressing
202
A. Tanttari et al. / Semantic Knowledge Modeling in Medical Laboratory Environment
bentsodiatsepin that appear in the Vaasa Central Hospital [5]. Using this model, we have come up with a following category of the laboratory exams related to bentsodiatsepin [15]: Laboratory test Medicine and poison test Bentsodiatsepin S-Diatsepam S-Alpratsolam S-Oxatsepam S-Nitratsepam S-Klobatsam S-Klonatsepam B-Drug_monitoring U-Drug_screening (qual) U-Drug_screening U-Drug, check and screening Figure 2.
Categories for bentsodiatsepin in lab exams
Conceptually this is a single heritance based concept system. When exams are ordered from the laboratory in the real life cases the measurement of bentsodiatsepin can be requested by various exam names. To cover these cases we have to use a multiple heritance description, where various super classes inherit two or more properties from the higher order concepts in the system, as shown in Figure 3. Laboratory test Medicine and poison test Bentsodiatsepin Diatsepam Name of the medicine Diapam Orion Pharma Diazepam Desitin Desitin Gastrodyn Comp Leiras Medipam Ratiopharm Relapamil Orion Pharma Setsolid Alpharma Setsolid Novum Alpharma Setsolid Prefill Alpharma Vertipam Orion Pharma Name of the test S-Diatsepam B-Drug_monitoring U-Drug_screening (qual) U-Drug_screening U-Drug, check and screening Figure 3.
Inherited categories for bentsodiatsepin in lab exams
The building of these hierarchical systems can either proceed as top down development process or as bottom-up development process. In the top down approach classes are refined into smaller, more specific classes. The higher concept hence processes less features than the lower ones. Depending on the case, we might also take an intermediate approach, where we develop the classification starting from the middle of the hierarchy with an origin that is highly meaningful from the domain context point of view. In general, this is done by a combination or scenario based development, see [8, 10].
A. Tanttari et al. / Semantic Knowledge Modeling in Medical Laboratory Environment
203
3. Using Semantic Knowledge Approach with Laboratory Documentations As mentioned earlier, various semantic web approaches have permeated most IS application, such as the use of XML as data layer in classical IT applications [14]. In the next section we describe how the satellite model of Figure 1 is translated to its technical implementation using RDF and its related expression formulations. Next we will illustrate two simple examples of utilizing our satellite model of previous chapter in real life medical documentation: 3.1 Changes in patient treatment process by doctor In the medical laboratory, finding the most relevant test from a multitude of tests often creates practical problems for doctors. Using the topic or test name based classification requires additional information in order to be successful. Because of the continuous changes in naming, it is critical to keep the system up to date all the time. Doing incorrect or unnecessary exams is bad, both from the patient’s and economical points of views. The following diagram exemplifies the approach when the additional knowledge of the model is being used: Disease or trauma
Figure 4.
Name of the medicine
Active drug
Name of the laboratory
The process how the doctor can find a name of the correct laboratory test
How ever to maintain this kind of system is complicated due to the constant changes in the data. Besides new drugs and those withdrawn from the market the tests themselves are continuously improved and some tests become obsolete. To ease this maintenance the shared knowledge between the Pharmaceutical Information Centre and laboratories is crucial. Here, again, semantic web provides means of sharing and combining the information sources. This approach is well accepted and used in the research of advanced knowledge retrieval on the web in general as well [21]. 3.2 Finding a lab exam related to a specific medicine When treating a patient the doctor has to know the link between the patient’s disease and related drugs. In reality, finding this link is a manifold communicative and systematic process between the domain expert and a patient with his/her own needs and ideas. Overall this process can be generically described as below: Name of the medicine
Figure 5.
Name of the laboratory test
The correct laboratory test related to a specific medicine via the model
When one wants to enhance this process with computer based knowledge, the conceptual model has to be both simple, easy to maintain and apt for automated processing. Here the key role of semantic metadata is most important. From the practical point of view the span of the visible concepts should not be too wide or deep. From the patient records the doctor will possess the knowledge of all the drugs that the patient is using at that moment. So, in most simple case, it would be sufficient to combine this knowledge with its specifically related laboratory tests to reduce the complexity and improve the quality of the medical care by simple automation.
204
A. Tanttari et al. / Semantic Knowledge Modeling in Medical Laboratory Environment
Various expert systems have been already developed using this approach (see [13]), but in our case, we want to improve the explicit usability of the deeper knowledge provided by the higher interlinking of these two variable information sources using semantic web methodologies and tools. In practice, this would require a search engine that would recommend a suitable drug based on the patient’s blood test and medical history as demonstrated in Figure 5. Of course, there exists a multitude of other viable medical cases for using semantic web in medical IS beyond these two cases mentioned here. 4. Using RDFS/RDF with Satellite Model for Practical Laboratory Work 4.1 Presenting the knowledge of laboratory data for knowledge processes For implementing the case of section 3.2, we next turn to the RDFS/RDF based knowledge descriptions of the laboratory documentation. As an example we show how RDFS/RDF statements related to finding the laboratory test will be written. Statements of this kind will form the basis for our systemized medical ontology: Table 1. Thesaurus linking medicines and lab exams
Name of the medicine Diapam Orion Pharma
Laboratory test S-Diatsepam
Frisium Sanofi-Aventis S-Klobatsam
This is directly derived from the satellite model of Figure 1. Here from the resource, property type and its values we can form the following sentences in English: - Diapam Orion Pharma named drug is researched by an exam named Sdiatsepam in the laboratory - Frisium Sanofi Aventis drug is researched by an exam named S-klobatsam Using N3 (Notation 3), which is a non-technical presentation of these so called RDF triples spells as: - [<#drug>”Diapam Orion Pharma”;<# exam >”S-Diatsepam”] - [<#drug >”Frisium Sanofi Aventis”;<# exam >”S-Klobatsam”], where # identifies a URI-address. For ultimate clarity these can be automatically reformatted as diagrams:
http://www.Pharmaceutical_Information_Centre/Na of_the_medicine/ Diapam Orion Pharma
S-Diatsepam
laboratory_handbook="http://www.vshp.fi/laboratory_handbook/Name of the laboratory test# http://www.Pharmaceutical_Information _Centre/Name_of_the_medicine/ Frisium Sanofi Aventis
S-Klobatsam
laboratory_handbook="http://www.vshp.fi/laboratory_handbook/Name of the laboratory test# Figure 6. Two triplets for finding exam names based on medicine names
A. Tanttari et al. / Semantic Knowledge Modeling in Medical Laboratory Environment
205
The above three presentations are used in the application domain by the experts as well as providing knowledge aware applications for describing the domain and case related processes for chapter 3. In building semantic applications various programming tools, like Jena, Joseki and JADE, allow building agent based systems that use the knowledge in dynamic manner beyond the primitive searching discussed in sections 3.1 and 3.2. 4.2 Example of using RDF for the knowledge of laboratory data and processes Using the previously discussed top down approach in detailing the information of the satellite model, we will next give the technical details of RDF as a practical example. When RDF bag elements are applied for our example data, we obtain the following RDF graph: rdf:bag
rdf.type
S-Diatsepam
rdf:_5
examines Diapam Orion Pharma
Name of the laboratory test
B-Drug_monitoring
rdf:_1 rdf:_2
U-Drug_screening_(qual) rdf:_3 rdf:_4
U-Drug_screening
U-Drug, check and screening
Figure 7. Description of a laboratory exam with RDF:bag as a graph
This diagram shows that the effect of the drug Diapam Orion Pharma can be examined by five different toxicity tests. The presented example gives hints as how the RDF- based knowledge could be used in the real medical cases with processes of section 3, see [19] for more details. 4.3 Presenting more complicated relations with RDFS graphs In XML usage the evolutionary improvement of information abstraction leads to the use of XML Schemas. Likewise in the semantic web setting, the growing of the RDF information will enable the developers of the documentation system to see the deeper relations between drugs and their treatment with related medical processes. For systematically describing these relations we will use RDF Schemas (RDFS) analogously to XML Schemas [17, 18]. The most important conceptual relation that RDFS represents is hypogyny. Hypogynies can be described using classes, instances and properties in RDFS (RDF-Schema). Figure 8 shows an example of multiple inheritance between the concepts in our case study with RDFS:
206
A. Tanttari et al. / Semantic Knowledge Modeling in Medical Laboratory Environment
rdf:Property Medicine and oison test
rdf:type
rdf:type
rdfs:domain rdfs:subClassOf
rdf:type rdfs:domain
Name of the laboratory test
name
rdfs:range rdfs:subClass
drug
Bentsodiatsepin
Diatsepam
person in charge _name rdfs:subClassOf
Alpratsolam
rdfs:range rdf:type
Oxatsepam
Name_of_the_medicine
rdf:type rdf:type
rdf:type rdf:type Rdfs:Class
Figure 8. RDF-schema for the ontology of bentsodiatsepin
Above bentsodiatsepin is defined as a subclass of ”Name of the laboratory test”. Each class typically contains one or many instances, for example this test is related to several other “Diatsepam” related tests. This is enabled as any resource can be an instance of several classes. With this feature any bentsodiatsepin related tests can be found with related medicines. The role of other properties is to describe attributes or relations to other resources. In RDFS we specify the scope or domain for these attributes with rdfs:domain values. This domain is also a resource by itself, which can thus appear as a subject for another property. Range specified with rdfs:range can appear as an object of an entity. For example, in our case, any medicine can appear as a subject for a toxic lab test and brand of medicine can appear as an object in related domain knowledge sentences written in RDFS. These will form the basis for the logical processing of the RDF statements of section 4.2. Ultimately, the domain inherent rules and processes (contained in the classical information systems) could be technically spelled out as OWL rule based sentences, providing a basis for the dynamic operation of the knowledge agents in knowledge processing systems beyond the classical expert systems, see [7, 21].
5. Conclusions In this paper we have demonstrated our approaches in using semantic web in the context of medical laboratory tests. It shows the principal advantage of knowledge analysis and provides a technological basis for developing knowledge supported and intensive medical treatment processes and systems. In subsequent papers we will address the medical treatment processes in general and their IS system using semantic web based tools and technologies. As the frameworks and approaches of shared knowledge get more popular, we will continue the research in systemizing the medical processes via their extended functional usage using these semantic web based knowledge models and descriptions together with web services on the software components. After this, our focus will move towards expert assistance, hospital process automation and later agent based approaches in consuming the already vast existing digital knowledge of medicine and its meaningful sharing in various medical cases.
A. Tanttari et al. / Semantic Knowledge Modeling in Medical Laboratory Environment
207
References [1] [2] [3] [4] [5] [6] [7] [8]
[9] [10] [11] [12] [13] [14] [15]
[16]
[17]
[18] [19] [20]
[21]
Berners-Lee, Tim (2003): Web Services - Semantic Web. Architectural layers to International WWW Conference. Daconta, Michael C, Leo J. Obrst & Kevin T. Smith (2003): The Semantic Web. John Wiley & Sons, Inc. ISBN 0-471-43257-1. Dublin Core Metadata Initiative (2006): DCMI Metadata Terms. DCMI Recommendation. URL:< http://dublincore.org/documents/dcmi-terms/> Hyvönen, Eero (2005): Finnish semantic web ontologies (FinnONTO-project) (in Finnish). Documentation for laboratory exams in Vaasa Central Hospital, the laboratory of clinical chemistry and microbiology (in Finnish and Swedish): Handbook of laboratory exams (1999): Vaasa Central Hospital, the laboratory of clinical chemistry and microbiology (in Finnish). Luck Michael, Ashri Ronald, D’Inverno Mark:. Agent-based Software Development, Wiley, 2004 Noy, Natalia L. & Deoborah McGuiness (2001): Ontology Development Guide 101: A Guide to Creating Your First Ontology. Medical Subject Headings: http://www.nlm.nih.gov/mesh/ Montague Institute Review (2001): Managing taxonomies strategically. Open Biomedical Ontologies: http://obo.sourceforge.net/ Pharmaca Fennica I (2005): Pharmaceutical Information Centre Inc. ISBN 0355-7472. S. Russell, P. Norvig (2005): Artificial intelligence, A modern approach, Prentice-Hall, 1995 Salminen, Airi (2005): Building Digital Governement by XML. University of Jyväskylä. The Association of Finnish Local and Regional Authorities (2006): Nomenclature for laboratory examinations 2006 Social- and health care, Classifications and nomenclature (in Finnish). Tanttari Anne (2006): Semantic modelling and utilization of concept definitions: considered laboratory handbook applications for medicine and poison testing in laboratories of the Vaasa Central Hospital (Master thesis in Finnish) University of Vaasa, 2006 World Wide Web Consortium (2000): Resource Description Framework (RDF) Schema Specification 1.0. Candidate Recommendation 27 March 2000. Editors: Dan Brickley, University of Bristol & R.V. Guha, Epinions. World Wide Web Consortium (2004): RDF Vocabulary Description Language 1.0: RDF Schema. World Wide Web Consortium (2004): RDF Primer. W3C Recommendation 10 February 2004. Editors Frank Manola & Eric Miller. World Wide Web Consortium (2004c): RDF Semantics. W3C Recommendation 10 February 2004.Editor Patrick Hayes. Series Editor Brian McBride. Hewlett-Packard Laboratories. World Wide Web Consortium (2004): Semantic Web emerges as commercial-grade infrastructure for sharing data on the Web. [online].
208
Information Modelling and Knowledge Bases XIX H. Jaakkola et al. (Eds.) IOS Press, 2008 © 2008 The authors and IOS Press. All rights reserved.
Towards Automatic Construction of News Directory Systems Bin Liu, Pham Van Hai, Tomoya Noro, and Takehiro Tokuda {ryuu, hai, noro, tokuda}@tt.cs.titech.ac.jp Department of Computer Science, Tokyo Institute of Technology Meguro, Tokyo 152-8552, Japan
Abstract. Currently news sites and news index sites basically provide streams of news with simple classifications or keyword based searching with publication data. When we try to search for news involving a number of potential keywords or unknown keywords, then the task becomes manually tedious or almost impossible. We present automatic methods for constructing news directory systems which contain collections of news index information with flat or hierarchical classification structures. This directory structure enables us to reach the news articles without knowing the keywords exactly. We implemented and evaluated one sample news directory system.
1 Introduction On the Web it is not difficult for an ordinary user to manually access to a small number of domestic news sites in various countries/regions, such as New York Times, Guardian Unlimited and Straits Times, or to access to a number of global news sites covering various parts of our earth from their eyes, such as CNN International, BBC world, and Reuters. Also news index sites such as Google News US, Google News Canada and Google News Australia, tailored to concerns of intended audience countries/regions, provide index information to a large number of related news source sites. However, these news sites and news index sites provide two basic methods for access. Namely they provide streams of news with simple classifications such as Asia/Pacific section and Business section, or keyword based searching with publication data such as publication date and publisher name. Hence, if we would like to ask a global question, not tailored to particular audience countries/regions, involving a number of potential keywords or unknown keywords, keyword based searching for news articles may be manually tedious or almost impossible. For example, the question may be ”what kind of country/region names are frequently mentioned together with particular disease names at news sites in our world?” We present an automatic approach to these questions based on the idea of news directory systems. A news directory system is a collection of news index information automatically retrieved from various news sites on our earth and automatically classified into a flat or a hierarchical directory system. Users can customize directory systems, design the structures, and give definitions to directories for themselves. Index information would be classified into these directories automatically so that users can find information they need more quickly with these systems. We implemented and evaluated one sample news directory system. This
B. Liu et al. / Towards Automatic Construction of News Directory Systems
209
news directory systems allow us, for example, to take a look at news articles containing typical disease names such as Bird flu or AIDS in each instance subdirectories under Disease directory. The directory structure also enables us to reach the news articles without knowing the keywords exactly. The organization of the rest of this paper is as follows. In Chapter 2 we explain the overview of news directory systems. In Chapter 3 and 4 we respectively give methods for automatic construction of directory structures, methods for directory definitions and automatic placement. In Chapter 5 and 6 we explain our experimental evaluation, concluding remarks and future work respectively.
2 A News Directory System A news directory system has following subsystems for a user to take a look at news article index information automatically placed according to the given directory structures and the definitions of articles to be contained in each directory.
Figure 1: System Structure
Subsystem 1 Automatic retrieval of news article index information from news sites. Currently, we get the index information in two ways. To RSS news sites, we use RSS to retrieve news article pages. To those not providing RSS, we utilize the URL structure of news sites to retrieve news article pages. Subsystem 2 Definition of a directory structure with one level flat classification or multilevel tree classification. We can freely construct the directory structure as we wish or we can also use directory structures similar to that of WordNet [6] or Wikipedia [5]. Subsystem 3 Handling of definitions of news articles to be contained in each directory. We need to give each directory a definition with keyword expressions to specify articles.
210
B. Liu et al. / Towards Automatic Construction of News Directory Systems
Subsystem 4 Automatic placement of news article index information in each directory. Once the directories’ structures and definitions are given, the system will classify those index information collected and place them into corresponding directories automatically. Subsystem 5 A subsystem for query processing and visualization. Users can search for news articles they are interested in by following the directory structures or give search keywords directly. The search results can be also shown in a map. The structure of our news directory system is shown in Fig. 1.
3 Directory Structures In news directory systems we use one-level flat directory structures or multi-level tree directory structures. Typical examples of one-level flat directory structures may be as follows. • Classification of natural disasters such as typhoon and earthquake. • Classification of human diseases such as diabetes and malaria. Typical examples of multi-level tree directory structures may be as follows. • Classification of locations such as countries/regions on the earth and outside of the earth. • A small classification tree constructed from the large classification tree such as WordNet or Wikipedia classification.
Figure 2: Directory Disease
Figure 3: Directory Countries/Regions
An example of one-level flat directory structure is shown in Fig. 2 and an example of multi-level tree directory structure is shown in Fig. 3. Users can also build their original directory structures manually. Here we will give some methods to bulid directory structures with exsiting resources. Method 1 We use open knowledge collection of classifications by humans, such as Wikipedia and WordNet, to build an initial collection of instance names belonging to one category. Method 2 Our method of building multi-level tree directory is as follows. We need a small set of basic words. Such a set of basic words may be subject words in New York Times Topics Index or a subset of Longman defining vocabulary [4] or a subset of Oxford defining vocabulary [3]. For a given set of basic words we construct a small classification tree as follows.
B. Liu et al. / Towards Automatic Construction of News Directory Systems
211
1. We retrieve full paths of all basic words in the WordNet tree. 2. We construct the initial small tree using the full paths obtained in the step 1. 3. We construct the small tree by deleting all non-basic words having exactly one son node from the initial small tree. A process of construction of a multi-level tree directory is shown in Fig. 4.
Figure 4: Composition of a small classification tree
4 Directory Definition and Automatic Placement We need definitions of news articles to be contained in each directory and an automatic method of placing those news article index information into corresponding directories. 4.1 Directory Definitions Our default definition of a news article A to be contained in a directory B is that the article A has an occurrence of the word B. In addition to default definitions of single word occurrences, we may use explicit definitions of a news article in a directory using the expressions defined by following extended context-free syntax rules with repetition operator {} representing zero or more times of repetitions. expression → (term) {OR (term)} term → factor {AND factor} factor → (phrase)|(NOT phrase) phrase → word {SPACE word} word → character {character}
212
B. Liu et al. / Towards Automatic Construction of News Directory Systems
This expression allows us to define news articles having slightly more complicated word occurrences. For example, we may write a definition using the following expression. ((football) AND (NOT american football)) OR ((soccer)) This expression means that an article A is to be contained in the directory, if A contains the word ”football” but not ”american football” or A contains the word ”soccer”. The same expression may be written briefly as follows. 1. football AND (NOT american football) 2. soccer 4.2 Automatic Placement The task of automatic placement consists of two phases. In the first phase, we construct two finite-state automata M1 and M2 . In the second phase we actually classify news articles into directories using automata M1 and M2 . The automaton M1 recognizes each phrase using the transition by one character. The automaton M2 classifies a news article into corresponding directories according to expressions with the acceptance/non-acceptance result of each phrase by the automaton M1 . 4.2.1 The First Phase
We construct two automata M1 and M2 as follows. M1
1. We collect all defined phrases d1 , d2 , ..., dn consisting of characters and construct corresponding finite-sate automata M11 , M12 , ..., M1n , which have transition labels of one character and recognize defined phrases d1 , d2 , ..., dn respectively. 2. We construct a finite-state automaton M1 by applying subset construction method to the set of automata M11 , M12 , ..., M1n .
M2
1. We collect all expressions e1 , e2 , ..., en consisting of defined phrases and decompose each expression to terms t11 , t12 , ..., tn1 , ..., tnm 2. For each term, we construct a sequence consisting of sorted factors. Using the sequences, we construct corresponding finite-sate automata M21 , M22 , ..., M2k whose transition labels are phrase or NOT(phrase). 3. We construct a finite-state automaton M2 by applying subset construction method to automata M21 , M22 , ..., M2k . For the sample expressions of Section 4.1, we can construct M1 and M2 as follows.
B. Liu et al. / Towards Automatic Construction of News Directory Systems
213
4.2.2 The Second Phase
In the second phase we actually classify news articles into directories using automata M1 and M2 . Recognition of a phrase by M1 We run the automaton M1 with the initial control point in the initial state of M1 as follows. • If a phrase consists of one word, then the behavior of M1 is same as an ordinary automaton. • If a phrase consists of two or more words separated by spaces or other delimiters, then the behavior of M1 is as follows. – Each time we meet with a delimiter, then we introduce one more control point for recognizing the remaining postfix of the phrase from the initial state of M1 . Classification of news articles by M2 We run the automaton M2 with the initial control point in the initial state of M2 as follows. The input string consists of sorted phrases accepted by the automaton M1 . Each term of expressions has corresponding directories. If a control point reaches the end of the final phrase of a term by looking at the entire input string, then we associate the article with the corresponding directories. Otherwise, no corresponding directories exist. The basic behavior of M2 is as follows. If a state S has the control point for the first time, and the state S has a number of transition labels L1 , ..., Ln and corresponding next states N(L1 ), ..., N(Ln ), then we create one copy of the control point in each next state of S and delete the control point of S . Additional behavior is determined according to the transition label L and the first phrase p of the input string as follows. • If the transition label L is p, then the input string becomes the rest of the input string and the control point is in N(L) as above. • If the transition label L is NOT p, then we delete the control point in the next state N(L). • If the sorting ordering of the phrase of the transition label L is smaller than that of p, then we delete the control point in the next state N(L). • If the sorting ordering of the phrase of the transition label L is larger than that of p, then the input string becomes the rest of the input string and we move the position
214
B. Liu et al. / Towards Automatic Construction of News Directory Systems
of the control point from the next state N(L) to S . This control point may go to the next state N(L) when the first phrase of the input string becomes L after deletion of first phrases.
5 Experimental Evaluation 5.1 Construction of Directory Structures We constructed a sample news directory system having country/region directories, disease directories, natural disaster directories, energy resource directories, and sport directories. For our future use we also constructed a small classification tree of 885 nodes with 624 basic words from Longman defining vocabulary and 261 non-basic words from WordNet. Parts of WordNet tree and our constructed small tree near ”animal” are shown in Fig. 5 and 6. This small tree may serve us as a small classification tree of news articles.
Figure 5: A WordNet tree near ”animal”
Figure 6: A small classification tree near ”animal”
5.2 Automatic Placement We automatically classified 5,657 news articles collected from June 2006 to July 2006 from 21 news sites of 17 countries/regions into a countries/regions directory structure. We manually evaluated the precision rate and recall rate of our automatic placement method using country/region classification of 500 news articles as shown in Table 1. Of automatically classified 500 articles, 453 articles are appropriately placed. 12 articles mentioning country/region names are not classified into any country/region, because our definition of country/region names was primarily based on UN list of country names. 35 articles not mentioning country/region names are classified into countries/regions, because company names, event names, and news source names may contain country/region names. 5.3 Visualization and Analysis Out news directory system has a visualization subsystem so that users can understand the result visually. For example, we can represent the frequency level of co-occurrence of country/region names and particular words such as Bird flu on a world atlas using Google Maps
B. Liu et al. / Towards Automatic Construction of News Directory Systems
215
Table 1: Result of Automatic Classification 500 articles articles classified appropriately inappropriate articles not classified misclassified 453 12 35
as shown in Fig. 7. The co-occurrence of country/region names and Bird flu in a news article does not necessarily mean that Bird Flu epidemic is taking place in that country/region. However, this map shows that some country/region names are more frequently mentioned together with Bird flu than other country/region names.
Figure 7: A visualization map
Based on 5,657 news articles, disease name such as Bird flu is most frequently mentioned in countries/regions such as Indonesia and China. While disease name such as Cancer is most frequently mentioned in countries/regions such as United States and Australia. Table 2 shows the frequency of country/region names with some of human disease names and natural disaster names. Comparisons of our approach with existing approaches are as follows. For the classification of news articles, Bayesian classifications [2] may be used. However, the result of Bayesian classification is not deterministic or not predictable in general. We need to make our system’s behavior predictable. String matching algorithms such as Aho-Corasick algorithm [1] may be used for the classification of articles into directories. However string matching algorithms usually detect the occurrence of, for example, ”pen” in the word ”pencil” of a text. We need to avoid this partial matching in our system.
6 Conclusion We have presented automatic methods for constructing news directory systems. Our news directory systems allow us to search for news involving a number of potential keywords or unknown keywords.
216
B. Liu et al. / Towards Automatic Construction of News Directory Systems
Table 2: Frequency of country/region names together with particular words Category Bird flu Country Code IDN CHN THA VNM USA ESP AUS MYS IND LAO GBR NER HUN ZMB MMR KOR Article Count 72 35 28 14 12 7 7 5 5 4 4 3 3 3 3 2 Category Cancer Country Code USA AUS GBR KOR CHN ITA SGP THA FRA JPN LBN AUT VNM SWE SYR IRL Article Count 33 26 11 6 6 6 5 4 3 3 3 3 3 2 2 2 Category Tsunami Country Code IDN AUS THA JPN USA SGP DEU MYS VNM NLD GBR PHL IND CHN LKA FRA Article Count 147 7 7 4 4 4 3 3 3 2 2 2 2 1 1 1 Category Earthquake Country Code IDN CHN PHL USA JPN PAK SGP IND IRN TUR KWT AUS FRA TON EGY GIN Article Count 142 17 15 15 14 12 8 7 6 5 5 5 2 2 2 1
As our future work, we will extend our approach as follows. Improvement of precision for automatic classification According to the experiment, the rate of appropriate classification is 90%, we could improve it if we make the system recognize proper nouns, and we can also have more precision results if we give more definitions to directories in the system. Automatic definitions Currently, we define every directory in the system manually, it is really a costly and tedious work. But it is also one of the most important steps which will affect the automatic classification directly. If we use the relations between words in WordNet, it will help us in giving directories definitions. Multi-lingual searching We can construct a multi-lingual news directory system containing English, French, Chinese and Japanese news index information with the same classification structures, so that we can reach French news articles with the help of directory structures, and get ideas of the original contents with the help of Google Language Translation Tools from French to English.
References [1] Alfred V. Aho and Margaret J. Corasick, Efficient string matching: an aid to bibliographic search, CACM, 18(6), 333-340, June 1975. [2] Jennifer Hoeting, David Madigan, Adrian Raftery and Chris Volinsky, Bayesian Model Averaging, Statistical Science 14, 382-401, 1999 [3] A. S. Hornby and Michael Ashby, editors. Oxford Advanced Learner’s Dictionary of Current English. Oxford University Press, 2005. [4] Paul Proctor, editor. Longman Dictionary of Contemporary English. Longman, 2005. [5] Wikipedia, http://en.wikipedia.org/wiki/Main Page [6] WordNet, http://wordnet.princeton.edu/
Information Modelling and Knowledge Bases XIX H. Jaakkola et al. (Eds.) IOS Press, 2008 © 2008 The authors and IOS Press. All rights reserved.
217
A System Architecture for the 7C Knowledge Environment Teppo RÄISÄNEN, Harri OINAS-KUKKONEN Department of Information Processing Science, University of Oulu, Finland [email protected], [email protected] Abstract. This paper presents an information system architecture for the 7C model for organizational knowledge creation and management. The architecture is derived from the requirements that the 7C model posits. The architecture presented here comprises three layers: the conceptual layer, which discusses fundamental principles of the model, the technology layer, which tackles potential implementation technologies for the environment, and the application layer, which describes possible applications in the environment.
1 Introduction Knowledge management has received great attention both among practitioners’ and researchers’ literature for a longer period of time (see, e.g. [1][19][22][26][30][41]). More recently, collaborative approaches for managing knowledge have been proposed [44], suggesting that new knowledge is being created in group-efforts among many people instead of a few experts only [44]. This paper approaches knowledge management through a conceptual framework known as the 7C model [37]. This model suggests that knowledge is produced through the interaction of individual and social knowledge, as well as explicit and tacit knowledge. As the 7C model puts special emphasis on the social aspects of the knowledge management we will try to identify and analyze those new technologies that offer support for them. The research approach adopted for this paper is design science [25][20], in which IT artifacts are build and evaluated. March and Smith [25] recognize four types of design science products, namely constructs, models, methods and implementations. This paper describes a construct, namely an overall information system architecture for the 7C model. More specifically, systems development as a research methodology consists of five parts [31]: 1) constructing a conceptual framework, 2) developing system architecture, 3) analyzing and designing the system, 4) building the (prototype) system, and 5) observing and evaluating the system. In line with this definition, the research described in this paper is part of a larger system development research effort. According to March and Smith constructs “form the vocabulary of a domain”, and “they constitute a conceptualization used to describe problems within the domain and to specify their solutions” [25]. The 7C conceptual framework has been originally described in [37]. The contribution of this paper lies in the system architecture, which together with the conceptual framework, may be regarded as a whole construct [25]. Later, following the framework presented here the 7C knowledge environment will be implemented and experimented. Nunamaker et al. [31] define that system architecture is supposed to: 1) define a unique architecture design for extensibility, modularity, etc., and 2) define functionalities of system components as well as interrelationships between them. They also state that careful system requirements definition should be made and that the requirements should be
218
T. Räisänen and H. Oinas-Kukkonen / A System Architecture for the 7C Knowledge Environment
measurable. For this reason, we aim at identifying the requirements for the overall 7C architecture, and then present the architecture using layers, integrating the functionalities and interrelationships of system components within the architecture. The paper is organized as follows. Chapter 2 describes the 7C conceptual framework. Chapter 3 analyses the framework in order to define requirements for the 7C information system architecture. Chapter 4 summarizes the requirements to recognize such concepts that the architecture must implement. Chapter 5 presents possible implementation technologies which are able to meet the concepts. Chapter 6 and 7 discuss example applications and the contribution of the paper. And finally, chapter 8 concludes the paper.
2 The 7C Model in a Nutshell The 7C model [37] for understanding organizational knowledge creation suggests that the following seven Cs play a critical role in the creation of organizational knowledge: Connectivity, Concurrency, Comprehension, Communication, Conceptualization, Collaboration, and Collective intelligence. Technologically, the benefit is realized through the fluent connectivity that the Internet technology provides with information and people for potentially several concurrent users (the 1st and 2nd Cs). The World Wide Web and its hypertext functionality to promote options and allow freedom of choice with contextual support provides users with a rich environment for comprehending (the 3rd C) and communicating (the 4th C) the information they find. Knowledge is conceptualized (the 5th C) as knowledge artefacts, which serve as a collaboration vehicle through interaction between information producers and consumers, within a team of co-workers or among other stakeholders (the 6th C). All of these six preceding Cs contribute to the growth of “togetherness” or collective intelligence (the 7th C) [37]. The creation of organizational knowledge is not a linear process, but rather a multicycle spiral process [37]. See Fig. 1. The framework assumes that connectivity of all stakeholders with the joint information space and people potentially concurrently is provided in a technologically sound manner, e.g. through the Web, Internet, wireless, mobile and other technologies. The 7C model follows Nonaka and Takeuchi [30] in that the integration of individual and social orientations (in their terminology individual and organizational) are emphasized, and that knowledge is assumed to be created through interaction between tacit and explicit knowledge. The model follows Engelbart [13] in the outcomes of the Comprehension, Communication and Conceptualization sub-processes. Individual
Social
Kw transfer Communication Tacit knowledge Comprehension Explicit knowledge
Kw creation
Collective intelligence
Kw creation
Conceptualization
Kw Collaboration application
Figure 1: Organizational knowledge creation [37].
T. Räisänen and H. Oinas-Kukkonen / A System Architecture for the 7C Knowledge Environment
219
The four most central sub-processes in the knowledge creation are [37]: x Comprehension – a process of surveying and interacting with the external environment, integrating the resulting intelligence with other project knowledge on an ongoing basis in order to identify problems, needs and opportunities; embodying explicit knowledge in tacit knowledge, “learning by doing”, re-experiencing. x Communication – a process of sharing experiences between people and thereby creating tacit knowledge in the form of mental models and technical skills; produces dialog records, which emphasize the needs and opportunities, integrating the dialog along with resulting decisions with other project knowledge on an ongoing basis. x Conceptualization – a collective reflection process articulating tacit knowledge to form explicit concepts and rationale and systemizing them into a knowledge system; produces knowledge products of a project team, which form a more or less comprehensive picture of the project in hand and are iteratively and collaboratively developed; may include proposals, specifications, descriptions, work breakdown structures, milestones, timelines, staffing, facility requirements, budgets, etc.; rarely a one-shot effort. x Collaboration – a true team interaction process of using the produced conceptualizations within teamwork and other organizational processes. Each of the sub-processes may also be regarded as the building of an artifact and reasoning why it has been built the way it has, i.e. capturing the knowledge rationale. Repeatedly going through these phases in a seamless and spiral-like way leads into the growth of collective intelligence. Support for capturing deep individual thinking and recording the dialog between team members may help create truly innovative knowledge products. The learning involved in the comprehension and communication processes is closely related to the attitudes of the participants, i.e. whether they understand their weak points in the sense of individual learning styles, for example. In spite of receiving a lot of attention recently among practitioners, relatively little organizational knowledge management research has discussed the evaluation of the suggested solutions [38]. Evaluating may be carried out at the individual, work unit (group, team, or department), or overall organizational levels. The 7C model shares the view of King and Ko [22] to knowledge in that knowledge surpluses data and information, and thus even if it emphasizes knowledge content, it also addresses the link from knowledge back to re-shaping data and information (cf. [41]). The increase of sharing and dissemination of information and the increase in varied interpretations are obvious and, as a matter of fact, by no means the most important measures for the success of knowledge management solutions [38]. The truly important measure is the identification of underlying non-obvious, complex problems and issues. This may help better formulate the problems and issues the organization is facing, has faced or will face. Naturally, means for solving these problems are urgently needed. By emphasizing the identification of the key organizational issues and focusing more clearly on solving these instead of something else, the organization also becomes less dependent on its individuals. At the same time the corporate or collective intelligence grows by the transfer of ideas, experience and best practices, and the individuals become more confident at their daily work [38]. An example of these, in particular Collaboration and Conceptualization, is the role of argumentation or design rationale in systems development (cf. [34]).
220
T. Räisänen and H. Oinas-Kukkonen / A System Architecture for the 7C Knowledge Environment
3 Requirements for the 7C Information System Architecture The purpose of this paper is to develop an information system architecture that follows the 7C conceptual framework. According to Nunamaker et al. [31], system architecture should be designed for extendibility and modularity. This is supported by presenting the architecture in three layers: application layer, technology layer and conceptual layer. Extendibility is supported by separating possible applications and technologies from the key concepts presented in the conceptual layer. The key concepts provide those underlining principles upon which the architecture, and the 7C model, builds upon. Modularity is supported by defining the structure of each layer. As new technologies are developed, they can be included in the technology layer if they support the identified key concepts. First, to recognize the key concepts for the system architecture, we will aim at identifying requirements posed by the 7C model itself.
3.1
Connectivity
The fluent connection provided by Internet technology is the basis of the 7C model. The users must have access to the system whether they are working at home or in the office. For example, language context processes of Communication and Comprehension rely heavily on Internet technology to provide a connection. This connection can be to people (Communication) or to knowledge (Comprehension). The connection to the Internet provides users a space in which they can communicate and interact regardless of time or place. Connection may also be improved through multiple access points in the system (e.g. mobile access) in such a manner that users are able to stay connected even when on the move.
3.2
Concurrency
Concurrency refers to the fact that the system may have several concurrent users, which, in some cases, may be interested to work with the exact same knowledge artifacts. Thus, proper concurrency control must be taken care of. Internet technology provides a good start for the Concurrency. However, Concurrency may be supported to a greater extent through providing another access point to the system. For example, a mobile access to the system for those on the move may enhance their participation for the knowledge creation processes. Providing mobile access should require no client application to be installed for the mobile device (and, in a matter of fact, for the desktop either). The system should be able to be used with any device that has a modern browser.
3.3
Comprehension
Comprehension is a process of “surveying and interacting with the external environment, integrating the resulting intelligence with other (…) knowledge” [37]. It is the process of embodying explicit knowledge into tacit knowledge. Different knowledge artifacts are created and stored for gaining collective intelligence. The user must be able to browse these artifacts and organize them as (s)he sees fit. Through browsing and organizing existing explicit knowledge, the user is able to “identify problems, needs and opportunities”, and thus learn by doing [37]. This interaction should go deeper than just browsing and organizing. The user should be able to ‘play with’ the existing knowledge. For example, the user should be able to integrate and link different pieces of knowledge, to edit or highlight
T. Räisänen and H. Oinas-Kukkonen / A System Architecture for the 7C Knowledge Environment
221
texts and graphics, or to take an audio file and embed it within a video. In any case the interaction should go deeper than just browsing of static Web pages or the generation of dynamic Web pages through user defined queries. Another way to support deeper understanding would be to allow users to see (potentially any kind of) similarities between knowledge artifacts, in particular between different pieces of knowledge. An associative link [6] between two knowledge objects would explain the user that these objects are somehow related or that they have something in common. Providing this information may trigger the user to understand something totally new. Links may also be typed and they may have attributes [6]. Typed links may help users organize information more effectively and, more importantly, “lend context for readers” to boost Comprehension [6]. Guided tours or paths [6] are examples of providing such a context.
3.4
Communication
Communication is the process of “sharing experiences between people and thereby creating tacit knowledge in the form of mental models and technical skills” [37]. Tacit knowledge an individual possesses may be transferred to other individuals or to a group of individuals. While the transfer of codified knowledge (electronic documents or pictures, for example) is easy to support with computerized information systems, supporting the transfer of tacit knowledge is much more difficult. Asynchronous communication must be supported: users are not always online at the same time, but they must still be able to discuss issues through the knowledge support system. In the 7C model controlling concurrency means supporting the co-presence of users in the virtual space. Even though the knowledge workers may be located in different places, they can still be connected to the same work processes. Co-presence may require some support for synchronous communication, in which knowledge transfer may be enhanced through real-time communication. Marwick [26] argues that in text-based chats, people use such a kind of informal dialog that can help the emergence of new tacit knowledge. Another aspect that speaks for text-based communication is the fact that we can relatively easily search, navigate, and visualize previous text-based communications. We can also add structure for text-based conversations: summarize, highlight, link and annotate them [14]. For example, discussion stored in a XML file may include meta-information on it (i.e. metadata such as date, topic, participants etc.), aw well as the actual content of the discussion. Annotating a certain part of the conversation can be done simply by adding a new tag into a specific spot in the file. With structure visualizations these discussions become relatively lively, and annotations and links between discussions may be displayed when needed. For communications stored in video or voice, this becomes much more difficult. Nevertheless, video, voice and pictures are important in tacit knowledge transfer. Tacit knowledge is often deeply rooted in visual and other bodily senses [29]. According to Nonaka [29] tacit knowledge can be acquired without language, through observations, imitations and practice. Tacit knowledge gained through visual observation may be impossible to articulate and transfer without some visual stimuli to trigger and help the transfer process. Thus it should be possible to use different kinds of multimedia objects (video, sound, pictures etc.) to enrich text-based discussions. After all, the things that are communicated are more important than how they are communicated. The 7C model states that Communication is a process of sharing tacit knowledge, particularly experiences. Typically, information communication technologies provide a means for communication, but they also have an effect on what users
222
T. Räisänen and H. Oinas-Kukkonen / A System Architecture for the 7C Knowledge Environment
communicate [40]. According to the 7C model users should be encouraged, or even persuaded, to share their knowledge and experiences with co-workers in organizational settings. For persuasive purposes, information systems can be regarded as tools, social actors, or as media [16]. As a tool, an information system may persuade by making the sharing of knowledge easier. As a social actor, it may reward the user and provide social acceptance. And as a medium, it may provide people vicarious experiences that motivate them to share information. One problem for tacit knowledge sharing and formation is the potential lack of trust among participants [26]. Especially in virtual environments where the lack of past or future association (face-to-face meetings, for example) decreases the potential existence of trust [21]. One solution for building trust online is to create online communities [4]. Virtual environments may help share some experiences. If a past experience was “learned the hard way” (which may have seen an embarrassing or even humiliating personal experience) sharing such a lesson requires not only trust, but personal courage as well. If no past or future connections among participants exist, sharing such experiences might be easier. On the other hand, if the users know each other, there should be a way to share experiences anonymously. Even though the Communication process is probably the easiest C to support, there are still potential problems with it. Understandably, the sharing of tacit knowledge is more complicated than the sharing of explicit knowledge. In a matter of fact, instead of only supporting the sharing of knowledge for other stakeholders, a support environment should also support the acquisition of knowledge by individual users. A critical, social requirement for the environment such as discussed in this paper is to ensure that users end-up sharing their knowledge and experiences. Special emphasis should be put on such knowledge and experiences that other users do not know. Another important requirement is that the communications are stored in a well-defined, text-based format, such as XML or its variants. In this manner, the communications can best support the full 7C knowledge creation cycle, and information may be reused in the Comprehension and Conceptualization phases more easily and to a larger extent than if they were in some other formats, such as audio.
3.5
Conceptualization
Conceptualization is the “collective reflection process articulating tacit knowledge to form explicit concepts and systemizing the concepts into a knowledge system” [37]. It is the process of transforming tacit knowledge into explicit, and it is probably the least researched area of the 7C processes. This may also be why the existing systems and tools offer little support for it. According to Nonaka [29], the first step in transforming tacit knowledge into explicit knowledge is the use of metaphors. Moreover, the use of metaphors “constitutes an important method of creating a network of concepts which can help to generate new knowledge about the future by using existing knowledge” [29]. It is a creative, cognitive process which relates concepts that are far apart in an individual’s memory. When two concepts are presented in a metaphor, “it is possible to (…) make comparisons that discern the degree of imbalance, contradiction or inconsistency involved in their association” [29]. Nonaka also states that contradictions incorporated in metaphors may be harmonized through the use of analogies. Association of meaning by metaphors is mostly driven by intuition and involves images, whereas association of meaning through analogy is more structural and functional, and is carried out through rational thinking.
T. Räisänen and H. Oinas-Kukkonen / A System Architecture for the 7C Knowledge Environment
223
Conceptualization is a collective process and it requires some sort of consensus about the explicit concept being formed and systemized. This might mean that people have different opinions and ideas about the concept at hand. In that case, reaching a consensus (or compromise if the ideas are too far apart) might need a strong argumentation. If we are to get others to accept a radical idea (or at least to accept the existence of differing opinions) we must show why they should do so. Capturing design rationale in systems development may be used to accomplish just this. Design rationale means the understanding of why an artifact has been designed the way is has [34]. Capturing the rationale behind explicit concepts may lead to “clarity of thinking and augmentation of (…designer’s…) memory” as well as to better communication [34]. With argumentation, we may try to understand the specific elements of each others’ concepts, and perhaps even try to persuade others into accepting our viewpoints, or in other words to conceptualize “knowledge rationale”. If we can argue the explicit knowledge created in the Conceptualization process, we then have a chance of understanding the tacit knowledge behind it. In this way the arguments behind the knowledge help us in Comprehension, Communication and Conceptualization, making knowledge rationale one of the key concepts of the 7C architecture. The outputs of the Conceptualization process are the explicit concepts (basically this can be any explicit knowledge object) backed up with rationale arguing (against or for) the concepts. Visualizing and linking these concepts to each other may help in Comprehension and Collaboration. In the 7C model Conceptualization is a collective process, and the use of metaphors and analogies could facilitate the formation of explicit concepts. Visualization of metaphors and linking them through analogies may provide a way for new concepts to emerge. By utilizing knowledge rationale one may help others to understand his/her reasoning, thus helping Comprehension, Communication, and Conceptualization (and indirectly also Collaboration).
3.6
Collaboration
7C Collaboration process is a “true team interaction process of using (…) conceptualizations within teamwork” [37]. As discussed in the Communication process, a shared virtual environment must be provided for the team to work in. The most important aspect of Collaboration process is to support the coordination and distribution of work. Users should be able to know who is doing what and with whom. An essential aspect for the Collaboration process is that it must provide ways to utilize the produced conceptualizations. Thus, the users should be able to decide who works with whom and with what conceptualization. The actual outcomes of the Collaboration may vary depending on the job at hand but the shared virtual environment provides a good starting point for teamwork. Browsing previous cases, e.g. conceptualizations in use, and reusing the work already accomplished should also be possible.
3.7
Collective intelligence
Going through the Conceptualization, Communication, Conceptualization and Collaboration phases several times in a seamless spiral-like way leads into the growth of Collective intelligence [37]. While organizations create new knowledge, they also forget it [1][3][11]. That is why the storage, organization, and retrieval of organizational knowledge are important [42].
224
T. Räisänen and H. Oinas-Kukkonen / A System Architecture for the 7C Knowledge Environment
In the 7C architecture, it is important that all the knowledge artifacts created in any subprocess are stored. These knowledge artifacts can be anything between discussions in the Communication process and metaphors in the Conceptualization process. Equally important is that the stored knowledge artifacts can be retrieved whenever needed.
4 From the requirements to the architecture – key concepts In the discussion above, 22 requirements for the 7C architecture were identified. The requirements are summarized in Table 1. With these requirements we aim at capturing the essence of the 7C model and identifying the key concepts underlying the C’s. Requirements R1 and R3 state that the 7C architecture must be designed as a Web information system that also takes into count possible mobile users. In this way the architecture can provide the best possible support for Connectivity and Concurrency. Without it much of the potential of 7C may be lost. Multiple users working on the same knowledge artifact requires concurrency control (requirement R2). Comprehension requires that the users must be able to interact (browse, search, read, requirements R4-R6) with existing knowledge artifacts and their metadata in order to comprehend or learn from them. This is essential for new tacit knowledge to emerge as merely providing static information is not enough to truly support Comprehension. Table 1. Requirements for the 7C information system architecture. 7 C’s Connection Concurrency
Comprehension
Communication
Conceptualization
Collaboration
Collective Intelligence
Requirements R1: must be designed as a Web information system R2: must provide concurrency control for managing simultaneous users working with the same knowledge artifacts R3: should be designed mobile aware R4: must provide a way to interact, browse and search the knowledge artifacts and metadata concerning the knowledge artifacts R5: must provide a way to reorganize stored knowledge artifacts R6: should provide a way to interact with the knowledge rationale R7: must enable the sharing of knowledge and experiences R8: must support asynchronous text-based communication R9: should support synchronous communication R10: should support user communities and increase of trust among users R11: should be able to share experiences anonymously R12: must support the definition of knowledge concepts R13: must support the capture of rationale behind the explicit concepts R14: should support the use of metaphors to recognize contradictions R15: should support the use of analogies to resolve the contradictions R16: should support the visualization of concepts R17: should support the linking of concepts R18: must provide a shared virtual working environment R19: must support the coordination and distribution of work R20: should support the use of visual conceptualizations R21: must store all knowledge artifacts created in any 7C process R22: must provide a way to retrieve stored knowledge artifacts
Communication requires that the users can share their experiences or tacit knowledge (R7). Without it no transfer of knowledge will take place. The feeling of community could be used to further enhance this (R10). Much of the knowledge transfer should be text-based so that the previous communications may be easily stored, visualized and searched (R8). To further increase tacit knowledge transfer synchronous communication may be used (R9). Sharing of past experiences “learned the hard way” could be facilitated by allowing users to do it anonymously (R11).
T. Räisänen and H. Oinas-Kukkonen / A System Architecture for the 7C Knowledge Environment
225
Conceptualization means definition of explicit concepts (shift from tacit to explicit). The architecture should support this by defining the knowledge concepts (R12). Resolving between differing opinions or ideas about concepts becomes important (R13). Argumentation behind explicit knowledge is also vital for Comprehension (reading the argumentation might help the reader to understand the tacit knowledge behind the argumentation). Conceptualization might also be enhanced with the use of metaphors and analogies (R14, R15). All this might be facilitated by allowing the visualization and linking of concepts (R16, R17). Collaboration requires a shared working environment (R18). Without it doing any collaborative work is impossible. Collaborative work also requires coordination and distribution of work tasks (R19), so that work is efficient and users know what they should be doing. Also the users should be able to collaborate by using the conceptualizations in their work (R20). Finally, the processes produce knowledge artifacts that must be stored and retrieved as needed (R21, R22). Without the ability to store and retrieve the knowledge, there would be no organizational memory and the knowledge created would be quickly lost. From the requirements, we can recognize key concepts for the 7C architecture. The first is the knowledge rationale. As the 7C is a model for understanding organizational knowledge creation, knowledge and how it is represented is essential. Knowledge rationale means backing up the explicit knowledge objects with solid argumentation. The second is the use of hypertext functionality, i.e. features such as linking, and metadata. The third is the concept of mobile aware Web information system which supports the Concurrent Connection required by the 7C model. Because the 7C is a model for organizational knowledge creation and management, knowledge and how it is represented are crucial for it. This paper proposes that the rationale behind knowledge, i.e. knowledge rationale, should be treated equally important to the knowledge itself. This means that any produced concept of knowledge is stored with argumentation for it. This helps in many ways. For example, if another similar knowledge concept is being produced existing argumentation may be checked to understand why a certain knowledge concept is defined the way it is, or argumentation that has been found valid in one case may be found valid in the other case, too. It may also be possible to find knowledge traces in these argumentations, and this rationale might even include some of the tacit knowledge associated with the task at hand. This might help managing the organizational memory also. For example, the piece of explicit knowledge could be an important decision, e.g. whether or not a company should expand to new markets, based on a collection of facts, e.g. an analysis by consultants. If the question at hand is argued for and against, the ultimate decision will be easier to make. Often this argumentation holds much of the knowledge, and it is imperative for the organization that it is stored with the knowledge as it may be more important to trace the arguments than to know the exact decision. In the 7C model, knowledge rationale is embedded in Comprehension, Communication, Conceptualization, and Collaboration sub-processes. Each of these may produce new artifacts and new knowledge. For example, in Conceptualization, the produced concepts can be seen as explicit knowledge in the form of proposals, specifications, descriptions, work breakdown structure, etc., and the rationale behind the knowledge. The knowledge rationale is in the very heart of 7C architecture, and all of the processes deal with it in one way or the other. Knowledge rationale can be seen as an addition to Conversational Knowledge Management (CKM) [10][44]. In CKM knowledge is created and shared through questions and answers. This is typically done through email lists, discussion forums, or similar. Knowledge rationale adds the element of argumentation
226
T. Räisänen and H. Oinas-Kukkonen / A System Architecture for the 7C Knowledge Environment
to CKM. In Conceptualization the question-answer pair would not capture all relevant knowledge. While it is relatively easy to capture explicit knowledge in question-answer pairs, capturing tacit knowledge is more difficult. In knowledge rationale one question can have many answers and each answer can have arguments against or for made by different people [34]. In this way, conversations carried out in the Conceptualization become dynamic and natural, and the arguments may embed tacit knowledge regarding the question-answer pair at hand. CKM can also been as a way to transfer existing explicit knowledge to others, i.e. mainly the responder transferring his/her knowledge to the individual asking the question (and to others who read the questions and answers). In knowledge rationale, there is a better chance for new knowledge to emerge. New knowledge might emerge in the dialog between the arguments for and against as the users would have to come up with better arguments to counter other people's arguments. The same thing could also happen in CKM but knowledge rationale persuades users to do this through argumentation Besides linking and metadata discussed earlier the interaction capabilities provided by hypertext functionality are also important for the 7C model. They provide the means for "survaying and interacting with the external environment, integrating (...) intelligence (...), identify problems, needs and opportunities" [37]. Without the ability to interact with knowledge objects we loose some of the ability to "learn by doing" and re-experiencing [37]. As such the hypertext functionality is very important for the Comprehension. To allow the users to truly interact with the existing knowledge, hypertext must be provided in a richer way than with static Web pages or even with dynamic Web pages (i.e. Web pages are created according to the users actions). The users should be able to edit, comment, link and create the Web pages as they see fit. With this kind of functionality we may even further facilitate the Comprehension. Hypertext functionality is also useful for Conceptualization and Collaboration, too. We can use linking and annotation to help the use of metaphors, for example. As another example structure-based query can support knowledge rationale. As knowledge is saved with its reasoning, knowledge-based search is not enough: there also has to be the capability to investigate the rationale. Annotations [6] attached to knowledge can be used as the rationale. In Collaboration we can interact with the produced concepts to perform the work at hand and use them within teamwork [37]. The Concurrent Connection is realized through the concept of mobile aware Web information system [35]. A mobile aware Web information system (MAWIS) is a Web information system that has been designed with its potential usage through wireless interfaces in mind. Wireless interface refers to different mobile devices such as PDA’s, mobile phones, etc. With the concept of MAWIS, we can improve the connectivity as well as the number of concurrent users. In doing this, the separation of content from its presentation becomes essential.
Knowledge Rationale Explicit knowledge objects, rationale behind the objects
Hypertext functionality Linking, metadata, hypertext
Mobile aware Web information system Concurrent Connection to Information and People
Figure 2. Key concepts – conceptual layer of the 7C architecture.
T. Räisänen and H. Oinas-Kukkonen / A System Architecture for the 7C Knowledge Environment
227
To summarize, the key concepts of knowledge rationale, hypertext functionality and mobile aware Web information systems form the conceptual layer of the 7C architecture. It is shown in Figure 2. Knowledge rationale is perhaps the most important concept in the 7C system architecture. According to the 7C model, the outputs of Conceptualization are explicit concepts. In our architecture the explicit concepts consist of explicit knowledge and arguments behind them. The Connectivity and Concurrency suggest that the system should be designed as a mobile aware Web information system to increase the concurrent connection to information and to people. And lastly, the hypertext functionality serves as a basis for all 7C processes and it allows the use of linking and metadata, and user interaction with knowledge objects stored in the system thus helping Comprehension and Conceptualization.
5 Implementation considerations All key concepts of the architecture imply some specific technological needs for the implementation. Some technologies meet these needs better that others. For example, one core competency of Web 2.0 is to “harness the power of collective intelligence” [33]. This will go hand in hand with the 7C model. On the other hand, some other technologies seem to emphasize aspects that are not so suitable for the 7C model. We will first go through technologies that support hypertext functionality, in particular Web 2.0 technologies, as they will work also with the other concepts. Then we’ll look at the technologies that support the knowledge rationale, followed by technologies that support the mobile aware Web information systems. Finally, other possible technologies that might fit the overall 7C framework will be discussed.
5.1
Technologies supporting hypertext functionality
Web 2.0 [33] refers to a perceived or proposed second generation of Internet-based services, such as social networking sites, wikis, communication tools, and tagging, that emphasize online collaboration and sharing of knowledge between users. Web 2.0 is not a technical standard but rather a buzzword for innovative applications that are made possible by the ever growing number of Internet technologies and the novel use of combining existing technologies. Some characteristics for Web 2.0 have been defined [33]: 1) Web as platform, 2) Architecture of participation, 3) Rich user experience, and 4) Social networking The Web as a platform allows applications to be delivered and used through a Web browser. There is no need for software releases, licensing or porting to different operating systems [33]. For example, people can use www.google.com with just about any device that has a Web browser and they need no software updates or separate payments. In the Web 2.0 the importance and usefulness of a service is emphasized. This is mainly because the business value comes from delivering services over the Web platform [33]. A typical service could be a search engine or an on-line auction site. New Web services are also emerging in the form of mashups: a combination of existing Web services to form a new value-added service, e.g. combining Google Maps1 with apartment rental and home purchase services to create an interactive housing search tool [33]. Web as a platform improves the Concurrent Connection: users can run the service any time, anywhere without the need of client software. 1
http://maps.google.com
228
T. Räisänen and H. Oinas-Kukkonen / A System Architecture for the 7C Knowledge Environment
Architecture of participation refers to the success of Web sites that promote user participation. For example, Flickr2 not only stores your photos, but it allows you to share them with others. Weblogs and Wikis also provide an example of participation. Weblogs, or “blogs” are frequently updated Web pages with a series of archived posts, typically in a reverse-chronological order [28]. They are primarily textual, but often they also contain photos or other multimedia content. They also may include hypertext links to other Internet sites (often to other blogs). While personal homepages and Web publishing are nothing new as such, it is the user participation that gives weblogs an edge: the audience can read the blog, but they can also comment them. Blog entries, their comments and comments-oncomments enable better participation. It is interesting to note that the thing that made blogging truly participatory was not just the ability to comment on other's texts but the introduction of two types of links, namely the permalink and traceback [8]. The permalinks gave each blog entry a permanent location at which it could be referenced and this allowed a blogger (the writer of a blog) to cite exact blog entries. A traceback allows a blogger to ping other weblogs by placing a reciprocal link in the entry they have just referenced [8]. Together permalink and traceback allowed weblogs to become parcipatory: a blogger would know when other blogger would cite and comment his texts and he could write a reply. Participation is very important for Communication, Conceptualization and Collaboration in 7C. Another important Web 2.0 technology that supports participation and is important in knowledge management (see [44][39]) is a Wiki. Wikis are collaborative tools that enable groups to jointly create content [43], and they differ from plain discussion forums in collaborative aspects. In Wikis, users can edit any knowledge stored in it, not just their own writings as in discussion forums. Leuf and Cunningham [23] define Wiki as "a set of linked Web pages, created through the incremental development by a group of collaborating users". Wikis are found to be a good way to support question-answer pairs of CKM [44][10] and thus should also support knowledge rationale. The collaborative nature of Wikis allows Web documents to be authored collectively, which fits very well with the 7C model. There are many Wiki software systems available as open source. These Wiki software systems differ from each other mainly in their special features. Some useful features could be voting, workflow management and file and image galleria [43]. Wikis would also take care of concurrency and versioning issues to avoid conflict or inconsistencies arising from multi-user capabilities [43]. For the 7C model, Wikis could be used as a platform for Collaboration and Comprehension, vehicle for Communication, argumentation and Conceptualization and as a knowledge repository for all the knowledge created in 7C processes. As such, Wikis seem to provide a natural way to implement 7C tools. The term “Rich user experience” as well as “Rich Internet Applications” (see [2]) refers to fact that Web-based applications are starting to offer GUI-style application experiences to users [33]. An example of such user experience is Google Maps. Typically, in map-based Internet application a user has to click on a hyperlink to scroll the map. In Google Maps the user can click on the map and scroll it using the mouse, i.e. in a similar fashion as he would do on a desktop application. Google uses AJAX (Asynchronous JavaScript with XML) [17][27] and this collection of technologies has become one of the key components of the Web2.0 applications [33]. AJAX incorporates “standards-based presentation using XHTML and CSS, dynamic display and interaction using the Document Object Model, data interchange and manipulation using XML and XLST, asynchronous data retrieval using XMLHttpRequest, and JavaScript” [17]. While none of these 2
http://www.flickr.com
T. Räisänen and H. Oinas-Kukkonen / A System Architecture for the 7C Knowledge Environment
229
technologies are new in themselves, it is the novel use of them together that supports the provision of a rich user experience in Web 2.0 applications. For the purposes in the 7C model, rich user experience may facilitate the visual representation of concepts and knowledge objects. This might have a positive effect on Comprehension as the user experience would not hinder the work. The same applies to some degree to Collaboration, too, as users would apply the produced concepts in their work. With the Web 2.0, social networking has also found its way into Web applications. Typically, social networking sites allow users to create and maintain a network of close friends or business associates for social and/or professional reasons. An example of such a Web site is LinkedIn3. It allows members to look for jobs, seeking out experts or to make contacts with other professionals through a chain of trusted connections [32]. For the 7C purpose, social networking could be used to seeking out expertise (as in LinkedIn). Attaching metadata in the form of keywords (called tags) to content is a common way of organizing content for future navigation, filtering or search [18]. With Web 2.0, collaborative form of this process called tagging or folksonomy has gained popularity [18][33]. In tagging, people not only tag information for themselves but for others, too. This works best when there is no authority to control the tagging and people can use tags as they see fit [18]: Somebody might tag a video about a man breaking his arm as “man” and “funny” while another user can tag the same video with “accident”, or a photo of a puppy could be tagged “puppy” and “cute” and the photo could be retrieved using either tag correspondingly. This allows multiple and overlapping associative linking [7] imitating the human brain rather than a formal categorization [33]. Tagging can help the user in Comprehension because (s)he can browse, search and categorize explicit knowledge objects (s)he (and others) tagged, and in Communication because he can see how others have tagged knowledge and because (s)he can share his tags with others.
5.2
Technologies supporting knowledge rationale
Semantic Web is a project which tries to facilitate information exchange by bringing structure to the meaningful content of Web pages [5]. This is done by putting documents with computer-processable meaning (semantics). Semantic Web is not a separate Web but an extension of the current [5]. In Semantic Web XML (eXtensible Markup Language) and RDF (Resource Description Framework) are used to describe the structure (XML) and meaning (RDF) of the information. Ontologies are collections of information that define relations among terms [5] and they are created with OWL Web Ontology Language. Together, these techniques form the basis of the Semantic Web. According to Berners-Lee et al. [5], "the real power of the Semantic Web will be realized when people create many programs that collect Web content from diverse sources, process the information and exchange the results with other programs". These programs are called agents and their “effectiveness (…) will increase exponentially as more machine-readable Web content and automated services become available” [5]. Semantic Web as such seems to give more power to the computers, e.g. putting documents into computer-processable form for software agents, whereas Web 2.0 relies on users working collectively, e.g. through tagging and social networks. Since knowledge creation is a collective and social process Web 2.0 technologies seem to be more important for the knowledge management purposes than those of the Semantic Web. That is also why we do not represent knowledge artifacts through Semantic Web ontologies but rather through argumentation in conjunction with Web 2.0 technologies. Knowledge (be it in any 3
http://www.linkedin.com
230
T. Räisänen and H. Oinas-Kukkonen / A System Architecture for the 7C Knowledge Environment
computerized form – text, pictures, audio, video) is stored with reasoning concerning the knowledge. Typically, argumentation (or design rationale) means the understanding of why an artifact has been designed the way is has [15]. To argue for knowledge, we will use the Question-Answer-aRgument (QAR) method [34] and apply its concepts to knowledge rationale. QAR has been chosen because of its inherent support for hypertext functionality (linking, annotating, hyperdocument structure, etc.) and simplicity. In matter of a fact, it has originally designed in order to simplify the explicit rhetorical structure of rationale capture [34]. One suitable Web 2.0 technology for knowledge rationale is provided by Wikis. Knowledge objects can be argued within Wiki pages using QAR. In this way Wiki users would contribute collectively in the forming of the rationale. One of the most important steps in implementing the 7C Knowledge Environment is the integration of Wiki and QAR: the users must be able to interact with the argumentation stored in QAR and the knowledge stored in the Wiki pages.
5.3
Technologies supporting MAWIS
A mobile aware Web information system is a Web Information System that has been designed with potential usage with wireless interfaces in mind [35]. For successful construction of mobile aware Web information systems, content and presentation (functionality) should be separated from each other [36]. This enables information exchange with other information systems and also makes customization towards wireless devices easier, which further increases support for Connectivity and Concurrency. One way to separate the content and presentation is using XML to define the content and document structure and a stylesheet language to define the presentation [24]. Often Cascading Style Sheet (CSS) or Extensible Stylesheet Language (XSL) is used for presentation. As XML, CSS and XSL are integral parts of AJAX implementing 7C as mobile aware should be rather straightforward.
5.4
Other technologies for implementing 7C
Other solutions besides AJAX have emerged to support rich user experiences. One such solution is Adobe Flex. Typical Flex applications consist of interface elements build with MXML (Macromedia Flex Markup Language) and interactivity designed with ActionScript [9]. With Flex it is possible to create Flash-based applications with features such as chat, real-time dashboards, messaging and data push services [9] that run in a Flash player embedded in the browser. These Flash applications have excelled in recent years in streaming video on-demand [12]. An example of Flash for video streaming is YouTube4. Besides just streaming video Flash also lets users create layered visual effects by combining video with text, vector graphics, and other elements [12]. This could help users to comment certain interesting parts of the video instead of just commenting the whole video. This would imply that a Flash player would be suitable for playing the videos stored in the 7C environment. A challenge for using solutions such as Flex is that they require a plug-in5 to work with. This does not enable the best possible connectivity since all users will not install the plug-ins needed. Also the use of plug-ins in mobile settings is often impossible. Another way to provide richer use experiences is by extending the browser through user interface markup languages [45]. One such markup language is XUL (XML User Interface Language 4 5
http://www.youtube.com A plug-in is a program that interacts with a Web browser to provide a certain function on-demand.
T. Räisänen and H. Oinas-Kukkonen / A System Architecture for the 7C Knowledge Environment
231
[45]). An example of such an extension is to add triple-click functionality to the application (e.g. trible-clicking would select the clicked paragraph of text). The problem with such extensions is that they need browser support, which conflicts with the goals of concurrent connection and mobile use. As the 7C environment will require visualization of concepts, one implementation solution could be to use AJAX technologies to support the business logic and Flex to support the visualization. The simultaneous use of multiple new technologies may cause additional challenges for mobile use as the browsers in mobile devices in most cases do not support the latest technologies. Nevertheless, mobile access should be provided even with limited functionality, because even if mobile devices do not provide a rich user experience they may still help in the communication to a great extent.
6 System applications The key concepts and the technologies recognized implicitly suggest a set of tools which match with the requirements presented in chapters 3 and 4. Since Comprehension and Conceptualization are the processes that have received the least attention in research literature, we aim at putting special emphasis on them. A specific support tool for Comprehension should allow rich interaction with the existing knowledge: the users should be able to browse, search and categorize knowledge and the knowledge rationale stored in the 7C environment. Using personal and shared tags supports Comprehension by providing the kind of associative linking which enables the user to recognize similarities and possibly to identify specific needs and opportunities as well as potential problems. A richer user experience provided through AJAX or Flash may facilitate this interaction even more as the user is able to ‘play with’ the knowledge in a richer way than with the normal interaction capabilities provided by the static Web pages. In fact, the richer the interaction the better the chances probably are for comprehending something new. By tagging a user may share associative links with other users. This may facilitate Comprehension and Communication. For example, navigating through pieces of knowledge that have been tagged in a similar manner forms a path [6]. This may provide context for deeper Comprehension, e.g. through recognizing similarities. In a matter of fact, tags, as well as other ways to support link typing, are in the very heart of both Comprehension and Communication subprocesses, and for this reason the 7C Knowledge Environment should support flexible linking through different types of links. User1 blog Entry 1
User3 blog
Entry 2
Entry 1 Entry 2
Entry n
Concept 1 Knowledge Object1
User2 blog Entry 1
Entry n
Rationale (QAR)
Entry 2
Entry n
Figure 3. Users can link their blog entries to other users’ blog entries and to other objects within the system.
Users also need the capability to write down their own thoughts and ideas about different knowledge objects. This may be done with a tool such as a weblog. Weblog entries should be able to link with anything within the system (see Figure 3). Writing and
232
T. Räisänen and H. Oinas-Kukkonen / A System Architecture for the 7C Knowledge Environment
reading blog entries may facilitate Communication, in particular when users comment other users’ blog entries. Users should be able to modify one’s own blog entries, but they should only be allowed to read and comment other users’ blogs. A tool that would support Conceptualization should enable users to collectively articulate tacit knowledge in order to form explicit concepts. This paper approaches these concepts through the knowledge rationale. Each concept consists of an explicit knowledge object and the rationale behind it (see “Concept 1” in Figure 3). A Conceptualization tool should allow people to edit the explicit knowledge as well as argue for or against the question-answer pairs in the QAR and attach these debates into the knowledge objects at hand. Users should also be able to link concepts and knowledge objects together to show associations between them [6], e.g. concept1 could be linked to concept2 or to knowledge object1, and knowledge object1 could be linked to knowledge object2 or to concept1, etc. It should be also possible to form different concepts from one knowledge object. The same knowledge object may be used in different situations and each situation may require different arguments. Thus, we can create many concepts from one knowledge object (see Figure 4). Concept 1 Rationale1 (QAR)
Knowledge Object1
Rationale2 (QAR)
Concept 2
Figure 4. One knowledge object can belong to many concepts.
Storing and retrieving knowledge is important because without it organizations would not have a memory, and knowledge would be forgotten as soon as it was not used anymore. In the proposed architecture all explicit knowledge artifacts created in any subprocesses must be stored. This includes communications in the Communication process, knowledge rationale and the concepts produced in Conceptualization, and so on. A knowledge repository tool has two main features. It must allow the knowledge to be stored and retrieved, and it should enable removing unnecessary or gratuitous knowledge, when seen fit. The easiest implementation of the knowledge repository tool would probably be to make it a Wiki [43]. A tool support for Collaboration must allow the use of explicit concepts created in Conceptualization as well as the reuse of previous work carried out. The collaboration should be based on a shared virtual environment which would form a basis for the whole toolset. This may be done through a Wiki, where users may work collaboratively with the Concepts produced. The Wiki should also support visualizing the conceptualizations. In addition, a Wiki should handle the organization and distribution of work.
7 Discussion The application layer of the 7C architecture is represented in Figure 5. In its simplest form, 7C environment is a Wiki that consists of users’ blogs and concepts produced as knowledge rationale. Users blog for Communication purposes. To further facilitate real-time communication additional tools, such as VoIP-based tools, may be implemented. Blogs can also support Comprehension as the users may write down their thoughts and ideas. Yet, most of the Comprehension support is provided by browsing, searching and categorizing the concepts. Tagging is a key technology for Comprehension as it enables to define associative links between the concepts. Comprehension is also supported by allowing users to read the rationale behind knowledge objects. Conceptualization is
T. Räisänen and H. Oinas-Kukkonen / A System Architecture for the 7C Knowledge Environment
233
supported through a Wiki, where users collectively debate (argue) over produced knowledge objects using QAR. Wiki also works as a vehicle for Collaboration.
User’s Blogs
Concepts Knowledge Objects
Rationale
Application layer
7C Wiki
QAR
Flex & Flash
Tagging
Technology layer AJAX, XML, CSS, XSL, JavaScript, VoIP
Knowledge Rationale Hypertext Functionality
Conceptual layer
Mobile aware Web information system
Figure 5. 7C Information System architecture.
Integrating Blogs or at least blog-like features into Wikis should be rather straightforward. In a lightweight solution, normal Wiki pages could be used as personal blogs, but to take full advantage of the 7C model, at least permalinks and traceback of the blog functionality must be included in the implementation. The implementation of the 7C Wiki should use the technologies of Web 2.0. Especially tagging is essential, since it allows the kind of associative linking that could help both Comprehension and Conceptualization. QAR is suitably lightweight in supporting rationale related to knowledge objects. Too complicated a method of including rationale to knowledge objects might discourage users, and they might end-up not using the tool. AJAX and the technologies included in it offer the kind of rich user experience that might facilitate Comprehension and Collaboration. XML offers the technology to capture the content of all the knowledge produced in 7C Information System. Different stylesheet languages (CSS and XSL, in particular) provide a way to represent this knowledge in any required form, e.g. mobile device or desktop computer. On the conceptual level of the proposed architecture we have the key concepts that influence both of the layers above it. In a way the key concepts in the conceptual layer are a summary of the whole 7C environment: A mobile aware Web information system providing the needed Concurrent Connection on top of which hypertext functionality provides the means for users producing the knowledge rationale. Table 2 presents the key 7C subprocesses and how they are supported by the 7C Knowledge Environment. Concurrent Connection is provided by designing the system as a mobile aware Web information system (basically a Wiki). For the Comprehension subprocess the user’s interaction with the existing knowledge should be as rich as possible. The rich user experience delivered possibly by AJAX may be a key to success as richer experience may foster Comprehension. (S)he should also be able to use associative linking (tagging) to identify similarities. For the Communication, traceback and permalink features provide better participation and thus help users to communicate as they know better when and how someone comments their texts. Conceptualization is probably the least researched part of knowledge management. We propose that knowledge rationale can be used to better
234
T. Räisänen and H. Oinas-Kukkonen / A System Architecture for the 7C Knowledge Environment
support forming of explicit concepts required by the model. Wiki technology seems to be a natural way to support Collaboration. However, implementing all of the 7C features on top of a Wiki may be challenging. Table 2. Support of the proposed architecture for the subprocesses of the 7C model. 7 Cs Connection Concurrency Comprehension Communication Conceptualization Collaboration
Collective Intelligence
How they are supported The system is designed as a mobile aware Web information system Wiki handles concurrency control. Mobile access improves the chances for concurrent users. The users can interact with the knowledge and arguments stored in the environment, e.g. editing, linking (including tagging), commenting, combining existing knowledge Users can blog to communicate about their experiences and to read other users’ experiences. The users can use QAR to argue for and against a question to define the explicit concepts in a form of knowledge rationale The 7C Wiki can be used as a platform for collaboration where users divide the work among them and use the produced conceptualizations to perform collaborative knowledge work. All the created knowledge is stored in the environment and it can be retrieved whenever needed, e.g. in the Collaboration process
The continuous use of the proposed knowledge environment (in which all of the created knowledge is stored) should improve the efficiency and capabilities of its users, and thus in time also the Collective Intelligence of the organization. The most critical part of the environment is the Knowledge Rationale and how it can capture the concepts created in Conceptualization. 8 Conclusion This paper presented the 7C information systems architecture. The architecture consists of three layers, supporting extendibility and modularity as the role of extendibility and modularity are essential in IS architectures [31]. The conceptual layer composes of the key concepts posited by the 7C model. The technology layer presents possible technologies that could be used to implement the key concepts. And finally, the application layer presents the working applications of the 7C environment. The 7C environment must enable users to communicate with each other (using permalinks and traceback) and to interact with the knowledge stored in it. This interaction should go deeper than just browsing the knowledge: The user should be able ‘play with’ the knowledge. This richer interaction may provide a way for comprehending something new. The Wiki technology nicely supports Collaboration. An example of a 7C Knowledge Environment would be a Wiki that supports knowledge rationale using QAR. As a future work, a toolset following this architecture should be implemented and experimented with. The most crucial parts of the 7C model as well as the proposed architecture are Comprehension and Conceptualization. Special emphasis should be put on implementing and testing those, in particular the capture of knowledge rationale through the QAR method as the conceptualizations are used or they interact with many 7C subprocesses. One potential way to study this is to implement QAR either through wikis or blogs. Another important aspect that needs further investigation is the use of linking and link types in Comprehension and Communication subprocesses, e.g. using tags to recognize similarities or guided tours for sharing experiences.
T. Räisänen and H. Oinas-Kukkonen / A System Architecture for the 7C Knowledge Environment
235
Acknowledgements: We would like to thank Seamus Hickey for his comments on improving the presentation.
References: [1] Alavi, M. & Leidner, D. E. (2001). Review: Knowledge management and Knowledge Management Systems: Conceptual Frameworks and Research Issues. MIS Quarterly, Vol. 25, No. 1, pp. 107-136. [2] Allaire J (2002). Macromedia Flash MX-A next-generation rich client. Technical report, Macromedia, March 2002. [3] Argote, L., Beckman, S. L., & Epple, D. (1990). The Persistence and Transfer of Learning in Industrial Settings. Management Science, Feb 1990, Vol. 36, No. 2, pp. 140-154. [4] Ba, S. (2001). Establishing online thrust through a community responsibility system. Decision Support Systems 31, pp 323-336. [5] Berners-Lee, T., Hendler, J., Lassila, O. The Semantic Web. Scientific American, May 2001, Vol. 284, Issue 5, p3443. [6] Bieber M., Vitali F., Ashman H., Balasubramanian V. & Oinas-Kukkonen H. (1997) Fourth Generation Hypermedia: Some Missing Links for the World Wide Web. International Journal of Human Computer Studies, Vol. 47, No. 1, p. 31-65. [7] Bieber M., Oinas-Kukkonen H. & Balasubramanian V. (1999) Hypertext Functionality. ACM Computing Surveys, Hypertext and Hypermedia Electronic Symposium, Vol. 31 (4es), December 1999. [8] Blood, R. (2004). How Blogging Software Reshapes the Online Community. Communications of the ACM, December 2004, Vol. 47, No. 12. [9] Borck, J. R. (2006). Flex 2.0 Enriches App Dev Experience. InfoWorld; Aug 14, 2006, Vol. 28, No. 33, pg 41-42. [10] Cheung, K. S K., Lee, F. S. L., Ip, R. K. F. & Wagner, C. (2005). The Development of Successful On-Line Communities. International Journal of The Computer, The Internet and Management, Vol. 31, No. 1 (January-April, 2005), pp. 71-89. [11] Darr, E. D., Argote, L. & Epple, D. (1995). The Acquisition, Transfer, and Depreciation of Knowledge in Service Organizations: productivity in Franchises. Management Science, Nov 1995, Vol. 41, No. 11, pp. 1750-1762. [12] Emigh, J. (2006). New Flash player rises in the Web-video market. Computer, Vol. 39, Iss. 2, pg. 14-16. [13] Engelbart, D. (1992). Toward High-Performance Organizations: A Strategic Role for Groupware. In Proceedings of the GroupWare '92 Conference, San Jose, CA, August 3-5, 1992, Morgan Kaufmann Publishers. [14] Erickson, T. & Kellogg, W. A. (2000). Social Translucence: An approach to Design Systems that Support Social Processes. ACM Transactions on Computer-Human Interaction, March 2000, Vol. 7, No 1, pp 59-83. [15] Fischer G., Girgensohn A., Nakakoji K. & Redmiles D. (1992). Supporting Software Designers with Integrated Domain-Oriented Design Environments, IEEE Transactions on Software Engineering, Vol. 18, No. 6, pp. 511-522. [16] Fogg, B. J. (2003). Persuasive Technology: Using Computers to Change What We Think and Do, Morgan Kaufmann Publishers, San Francisco, 2003. [17] Garrett, J. J. Ajax: A New Approach to Web Applications. http://www.adaptivepath.com/publications/ essays/archives/000385.php (visited 1.1.2007). [18] Golder, S., A. & Huberman, B., A. (2005). Usage patterns of collaborative tagging systems. Journal of Information Science, Vol. 32, No. 2, 2006, pp. 198–208. [19] Grant, R. M. (1996). Toward a Knowledge-based Theory of the Firm. Strategic Management Journal, Vol. 17, Winter Special Issue, 1996, pp. 109-122. [20] Hevner, A. R., March, S. T., Park, J. & Ram, S. (2004). Design Science in Information System Research. MIS Quarterly, March 2004, Vol. 28 No. 1, pp. 75-105. [21] Jarvenpaa, S. & Leidner, D. E. (1999). Communication and Trust in Global Virtual Teams. Organization Science, Vol. 10, No. 6, November-December 1999, pp. 791-815. [22] King W. R., and Ko D. (2001) Evaluating Knowledge Management and the Learning Organization: An Information/Knowledge Value Chain Approach. Communications of the AIS, Vol. 5, Article 14, May 2001. [23] Leuf, B. & W. Cunningham (2001). The Wiki Way: Collaboration and Sharing on the Internet. Reading, MA: Addison-Wesley. [24] Lie, H. W., Saarela, J. (1999). Multipurpose Web publishing using HTML, XML, and CSS. Association for Computing Machinery. Communications of the ACM; Oct 1999, Vol. 42, No. 10, pg: 95. [25] March, S. T., & Smith G. F. (1995). Design and Natural Science Research on Information Technology, Decision Support Systems, Vol. 15, pp. 251-266.
236
T. Räisänen and H. Oinas-Kukkonen / A System Architecture for the 7C Knowledge Environment
[26] Marwick, A. D. (2000). Knowledge Management Technologies. IBM Systems Journal, 2001, Vol. 40, No. 4, pp 814830. [27] Mesbah, A. & van Deursen, A. (2007). An Architectural Style for Ajax. Proceedings of the 6th Working IEEE/IFIP Conference on Software Architecture (WICSA'07). IEEE Computer Society, 2007. [28] Nardi, B., A., Schiano D., J. & Gumbrecht, M. (2004). Blogging as social activity, or, would you let 900 million people read your diary? CSCW’04, November 6–10, 2004, Chicago, Illinois, USA. [29] Nonaka, I. (1994). A Dynamic Theory of Organizational Knowledge Creation. Organizational Science, Vol. 5, No. 1, February 1994, pp. 14-37. [30] Nonaka, I. & Takeuchi, H. (1995). The Knowledge-Creating Company — How Japanese Companies Create the Dynamics of Innovation, Oxford University Press. [31] Nunamaker, J.F. Jr., Chen, M. & Purdin, T.D.M. (1991). Systems Development in Information Systems Research, Journal of Management Information Systems, Vol. 7, Issue. 3, pp. 89-106. [32] O'Murchu, I., Breslin, J. G., Decker S. (2004). Online Social and Business Networking Communities. Proceedings of the ECAI 2004 Workshop on Application of Semantic Web Technologies to Web Communities, Valencia, Spain, August 23-27, 2004. [33] O'Reilly, T. (2005). What Is Web 2.0 - Design Patterns and Business Models for the Next Generation of Software. O'Reilly Network. http://www.oreillynet.com/pub/a/oreilly/tim/news/2005/09/30/what-is-web-20.html (retrieved 01/01/2007). [34] Oinas-Kukkonen H. (1998) Evaluating the Usefulness of Design Rationale in CASE. European Journal of Information Systems, September 1998, Vol. 7, No. 3, pp. 185-191. [35] Oinas-Kukkonen, H. (1999) Mobile Electronic Commerce through the Web. Second International Conference on Telecommunications and Electronic Commerce (ICTEC ’99). Nashville, USA, October 6-8, 1999, pp. 69-74. [36] Oinas-Kukkonen H., Alatalo T., Kaasila J., Kivelä H. & Sivunen S. (2001) Requirements for Web Information Systems Engineering Methodologies. In M. Rossi & K. Siau (editors): Information Modelling in the Next Millenium, Idea Group Publishing, 2001. [37] Oinas-Kukkonen H. (2004). The 7C Model for Organizational Knowledge Sharing, Learning and Management. Proceedings of the Fifth European Conference on Organizational Knowledge, Learning and Capabilities (OKLC ' 04), Innsbruck, Austria, April 2-3, 2004. [38] Oinas-Kukkonen H. (2005) Towards Evaluating Knowledge Management through the 7C Model. Proceedings of the European Conference on Information Technology Evaluation, (ECITE ’05), Turku, Finland, September 29-30, 2005. [39] Raman, M., Ryan, T. & Olfman, L. (2005). Designing Knowledge Management Systems for Teaching and Learning with Wiki Technology. Journal of Information Systems Education, Fall 2005, Vol. 16, No. 3, pp. 311. [40] Te’eni, D. (2001). Review: A Cognitive-Affective Model of Organizational Communication for Design IT. MIS Quarterly, Jun 2001, Vol. 25, No. 2, pp 251-312. [41] Tuomi I. (2000) Data is More than Knowledge: Implications of the Reversed Knowledge Hierarchy for Knowledge Management and Organizational Memory, Journal of Management of Information Systems, (16)3, pp. 103-117. [42] Walsh, J. P., & Ungson, G. R. (1991). Organizational Memory. Academy of Management Review, Vol. 16, No. 1, 1991, pp. 57-91. [43] Wagner, C. (2004). Wiki: a Technology for Conversational Knowledge Management and Group Collaboration. Communications of the Association for Information Systems, Vol. 13, 2004, pp. 265-289 [44] Wagner, C. (2006). Breaking the Knowledge Acquisition Bottleneck Through Conversational Knowledge Management. Information Resources Management Journal, Jan-Mar 2006, Vol. 19, No. 1, pp. 70-83. [45] Wusteman, J. (2005). About XML: from Ghostbusters to libraries - the power of XUL. Library Hi Tech. Bradford: 2005. Vol. 23, Issue. 1, pp. 118-129.
Information Modelling and Knowledge Bases XIX H. Jaakkola et al. (Eds.) IOS Press, 2008 © 2008 The authors and IOS Press. All rights reserved.
237
Inquiry Based Learning Environment for Children Marjatta KANGASSALO, Eva TUOMINEN Department of Teacher Education, Early Childhood Education, FIN-33014 University of Tampere, Finland
Abstract. This paper describes development work on children’s science learning environments that utilize an inquiry-based learning approach as well as modern technological possibilities. The emphasis in the paper is on the theoretical and pedagogical starting points of inquiry learning and their application to two multimedia learning environments. The modelling of children’s exploratory learning in these environments is also described and discussed.
1.
Introduction
Thinking skills, as well as individual and collaborative exploratory activities, have ascended to a significant position in the rapidly changing technological environments of today. The opportunities of computer technology in the pre-school and primary school learning environments, integrated into the children’s spontaneous activities, have been studied for the past 17 years in the different research projects by Kangassalo and her research partners (e.g., Kangassalo 1991, 1992, 1997, 1998c; Kangassalo and Kumpulainen 2003; Kangassalo et.al. 2005). The purpose of these different research projects has been to discover pedagogic practices in which the opportunities provided by the new technology would support pedagogical activities in a natural and justified manner. In this article, we describe developmental work where the principles and theoretical starting points of inquiry and exploratory learning have been applied to modern technological possibilities for children’s science learning environments. Inquiry-based technological learning environments open opportunities for both the individual and collaborative knowledge construction process. It builds on learning where the learners’ own earlier knowledge and deeper understanding of the phenomena can be achieved together with the development of the learners’ learning and exploration skills. In the article, the theoretical and pedagogical starting points of inquiry and exploratory learning are described along with the developmental work and main research results of the PICCO research program (e.g., Kangassalo 1992, 1997, 2001) and the pedagogical scaffolding approach and examples of the Proagents learning environment (e.g., Kangassalo et al. 2005). At the end of the article, the ongoing research work of the PICCO research program, in which the entire modeling and description of the children’s exploration processes will be the main aim, is presented and discussed (e.g., Kangassalo and Kumpulainen 2004).
238
2.
M. Kangassalo and E. Tuominen / Inquiry Based Learning Environment for Children
Theoretical and Pedagogical Background
2.1. Exploratory Learning Exploring, wondering, and asking questions are natural ways for small children to acquire knowledge of the objects and phenomena surrounding them. Children's exploration utilizes all of the senses, and, gradually, also encompasses spoken language. A small child explores her environment in every way possible. She feels, touches, throws, chews, and follows with her gaze. This multimodal exploration produces information about the object using various senses. Multisensory information builds a rich conceptual basis for understanding the objects, relations, and phenomena in the physical world. Interaction between the child and the physical environment gradually extends into interaction with other children. Exploration takes place alone, together with other children, and with adults. The progress and the phases of exploratory learning may vary considerably depending on the age of the children, whether there are children of the same age in the small group, whether exploratory learning is carried out within a strategy or curricula, the size of the groups, and the number of adults guiding the action.
2.2. Characteristics of Exploratory Learning The objectives of exploratory learning can be roughly divided into two types: 1) objectives that aim at learning the phenomenon or object in question, and 2) objectives that focus on the exploratory action per se. From the viewpoint of the phenomena to be studied, the essential objective in learning is to understand the phenomena in question. Understanding, in turn, includes comprehending causal relations and explaining and foreseeing the changes taking place in the phenomena. Reaching the deeper level of learning and understanding requires grasping the theory behind the phenomena. Thus, the objective of achieving deeper understanding is grasping the key concepts of the phenomenon and their mutual relations on the abstract level where it is possible to explain individual events and phenomena within the framework of the generated conceptual and theoretical knowledge (e.g., Ausubel 1965, 1968; Hakkarainen et. al. 2004; Kangassalo 1997.) The structuring of knowledge and conceptual constructions is a step-by-step process. In the first phase of learning, it is essential that the student recognizes and becomes aware of her previous conceptions and knowledge of the object to be studied. Against the background of one's previous knowledge, it is possible to recognize the gaps and deficiencies in one's knowledge and one's explanations of the phenomenon. These gaps, in turn, guide the learner in the acquisition of new knowledge and thus in the gradual perception of the big picture, one piece at a time. In the different phases of knowledge construction, we often run into situations in which we find the inconsistencies and contradictions of our own explanations and conceptions in relation to the views presented in the material used to support the learning. Inconsistencies are also detected in relation to the conceptions of the other students. In these situations, the child faces the need for a conceptual change, either in a very radical form or as a partial change in her own knowledge and conceptual structures. It is possible to reach conceptual change in students' conceptual constructions of the phenomenon via multimodal learning and teaching practices and the students' own active exploration. In teaching and guiding the learning, it is essential to guide the students towards seeking solutions and answers for why-questions as well as for how and what-questions. Exploring the phenomena begins, particularly with small children, with the observation of phenomena and objects, and it aims at making sense of what exactly happens in the phenomenon, what changes, how and in what kind of circumstances the change takes place. After this modeling phase, explaining
M. Kangassalo and E. Tuominen / Inquiry Based Learning Environment for Children
239
and foreseeing the phenomenon become gradually possible (e.g., Vosniadou 1999; Vosniadou et. al. 2001.). The objectives related to the research of exploratory learning have to do with learning research skills, developing research strategies, learning the skills of exploring together, and developing metacognitive skills, which means that the pupils learn to recognize and analyze their ways of learning and thinking, discovering alternative opportunities for research and learning available, and becoming aware of what they have previously learned on the objects and phenomena to be studied. Along with the development of metacognitive skills, the children also learn to pose questions themselves, which helps them proceed with learning by evaluating the gaps and deficiencies in their own knowledge. Guidance and support from the teacher are very important in all the phases of exploratory learning and in actions aimed at enhancing the deeper understanding of both the students' exploratory operations and the phenomena themselves. Exploratory learning combines the learning of the phenomena and objects and the developing of research operations together with the teacher and the other students in a natural way. At its best, exploratory learning is a research process that generates new understanding and new information about the object to be collectively studied.
2.3. Supporting Explorations in Classrooms Knowledge restructuring and comprehension activity that aims at understanding more deeply the phenomenon in question are deliberate processes, and in many cases some cognitive and sociocultural support is necessary. In schools, teachers play an important role in motivating and supporting students to engage in continuous efforts to seek understanding and to revise their prior knowledge. In other words, to amplify students’ motivation “a teacher has to create and maintain a sociocultural environment that favors comprehension activity” (Hatano & Inagaki 2003, 409). In exploration-based science instruction, one important condition for comprehension activity to occur is that greater emphasis is placed on students’ thinking processes rather than on the need for correct answers, and that enough time is given for the exploration of key concepts in one subject matter area (Vosniadou et al. 2001). It has also been considered important that students are provided with opportunities to work with phenomena instead of only watching teacher demonstrations. Some cognitive scaffolding should be available to help the students find new and alternative ideas (e.g., Andre & Windschitl 2003; Hatano & Inagaki 2003). In this connection, fostering students’ metacognitive awareness is essential, since students themselves aren’t often aware how the previous beliefs constrain their learning (Vosniadou et al. 2001). The enhancement of metaconceptual awareness is possible, for example, by providing students with opportunities to create verbal expressions of their ideas, and by guiding them to elaborate their explanations and previous knowledge with regard to the phenomena in question. It has also been considered essential that the order of the acquisition of concepts in a given subject matter area receive attention. The teacher can take this into account when scaffolding the explorations. Meaningful and theoretically relevant experiences as well as providing models and external representations are important in clarifying scientific explanations. (Vosniadou et al. 2001.) Models and external representations offer students opportunities to explore aspects of phenomena in other forms than the purely linguistic and they can facilitate the comprehension of complicated phenomena by providing visual presentations of interrelations in phenomena.
240
M. Kangassalo and E. Tuominen / Inquiry Based Learning Environment for Children
2.4. Children’s Conceptual Learning and Computer Simulations In the past few decades, cognitive research on science education has focused on the acquisition of science concepts and conceptual change. Several studies have shown that children form intuitive conceptions and explanations about phenomena based on their everyday experience. These conceptions are often very different from the current scientific knowledge and, in addition, can be very resistant to change. (E.g., Vosniadou1991, 1999.) In recent years, research has progressed from describing children’s initial conceptions to the analysis of the processes that instructional interventions can bring (Caravita 2001). Attention has been drawn to environmental factors. The research findings of instructional interventions have shown that “concepts are embedded in rich situational contexts, in the tools and artefacts of the culture, and in the nature of symbolic systems used during cognitive performance”. (Vosniadou 1999, 9.) The recent research on learning environments and instructional approaches that could facilitate conceptual change has emphasized variables such as the role of metaconceptual awareness and students’ preconceptions, social interaction among students, student self-regulation and autonomy, exploratory activities, the meaningfulness of educational tasks and the use of external representations (e.g., Caravita 2001, Diakidoy & Kendeou 2001; Kangassalo 1997; Vosniadou, Ioannides, Dimitrakopoulou & Papademetriou 2001). Computer simulations can provide children with an exploratory learning environment (e.g., Kangassalo 1991, 1998d). They have also been considered one way of addressing children’s intuitive conceptions and teaching for conceptual change. This is true especially when simulations allow learners to perceive what usually can’t be directly observed and provide visual representations for a set of interrelated concepts (e.g., Snirr, Smith & Grosslight 1995). Earlier research findings on children’s exploratory learning and the development of their conceptual thinking in a simulation environment have been encouraging. For example, the findings from Kangassalo’s (1996, 1997, 1998c) research indicate that children’s independent exploration with a computer simulation, at the stage when they are spontaneously interested in the phenomena in question, can facilitate knowledge construction in the direction of currently accepted scientific knowledge. According to the research findings, children’s exploration process in the simulation contained, for example, “wandering here and there, investigating and seeking for something and experimenting with aim.” (Kangassalo 1994, 296.) In examining the children’s exploration process in relation to their conceptual model, it seemed that the more developed the child’s conceptual model was, the more there was experimentation and investigation with a purpose (Kangassalo 1994, 1997).
2.5. Pictorial Computer Based Simulation PICCO as an Exploration Tool Kangassalo (1997, 1998c) has examined how a computer based multimedia simulation program could support children’s conceptual development in astronomy. The program children used in the research was PICCO (Pictorial Computer Based Simulation Program) (Kangassalo 1991, 1998d). The use of the program was designed so that children could use it based on their own interests, questions and their spontaneous and independent exploration. The program has been used in research experiments where children explored the phenomena by following their own interests without adult supervision and at school, which supported normal everyday school learning situations. (E.g., Kangassalo 1997, 1998c.) In the PICCO program, the selected natural phenomenon was the variations of sunlight and the heat of the sun as experienced on earth in relation to the positions of the earth and sun in space. On the earth level, children can explore their natural surroundings, its phenomena and events (such as day and night and seasons) in a natural and realistic way. On the space level,
M. Kangassalo and E. Tuominen / Inquiry Based Learning Environment for Children
241
these phenomena are represented with the help of an analogue model. (Kangassalo 1997, 1998a.) All events and necessary elements in the simulation are represented as pictures and familiar symbols. The program doesn’t include pathways or rules on how the children should proceed. Children can explore the phenomena according to their own interests, either alone or with friends. (Kangassalo 1997.) In Kangassalo’s research (1997, 1998c), there were thirty-three children aged between six to eight years. Eleven of these children had the simulation available to use in a day care center after school for a four-week period. The children could use the program spontaneously and independently. The children did not have any formal instruction about these phenomena either before or during the research period. Twenty-two children used the program at school while simultaneously having a teaching period concerning astronomical phenomena. Before and after the use of the simulation all of the children’s conceptual models were elicited. In the elicitation of a conceptual model, attention was paid to the order, continuity and regularity of events of the natural phenomenon on the earth, the interconnections of the earth and sun in space, as well as the interrelations of phenomena on the earth and in space. In addition attention was paid to the size, form and distance between the earth and the sun. The eliciting was achieved using procedures where children modeled the phenomenon through action, pictorially and verbally. (Kangassalo 1997.) Children’s conceptual models were at very different levels before the use of the simulation. Some children’s models were quite well developed, while others’ were still rather undeveloped. Only a few children’s models contained misconceptions. When the children used the computer-simulation, some changes occurred in their models. The most fundamental change that occurred was that the interconnections of different aspects and phenomena began to be constructed. The changes seemed to occur largely through the progression of different phases in the direction of the currently accepted scientific view. The extent of construction varied in children’s conceptual models. When comparing the results between the two groups – between children who didn’t receive any teaching and children who had a teaching period during the use of the program – some very interesting differences were discovered. The children who received teaching had more difficulties in the integration of the succession of seasons and the alternation of light and dark on earth in the relationship between the earth and sun, than the children who explored the phenomenon independently with the PICCO program. This difference was due to the fact that the children who received teaching tried to integrate these phenomena simultaneously, while in the conceptual models of children who explored the phenomena independently using PICCO by first exploring the causes of succession of seasons then the children started to integrate the alternation of lightness and darkness on earth in relation to the earth and sun in space (Kangassalo 1998c). The conceptual change in children’s conceptual models of the selected natural phenomenon in the PICCO environment followed Thagard’s (1992) classification: The modification of point of view takes place, the level of abstraction becomes higher, and the addition, deletion and reorganization of information occurred. Reorganization could be further divided into the replacement of existing relations into new relations, the joining together of separate relations or the discovery of new relations (Kangassalo 1997, 1998c). Before using the PICCO simulation, children’s conceptual models formed a starting point from which the exploration of the phenomenon was activated. Children’s exploration contained goal oriented and systematic action, wandering, seeking for something, investigating, experimenting, finding amusement with the space shuttle and making up stories. Goals and the intensity of exploration could vary, even during the same exploration situation. Furthermore, the more developed and integrated the conceptual model, the more the children’s exploration contained goal-oriented investigation and experimentation. Children’s
242
M. Kangassalo and E. Tuominen / Inquiry Based Learning Environment for Children
exploration strategies also developed at the same time when children’s conceptual models were developing. (Kangassalo 1996, 1997.) On the basis of the theoretical, methodological approaches and research findings of the PICCO research program, a new learning environment Proagents has been developed, which utilizes an exploratory learning approach. Additionally, a pedagogical support system has been constructed inside the system.
2.6. Question-Driven Inquiry as a Pedagogical Approach The theoretical foundations for the developed Proagents exploratory learning environment are derived from the inquiry-models that emphasize the role of questions as a starting point for the inquiry. One such a model is the interrogative model of inquiry. This model was originally developed for the purposes of the philosophy of science (see e.g., Hintikka 1985, 1988; Sintonen 1990, 1999), but it has also been used to represent knowledge-seeking in educational contexts (e.g. Hakkarainen et al. 2004; Hakkarainen & Sintonen 2002). In the interrogative model, the scientific procedure is viewed as information seeking by questioning. More specifically, inquiry is defined as a series of questions the inquirer poses during his/her inquiry process, either to nature or to some other source of information. The inquirer tries to derive an answer to his/her initial question or problem by using his/her existing knowledge and by formulating and seeking answers to smaller questions. The acquisition of new knowledge raises new questions that have to be examined. By choosing the questions, the inquirer can direct the course of the inquiry according to his/her own plans (Hakkarainen et al. 2004; Hintikka 1985). According to the interrogative model, an inquiry can be conceived as a dynamic, question-driven process of understanding (Hakkarainen & Sintonen 2002). Applying this model means that a child’s learning is viewed as an active process guided by his/her own questions and previous knowledge. The selection of the approach is largely based on Kangassalo’s earlier research findings (1997, 1998c) with the PICCO-computer environment where a child has been seen to progress in his/her exploration process step-by-step on the basis of his/her earlier knowledge foundation concerning the phenomenon in question. In this project, we intend to continue the PICCO-project by constructing support for children’s explorations. The support aims at encouraging the formation of questions in a child’s mind as well as the process of seeking for answers and explanations.
3.
Natural Phenomena for Simulations
3.1. Selecting Natural Phenomena When selecting the natural phenomena for the simulation applications it was essential that the phenomenon was important and significant in everyday life. The simulated phenomena have to awaken sufficient interest in the children and efficiently utilize the possibilities offered by the computer technology. The phenomena chosen are those that can in no other way be easily and illustratively presented, such as phenomena linked with space and elementary astronomy. An important selection criteria for the chosen natural phenomenon is that through its conformities to natural law, it forms a clear, well-organized knowledge structure and theory, and that these aspects lay a strong and well-defined foundation for the modelling of the phenomena in the simulation application. (See Kangassalo 1992, 1996, 1998a, 1998b.) The pictorial computer simulation PICCO concentrates on the variations of sunlight and heat of the sun as experienced on earth in relation to the positions of the earth and the sun in
M. Kangassalo and E. Tuominen / Inquiry Based Learning Environment for Children
243
space. On the earth level, the simulation concentrates on phenomena, which are close to the everyday experiences of children, such as day and night, seasons, changes in the life of plants and birds, etc. On the space level, it is possibilities to explore, for example, the earth, the earth and the sun, the solar system, the planets, and the dimensions of the universe. In the simulation it is possible, for example, to explore the variations in a natural environment on the basis of the interconnections and positions of the earth and the sun in space. The selected phenomena include multilevel interrelationships and central concepts. The whole phenomenon is rather complex and abstract, but the phenomena are clearly integrated with each other and form a coherent theory. The simulation program has been implemented in such a way that the knowledge structure and theory of the phenomenon are based on events appearing together with the phenomenon in question, and these events are illustrated. In the simulation all events and necessary elements are represented as pictures and familiar symbols. PICCO is very easy to use and it does not assume an ability to read or write. (See Kangassalo 1992, 1996, 1997.) From the basis of the PICCO simulation, the phenomena selected for the Proagents simulation were the change of night and day, the change of seasons, the earth, the earth's atmosphere, the layers of the earth, the earth and sun, and the solar system and its planets.
3.2. Cognitive Requirements for Modelling and Simulation The cognitive requirements for modelling and designing the natural phenomena for computerbased learning environments are based on theories and concepts of cognitive psychology, cognitive science, socio-cognitive approach and science learning. The main aim is that a constructed learning environment could support children in forming integrated abstract conceptual structures and models of the selected natural phenomena and support them in continuous knowledge construction process concerning the phenomena in question. Thus, cognitive requirements have to be taken into account when selecting the natural phenomena, writing the manuscript, modelling the phenomena, and simulating the phenomena onto the computer, as well as displaying the simulation on the screen and using the computer simulation. (See Kangassalo 1992, 1997, 1998a, 1998b.) The phenomena have to be modelled for the simulation applications according to the theory and existing knowledge of the phenomena. This means, for example, that information and knowledge on the screen and in the pedagogical agents’ descriptions, explanations and guidance have been designed and implemented according to the present scientific knowledge. Additionally, the sounds of natural phenomena, such as birdsong, the sound of wind and waves, are the natural sounds of nature. This is important because of children’s knowledge construction process and the formation of information and conceptual structures. These are significant because integrated and organized information, as well as knowledge structures in human memory at the general level of the phenomenon in question, is important to effective and demanding thinking, continuous knowledge construction and the theory formation. In these applications these requirements have been taken into account. (See e.g., Kangassalo 1992, 1996, 1997, 1998a, 1998b.) From the users’ perspective, it is important that the use of the application is based on the users’ own activity. Children can proceed according to their own interests and ideas. In these applications, there are no paths or rules on how to explore and go forward. Children can use as much time as they like each time. All this provides the children with possibilities to explore the phenomenon any time as long as they want and in the order they wish. When the program is under the user’s control, it is possible for the user to concentrate on the phenomenon in question. A child’s own activity, attention and interest, supports the development and construction of conceptual structures of the phenomenon within children. The more
244
M. Kangassalo and E. Tuominen / Inquiry Based Learning Environment for Children
complicated the phenomenon is, the more important is a child’s own activity and interest in analyzing and organizing information and its storing into the memory. (See e.g., Kangassalo 1997.)
3.3. Modelling the Phenomena Modelling the natural phenomenon for the computer simulations means constructing a presentation - a model - of the phenomenon. The modelling of the phenomenon is carried out by describing and presenting the core features and central events in the phenomenon such as they are in the phenomenon. By means of the model constructed onto the computer, the phenomenon can be imitated and simulated, that is to say that it is a simulated model of the phenomenon. (See e.g., van Gigch 1991, 122; Roberts, Andersen, Deal, Garet, and Shaffer 1983, 3; Rothenberg 1989, 75-82.) A simulated model of a phenomenon means that the phenomenon's events, objects, their characteristics and mutual time and space-related relations and changes within them are included in the model. The simulation model of the phenomenon is seen on the screen pictorially. The imitation of the phenomenon, by means of pictures, is constructed to be performed and manipulated via the computer. (Kangassalo 1997.) The modelling process for the PICCO simulation is described in detail in Kangassalo’s (e.g., 1997, 1998a, 1998b) articles and report. In the next chapters, the central features of the modelling process concerning the phenomena selected for the Proagents simulation will be described.
4.
The Proagents Learning Environment
4.1. Proactive Pedagogical Support System in the Environment In designing and developing the Proagents simulation system, the research done with the PICCO-research program has been continued. The aim was to design a multimodal computer simulation environment that would support pre- and primary school aged (6 to 8 years old) children’s exploratory and conceptual learning in the domain of astronomy. The computer simulation provides the children with an exploratory learning environment where they can explore the selected phenomena according to their own interests and questions. Children’s own questions are considered as a starting point for explorations. A child’s learning is viewed as an active process guided by his/her own questions and previous knowledge (see e.g., Lonka, Hakkarainen, Sintonen 2000). To achieve progress and deeper understanding, children need guidance and support for their exploration. In the system proactive pedagogical agents have been used to scaffold each child’s inquiries in the simulation by making questions and encouraging child’s own questioning and hypothesis formation as an aim to guide the child’s exploration process towards scientific inquiry. Agents’ operations in this system will be based mainly on auditory and haptic feedback, since the system has been developed both for sighted and visually impaired children. The interrogative model of inquiry as well as ideas from the progressive inquiry approach have been applied here as a pedagogical framework in creating and developing the proactive support for child’s explorations. As a principle, it is considered important that the support doesn’t replace the child’s own thinking, but rather guides the child to think and to explore the phenomena more deeply and extensively. Moreover, during the exploration process, the proactive agents scaffold the child by asking questions and posing problems that stimulates the child’s thinking and the formation of questions in child’s mind.
M. Kangassalo and E. Tuominen / Inquiry Based Learning Environment for Children
245
Proactive pedagogic agents support children’s explorations by encouraging a child’s own questioning, directing a child’s attention to objects and their relationships in phenomena, making questions and suggestions, guiding from familiar everyday phenomena, and progress gradually to more complicated topics and the causes and explanations of the phenomena. The agents scaffold each child with respect to his/her own capabilities and exploration paths. The agents don’t make any decisions for the child or force him or her into any particular exploration path. At any moment a child can choose either to listen to what the agents ask or suggest or to ignore. As a child’s explorations proceed, the agents’ support may decrease step by step. The agents’ support is based on a child’s explorations and user profiles. Very important in the agents’ action is the right timing and the form of the support and questions. The construction of the rules of proactive pedagogic agents has been designed and tested carefully. Children need guidance and support for their explorations for achieving progress and deepening their understanding. In the Proagents system, proactive agents have been constructed to support children’s questioning and explorations. Proactivity in this system can be defined as anticipative support. It takes into account the user and situation, predicts the users’ intentions, and acts accordingly. (See e.g., Tuominen, Peltola and Kangassalo 2003; Tuominen 2003, 2006.) The pedagogical agents have different imaginary characters and different names and voices. The entire learning environment has been constructed so that the narration and play are essential parts in children’s explorations. These elements form an important pedagogical support system for children’s exploration, science learning and thinking. This is because children’s thinking takes place in the form of continuing events, and fairy tales and stories support in recognizing and keeping in mind wholeness. In addition the meaning of different imaginary characters is to help children in analyzing and recognizing each individual themes in the application and this again helps children in navigation and the formation of interrelationships of different phenomena. (See e.g., Kangassalo 1997; Tuominen, Peltola and Kangassalo 2003.) The agent’s suggestions assist the child to find the central phenomena and concepts that are connected to the each application that the child is currently exploring. They also guide the explorations from one theme to another, and in this way support the finding of the relations and explanations in the selected phenomena. From the perspective of conceptual learning, the agents guide the explorations from familiar everyday observations towards the causes and scientific explanations of phenomena. For example if the child is exploring the solar system, and has already examined the different planets and their properties, an agent might suggest that the child if she or he seek out the planet earth. After that the agent may challenge the child to think why only earth has people and animals on it, and suggest if the child would like to explore the earth even closer. Furthermore, when exploring the earth an agent may direct the child’s attention to earth’s gravity (did you notice when you travelled with your space shuttle that you were pulled to the earth’s surface?), challenge the child’s thinking with arguments (what happens to different objects when you throw them into air or drop them?) and offer explanations on gravity. (See Kangassalo, Peltola, Tuominen 2003/2004.) In summary, the agents try to guide the children to elaborate their previous knowledge through their questions and suggestions, and encourage them to examine the properties and relations in phenomena. The proactive agents also aim to help the children become conscious of their own exploration and thinking. The agents allow children to explore and proceed in different ways, and the child him/herself can continuously choose either to listen to what an agent wants to say or to ignore him.
246
M. Kangassalo and E. Tuominen / Inquiry Based Learning Environment for Children
4.2. Proagents as an Exploration Environment In the Proagenst learning environment a child can explore the earth and its rotation around its axis, the solar system and its planets, the revolution of the earth around the sun and the atmosphere and core of the earth. These two latter ones were not yet at children’s use in the carried out research experiment. Because the environment has been constructed also for visually impaired children’s use, the exploration occures by using the stylus which gives haptic feedback (the Phantom stylus, http://www.sensable.com), see Figure 1. In the environment haptic feedback is supported by auditive feedback. The following description is based on the manuscript of the Proagents – learning environment (Kangassalo, Peltola, Tuominen 2003/2004; Tuominen 2006).
Figure 1. Phantom device in a child’s use
A child starts an exploration from the hexagonal research station, where in each corner is the door to the exciting research world, so-called mini-world. When a child would like to explore the solar system and its planets, at the door of the solar system the pedagogical, proactive agent Antti Astronaut welcomes an explorer. Antti Astronaut guides in the solar system from a planet to another planets, asks if a child would like to know more about planets or if she would like to follow the revolution of the different planet around the sun. As a child touches one of the orbits, the agent tells the child which planet's orbit it is. The orbit can be felt under the haptic stylus as a groove when a child is moving along the groove. When a child is exploring the orbit of the certain planet and she is just touching the planet, the program tells her what planet it is and where it is located in relation to the sun. It is also possible to listen to more information on each of the planets and the sun. The agent Antti Astronaut guides, gives more information and makes questions, if a child would like to choose the listening of Antti Astronaut. When a child would like to continue from the solar system to another mini-world, she presses a button in the system and returns to the research station and chooses a new mini-world. In the earth mini-world Earth Giant and his finger is guiding the exploration of the earth and its surface. The earth can be felt as a three-dimensional round object. A child can feel by using the finger of the Earth Giant the different forms of the surface. When touching the surface of the earth, it is possible to feel the differences between solid ground and the oceans. The ground feels hard and uneven, while the oceans are more even areas. It is also possible to feel the biggest mountain ranges. When touching the surface of the earth, a child can hear sounds of human inhabitation and the nature. At the sea it is possible to hear typical sounds to the ocean (waves, seagulls). A child may also wish to listen to which continent or ocean she is
M. Kangassalo and E. Tuominen / Inquiry Based Learning Environment for Children
247
currently exploring. The earth also has gravity that can be felt with the stylus as a light pull towards the earth. It is also possible to rotate the earth around its axis and to feel the moving of the earth under the finger of Earth Giant (stylus). As the visual feedback, it is possible to see the round globe and the Phantom stylus on the screen. In the mini-world, the earth and the sun, a child can follow the revolution of the earth around the sun. When a child is moving around the sun on the earths’ orbit she can listen typical sounds to different seasons in Finland. The sounds are located on the certain area on the orbit when the earth is revolving around the sun just at that seasonal time. The sounds of the seasons follow the time period of the seasons in Finland during the earth’s revolution on its orbit around the sun. Sunny Anneli, the pedagogical agent of the mini-world, tells about the seasons in Finland and asks, if a child would like to listen to more information about the things which cause the seasons in Finland during the earth’s revolution around the sun. Sunny Anneli also asks, if a child would like to listen some questions. Anneli asks, for example, how long the earth’s revolution around the sun takes in real time. She also tells about the sun’s light and warmth in each season. The sun, the earth, and the earth's orbit are shown as visual feedback on the screen To the mini-world called the research laboratory there is the collection of the different question and tasks for the child. The questions concern the things in the mini-worlds and the information and knowledge of their pedagogical agents. Next there are some examples of the questions, “if you throw a toy car from a tower, does it start floating in space” or “how long it takes when the earth is revolving around the sun”, which causes the variation of the seasons in Finland” and so on. If a child's answer is wrong, the program asks a child to think about it once more and listen to the question again. In the bowels of the earth mini-world a child can explore the insides of the earth. The various layers of the earth are represented as a cross-section of the northern hemisphere. The layers can be explored by touching them with the stylus. The top layer is hard and 'stony'; when descending to the interior, the layers become softer and softer. The haptic feedback inside the earth simulates the liquid core of the earth. As a visual feedback, the child may see a cross-section of the various layers and the Phantom stylus. In this mini-world the pedagogical agent Mr. Kairanen guides the child and they use a drilling machine for moving. In the atmosphere mini-world the child may study the earth's atmosphere from the surface of the earth to the upper layers of the atmosphere. The mini-world is presented similarly to the bowels of the earth mini-world, as a cross-section in which the bottom denotes the ground and the top of the screen (and the touching area) is the topmost border of the atmosphere. Exploring the mini-world is first and foremost based on auditory feedback. As a child is moving at the bottom of the touching area (”near the ground”), she can hear people's voices, birdsong, and the leaves of trees moving. As a child moves upwards, the sounds of airplanes and typical sounds to the wind can be heard. Further up in the atmosphere, the voices grow softer and disappear, until on the outside of the atmosphere there is silent space. The haptic feedback is almost unnoticeable and light, and it aims to create a tangible ”feeling of air”. A child can freely move using the stylus in the different layers of the atmosphere. The program also tells the child, when the child herself so desires, about the characteristics and the importance of the atmosphere. The moving in the mini-world occurs by using a space lift and the guide in this world is Iiro Ilmarinen.
4.3. Analyzing Children’s Inquiry: An Example An example of one child’s inquiry as he used the program is described in this section. The description is based on micro-level analysis of the inquiry process. It includes the analysis of the child’s exploration times in the micro-worlds and observations on the child’s questions
248
M. Kangassalo and E. Tuominen / Inquiry Based Learning Environment for Children
during the exploration. The agents’ messages and the researcher’s guidance during the use of the system are also examined. The child, whose inquiry is described next, used the system at the school for visually impaired children in Jyväskylä, Finland. He used the system twice, approximately 43 minutes on the first day and 37 minutes on the next day. The researcher sat next to the child and assisted the child as he used the program. Before and after the use of the system the child’s conceptual model was evaluated. The evaluation was based on Kangassalo’s (1997) study on the formation of children’s conceptual models of a natural phenomenon when using PICCO. On the basis of the first evaluation it was possible to say that the child had already formed models of spherical earth and sun before the use of the program. He knew all the planets by name and could arrange them in order starting from the sun. The order and the regularity of the times of day and the seasons were also organized on the level of the surface of the earth. The connections between the surface level of the earth and the mutual relations between the sun and the earth were very weak, however. For example, when thinking about the changing of day and night, the child thought that at night time the sun sets towards the equator, and during the day, the sun rises high up in the sky. When asked about causes of the seasons, the child first said that it’s because “the cold and warm waves hit”. He also explained that the earth ”turns to winter” and showed this by rotating the modelled earth back and forth against the table. The child was very active and concentrated while using the program. He thought long and hard about what he wanted to study next. He started his exploration in the solar system micro-world. This was also his most explored area, a total of 26 minutes, Table 1. The fact that the child started his exploration from the solar system micro-world corresponds to the previous studies (e.g., Kangassalo 1997) with the PICCO-environment where it was found that the children’s conceptual models formed a starting point from which the exploration of the phenomenon was activated. In this case, the child already knew the planets in the solar system while the mutual connections of the earth and the sun were quite disorganized.
Table 1. The exploration times in each mini-world Mini-world Solar system The earth The earth and the sun Research station
Time of exploration (total) 26 min 9 min 10 min 17 min
At the research station the child usually wandered around the different corners of the station and listened many times as the program told what could be explored in each micro-world. This also shows in the large amount of time he spent at the station (17 minutes), Table 1. He expressed many times that he would like to explore the atmosphere, but unfortunately that particular micro-world wasn’t yet constructed at the time of the research experiment. At the solar system micro-world the child’s exploration included wandering as he went through the planets, and sometimes he said aloud things like: ‘Let’s see if it would tell about Pluto’. During his exploration he also sought for different planets, and investigated the temperatures of the planets. He also actively investigated the surface features of the earth in the earth micro-world. The child was also interested in the program and its operations and experimented often what would happen if he, for example, pressed objects with the stylus. In the exploration the following strategies could be observed: wandering, seeking for something, investigating and experimenting (cf. e.g., Kangassalo 1997).
249
M. Kangassalo and E. Tuominen / Inquiry Based Learning Environment for Children
Some signs of reflection could be observed during the child’s exploring process as the child said, for example, ‘I don’t need to explore Pluto’. Reflection concerning the exploring process appeared also as the child started to rethink his earlier speculations about the duration of the earth’s revolution around the sun. The child had earlier, when exploring the earth’s revolution around the sun in the earth-sun micro-world, said that he thought that it takes the earth two months to go round the sun. The reflection started after hearing an agent’s message concerning the earth’s circumference: An Agent: You are now exploring the earth, the planet in which we live. The earth's circumference is approximately 40,000 kilometres. This means that if you could travel around the world by car, you would have to sit still for three weeks in a row. C: It takes less time than the round of the sun. I think I guessed a bit wrong. R: Well, what do you think about it now? C: Now I think if you took an airplane it'd take only two weeks. The example shows also that the duration of the earth’s revolution around the sun hadn’t yet been organized in child’s mind despite the active exploration of the earth-sun –micro world. It may be that the exploration of this micro-world is too confusing without a more developed model of the mutual relations of the earth and the sun in space and its connection to the seasons. As discussed in section 2.6, the pedagogical approach of the simulation program is based on interrogative model of inquiry, which means that inquiry is viewed as a series of questions the inquirer poses during his/her inquiry process. Next the questions the child asked during his inquiry are examined (see Table 2). The child asked a lot of questions during his exploration process. Most of the questions concerned the program and its operations. That is understandable because the child had to operate without visual feedback, and the equipment and the program was new to the child. There were eight (8) questions on the first exploration time and nine (9) questions on the second exploration time that clearly concerned about the phenomenon and its exploration. Most of these questions were fact-seeking questions (i.e. When have we explored the whole earth? Which planets have many moons? Are there two hundreds degrees?) that could be answered by providing factual information. During the exploration there were no questions concerning the phenomena that could be categorized as explanation seeking (how or why questions). A few of these questions could, however, be found in the evaluation situations that were conducted before and after the exploration. It seems that - on the level of questions that the child posed – the inquiry concentrated very much on the program and its operations.
Table 2. The child’s questions during the exploration The child’s questions during the exploration closed question: if/whether… what where, when how, why
Other
41
Exploration/ Phenomenon 8
13 6 8
4 5 -
5 1 -
The device
The program
6 2 1 4
7
During the child’s exploration there were a total of 16 messages from the agent, Table 3. The messages came mainly in the solar system micro-world (13 of the 16 messages). Of these 16 messages, there were only two messages that the child did not choose to hear. It may
250
M. Kangassalo and E. Tuominen / Inquiry Based Learning Environment for Children
be that one of the rejections was because the child wanted to test the agents. The other message that the child rejected came when the child was exploring the planets in the solar system. In most of the messages, the agent provided more information and explanations about the phenomena. The same message often came twice due to a failure in the system. Table 3. Agent’s messages and the child’s responses TIME (s) 0 508 1647
MICROWORLD
Research station Solar system
1659 1668
1947 1997
0 826
The Earth
890 1302
Solar system
1417 1426 1437
1457
1558 1709
1770
AGENT’S MESSAGE
1st exploration time Agent tells: Distances in solar system. Agent tells: Planets revolution around the Sun. Agent tells: The heat of the Sun The agent tells: The Sun’s distance from the Earth Agent tells: Distances in solar system Agent tells: The biggest planet is Jupiter. 2nd exploration time Agent tells: The circumference of the Earth
Message not received. Agent tells: The biggest planet is Jupiter. Message not received. Agent tells: The heat of the Sun. The agent tells: The Sun’s distance from the Earth. Agent tells: Planets revolution around the Sun. Agent tells: Distances in solar system. Agent tells: Planets revolution around the Sun. And asks: how long it takes the Earth to go round the Sun. Agent tells: Planets moons.
CHILD’S REACTIONS/ RESPONSES
CHILD’S EXPRESSION
-
-
-
-
-
-
The child comments the message.
“It would take a long time.”
-
-
-
-
Re-evaluating earlier thinking
“It takes less time than the round of the sun. I think I guessed a bit wrong.” “I look now at “no”.” “What would it say if I press “yes”?”
Testing the agents Testing the agents
-
-
-
-
-
An intentional choice to hear the agents. -
“I press “yes” so that I can hear information.” -
-
-
Elicitates a question from the child.
“What planets have many moons?”
M. Kangassalo and E. Tuominen / Inquiry Based Learning Environment for Children
251
When examining the messages in relation to child’s exploration process as a whole, it could be observed that there were during the second exploration time a phase where the child obviously tested the agents and afterwards, the using of agents changed intentional (“I press yes so that I can hear information”), Table 3. In addition, after hearing an agent tell about the circumference of the earth, the child started to rethink his earlier ideas about the duration of earth’s revolution around the sun. One of the agents’ messages also elicited a question from a child (“what planets have many moons?”) that could have been explored further. It seems that the child became more interested in the agents and their messages during the exploration, and was able to include them better as a part of his inquiry as the exploration proceeded. During his second exploration time the child even expressed a wish to hear an agent when he was in solar system micro-world and explored the sun. At that time he said: “If a message comes here, I will press “yes””. The researcher’s guidance focused largely on the using of the program. The researcher, for example, told the child how he could operate in the micro-world and what could be explored there. She also assisted in the use of the stylus, as it is important to keep the stylus in a right position to be able feel to objects properly. With regard to the phenomena, the researcher assisted the child in finding different objects in the micro-worlds and directed the child’s attention to central objects or features of micro worlds. She also offered explanations of phenomena and asked some questions. Table 4 presents the most essential aspects of the researchers guidance with regard to the phenomena and its exploration in each micro-world.
Table 4. Researcher’s guidance concerning the exploration of phenomena in different micro-worlds. SOLAR SYSTEM st 1 exploration time Assists in finding planet.
Assists the child find a planet. Guides the child to go along an orbit to find a planet Assists the child to find a planet.
Assists the child to find a planet. Guides the child to go along an orbit to find a planet Assists the child to find a planet. 2nd exploration time Asks: What would happen if the earth was there where mercury is? (The child doesn’t answer.) Explains the planets motions and asks if the child remembers how the earth moves in other micro worlds. (An agent’s message interrupts – the child doesn’t
THE EARTH st 1 exploration time Directs the child’s attention to the different sounds that can be heard when exploring the surface of the earth. Explains about the earth’s rotation Directs the child’s attention to feeling different surfaces. Asks about meaning of the “ticking” noise when rotating the earth. (The child doesn’t answer.)
nd
2 exploration time Asks about meaning of the “ticking” noise when rotating the earth. (The child doesn’t answer.) Asks once more about meaning of the “ticking” noise when rotating the earth. (The child doesn’t answer.)
THE EARTH AND THE SUN st 1 exploration time Directs the child’s attention to the sounds of seasons.
Assists the child to find the earth. Directs the child’s attention to the sounds of seasons. Assists the child to the orbit of the earth.
2nd exploration time Asks the child if he could find the orbit of the earth.
Asks the child to name the different seasons when the corresponding sounds are heard when circling the orbit.
252
M. Kangassalo and E. Tuominen / Inquiry Based Learning Environment for Children
answer.) Tells about planets distances.
Helps the child to find the earth.
Explains the meaning of the ticking noise as the child’s rotates the earth. Suggests that the child rotates the earth and examines the changes of day and night.
Asks about duration of the earth’s revolution around the sun. Guides the child to circle the orbit again and listen to the sounds of seasons. Asks again about the duration the earth’s revolution around the sun.
As can been seen in Table 4, in the solar system micro-world the researchers guidance mostly concentrated on assisting the child to find the planets he wished to explore. She also asked a few questions to which, however, the child didn’t answer. When the child was exploring the earth micro-world, the researcher directed the child’s attention to feeling the earth’s surface. She also explained the earth’s rotation around its axis, and tried to describe how in this program the ticking sound meant the child is rotating the earth. After that, she tried many times to get the child to think about why there was a ticking sound when the child rotated the earth. The child didn’t answer the researcher’s questions, maybe because the question itself can be considered a bit confusing. On the basis of the evaluation of the child’s conceptual model, it is also possible that the question was too difficult for the child as the child’s model concerning the changing of day and night was largely based on model where the sun sets and rises. In the earth-sun micro-world the researcher first directed the child’s attention to the different sounds of the seasons that could be heard when circling the sun. At the second exploration time she tried to direct the child’s thinking to the duration of the earth’s revolution around the sun on the basis of seasonal sounds heard when circling on the earth’s orbit. The duration of the revolution and its connection to the changing of seasons didn’t, however, outline for the child. Next, the changes that took place in child’s conceptual model after using the simulation program are briefly examined. There were some small changes that could be observed in the child's conceptual model concerning the phenomenon. The model that was based on the setting and the rising of the sun was left out or grew weaker: in the evaluation after using the program, the child no longer knew why the times of the day change and was not able to show this with the modelling clay. Another slight change that took place in the child's model is that the earth started to rotate around its axis. However, the child mostly associated this with the change of seasons. To summarize, it could be said that after using the program, some erroneous associations disappeared, and on the other hand, some still erroneous ones were being formed.
4.4. Environment Supporting Children’s Conceptual Thinking Studying how children explore the phenomena with the learning environment has been only one of the goals in our research. One of the main aims in the construction of the learning environment has been to support children’s conceptual thinking and learning with regard to the selected natural phenomenon. Our research has focused especially on the formation of visually impaired children’s conceptual models of the phenomenon in question in a situation where children are using the learning environment. Two 7-8-year-old visually impaired children participated at the research experiment at the school for visually impaired children and a third child in the usability laboratory a few months later. The children were interviewed both before and after using the program. The aim of the
M. Kangassalo and E. Tuominen / Inquiry Based Learning Environment for Children
253
interviews was to elicit the children’s conceptual models with regard to the chosen astronomical phenomena. The eliciting of conceptual models was based on earlier research conducted by Kangassalo (1997). The interviews were videotaped so that both operative and verbal expressions of the child could be taken into account when analyzing the data. The research data also included video recordings and log files of the children’s use of the program, and the questionnaire for the children’s parents. The video recordings of the children’s interviews were transcribed and analyzed in order to study the children’s conceptual thinking and conceptual change. The analysis of the exploration processes was based on log files and video recordings of the situations where children used the system. The log files provided information about the child’s exploration pathways: how long and in which order the child explored the micro worlds and what kind of things was explored. The child’s comments, questions, and other expressions as well as the researcher’s guidance could be observed from the videotapes. To obtain an accurate picture of the child’s exploration process, these video recordings were also transcribed next to the log files. With the collected data, it was possible to describe the children’s knowledge construction processes during the research period and also to examine the nature of children’s exploratory action in the constructed environment. (Tuominen 2006.)
5.
Modelling Children’s Exploration and Learning
The PICCO research program continues to study the development of the conceptual learning and thinking of 5- to 10-year-old children regarding selected natural phenomena. It examines children’s inquiry learning, reciprocal interaction as well as the potential of the learning environment in such activity environments where children can use information technology in addition to traditional materials and tools. With regard to the development of children’s conceptual thinking on natural phenomena, the specific research object is the formation of children’s conceptual models of natural phenomena, conceptual changes and the construction of knowledge (e.g., Kangassalo and Kumpulainen 2003, 2006.). In order to obtain a coherent picture of children’s conceptual and exploratory learning, social interaction and their meaning in children’s conceptual thinking and knowledge construction, it has been necessary to synthesize the collected empirical data into a coherent form. Specific description techniques will be developed for this purpose. The description techniques aim at describing children’s conceptual learning and thinking from the bases of the exploratory learning approach by taking into account children’s self-regulation and meta cognition, peer activities and adults’ (and agents’) scaffolding processes in the situated context. Due to the developmental age of the children as well as due to the tool rich learning context, the analysis techniques try to capture children’s activities from a holistic viewpoint by concentrating on modelling operative, non-verbal and verbal expressions. In our research project we would like to know how children’s knowledge construction process, social interaction and children’s exploration processes are developing and integrating and we try to find interrelationships social and cognitive activities in the development of understanding of the phenomena in question. The final aim will be to develop the description technique by which it is possible to simulate and animate dynamically children’s conceptual thinking and learning from multiple dimensions by taking into account the dynamic and situated nature of conceptual thinking and learning. (Kangassalo and Kumpulainen 2003, 2006.) In this article, the theoretical approach of the inquiry learning, its application to the simulation programs and first steps for analyzing and modelling children’s explorations has been described.
254
M. Kangassalo and E. Tuominen / Inquiry Based Learning Environment for Children
Acknowledgements We thank the children, their teachers and parents for participating in our experiments, the PICCO (115161) and Proagents (202179) research groups of Early Childhood Education, the Department of Teacher Education, University of Tampere for the theoretical and pedagogical developmental work, and the research group of the Department of Computer Sciences (202180) for the technical realization of the Proagents application. The mentioned projects are funded by the Academy of Finland.
References Andre, T. & Windshitl M. 2003. Interest, Epistemological Belief, and Intentional Conceptual Change. In G. M. Sinatra & P.R. Pintrich (Eds.) Intentional Conceptual Change. Mahwah, NJ: Lawrence Erlbaum Associates, 173197. Ausubel, David P. 1965. Introduction. In Richard C. Anderson and David P. Ausubel (Eds.) Readings in the Psychology of Cognition. New York: Holt, Rinehart and Winston. Ausubel, D. P. 1968. Educational Psychology: a Cognitive View. New York: Holt, Rinehart and Winston. Caravita 2001. A re-framed conceptual change theory? Learning and Instruction, 11, 421-429. Diakidoy, I-A. Vosniadou, S. & Hawks, J.D. 1997. Conceptual change in astronomy: Models of the earth and of the day/night cycle in American-Indian children. European Journal of Psychology of Education, XII (2), 159184. van Gigch, John P. 1991. System Design Modeling and Metamodeling. New York and London: Plenum Press. Hakkarainen, K. & Sintonen, M. 2002. The interrogative model of inquiry and computer-supported collaborative learning. Science & Education, 11, 25-43. Hakkarainen, K., Lonka, K. & Lipponen, L. 2004. Tutkiva oppiminen. Järki, tunteet ja kulttuuri oppimisen sytyttäjinä. 6. uudistettu painos. [Progressive Inquiry: Reason, emotions and culture as the initiators of learning.] Porvoo: WSOY Hatano, G. & Inagaki, K. 2003. When is conceptual change intended? A cognitive-sociocultural view. In G. M. Sinatra & P.R. Pintrich (Eds.) Intentional Conceptual Change. Mahwah, NJ: Lawrence Erlbaum Associates, 407-427. Hintikka, J. 1985. True and false logic of scientific discovery. In Hintikka, J. (Ed.) Logic of discovery and logic of discourse. New York: Plenum. Hintikka, J. 1988. What is the logic of experimental inquiry? Synthese, 74, 173-190 Kangassalo, M. 1991. PICCO. The pictorial computer simulation of a selected natural phenomenon for children’s use. Computer program. The grant of the Academy of Finland. Kangassalo, M. 1992. The pictorial computer-based simulation in natural sciences for children’s use. In Ohsuga, S., Kangassalo, H., Jaakkola, H., Hori, K. and Yonezaki, N. (Eds.) Information Modelling and Knowledge Bases III: Foundations, Theory and Applications. Amsterdam: IOS Press, 511-524. Kangassalo, M. 1994. Children’s independent exploration of a natural phenomenon by using pictorial computerbased simulation. Journal of Computing in Childhood Education 5 (3/4), 285-297. Kangassalo, M. 1996. Picco as a cognitive tool. In Y. Tanaka, H. Kangassalo, H. Jaakkola & A. Yamamoto (Eds.) Information modelling and knowledge bases VII. Amsterdam. IOS Press, 344-357. Kangassalo, M. 1997. The formation of children’s conceptual models concerning a particular natural phenomenon using PICCO, a pictorial computer simulation. Acta Universitatis Tamperensis 559. University of Tampere. 188 pp.
M. Kangassalo and E. Tuominen / Inquiry Based Learning Environment for Children
255
Kangassalo, M. 1998a. Luonnonilmiö kuvalliseksi tietokonesimulaatioksi. Valitun ilmiön mallintaminen, simulaation ja kuvaruudun näyttöjen suunnittelu. [A Natural Phenomenon into a Pictorial Computer Simulation: Modelling the Selected Phenomenon and Designing the Simulation and the Computer Screen]. Tampereen yliopiston julkaisusarja TAJU. Tampere. 129 p. Kangassalo, M. 1998b. Modeling a Natural Phenomenon for a Pictorial Computer-Based Simulation. In Kangassalo, H., Charrel, P-J, Jaakkola, H. (Eds.) Information Modelling and Knowledge Bases IX. IOS Press, Amsterdam, 239-254. Kangassalo, M. 1998c. Conceptual change in astronomical phenomena using PICCO. EARLI, Second European Symposium on Conceptual Change, Madrid, November 6-8, 1998. Kangassalo, M. 1998d. PICCO Day and Night, Seasons – the Earth and the Sun. Cd-rom. ISBN 951-98035-0-5. Piccos Programs Ltd. Kangassalo, M. 2001. Explorative learning in PICCO -environment. In Hannu Jaakkola, Hannu Kangassalo and Eiji Kawaguchi (Eds.) Information Modelling and Knowledge Bases XII. Amsterdam: IOS Press, 259-261. Kangassalo, M. and Kumpulainen, K. 2003. The Dynamics of Children's Science Learning and Thinking in a Social Context of a Multimedia Environment. In Hannu Kangassalo, Eiji Kawaguchi, Bernhard Thalheim and Hannu Jaakkola (Eds.) Information Modelling and Knowledge Bases XIV. Amsterdam: IOS Press, 188-197. Kangassalo, M. and Kumpulainen, K. 2004. Methodological Tools for Describing Children’s Knowledge Construction Process in Multimedia Environment. In Yasushi Kiyoki, Eiji Kawaguchi, Hannu Jaakkola, Hannu Kangassalo (Eds.) Information Modelling and Knowledge Bases XV. Amsterdam: IOS Press, 119-122. Kangassalo, M. and Kumpulainen, K. 2006. Investigating the ecological dynamics of children’s conceptual models in technology-enriched science classroom. A paper presented at the Annual American Educational Research Association Conference, San Francisco, April 2006, CA. Kangassalo, M., Raisamo, R., Hietala, P., Järvi, J., Peltola, K., Saarinen, R., Tuominen, E., Hippula, A. 2005. Proactive Agents That Support Children’s Exploratory Learning. In Kiyoki, Y., Wangler, B., Jakkkola, H., Kangassalo, H. (Eds.) Information Modelling and Knowledge Bases XVI. IOS Press, Amsterdam, 123-133. Lonka, K., Hakkarainen, K., Sintonen, M. 2000. Progressive inquiry learning for children – Experiences, Possibilities, Limitations. European Early Childhood Education Research Journal. Vol. 8, 7-23. Roberts, Nancy, Andersen, David F., Deal, Ralph M., Garet, Michael S. and Shaffer, William A. 1983. Introduction to computer simulation: The system dynamics approach. London: Addison-Wesley. Rothenberg, Jeff 1989. The Nature of Modeling. In Lawrence E. Widman, Kenneth A. Loparo and Norman R. Nielsen (Eds.) Artificial Intelligence, Simulation, and Modeling. New York: John Wiley & Sons, 75-92. Sintonen, M. 1990. How to put questions to nature. In D. Knowles (Ed.) Explanation and its limits. Royal Institute of Philosophy lecture series; 27. New York: Cambridge University Press, 267-284. Sintonen, M. 1999. Why questions, and why just why-questions? Synthese 120, 125-135. Snirr, J., Smith, C. & Grosslight, L. 1995. Conceptually Enhanced Simulation: A Computer Tool for Science Teaching. In D.N. Perkins, J.L. Schwartz, M.M. West & M.S. Wiske (Eds.) Software Goes to School. Teaching for Understanding with New Technologies. New York. Oxford University Press, Inc. Thagard, Paul 1992. Conceptual Revolutions. Princeton, NJ: Princeton University Press. Tuominen, E., Peltola, K. and Kangassalo, M. 2003. Proactive Agents Supporting Children’s Exploratory Learning. JURE pre-conference, 25th – 26th of August, 2003, Padova, Italy. Tuominen, E. 2003. A Visually Impaired Child, Proactive Agents and Conceptual Learning. At 13th Annual Conference on Quality in early Childhood Education, EECERA Conference, Glasgow, Scotland, September 3-6, 2003.
256
M. Kangassalo and E. Tuominen / Inquiry Based Learning Environment for Children
Vosniadou, S. 1991. Conceptual Development in Astronomy. In Shawn M. Glynn, Russell H. Yeany and Bruce K. Britton (Eds.) The Psychology of Learning Science. Hillsdale, NJ: Lawrence Erlbaum, 149-177. Vosniadou 1999. Conceptual Change Research: State of Art and Future Directions. In Schnotz, W., Vosniadou, S. & Carretero, M. (Eds.) New Perspectives on Conceptual Change. Oxford, UK: Elsevier Science, 3-14. Vosniadou, S., Ioannides, C., Dimitrakopoulou, A. & Papademetriou, E. 2001. Designing learning environments to promote conceptual change in science. Learning and Instruction, 11, 381-419. Unpublished manuscript Kangassalo, Peltola, Tuominen 2003/2004. The manuscript of the Proagents learning environment. Department of Teacher Education. Early Childhood Education. University of Tampere, Finland. Tuominen, E. 2006. The Development of the Conceptual Models by Visually Impaired Children concerning the Selected Astronomical Phenomena in the Multimedia Learning Environment. The Manuscript of the Doctoral Dissertation, the Department of Teacher education, Early Childhood Education, University of Tampere.
Information Modelling and Knowledge Bases XIX H. Jaakkola et al. (Eds.) IOS Press, 2008 © 2008 The authors and IOS Press. All rights reserved.
257
A Perspective Ontology and IS Perspectives Mauri LEPPÄNEN Department of Computer Science and Information Systems P.O. Box 35 (Agora), FI-40014 University of Jyväskylä, Finland [email protected] Abstract. Information processing is often such a large and complex artifact that it goes beyond a human being’s capacity to conceive, model and develop it with all of its aspects at a time. For this reason, it is typical for one to focus on some aspects of it in one time and on other aspects in another time, depending on the problem at hand. For recurrent situations it is necessary to have a structured and well-defined set of perspectives which guide selections and shifts of focuses. This paper presents a light-weight perspective ontology which provides a set of well-defined perspectives, established on three dimensions, to conceive issues in information processing in an organized manner. The perspective ontology can be applied on different information processing layers, such as information systems (IS), information systems development (ISD) and method engineering (ME). To demonstrate the applicability of the ontology, it is used to derive a set of IS perspectives with basic IS concepts and constructs. The IS perspectives are then deployed as a framework in a comparative analysis of current perspectives in the IS literature.
Introduction Information systems (IS) are large and complex artifacts. That is why it has become a commonplace to decompose information systems development (ISD) work into activities, tasks and operations in such a way that in each of them it is possible to focus on certain features of an IS. To make an ISD process, in this sense, more structured and manageable, a number of perspectives, views and viewpoints have been proposed [1, 9, 13, 21, 22, 27, 45, 46, 51, 53, 55, 60, 62]. These are used, not only in structuring ISD processes, but also in specifying quality criteria for IS and contingency factors for ISD efforts. Perspectives are not advantageous merely in the IS context. They benefit considerations in other fields of information processing as well. ISD methods, for instance, are often quite complicated. To integrate, customize, configure and implement an ISD method for the use of an organization, or a project, it is necessary to focus on some specific features of the method at a time. Some of those features may relate, for instance, to the semantic contents of the method. Method engineering (ME), in turn, is commonly carried out with the support of some methodical artifacts, for example ME strategies (e.g. [49]), meta models (e.g. [25, 18]), ME techniques (e.g. [26, 50]) and ME steps (e.g. [30, 52, 56]). The development of these kinds of ME artifacts also benefit if there are well-defined perspectives which can direct one to pay attention to particular features of an ME artifact at a time. Sets of perspectives have been discussed since the 1970’s. However, most of them have no theoretical basis justifying how the perspectives in the sets are related to one another and how they should be used in a rigorous manner. In addition, perspectives are not clearly defined. They are also, with only a few exceptions, IS specific, meaning that they have been engineered merely for the development and evaluation of an IS. To our knowledge, there is only one presentation [16] which suggests a set of views for engineering ISD methods.
258
M. Leppänen / A Perspective Ontology and IS Perspectives
These views are not, however, grounded on any theory, and it is not made explicit how the views are related to one another. To our view, there is a need for a consistent set of well-defined perspectives that can be used as a common framework in the consideration of information processing on the IS, ISD and ME layers. The higher the concerned layer is, the more necessary it is that the same kinds of perspectives are available for considerations on the lower levels as well. For instance, in method engineering the perspectives are used to integrate and organize method components (i.e. models and techniques) taken from existing ISD methods. In this process it is necessary to know, among other things, which features of the IS are of relevance from which viewpoint of method components, and in which order the features should be discussed during this process. To have this kind of shared set of perspectives, we need a sound conceptualization of issues related to information processing. Ontologies are kinds of frameworks unifying different conceptions and serving as a basis of common understanding. More specifically, an ontology is an explicit specification of a shared conceptualization of some part of reality that is of interest [cf. 14]. The part of reality we are here interested in concerns information processing. The purpose of this paper is to present a light-weight perspective ontology which provides a set of well-defined perspectives to conceive, understand, structure and represent aspects of information processing in an organized manner. The perspective ontology is aimed to be general enough to support considerations on three layers, namely the IS, ISD and ME layers. The concepts and constructs in the perspective ontology have been defined in a deductive and an inductive manner. Following an iterative procedure based on [58] and [11], we first determined the purpose, domain and scope of the ontology. Second, we searched for theories that address the domain (i.e. information processing) and provide grounds for specifying perspectives. Third, we analyzed existing presentations for views, viewpoints and perspectives to find out whether some of them could be integrated, as such or adapted, into our ontology. Fourth, we defined the basic concepts and constructs of the ontology, including the criteria, dimensions and perspectives. Fifth, we evaluated the perspective ontology on the basis of quality criteria (e.g. [4, 15, 57]) in several stages. This included applying the ontology to derive the perspectives for IS, ISD and ME. In order to have a more detailed view of the perspectives on the IS layer, called the IS perspectives, we defined a comprehensive set of concepts and constructs referring to essential aspects of the IS from each of the IS perspectives. Furthermore, we made a comparative analysis of current IS perspectives to show how sets of IS perspectives presented in the IS literature compare to one another, and to our IS perspectives. The rest of the paper is organized into five sections. In Section 1 we define the basic concepts related to information processing and consider them in relation to contexts and processing layers. In Section 2 we define the notion of perspective ontology established on one or more dimensions, specify five perspectives and show how they are applied on the IS, ISD and ME layers. In Section 3 we define for each IS perspective an array of IS concepts and constructs to be used when applying the perspective. In Section 4 we deploy the IS perspectives to compare and analyze sets of perspectives presented in the literature. The paper ends with a summary and conclusions. 1. Information Processing Reality is anything that exists, has existed or will (possibly) exist. The subjective reality is the result from our mental processes [3, 39]. The physical reality is the source of sense data, which we obtain, and it is thus external to us. A thing means any phenomenon in reality whether subjective or physical. Based on semiotics, there are three kinds of things,
M. Leppänen / A Perspective Ontology and IS Perspectives
259
concepts, signs and referents. Concepts are mental things, words of mind [19]. A sign is any thing which can stand for something else. A referent is a thing to which a concept refers. Predicates are properties of things that are used to characterize things. They determine the applicability of a concept. The human mind produces a variety of conceptions about the same thing in the physical reality, depending on a point of view adopted. Using a point of view, some things and some properties of the things are selected because they are more relevant than the others. A universe of discourse (UoD) is a part of the subjective reality that becomes relevant from the point(s) of view adopted. To derive and relate the points of view, some framework is commonly deployed. A framework is a thing that guides a human being to select the points of view that are the most appropriate for the case or the problem at hand. A framework can be intuitive or formally established, vague or rigid. Human and social actions are based on expertise and its accumulation through thinking and communication processes. Expertise is knowledge which is a relative stable and sufficiently consistent set of information objects owned by single human beings (cf. [10]). Knowledge represented in a language is called data [10]. Information is a knowledge increment brought about by receiving data, by observing reality, or by inner thinking processes by which a human being organizes, compares and assesses her/his knowledge (cf. [10]). Information processing means actions by which information is created, collected, stored, processed, presented, disseminated and interpreted. Next, we elaborate the notion of information processing by considering it (1) as a context, (2) on a specific layer, and (3) in relation to other contexts (Figure 1). The discussion is based on the ontological framework, called OntoFrame [30], which contains the ontologies for the context (the context ontology), the layers (the layer ontology) and the perspectives (the perspective ontology). In the following we first give a brief introduction to the first two ontologies and then define the perspective ontology in Section 2. Context Ontology
Information Processing
Layer Ontology
P T
ME
Ar An
L
OS
F
IPS
US
O
ISD IS
Perspective Ontology S
P
D
I C
Figure 1. Framework to elaborate the notion of information processing
To have a better understanding of information processing it should be considered as something that comprises not merely actions and targets of actions but also actors, motivations, facilities and so on. Shortly, information processing should be seen as a context with all its contextual features. We have defined, based on case grammar [12], pragmatics [36], and activity theory [8], the contextual approach and the context ontology in [30]. The contextual approach has been earlier applied to enterprises [31], ISD [35], method integration [34] and method engineering [32]. Here, we apply it to information processing in general. According to the contextual approach any context can be conceived through concepts and constructs which belong to eight contextual domains: purpose, actor, action, object, facility, location and time. The domains are defined as follows:
260
M. Leppänen / A Perspective Ontology and IS Perspectives
x
Purpose domain consists of those concepts and constructs which refer, directly or indirectly, to goals, motives, or intentions of someone or something. x Actor domain encompasses those concepts and constructs, which refer to individuals, groups, positions, roles, or organizations. x Action domain is composed of those concepts and constructs which refer to functions, activities, tasks, or operations carried out in the context. x Object domain comprises those concepts and constructs which refer to something which an action is targeted to. The objects can be goods or services, material or informational. x Facility domain consists of those concepts and constructs which refer to means, whether a tool or a resource, by which something can be done or is done. x Location domain is composed of those concepts and constructs which refer to parts of space occupied by someone or something. The location is physical, like a room or a building, or logical, like a site in a communication network. x Time domain includes those concepts and constructs which refer to temporal aspects in the context. In addition to the domain-specific concepts and relationships presented above there are a number of inter-domain relationships. Second, we consider information processing on three layers that are information system, information systems development and method engineering. An information system (IS) is a context which provides information to its utilizing system. An information systems development (ISD) means a context which carries out ISD actions, ranging from requirements engineering to implementation and evaluation of an IS, in order to contribute to a renewed or a new IS. A method engineering (ME) means a context which performs ME actions to develop, customize, configure and implement a new or an improved ISD method. A context, here called by the general term information processing system (IPS), is always associated to two other contexts, namely (a) the one which information is about, and (b) the one which utilizes information provided by the IPS. These contexts are called the object system (OS) and the utilizing system (US), correspondingly. The OS and the US have specific meanings depending on the layer on which the IPS is. On the ME layer, the IPS produces prescriptions for the next lower layer (ISD) to facilitate it to produce, efficiently and effectively, prescriptions for the lowest layer (IS) so that it can satisfy the goals and needs of its utilizing system (USIS). Thus, the USME consists of those ISD’s, IS’s, and USIS’s that are related to the IPS. Correspondingly, the USISD is composed of the related IS’s and their utilizing systems. Information objects at the ME layer refer to the prior ISD contexts and the current ISD, as well as to their US’s and OS’s (i.e. USISD and OSISD). The prior ISD contexts mean those ISD contexts in which the method under construction has been earlier deployed. The object system of the ISD (OSISD), in turn, comprises the existing IS and a new IS, as well as their US’s and OS’s (i.e. USIS and OSIS). To conclude, the IPS means a context, which is highly related to the two other contexts (OS, US), is located on some of the three processing layers, and is conceptualized through the concepts and constructs of seven contextual domains. 2. Perspective Ontology In this section we first define the general notions of perspective and system of perspectives, then specify five perspectives, and lastly show how these perspectives are applied on three processing layers.
261
M. Leppänen / A Perspective Ontology and IS Perspectives
2.1
A System of Perspectives
Due to the complexity of reality, a human being tends to focus on some specific aspects, depending on a point of view adopted. In everyday life, a point of view can be situational and intuitive, established in an ad hoc fashion. However, for recurrent situation it is necessary to have structured and pre-defined viewpoints. Especially this holds for information processing systems such as ISD and ME in which abstract thinking is commonplace and which involve a large number of people in close cooperation. We define a perspective to mean such a strictly defined point of view and a system of perspectives to stand for perspectives with specified relationships. The perspective ontology provides a system of well-defined perspectives established on certain dimensions to conceive aspects of information processing in an organized manner (Figure 2).
Framework
System of perspectives 1..*
*
1..* UoD
1..*
1 conceivedFrom 1..*
Criterion 1 conceivedFrom 1..* 1..*
basedOn 1 1
1..*
1..* Point of view
Systelogical
Perspective
Infological
Conceptual
establishedOn 1..*
Datalogical
Dimension
Physical
Figure 2. Perspective ontology
The perspectives should be defined in a way which (a) enables decisions on which aspects are relevant and which aspects should be ignored from each of the perspectives, (b) relates the perspectives to one another in a rigorous manner, and (c) applies to a structured consideration of information processing on the IS, ISD and ME layers. Perspectives are commonly defined based on certain criteria or principles derived from theories relevant to the domain. The IS literature recognizes three theories to be such ones, namely semiotics (e.g. [23]), systems theory (e.g. [21, 43, 44]) and formal logic (e.g. [40]). Here, we first apply semiotics [47]. It distinguishes between linguistic expressions, on the one hand, and conceptual constructs signified by the expressions, on the other hand. This division results in a dichotomy-like dimension which has two ends, linguistic and conceptual. Second, complexity makes it difficult to perceive and understand information processing, if not decomposed and specialized into more perceivable parts [28, 43]. Decomposition and specialization are principles inverse to the first-order abstraction [33]. These two principles form our second dimension. The third dimension is based on the predicate abstraction with the criterion of realization independence [33]. It enables the partitioning of the predicates of information processing into predefined sets depending on how closely the predicates are related to realization. At the one end, information processing is viewed to be fully independent from realization, while at the other end one concentrates, in particular, on physical predicates, including those of individual persons and groups, detailed procedures, concrete data files and documents in certain spatiotemporal space.
262
M. Leppänen / A Perspective Ontology and IS Perspectives
To conclude, the system of perspectives in the perspective ontology is established on three dimensions: (a) the linguistic – conceptual dimension, (b) the first-order abstraction dimension, and (c) the predicate abstraction dimension. Figure 3 presents the perspectives along the three dimensions in relation to the IPS, the US and the OS. In the next section we define the perspectives and discuss them on the basis of this figure.
Figure 3. Dimensions and perspectives
2.2. Definitions of the Perspectives The system of perspectives is composed of five perspectives that are: systelogical, infological, conceptual, datalogical, and physical perspectives. The term ‘systelogical’ was introduced in [60], although in a slightly different meaning. The terms ‘infological’ and ‘datalogical’ were originally coined in [54] and [29] about in the same meanings as we use them here. Because we consider the perspectives as parts of the generic ontology, we define them in general in relation to the information processing system (IPS). According to the systelogical perspective the IPS is considered in relation to its utilizing system (US). The IPS has no value or purpose by itself. It becomes desired and necessary through the support it provides to its utilizing system. Hence, organizational, social, economic and informational impacts of the IPS on the utilizing system form the essence which the systelogical perspective is interested in. The generic question to be answered from this perspective is “Why”. To put it more precisely, applying the systelogical perspective means considering the following issues: x Why does the IPS exist? x What kind of utilizing system does it have? What are its objectives, actors, actions, events, rules and objects on a general level? x What information services does, or should, the IPS provide, for whom and for which actions in the US? According to the infological perspective the IPS is seen as a functional structure of information processing actions and information objects, independent from any representational and implementational features. The IPS is regarded as a context, in which the given mission is pursued by actions related to one another through information flows. The generic question to be answered is “What”. This means in more detail:
M. Leppänen / A Perspective Ontology and IS Perspectives
263
x What information is processed in the IPS and why? x What are the actions and rules of information processing? According to the conceptual perspective the IPS is considered through the semantic contents of information it processes. This means that whereas the infological perspective is based on linguistic terms, the conceptual perspective concentrates on the understanding of the meaning of those things in the OS which linguistic terms signify. The question to be answered is “What does it mean?” To put it more precisely, the conceptual perspective is interested in the following issues: x What is the meaning of the information processed in the IPS? x What does the information signify? x What kinds of structural and dynamic constraints are valid in the OS? From the datalogical perspective the IPS is viewed, through representation-specific concepts, as a context, in which actors work with facilities to process data. This implies that the perspective makes a difference between two parts: a human information processing system (HIPS) and a computerized information processing system (CIPS). Considerations cover all those contextual non-physical phenomena that are relevant to the execution of data processing actions within and between those parts (cf. user interface). The datalogical perspective is interested in “How” questions such as: x How is information represented in data in the IPS? x How are the rules of information processing derived from US rules and formulated into concrete work procedures and algorithms? x How do the users and the CIPS communicate with each other? The physical perspective ties the datalogical concepts and constructs to a particular organizational and technical environment, showing how the IPS looks like and behaves when it is implemented. It answers, for example, the following questions: x Who are those actors carrying out actions in the HIPS, how and when they act, and where are they located? x Where and how are the data stored? x How are the facilities used and by whom? x What hardware and software are used, and how are they related? Now we can discuss more closely the relationships between the perspectives and the dimensions (see Figure 3). The systelogical perspective provides the point of departure for considerations about the IPS. The main focus of this perspective is on the US, and the IPS is viewed as something which only provides services for its US. Changing the perspective from systelogical to infological means a shift along the first-order abstraction dimension: the IPS seen as a “black box” is now conceived as a context that is composed of purposes, actions and objects. Compared to the infological perspective, the application of the datalogical perspective and the physical perspective means moves along two dimensions, along the first-order dimension, on one hand, and along the predicate abstraction dimension, on the other hand. The purposes, the actions, and the objects are, in the first stage, decomposed and specialized into smaller “pieces”. In addition, actors and facilities are, on a general level, recognized. In the second stage, the process of decomposing and specializing continues and more and more realization-specific aspects of the IPS and its components are recognized. The three perspectives (i.e. the infological, datalogical, and physical perspectives) constitute a “hierarchical system of stratified levels” as defined by Mustonen [44]. The conceptual perspective is based on the use of the linguistic - conceptual dimension. While the other perspectives consider linguistic objects, the conceptual perspective focuses on their conceptual contents.
264
M. Leppänen / A Perspective Ontology and IS Perspectives
2.3. Perspectives on the Processing Layers The perspectives were above defined on a general level with the intention that they apply to any processing layer. Table 1 summarizes how they can be elaborated for each of the processing layers. Because it is not possible here to go into details, we only give some general comments on the table. We can see that regardless of which layer is concerned the systelogical perspective means looking from the viewpoint of the utilizing system, the infological perspective considers the intentions, actions and objects of the IPS, and the conceptual perspective is interested in the contents of the information objects of the IPS. Furthermore, the datalogical perspective elaborates the conceptions about the intentions, actions and objects, and extends to consider actors and facilities of the IPS as well. The physical perspective covers also the physical features of the IPS. In the next section we consider the perspectives on the IS layer, called the IS perspectives, more closely. The ISD perspectives and the ME perspectives are discussed more deeply in [30]. Table 1. Perspectives on three processing layers (L = Layer) L ME
ISD
IS
Systelogical Considers what services the ME provides to the USME (i.e. the ISD’s, the IS’s and the USIS). Considers what services the ISD provides to the USISD (i.e. the IS and the USIS). Considers services the IS provides to the USIS.
Infological Considers intentions, functional structures and information objects in the ME. Considers intentions, functional structures and information objects in the ISD. Considers intentions, functional structures and information objects in the IS.
Conceptual Considers the semantic contents of information objects in the ME (i.e. the OSME composed of the ISD’s, the IS’s and the OSIS’s). Considers the semantic contents of information objects in the ISD (i.e. the OSISD composed of the IS’s and the OSIS’s). Considers the semantic contents of information objects in the IS (i.e. the OSIS).
Datalogical Considers ME actors, ME actions, ME objects, ME facilities and their interplay on a general level. Considers ISD actors, ISD actions, ISD objects, ISD facilities and their interplay on a general level. Considers IS actors, IS actions, IS objects, IS facilities and their interplay on a general level.
Physical Considers the ME as a physical and technical construct in organizational and technical environment. Considers the ISD as a physical and technical construct in organizational and technical environment. Considers the IS as a physical and technical construct in organizational and technical environment.
3. IS Perspectives In this section we define concepts and constructs through which the IS can be perceived from the IS systelogical, IS infological, IS conceptual, IS datalogical, and IS physical perspectives. The emphasis of our discussion is on the first three perspectives. Defining the IS perspectives gives a concrete example of how to apply the perspective ontology. 3.1 IS Systelogical Perspective From the IS systelogical perspective the IS is seen to be in relation to its utilizing system (USIS). The utilizing system may be a business system such as a manufacturing department, or a public organization such as a library maintaining and lending copies of publications. There are several approaches to viewing the utilizing system (e.g. an enterprise modeling view [24, 38], a business process modeling view [41, 42, 48] with business rules [20], or an
265
M. Leppänen / A Perspective Ontology and IS Perspectives
organizational communication view (e.g. [6]). Each approach applies different concepts and constructs to conceive and structure things in the utilizing system. We integrate the IPO (Input-Process-Output) approach, commonly applied in enterprise modeling and business process modeling, with the main concepts of the purpose domain and the actor domain. Depending on the nature of the IS, we have two somewhat different viewpoints on the USIS. If the IS is a computerized information system (CIS), the IS is seen as a tool used in the USIS. If the IS contains a human information system (HIS) as well, the IS is seen as a context providing information services to the USIS. Here, we model the IS systelogical perspective from the former viewpoint (the tool viewpoint) (see Figure 4). US organization
1 1..*
US actor
US org. unit
1 1..* 1..* US human actor
US event
*
raisedBy
US position
1..*
1 supervision
1..*
US role
*
*
*
*
occupiedBy
US condition
US rule
1..* 1..* governs 1..* *
* 1..* * responsibleFor
strivesFor 1..*
US action
*
*
1..*
uses
* input output 1..* 1..* User
* supports
US purpose
performs
US object
*
US tool
*
US resource
*
Material
Informational
CIS
US facility
Figure 4. IS systelogical perspective (the tool viewpoint)
A US organization is an organization (i.e. an enterprise, a department or some other administrative arrangement), which utilizes, or is going to utilize, an IS. It consists of US organizational units, which in turn are composed of US positions. A US position is a post of employment occupied by one or more US human actors. US positions are composed of US roles with responsibilities and authorities to conduct certain US actions. A US action is an action, which strives for one or more utilization purposes. US actions are governed by US rules. US rules are composed of certain parts in accordance with the so-called ECAA structure [20]: US event, US condition, thenUSAction and elseUSAction. Conducting US actions may raise new US events that possibly trigger other US actions. The US purposes mean goals for business processes and/or reasons for setting up those goals. The US actions use US objects as their inputs and may produce US objects as their outputs. The US objects can be material (e.g. machines, components, bridges and china) or informational (e.g. insurance contract, payment and reorder). The US actions are partly performed by US tools (e.g. lathe, circular saw and nailer). Some of the US tools are computerized information systems (CIS) supporting US actions. A US actor conducting US actions with the support of a CIS is called a user. The US actions consume US resources, such as money, energy, goods, and manpower.
266
M. Leppänen / A Perspective Ontology and IS Perspectives
3.2. IS Infological Perspective In IS modeling there are two prevailing approaches, the structured approach [61] and the object-oriented approach [e.g. [2]). Here, we apply the structured approach. Based on that the IS infological perspective sees the IS to be a functional structure of information processing actions and information objects (see Figure 5). No attention is given to how the information objects are represented or implemented. This means that the “black box” conceived from the IS systelogical perspective is “opened” to reveal the aspects of the IS within three contextual domains: purpose, action, and object. The concepts in the purpose domain are used to specify why information is processed. The concepts in the action domain are used to conceive action structures needed to produce information objects. Correspondingly, the information objects are decomposed, classified, and structured with the concepts and relationships in the object domain. Sequence str
Selection str
Iteration str.
Control str.
Decomposition str.
influence *
*
IS goal
predAbstract 0..1
versionOf *
IS purpose
* 0..1
1..* *
IS object supports * *
input * output * copyOf * 0..1
IS action *
* refinement * dueTo * *
IS action str. * strivesFor 1..* * raisedBy
IS reason
* IS event
* governs * IS rule
Transient
IS condition
Permanent
Figure 5. IS infological perspective
IS purposes mean IS goals for information processing and/or reasons for setting up those goals. An IS goal is a desired state of affairs in the IS [38]. IS reasons can be functional or non-functional requirements for information processing, problems in prevailing information processing, strengths and weaknesses in, and/or opportunities for and threats against the existing or a planned IS. The IS goals are related to one another through complex influence and refinement relationships [24]. In striving for the IS purposes, IS actions use information objects, called IS objects, as inputs and produce IS objects as outputs. The range of various types of IS actions is large. An IS action can mean, for instance, collecting, storing, processing, transmitting, coding, encoding, arranging, locating, discovering, interpreting, integrating, reviewing, testing, approving, or editing information. The action structures relevant from the IS infological perspective are the decomposition structure and the control structures. The decomposition structure splits IS actions into IS functions, IS activities, IS tasks, and IS operations. The control structures enable to recognize sequence, selection and iteration relationships between the IS actions. The IS actions are governed by IS rules. An IS rule is composed of IS events, IS conditions, thenISActions and elseISActions. The IS rules can be classified in many ways.
M. Leppänen / A Perspective Ontology and IS Perspectives
267
First, there are dynamic and static rules. The dynamic IS rules restrict or guide IS actions and IS events. The static IS rules restrict IS objects. Examples of the IS rules are: back-ups of the files should be run once a week; a social security number of a person cannot be changed; a salary of an hourly paid employee is derived by the rule ‘Salary := number of hours x hourly fee’. The first example is a business rule, the second rule is an integrity constraint [7], and the last rule is an example of a derivation rule. An IS object is in the form that is free from any representational and implementational aspects. An IS object is transient or permanent. A transient IS object lasts only a short time (e.g. a reply to a routine request). A permanent IS object is valuable enough to “live” longer (e.g. personnel information, vehicle information). The IS objects are interrelated in many ways. They are composed of other IS objects. Producing them is supported by other IS objects (cf. derivation of the monthly salary from hourly fee and number of hours). An IS object can also be a version of, a copy of, or an (predicate) abstraction from, another IS object. 3.3. IS Conceptual Perspective The IS conceptual perspective considers the semantic contents of the IS objects, meaning that the structure and behavior of those things in the OSIS which are signified by the IS objects are revealed. Thus, the IS conceptual perspective addresses the so-called deep structure of the IS [59]. There are several approaches to OSIS modeling. Some of them are structural, such as the ER approach [5] and the ORM approach [17], some others (e.g. the object-oriented approach [2]) cover dynamics of the OSIS as well. We prefer the ER approach to the ORM approach and other attribute-free approaches, because we consider it important to separate between entities and attributes. We also want to make a clear distinction between the static features and the dynamic features of the OSIS, unlike the object-oriented approach. Hence, the IS conceptual perspective is based on the ER approach (the structural view) and the state machine (the dynamic view) (Figure 6). According to it, the OSIS is composed of related things that are either entities or relationships having states and affected by state transitions.
Figure 6. IS conceptual perspective
An entity means any perceivable thing in the object system with an independent existence (cf. [7]). Only those things that are relevant and “independent” enough to be signified by
268
M. Leppänen / A Perspective Ontology and IS Perspectives
the IS objects are regarded as entities. An OS relationship between two or more entities means any relevant connection, association or the like between entities. The OS relationships include the abstraction relationships (e.g. classification, generalization, composition, grouping) defined in the abstraction ontology [33]. An attribute is a relevant predicate used to characterize an entity or an OS relationship. A particular entity (and OS relationship) has zero, one or more attribute values for each of its attributes. An OSIS construct means a conceptual construct composed of specific entities related to one another through OS relationships and characterized by specific attribute values. An OSIS state means a state of the object system or its parts, composed of OSIS constructs. An OSIS transition means a transition from one OSIS state, called the pre-state, to another OSIS state, called the post-state [10]. An OSIS transition can involve entities (e.g. the birth of a child), OS relationships (e.g. the divorce) and/or attribute (e.g. the quantity available). The transitions constitute the potential OSIS behavior. OSIS transitions can be composed to establish OSIS transition structures. An OSIS event means an event which may trigger an OSIS transition from the pre-state to the post-state and which may be caused by another OSIS state transition. 3.4. IS Datalogical Perspective and IS Physical Perspective From the IS datalogical perspective the IS is viewed, through representation-specific concepts, as a context, where IS actors work with IS facilities to process IS data. Thus, the IS objects, seen as information objects from the IS infological perspective, are here considered to be data objects represented in some non-formal, semi-formal or formal language(s). There are also special IS actions which transform data objects from one form to another. Although no reference is made to data carriers or other physical features of the IS, the IS datalogical perspective enables to make a difference between a human information system (HIS) and a computerized information system (CIS). To conceive the interaction between and cooperation among these two parts, we also distinguish user interface (UI). Each of these parts is conceptually quite large. Due to the scarcity of space, we only present the model of the IS datalogical perspective in Appendix (Figure A.1). The IS physical perspective considers the IS with all its physical aspects. It ties the IS datalogical concepts and constructs to a particular organizational and technical environment, showing how the IS looks like and behaves when it is implemented. The IS contains the HIS, and possibly the CIS and the UI. For all these parts, a highly detailed and realization-dependent view is provided by this perspective. Figure A.2 in Appendix presents concepts and constructs referring to a part of the CIS from the physical perspective. 3.5. Relationships between the IS Perspectives In the previous sections we have considered the contextual concepts and relationships within each of the IS perspectives. Here, we relate the IS perspectives to one another through main inter-perspective relationships. The perspectives have been established along three dimensions. Based on the discussion in Section 2.1, we can describe the relationships between the IS perspectives as shown in Figure 7. The small rectangles inside the IS systelogical, IS infological, IS datalogical and IS physical perspectives stand for information objects which signify conceptual constructs in the object system (cf. the IS conceptual perspective). The common denominator between the IS systelogical perspective and the IS infological perspective is the IS, implying that moving from the former perspective to the latter means that the IS, first seen as a black box,
M. Leppänen / A Perspective Ontology and IS Perspectives
269
is “opened” in order to expose IS purposes, IS actions, IS objects and relationships between them. In this process, the principles of decomposition and specialization are mainly applied.
Figure 7. Relationships between the IS perspectives
The IS infological, IS datalogical and IS physical perspectives are parts of a hierarchical system of perspectives within which the relationships are based on the same criterion of realization dependence. This means that in moving downwards in the hierarchy the conceptions about the IS first become representation-specific (cf. the IS datalogical perspective) and then implementation-specific (cf. the IS physical perspective). In parallel to this, the conceptions of the IS are concretized by decomposition and specialization. Each of the aforementioned IS perspectives recognizes information objects. In the IS systelogical perspective they are called informational US objects. The IS infological perspective identifies the IS objects, and the IS datalogical perspective views them as digital or non-digital data objects. Data files, data records and data fields represent the conceptions of IS objects from the IS physical perspective. In all those cases, there are ‘signifies’ relationships between the information objects and the things conceived as OSIS constructs from the IS conceptual perspective. Through these relationships it is possible to make sense of the meanings of the information objects. Still one type of a generic relationship can be found between the IS perspectives. If the OSIS overlaps with the USIS or the IS, there is ‘abstractedFrom’ relationships between the OSIS and the USIS, in the first case, and between the OSIS and the ISIS, in the second case. By this abstraction, most of the contextual aspects of the USIS (ISIS) are ignored in order to establish OSIS constructs composed of entities, OS relationships and attribute values. For example, US actions such as hiring and firing an employee are abstracted to OS events affecting on the OS state of a particular employee.
270
M. Leppänen / A Perspective Ontology and IS Perspectives
4. Comparative Analysis of Current IS Perspectives The IS literature provides a large variety of IS architectures (e.g. [53, 62]), IS frameworks (e.g. [21, 45, 46, 55]), reference models (e.g. [22]) and IS meta models (e.g. [13]) that are based on views, perspectives and viewpoints on the IS. For simplicity, we call these presentations the frameworks, and the views and viewpoints the perspectives. The purpose of this section is to make a comparative analysis of frameworks to find out which kinds of perspectives, underlying criteria and dimensions they propose and how they relate to our IS perspectives. For the analysis, we selected those frameworks (a) which have been developed for a comprehensive analysis and/or comparison of the concepts in the fields of IS and/or ISD, and (b) in which systems of perspectives have been clearly specified. These frameworks are (in temporal order): Welke [60], Olive [45], Essink [9], Olle et al. [46], Iivari [21], Sol [51], Sowa et al. [53], van Swede et al. [55], Freeman et al. [13], Avison et al. [1] and ISO [22]. Table 2 summarizes the results of the analysis. The dimensions, along which the perspectives have been established in the frameworks, are called “levels of abstraction” [45, 9, 21, 13], “design levels” [53], “aspects” [46], ”views” [1], “viewpoints” [22], “subproblems” [51]) or “perspectives” [55, 60]. The frameworks apply different criteria to distinguish between and relating the perspectives. Sowa et al. [53] and van Swede et al. [55] have based their perspectives on views of stakeholders. Iivari [21] has derived his levels of abstraction from abstractions of the host organization, the universe of discourse, and technology. Welke [60] has built the perspectives on consequences that changes in the existing IS result in the object system, the use of information, and the data processing sequences. Avison et al. [1] argue that their five views are needed to answer the vital questions of users. Freeman et al. [13] compare their levels of abstraction with the phases of software development. Sol [51] has made his division on the basis of the kinds of problems that must be solved during the ISD. Some frameworks (i.e. [45, 22]) give neither explanation for the perspectives nor apply any explicit underlying criteria. The correspondences of the perspectives in the frameworks to our perspectives are indicated by the markings ‘X’ (strong) and ‘x’ (weak) in Table 2. The following remarks can be made on them. In all the frameworks the upper perspectives relate to the US and the lower perspectives are more technology-specific. The perspectives between the extreme ends are defined through “independence” from something (e.g. from “the object system” in [45]), and from the technology in [21]). The frameworks differ from one another in the emphasis they give on the perspectives. Welke [60] and van Swede et al. [55], for instance, focus on the upper perspectives in their frameworks, and Avison et al. [1] and ISO [22] suggest more perspectives for the consideration of technological issues. Conceptual issues are included in the topmost perspectives ([9], and partly in [46]), or as it is the case more commonly, in the next lower perspectives. The framework of [45] is the only one providing a special perspective for perceiving the conceptual aspects of the IS. The framework of van Swede et al. [55] does not to consider IS conceptual issues at all. Iivari [21], Avison et al. [1], Freeman et al. [13], Sowa et al. [53] and Olle et al. [46] pay special attention to user interface. Essink [9] mentions it only incidentally, and the frameworks of [60] and [51] are too general to recognize it. In our view, it is important to have separate perspectives for each set of different aspects of the IS. Therefore, the IS systelogical perspective is needed to consider the IS in relation to the US. The IS conceptual perspective is necessary to address the conceptual contents of the IS objects. Unlike Avison et al. [1], Sowa et al. [53], Sol [51], Freeman et al. [13] ISO [22], we see it vital to clearly differentiate between the infological perspective that represents the “linguistic world”, and the IS conceptual perspective which stands for the “conceptual world”. On the predicate abstraction dimension, at least three related
271
M. Leppänen / A Perspective Ontology and IS Perspectives Table 2: A summary of the comparative analysis of the perspectives (S = systelogical, I = infological, C = Conceptual, D = Datalogical, P = physical) Frameworks Welke [60] - Systelogical perspective - Infological perspective - Datalogical perspective Olive [45] - External level - Conceptual level - Logical level - Architectural level - Physical level Essink [9] - Object system modelling - Conceptual IS modelling - Data system modelling - Implementation modelling Olle et al. [46] - Business analysis stage - System design stage - Construction design stage Iivari [21] - Organizational level - Conceptual/infological level - Datalogical/technical level Sol [51] - Systelogical problems - Infological problems - Datalogical problems - Technological problems Sowa et al. [53] - Scope level - Enterprise model level - System model level - Technology model level - Components level van Swede et al. [55] - Business perspective - Information perspective - Functionality perspective - Implementation perspective Freeman et al. [13] - World level - Conceptual level - Design level - Implementation level ISO [22] - Enterprise viewpoint - Information viewpoint - Computational viewpoint - Engineering viewpoint - Technology viewpoint Avison et al. [1] -Human-activity view -Information view -Socio-technical view -HCI-view -Technical view
Criteria Changes in IS and/or its usersubsystem should be addressed from several perspectives ([60] p. 150) “The model at the highest level is the most general and those at the lower level are more detailed” ([45] p. 63)
S
I
C
X
x X
x
X
D
P
x X
x x
x X X X X
“..are classes of problems that are relevant from a specific view on IS’s” ([9] p. 356)
x
X X
x X X
Not clearly specified X
X X X
Derived from abstractions of the host system, the UoD and technology
X X
X
x X
X
Not clearly specified X X
x X X
Levels correspond to views of specific stakeholders
X x X
x x
X X X
Perspectives correspond to views of specific groups of people
“..loosely corresponds to the phases of software development” ([13] p. 287)
X X
x x
x X
X
X X
X X X
“..to focus on particular concerns within a system” ([22] Section 3.2.7)
“..necessary to form a system which is complete in both technical and human terms”
X X
X X
X X X
X X
X X X
X X
X
272
M. Leppänen / A Perspective Ontology and IS Perspectives
perspectives can be clearly distinguished. The first one (infological) is independent from representational and realization-dependent aspects. The second one (datalogical) is independent from realization-dependent aspects. The third one (physical) recognizes all the concrete issues related to a specific realization. 5. Summary and Conclusions In this paper we have presented a light-weight perspective ontology, which defines and organizes concepts and constructs by which diverse aspects of information processing can be categorized and examined according to the perspective(s) depending on the problem at hand. The ontology defines five perspectives established on the well-defined criteria and dimensions. The perspective ontology has been anchored on relevant theories (i.e. case grammar [12], pragmatics [36], activity theory [8], semiotics [47] and systems theory [43]) and engineered in accordance with the guidelines of ontology engineering [11, 58]. The paper has demonstrated how the perspectives can be applied in the contexts of IS, ISD and ME. To further concretize the conceptions of the IS perspectives, the basic IS concepts and IS constructs have been modelled and defined for each of the IS perspectives. The IS perspectives have also been used as a framework in a comparative analysis of relevant works in the IS literature. To our best knowledge there are no earlier suggestions for a perspective ontology with the same kind of purpose. The term “perspective ontology” does exist but in different meanings. For instance, in the philosophy there is the well-known Zhuang Zi’s perspective ontology according to which the being and identity of an entity is contextually situated and perspective-dependent [37]. On a general level, perspectives are discussed as levels of abstraction by several researchers (e.g. [23, 43, 44]) but they do not go into detail as we do in this paper. The perspective ontology can be deployed as groundwork to elaborate current frameworks and frames of reference, especially when it comes to the underlying criteria and dimensions. The perspectives can also be used as a foundational structure in analyzing the conceptual contents of ISD methods, and in engineering new methods by integration, adaptation and customization. Furthermore, they can be applied to recognize and categorize diverse contingency factors related to the IS, ISD and ME. In future research we concentrate on the refinement of a so-called perspective-based approach to method engineering [32] in which the perspective ontology is used as a cornerstone for a conceptual framework of ME. References [1] [2] [3] [4] [5] [6] [7]
Avison, D., Wood-Harper, A., Vidgen, R. & Wood, R. 1996. Multiview: a further exploration in information systems development. Maidenhead: McGraw-Hill. Booch, G., Rumbaugh, J. & Jacobson I. 1999. The Unified Modeling Language – user guide. Reading: Addison-Wesley. Bunge, M. 1977. Treatise on basic philosophy, Vol. 3: Ontology I: The furniture of the world. Dortrecht: D. Reidel Publishing Company. Burton-Jones, A., Storey, V., Sugumaran, V. & Ahluwalia, P. 2005. A semiotic metric suite for assessing the quality of ontologies. Data & Knowledge Engineering 55(1): 84-102. Chen, P. 1976. The entity-relationship model – toward a unified view of data. ACM Trans. On Database Systems 1(3): 9-36. Dietz, J. 2003. The atoms, molecule and fibers of organizations. Data & Knowledge Engineering 47(3): 301-325. Elmasri, R. & Navathe, S. 2000. Fundamentals of database systems. 3rd edition, Reading: AddisonWesley.
M. Leppänen / A Perspective Ontology and IS Perspectives [8] [9] [10] [11] [12] [13] [14] [15] [16] [17] [18] [19] [20] [21] [22] [23] [24] [25] [26] [27] [28] [29] [30] [31] [32] [33] [34] [35]
273
Engeström, Y. 1987. Learning by expanding: an activity theoretical approach to developmental research. Helsinki: Orienta-Konsultit. Essink, L. 1988. A conceptual framework for information systems development methodologies. In H. J. Bullinger et al. (Eds.) Information Technology for Organizational Systems. Amsterdam: Elsevier Science Pub., 354-362. Falkenberg, E:, Hesse, W., Lindgreen, P., Nilsson, B., Oei, J. L. H., Rolland, C., Stamper, R., van Asche, F., Verrijn-Stuart, A. & Voss, K. 1998. A framework of information system concepts, The FRISCO Report (Web edition), IFIP. Fernandez-Lopez, M., Gomez-Perez, A., Pazos-Sierra, A. & Pazos-Sierra, J. 1999. Building a chemical ontology using METONTOLOGY and the ontology design environment. IEEE Intelligent Systems & Theory Applications 4(1): 37-46 Fillmore, C. 1968. The case for case. In E. Bach & R. T. Harms (Eds.) Universals in Linguistic Theory. New York: Holt, Rinehart and Winston, 1-88. Freeman, M. & Layzell, P. 1994. A meta-model of information systems to support reverse engineering. Information and Software Technology 36(5): 283-294. Gruber, T. 1993. A translation approach to portable ontology specification, Knowledge Acquisition 5(2): 119-220. Gruber, T. 1995. Towards principles for the design of ontologies used for knowledge sharing. International Journal of Human-Computer Studies 43(5/6): 907-928. Gupta, D. & Prakash, N. 2001. Engineering methods from method requirements specifications. Requirements Engineering 6(3): 135-160. Halpin, T. 1998. ORM/NIAM Object-Role Modelling. In P. Bernus, K. Mertins & G. Schmidt (Eds.) Handbook on Information Systems Architecture. Berlin: Springer-Verlag, 81-101. Harmsen, F. 1997. Situational method engineering. University of Twente, Moret Ernst & Young Management Consultants, The Netherlands, Dissertation Thesis. Hautamäki, A. 1986. Points of views and their logical analysis. Helsinki: Acta Philosophica Fennica, Vol. 41. Herbst, H. 1995. A meta-model for business rules in systems analysis. In J. Iivari, K. Lyytinen & M. Rossi (Eds.) Advanced Information Systems Engineering. LNCS 932, Berlin: Springer, 186-199. Iivari, J. 1989. Levels of abstraction as a conceptual framework for an information system. In E. Falkenberg & P. Lindgren (Eds.) Information System Concepts: An In-Depth Analysis. Amsterdam: Elsevier Science Pub., 323-352. ISO 1996. Information Technology – Open Distributed Processing - Reference Model: Overview, 10746-1. Kangassalo, H. 1980. Structuring principles of conceptual schemas and conceptual models. Report A85, Department of Mathematical Sciences, University of Tampere, Finland. Kavakli, V. & Loucopoulos, P. 1999. Goal-driven business process analysis application in electricity deregulation. Information Systems 24(3): 187-207. Kelly, S., Lyytinen, K. & Rossi, M. 1996. MetaEdit+: a fully configurable multi-user and multi-tool CASE and CAME environment. In Y. Vassiliou & J. Mylopoulos (Eds.) Proc. of the 8th Conf. on Advanced Information Systems Engineering (CAiSE’96). Berlin: Springer, 1-21. Kinnunen, K. & Leppänen, M., 1996. O/A matrix and a technique for methodology engineering. Journal of Systems and Software 33(2): 141-152. Kruchten, P. 1995. Architectural blueprints – The “4+1” view model of software architecture. IEEE Software 12(6): 42-50. Langefors, B. 1971. Theoretical analysis of information systems. Lund, Sweden: Studentlitterature. Langefors, B. & Sundgren, B. 1975. Information systems architecture. New York: Petrocelli. Leppänen, M. 2005. An ontological framework and a methodical skeleton for method engineering, Dissertation thesis, Jyväskylä Studies in Computing 52, University of Jyväskylä, Finland. Leppänen, M. 2005. A context-based enterprise ontology. In G. Guizzardi & G. Wagner (Eds.) Proc. of the EDOC International Workshop on Vocabularies, Ontologies and Rules for the Enterprise (VORTE’05), Enschede, The Netherlands, CTIT Workshop Proceedings, 17-24. Leppänen, M. 2006. Conceptual evaluation of methods for engineering situational ISD methods. Software Process: Improvement and Practice 11(5): 539-555. Leppänen, M. 2007. Towards an abstraction ontology. In Y. Kiyoki, H. Kangassalo & M. Duži (Eds.) The 16th European – Japanese Conference on Information Modelling and Knowledge Bases (EJC 2006), to be printed by IOS in the Series of the “Frontiers on Artificial Intelligence”. Leppänen, M. 2007. A contextual method integration. In Proc. of the 15th Int. Conf. on Information Systems Development (ISD 2006). Springer-Verlag, Berlin (in print) Leppänen, M. 2007. Towards an ontology for information systems development – A contextual approach. In K. Siau (Ed.) Contemporary Issues in Database Design and Information Systems Development. Idea Group Inc. (in print)
274 [36] [37] [38] [39] [40] [41] [42] [43] [44] [45] [46] [47] [48] [49] [50] [51] [52] [53] [54] [55] [56] [57] [58] [59] [60] [61] [62]
M. Leppänen / A Perspective Ontology and IS Perspectives Levinson, S. 1983. Pragmatics, London: Cambridge University Press. Li, C. 1999. The Tao encounters the West. Explorations in comparative philosophy. Albany: State University of New York Press. Loucopoulos, P., Kavakli, V., Prekas, N., Rolland, C., Grosz, G. & Nurcan, S. 1998. Using the EKD approach: the modeling component. ELEKTRA – Project No. 22927, ESPRIT Programme 7.1. Lyons, J. 1977. Semantics. Volume I-II, Cambridge: Cambridge University Press. Maibaum, T.S. 1986. Role of abstraction in program development. In H. Kugler (Ed.) Information Processing 86, Amsterdam: Elsevier Science Pub, 135-142. Melao, N. & Pidd, M. 2000. A conceptual framework for understanding business processes and business process modeling. Information Systems Journal 10(2): 105-129. Mentzas, G., Halaris, C. & Kavadias, S. 2001. Modeling business processes with workflow systems: an evaluation of alternative approaches. International Journal of Information Management 21(2): 123-135. Mesarovic M., Macko D. & Takahara Y. 1970. Theory of hierarchical, multilevel, systems. New York: Academic Press. Mustonen, S. 1978. Tavoitteisen järjestelmän kyberteettinen analyysi päätäntäteorioiden ja systemoinnin metatutkimuksessa (in Finnish). Report A7, Institute of Data Processing Science, University of Oulu, Oulu. Olive, A. 1983. Analysis of conceptual and logical models in information systems development methodologies. In T. Olle, H. Sol & C. Tully (Eds.) Information Systems Design Methodologies: A Feature Analysis. Amsterdam: Elsevier Science Pub., 63-85. Olle, T., Hagelstein, J., MacDonald, I., Rolland, C., Sol, H., van Assche, F. & Verrijn-Stuart, A. 1988. Information Systems Methodologies – A Framework for Understanding. 2nd edition. Reading: Addison-Wesley. Peirce, C. 1991. The essential Peirce, Vol. 1, edited by N. Houser & c. Kloesel. Blooomington: Indiana University Press. Phalp, K. 1998. The CAP framework for business process modeling. Information and Software Technology 40(13): 731-744. Ralyté, J., Deneckere, R. & Rolland, C. 2003. Towards a generic model for situational method engineering. In J. Eder & M. Missikoff (Eds.) Proc. of the 15th Int. Conf. on Advanced Information Ssystems Enginering (CAiSE’03). LNCS 2681, Berlin: Springer-Verlag, 95-110. Slooten van, K. & Hodges, B. 1996. Characterizing IS development projects. In S. Brinkkemper, K. Lyytinen & R. Welke (Eds.) Proc. of the IFIP TC8 WG8.1/WG8.2 Working Conf. on Method Engineering: Principles of Method Construction and Tool Support. London: Chapman & Hall, 29-44. Sol, H. 1992. Information systems development: a problem solving approach. In W. Cotterman & J. Senn (Eds.). Challenges and Strategies for Research in Systems Development. New York: John Wiley & Sons, 151-161. Song, X. 1997. Systematic integration of design methods. IEEE Software 14(2): 107-117 Sowa, J. & Zachman, J. 1992. Extending and formalizing the framework for information system architecture. IBM Systems Journal 31(3): 590-616. Sundgren, B. 1975. Theory of data bases. New York: Petrocelli/Charter. Swede van, V. & van Vliet, J. 1993. A flexible framework for contingent information systems modeling. Information and Software Technology 35(9): 530-548. Tolvanen, J.-P. 1998. Incremental method engineering with modeling tools – Theoretical principles and empirical evidence. Dissertation Thesis, Jyväskylä Studies in Computer Science, Economics and Statistics, No. 47, University of Jyväskylä, Finland. Uschold, M. 1996. Building ontologies: towards a unified methodology. In Proc. of 16th Annual Conf. of the British Computer Society Specialist Group on Expert Systems. Cambridge, UK Uschold, M. & King, M. 1995. Towards a methodology for building ontologies. In Workshop on Basic Ontological issues in Knowledge Sharing, held in conjunction with IJCAI’95, Montreal, Canada. Wand, Y. & Weber, R. 1995. On the deep structure of information systems. Information Systems Journal 5(3): 203-223. Welke, R. 1977. Current information system analysis and design approaches: framework, overview, comments and conclusions for large – complex information system education. In R. Buckingham (Ed.) Education and Large Information Systems. Amsterdam: Elsevier Science Pub., 149-166. Yourdon, E. 1989. Modern structured analysis. Englewood Cliffs: Prentice-Hall. Zachman, J. 1987. A framework for information systems architecture. IBM Systems Journal 26(3): 276-292.
275
M. Leppänen / A Perspective Ontology and IS Perspectives
Appendix IS action
1..* HIS purpose
strivesFor
UI
Dialog *
1..* HIS rule
1..*
* governs
* *
1..* 1..* responsibleFor
*
1..*
HIS action
*
Window
navigation
* operatesWith
1..*
input output * *
1..* *
* UI component
IS role Data object 1..* 1..* contains 1..* *
1 UI data
* IS position
UI data comp.
UI action comp.
supervision *
*
*
*
CIS rule
1..* nonDigital
UI state
1 IS org.unit
precedes
* *
1..*
*
*
CIS action
Transaction 1..*
*
1..*
*
*
UI transition
* causedBy *
1
governs
*
1
resultsIn
Digital
*
governs
1..*
*
IS organization
1..*
implements
presents 1..*
* triggers *
Algorithm
UI event
HIS
CIS
output input
Figure A.1. IS datalogical perspective (HIS = Human Information System, UI = User Interface, CIS = Computerized Information System)
1..*
1..* Data storage
allocated
Application SW
Memory device
1..* 1..*
Data file
Data base
SW component
1..*
*
Processor
1..* 1..*
1
1..* allocated
Record
Layer
1
1
Data message 1..*
1..*
1..* 1..* transmittedThrough 1..*
Data field
1..*
Node 1..*
1
connects 1..* 1..*
1..*
Communication line
SW architecture situated
1..* applies 1..*
1..* 1
1..*
1 1..*
Protocol
PhysicalLocation
HW architecture
CIS 1..*
Figure A.2. IS physical perspective covering a part of the CIS
276
Information Modelling and Knowledge Bases XIX H. Jaakkola et al. (Eds.) IOS Press, 2008 © 2008 The authors and IOS Press. All rights reserved.
The improvement of data quality – a conceptual model Tatjana WELZER, Izidor GOLOB, Boštjan BRUMEN, Marjan DRUŽOVEC, Ivan ROZMAN Faculty of Electrical Engineering and Computer Science, University of Maribor Smetanova 17, Maribor, Slovenia {welzer, izidor.golob, bostjan.brumen, marjan.druzovec, ivan.rozman}@uni-mb.si Hannu JAAKKOLA Tampere University of Technology Pori, Finland [email protected] Abstract. Usage of data in various areas and its electronic availability has upgraded the importance of data quality to the highest level. In general, data quality has at least a syntactic and a semantic component. The syntactic component is relatively easily reached, mostly supported by tools, while the semantic component requires further research. In many cases, data is taken from different sources which are distributed among enterprises and vary in levels of quality. Special attention needs to be paid to data upon which critical decisions are met. In the paper we will focus on data quality in connection with conceptual modeling, including reuse of models and/or parts of them and data policy for increasing the quality of data.
Introduction Database design is concerned with arranging data required by one or more applications in an organised structure. We are facing an increasing demand for more and more complex applications on databases. This rapid growth has stimulated the need for higher level concepts, tools and techniques for database design and development. At the beginning, when databases first entered the information system market, database designers needed to invest a lot of hard work to the development of databases which was then only supported by very rough tools. But nowadays, however, designing databases has become a popular activity, performed not only by database designers, but also by nonspecialists, raising the issue of a possible inflation of quality, either as far as the database itself is concerned or concerning the saved data in the database. The starting point for the design of a database is mostly some abstract and general description of the reality, namely a conceptual model which is developed in the first phase of the database design. In the context of the database design, the conceptual model has various usages [cf. Frost 1986]: x at the start of the database design, it should integrate various interests and views of the end user; x it is a useful description for communication with users as well as for communication with non-specialists;
T. Welzer et al. / The Improvement of Data Quality – A Conceptual Model
277
it helps the database designer to build a more durable database system; it enables efficient introduction of the already designed database. To achieve the above mentioned usages of the conceptual model as well as in order to design an effective and high quality model, we must take into consideration that the conceptual database design is extremely complex and iterative [Ramamoorty 1989]. It can be greatly simplified by using different database design aids (methodologies and tools). Design methodologies for conceptual design should be rigorous as well as flexible. The methodologies should be based on a formal approach and also be applicable to a variety of situations and environments. Additonally, they should take care also about the data data quality and consecutively on information quality. Many approaches to improve information quality have been developed in the last years and have been implemented in various situations. Most of these approaches are only vaguely familiar with the fact that data quality is a prerequisite for the final quality of information. Data are of high quality if they are suitable for being used in a database. Data appear to be suitable for usage if they are free of damages and possess desired features [7],[1]. In information technology, aggressive steps to improve the data quality are being done. In our contribution, we will concentrate on the problem of improving data quality on basis of developing a conceptual model. In the following chapters we will be concentrating on data quality in general, as well as on possible data policy. Further research and final remarks will be presented in the conclusion. x x
1. Quality of Data The quality of data limits the ability of the final user to make correct decisions, which can have fatal consequences. There are a number of indicators which relate to the quality of data: accuracy, integrity, consistency, accessibility, comprehensives, timeliness and completeness, among others. The data must follow business rules and be free from anomalies. Although being a subjective measurement, the user's satisfaction with the quality of the data and the information derived from it, is arguably the most important indicator of them all [12],[2]. There are many reasons why it is difficult to capture and maintain quality data. Some of the difficulties are process-related, some are human-related problems, and, furthermore, other obstacles have their source in technology itself. All of the problems result in bad data, either from the semantic or syntactic point of view. Usually, these two components are interrelated and inconsistent. Process-related problems are frequently caused by the user by entering the data into an operational system at the wrong point of the business process or by lacking the understanding for the meaning of the data [13]. Difficulties with employees entering incorrect data into systems can be decreased by changing the emphasis on pure speed of processing to the quality of processing, where quality is composed of both speed and accuracy. Regardless of the source of the problems, it is important to identify the source of the problem, analyze its impacts and, where possible, propose a solution. We have to be aware of the fact that usually the customers vary in their needs and this might lead to a conflict. Additionally, the customers´ needs change all the time and what was good enough one day (talking about suitable data quality), is simply not good enough the next day (does not meet the data quality) [8]. The issue of data quality is an issue particularly important in data warehouses and data mining, especially in combination with sensitive domains (e.g. medicine, energy, flights).
278
T. Welzer et al. / The Improvement of Data Quality – A Conceptual Model
The introduction of sensitive domains increased the priority of data quality, as the risks and costs of inadequate quality become more visible and more real, and after all, more expensive [5]. The problem of poor data quality is one of the most difficult problems to be solved while constructing any conceptual model [8]. Because of bad data quality, more time and money is spent than anyone assumed initially. Pyle [6] estimates that the data preparation sub-process can take up to 90% percent of the time and money available for the whole system development. Data quality is also connected to the conceptual model while during its development we are checking data through defining entities, relationships and especially attributes describing both of them. Mostly in this phase the emphasis is on the syntactic and semantic quality, but some authors define additional types of data quality (e.g. physical quality, perceived semantic quality, pragmatic quality, social quality, language quality and knowledge quality) (Krogstie, 1998).
2. Conceptual modeling and data quality As mentioned before, we are connecting conceptual modeling with data quality which is a very likely and natural connection. In our work we pointed out mainly the syntactic quality (correspondence between the model and the tool in which the model is written (Krogstie, 1998)) and semantic quality which is presenting the correspondence between the model and the domain. The rest of possible data quality parts are not the topic of this paper, but we would like to point out that in one way or another they are involved or connected mostly with the semantic quality. According to the before mentioned definition for semantic quality, we have to bring up two semantic goals: validity – all statements made in the model are correct and relevant to the domain (no invalid statements) and - completeness (also a component of a data policy structure): the model contains all statements that would be correct and relevant about the domain. So it becomes apparent that the role of the domain is very important with a strong focus on data quality. We have to be very careful with those domains which are related to very sensitive data like that in the medical environment. Medical environments require special taking care for data because of their dual nature. First, the business part is required, which is just as complex as the business part in any other enterprise. Second, the medical part needs to be interwoven with the business part. In a general business system, we usually do not have such a duality. The medical part has its own specifics and requirements and if it is, in general, possible to reuse business objects (data models, applications) from different enterprises for the business part of a medical system [11], [4] we cannot apply the same for the medical part alone. As mentioned, it is essential that the medical part is tightly connected with the business part. Decisions made in a medical environment are very sensitive because they affect the “business object” (the patient) directly. Any decision, being it managerial or medical, can have fatal and devastating consequences. Moreover, the data should provide a firm foundation for information retrieval and, furthermore, for knowledge discovery. From the data quality point of view, we have to assure quality in each of the sub-systems. Additionally, the final (integrated) medical system, composed of both the business and medical part, needs to be validated again concerning data quality. The integrated high quality systems are the desired goal of system designers. To reach the goal, the following steps have to be followed: x Separate data issues from more traditional technical issues and assign lead responsibility for data to someone within the medical community.
T. Welzer et al. / The Improvement of Data Quality – A Conceptual Model
279
x
As with all quality efforts, the needs have to be understood correctly. Additional explanations and comments are welcome. x Already existing data have to be checked again. The importance of additional checking is growing, as the sensitivity of the domain increases (medical environment). We want also to emphasize that an organization sometimes may be doing better not having certain data (responsibility of community, request for additional check) than having inaccurate data, especially if those relying on the data are not aware of its inaccuracy. For example, a hospital would be put to a better position not knowing a patient's blood type than wrongly believing it to be O+. A problem of semantic data quality is evident. How can all these problems be solved? One of the possible solutions is to introduce data policy.
3. Data Policy A policy is a plan, course of actions or set of rules intended to influence and determine decisions, actions and other matters [9]. The origin of defining data policy is the assumption that the responsibility for the quality of data has to be assigned to those who create the data or to those who are as close to data creation as possible. That means that the data policy supports the work of these people through suggestions and rules that they have to follow or at least take into consideration. With the aim to define easily used, but nevertheless, powerful policy, we are suggesting the following structure: x Introduction (purpose, audience, definitions, related work, basic approach, responsibility, comments) x Data policy (data policy and its benefits, components of a good data policy, needs/reasons for a successful data policy, comments) x Structure (objectives, categories of data policy) x Domain rules (specific environment rules) x Syntactic and/or other data quality types x Dimensions of data quality (relevance, availability, clarity of definition, comprehensiveness, accuracy, integrity, homogeneity, structural consistency, consistency (semantic consistency), accessibility, security, timeliness completeness, portability) x Others (definitions, homogeneity, naming, redundancy, comments…) x Policy management (responsible for reviews, schedule of reviews, recommendation for reviews, policy issuance and revision date, comments) We have already mentioned that the focus in this paper is on the dimension of data quality (availability, security, comprehensiveness, flexibility, appropriate use, semantic consistency, simplicity, relevancy, completeness, consistency, portability, naming, relevancy, concurrency, definitions, robustness, homogeneity, redundancy). Furthermore, we set out especially those dimensions (cf. the policy structure) which assure data quality by considering conceptual models and/or parts of them also concerning the system of reusability – reusable components are already existing models and/or just confirmed parts of them (Welzer, 2004): x Relevance - objects needed by the applications are included in conceptual models. x Clarity of definition - all terms used in the conceptual model are clearly defined. x Comprehensiveness - each needed attribute should be included.
280
T. Welzer et al. / The Improvement of Data Quality – A Conceptual Model
x x
Occurrence identifiably - identification of the individual objects is made easy Homogeneity, structural consistency - object level enables the uniformity of stored concepts. x Minimum redundancy - only checked conceptual models are included. x Semantic consistency - conceptual models are clear and organized according to the application domains. x Robustness, flexibility - through the reuse both characteristics are fulfilled. x Security – reusability is increasing security, even that each specific domain could have especially needs and demands. x Appropriate use – in general, this varies from domain to domain, but the reusability raises the trustworthiness x Availability – specific domains could have especially needs or demands for it. It becomes obvious that those dimensions are connected both to the data policy structure (categories) and to different quality types, especially the semantic one.
4. Conclusion Medicine is a specific environment and thus creates a form of knowledge discovery for data and about data. Not only that the data need to be accurate, valid and timely (the semantic component of the data quality), but also the structures, from which we obtain the required data, have to be valid and syntactically correct. Too often the structures are neglected or taken for granted. For instance, if the information or knowledge obtained from the data does not satisfy users’ wishes, needs and demands, the process needs to be reverted to the previous steps – obtaining the data. But this does not lead to the desired outcome, since the structures are not suitable for the task. We argue that the data (information, knowledge) quality heavily relies on the data policy whose structure is introduced in the paper. For future research more practical results of introducing data policy in specific environments (medicine) are expected to confirm the data policy structure or to introduce some changes in dimensions (additional dimensions, changes for existing dimensions, withdrawing of existing dimensions) and/or data quality types. The result will not so easily be reached because medical data are very sensitive data, as we have mentioned, and most of them are secure and protected in various ways. So we will get quite easily access to statistical data (public databases are available), but much more difficult to handle will be the situation with conceptual models on which we have to check our conclusions on data quality types, data policy structures and influence of the conceptual model on the data quality.
References [1] Welzer, T. and Rozman, I. (1998): Information Quality by MetaModel. In Proceedings of Software Quality Management VI. Quality improvement issues. 81-88. HAWKINS. C (eds). Springer. London. [2] Welzer, T., Brumen, B., Golob, I. and Družovec, M. (2002): Medical diagnostic and data quality. In Proceedings of 15th IEEE Symposium on computer-based medical systems. 97-101. KOKOL P., STIGLIC B., ZORMAN M. and ZAZULA, D. (eds) IEEE Computer society. Los Alamitos. [3] Welzer, T. and Družovec, M. (2000): Similarity search in Database Reusability – a Support for efficient design of conceptual models. In Contemporary Applications and Research Issues in Industrial Product Moddeling. 23-34. HELLO. P. and WELZER. T. (eds). University of Vaasa.
T. Welzer et al. / The Improvement of Data Quality – A Conceptual Model
281
[4] Freeman, P. (1987): Reusable Software Engineering Concepts and Research Directions. In IEEE Tutorial Software Reusability. FREEMAN. P. (ed.). IEEE. [5] English, L.P. (1999): Improving data warehouse and Business Information Quality. John Wiley & Sons. [6] Pyle, D. (1999): Data Preparation for Data Mining. Morgan Kaufmann Publishers, Inc., San Francisco, California, USA. [7] Redman, T.C. (1996): Data Quality for the Information Age. Artech House. [8] Redman, T.C. (2001): Data Quality, The field Guide. Digital Press, Boston, USA. [9] Whitham, M.E. and Mattord (2003): Principles of information Security. Thomson. Canada. [10] Reiter, R. (1987): A Theory of Diagnosis from First Principles. Artificial Intelligence 3(2):57-95. [11] Rine, D.C. (1997): Supporting Reuse with Object Technology. IEEE Computer, 30(10): 43-45. [12] Tayi, G.K. and Ballou, W (1998): Examining Data Quality. Communications of the ACM, 4(12):54-57. [13] Welzer T., Brumen B., Golob I., and Sanchez J.L., Družovec M. (2004): Diagnostic process from the data quality point of view, Journal of Medical Systems, Kluwer.
282
Information Modelling and Knowledge Bases XIX H. Jaakkola et al. (Eds.) IOS Press, 2008 © 2008 The authors and IOS Press. All rights reserved.
Knowledge Cluster Systems for Knowledge Sharing, Analysis and Delivery among Remote Sites Koji ZETTSU a , Takafumi NAKANISHI a , Michiaki IWAZUME a , Yutaka KIDAWARA a , and Yasushi KIYOKI a,b a National Institute of Information and Communications Technology, Japan b Keio University, Japan Abstract. We, NICT, recently started a new project for research and development of “knowledge cluster systems” for knowledge sharing, analysis, and delivery among remote knowledge sites. We introduce several key concepts of the knowledge cluster systems. The “Three-site model” for knowledge system architecture defines three roles of remote sites: knowledge capture, knowledge transfer, and knowledge provision, with respect to the lifecycle of knowledge communication. The “global knowledge grid” is as an infrastructure that is suitable for implementing knowledge cluster systems on the basis of the three-site model. The knowledge cluster systems build an evolving network of community knowledge by connecting heterogeneous knowledge bases. The “global risk management system” is being developed as an application of the knowledge cluster systems. Keywords. Knowledge cluster systems, three-site model architecture, global knowledge grid, connection of heterogeneous knowledge bases
Introduction In today’s networked society, knowledge-intensive work involves a significant amount of communication, coordination, and cooperation practices that cross the boundaries of organizations, counties, cultures, and/or disciplines. As a motivating example, let us consider managing risks against natural disasters like Tsunami, volcanic eruptions, or avian flu. Natural disasters cause damages in various fields like health, economy, natural ecosystems, and so on. Moreover, the damage may spread beyond national boundaries. Therefore, experts from various fields need to collect and analyze information related to disasters. In addition, disaster victims and relatives need to be adequately informed. We believe that it is important for next-generation knowledge systems to place a particular emphasis on knowledge communication in order to manage knowledge in a world of networks [1]. While knowledge systems will be characterized by weakly structured and less predictable processes, in order to make communication stronger, we need to share, analyze and deliver knowledge among
K. Zettsu et al. / Knowledge Cluster Systems for Knowledge Sharing, Analysis and Delivery
283
Figure 1. Basic concept of global risk management system (for hot mud flood disaster in Indonesia).
remote knowledge sites. We introduce a concept of “knowledge cluster systems”, and explain our efforts toward their realization, especially in the context of global risk management against natural disasters.
1. Background The knowledge cluster system project started from April 2006 as a five-year research project of the National Institute of Information and Communications Technology (NICT), Japan. The main objective is to research and develop nextgeneration knowledge infrastructure in the networked age. The knowledge infrastructure consists of three functional layers: distributed knowledge access, knowledge computing and analysis, and knowledge presentation media. The “Global risk management system” is proposed as the first application, which aims to evaluate the impacts of various risks caused by natural disasters in a global context. It is a good example of knowledge-communication-intensive work, as described above. From February 2007, NICT and Electronic Engineering Polytechnic Institute of Surabaya, Institut Teknologi Sepuluh Nopember (EEPISITS) in Indonesia started a joint project on the research and development of global risk management system for natural disasters, especially for hot mud flow disaster. The concept of the global risk management system is illustrated in Figure 1. The meta-level architecture of the global risk management is shown in Figure 2. The local risk management subsystem focuses on shallow but real-time analysis of local risks, while the meta-level risk management subsystem focuses on deep but non-real-time analysis of global risks. To design the technology that is used in knowledge cluster systems, NICT held the First International Workshop on Knowledge Cluster Systems in March 2007 at Kyoto, Japan. The participants were from EEPIS-ITS (Indonesia), Tampere University of Technology (Finland), Christian Albrechts University at Kiel
284
K. Zettsu et al. / Knowledge Cluster Systems for Knowledge Sharing, Analysis and Delivery
Meta-level Risk Management Subsystem NICT
Global Risk Management Knowledge Base
Globalize local problems
EEPIS-ITS
Monitor local disaster events
Exploit global solutions Local Risk Management Subsystem for Indonesia
Local Risk Management Subsystem for Japan
Indonesian Risk Management Knowledge Base
Japanese Risk Management Knowledge Base
(to be added)
Localize global solutions in dependence upon local situations Inform the general public about local situations
Figure 2. Meta-level architecture of global risk management system.
(Germany), VSB-Technical University of Ostrava (Czech Republic), Saga University, Keio University, and Kanagawa Institute of Technology (Japan).
2. Three-site Model Knowledge System Architecture In the networked society, various communities organize their own knowledge repositories, each of which aggregates perception, skills, training, common sense, and experience of a community of people. Knowledge cluster systems are used to facilitate knowledge communication across the boundaries of these communities. The lifecycle of knowledge communication consists of the following three phases: Knowledge capture: deriving knowledge from information as an understanding of the information depending on the discipline or context where it is used. Knowledge transfer: conveying (or projecting) the knowledge of one community to another community. It can be considered to be a process of transmission and absorption. Knowledge provision: providing actionable information available in the right format, at the right time, and at the right place. In knowledge cluster systems, the above three phases are defined as three different roles of remote sites. We propose the “three-site model” for the knowledge cluster systems architecture, in which remote sites play the above three roles and thus realize the knowledge communication. The three-site model in the global risk management system is illustrated in Figure 3. The sites located in/around the disaster area will play the “knowledge capture” role (site-1) in order to capture the knowledge about the disaster by col-
K. Zettsu et al. / Knowledge Cluster Systems for Knowledge Sharing, Analysis and Delivery
285
Site-2: Knowledge Transfer Knowledge Base
Site-1:Knowledge Capture Correlation
Site-3:Knowledge Provision
Disaster information Information analysis
Mobile Phones
sensor Sensor Manager
DB1
DB2
Web
GIS Visualization
Sensor Manager
3D Visualization
Sensor Manager
Observe social situations and natural environments
• Analyze disaster information from Site-1 • Evaluate risks by employing various knowledge bases • Send risk information to Site-3
Warning Center
Aggregate, select and convert risk information from Site-2 for intended recipients
Figure 3. Three-site model knowledge system architecture in global risk management system.
lecting and organizing the disaster information on site. The remote sites playing the “knowledge transfer” role (site-2) evaluates the impacts on various risks by analyzing the disaster knowledge from the site-1. In the course of the risk analysis, various communities (or domains) of knowledge are employed (e.g., healthcare knowledge, economic knowledge, and ecosystem knowledge). The risk knowledge discovered by the site-2 is sent to the remote sites playing the “knowledge provision” role (site-3). At the site-3, various risk knowledge are aggregated, selected, and converted into the right format for the intended recipients. For example, local disaster victims of a local disaster, will be informed of the first-aid actions in real time. On the other hand, policy decision makers will be provided with , comprehensive risk information on demand. In the course of the knowledge provision, the risk knowledge may be localized, personalized, and/or adapted on the basis of the situations of the intended recipients. Note that the application requirements and capability of the remote sites will significantly affect how the three roles are assigned to the remote sites. For example, to manage local risks in real time in the early phase of a disaster, all of the three roles may be assigned to the local site in a disaster area. As the damage spreads to other regions and/or various fields, the knowledge transfer role will be assigned to those remote sites, which have enough knowledge to evaluate the risks, and the knowledge provision role will be assigned to the remote sites influenced by the disaster. 3. Global Knowledge Grid: An Infrastructure for Knowledge Cluster Systems The “global knowledge grid” is an infrastructure for implementing knowledge cluster systems based on the three-site model. The concept of the knowledge grid
286
K. Zettsu et al. / Knowledge Cluster Systems for Knowledge Sharing, Analysis and Delivery
Figure 4. Global knowledge grid as an infrastructure for knowledge cluster systems.
has recently emerged as an integrated infrastructure for coordinating knowledge sharing and problem solving in distributed environments. The knowledge grid was originally presented as the implementation of parallel and distributed knowledge discovery (PDKD) on top of a computational grid[2]. The knowledge grid uses the basic functions of a grid and defines a set of additional layers to implement the functions of distributed knowledge discovery. The knowledge grid enables collaboration between knowledge providers who must mine data stored in different information sources, and knowledge users who must use a knowledge management system operating on several knowledge bases. Our intentions are (1) to make the knowledge grid accessible to anyone via a global network (i.e., the Internet) and (2) to implement an additional layer on top of the knowledge grid in order to provide the capability of knowledge communication. The additional layer, known as “knowledge cluster service layer”, comprises of software modules that provide the services defined in the three-site model (i.e., knowledge capture, knowledge transfer, and knowledge provision). The knowledge cluster services are developed by the knowledge grid nodes in parallel. The knowledge cluster service layer is organized on the basis of a serviceoriented architecture (SOA)[3]. The application of the knowledge cluster system is developed by composing the knowledge cluster services (e.g., service mash-up), as illustrated in Figure 4. Our challenge is to develop a mechanism for discovering optimal composition of the knowledge cluster services with respect to various requirements of knowledge communication. In contrast with traditional SOA-based applications like business transactions, in which service composition can be defined previously based on standardized business protocol (e.g., BPEL[4]), the knowledge cluster application requires dynamic assignment of knowledge cluster services. For example, in the global risk management system, the knowledge capture services will be assigned to the grid nodes in disaster area, while the knowledge transfer services will be on the grid nodes which have the knowledge for evaluating the risks, and
K. Zettsu et al. / Knowledge Cluster Systems for Knowledge Sharing, Analysis and Delivery
287
the knowledge provision services will be on the grid nodes near to the disaster victims. The knowledge cluster applications need to define the conditions under which the knowledge communications are activated or deactivated. At runtime, the knowledge cluster services that satisfy the conditions are bound dynamically. The dynamic binding of knowledge cluster services will be discussed in our future work.
4. The Web of Knowledge: Connecting Heterogeneous Knowledge Bases Cross-boundary knowledge communications builds an evolving network of community knowledge. In the same way as the World Wide Web, the knowledge cluster systems provide a framework of infinitely-evolving knowledge repository by connecting heterogeneous knowledge bases owned by different communities. A typical example of the connection is based on causal relation from one knowledge base to another knowledge base. For instance, a disaster knowledge base can be connected with healthcare knowledge base by establishing a causal relation in order to find diseases caused by specific disasters. In that way, “a web of knowledge” will be formed. Connecting two different knowledge bases requires a “bridge concept”. In conventional approaches, schema mappings and bridge ontology are typically used as the bridge concept. They try to pre-define universal relations between two different communities of knowledge, while it is quite difficult in most cases. As a result, conventional approaches can only work on a small scale. As a workaround, most recent approaches try to employ word-level universal relations, like those found in a thesaurus or on the WordNet lexicon. However, in these approaches, knowledge is ideally broken into a bag of words with losing its contextual information. In order to enhance the scalability of the knowledge base connection, we have put defining the universal relations to one side, and focused on finding contextdependent correlations between different communities of knowledge. For example, in the context of health risk against hot mud flood disaster, knowledge about volcanic gases in the disaster knowledge base may have correlations with knowledge about respiratory organs illness in the healthcare knowledge base. Especially, the correlation between hydrogen sulfide (H2 S) and pulmonary edema may be stronger than any other correlations. In this way, we intend to develop a mechanism for first managing the contexts for evaluating the correlations between different communities of knowledge, and second, measuring the strength of correlations in each context, simultaneously. We are now developing the above mechanism based on the semantic space model[5], and. how it works is shown in Figure 5 . In the semantic space model, knowledge is represented by a vector. The basic idea is to project the vector from one semantic space to another semantic space, then search the target space for the vectors highly similar to the projected vector. The strength of correlation is measured by the similarity value. The context is given by the vector projection function (e.g., causal relation matrix in Figure 5). The correlation measurement approach allows us to discover on demand the knowledge related to the given knowledge in the given context. It also allows ambiguity or uncertainty of the
288
K. Zettsu et al. / Knowledge Cluster Systems for Knowledge Sharing, Analysis and Delivery
Figure 5. Connecting heterogeneous knowledge bases by causal relations based on semantic (vector) space model.
bridge concept. We plan to include time scale and geographical scale in the bridge concept, because they have a high affinity with the correlation measurement approach.
5. Future Work In response to the results of the First International Workshop on Knowledge Cluster Systems, we have expanded the scope of our research and development to include the following topics under the collaborations with international partners. Software engineering for knowledge cluster systems: (1) software development framework for knowledge cluster services, and (2) service mediation for dynamic binding of knowledge cluster services. Towards “Web 3.0”: (1) collaboration architectures on demand with the collective intelligence, and (2) treatment of collaborative data with social interaction and community management. Quality-driven content and information mining: (1) knowledge discovery depending on the source characteristics, portfolio and tasks, intentions of the mining, and (2) development of content and information mining workbench.
K. Zettsu et al. / Knowledge Cluster Systems for Knowledge Sharing, Analysis and Delivery
289
Integration of sensor data analysis and knowledge analysis: (1) shallow analysis on real-time basis for warning systems, evacuation information systems, and (2) deep analysis on non-real-time basis for social/global impact analysis. Adequate and comprehensive information provision: (1) navigation using geographical data and navigation facilities based on mobile/ubiquitous technology, and (2) involving humanitarian organizations
6. Conclusions We introduced the knowledge cluster system project conducted by NICT, Japan. The main objective of the project is to create an infrastructure for knowledge communication and distribution in a world of networks. We proposed several key concepts of the knowledge cluster systems. The‘ ‘Three-site model” for knowledge system architecture defines three roles of remote sites: knowledge capture, knowledge transfer, and knowledge provision, with respect to the lifecycle of knowledge communication. The “global knowledge grid” is being developed as an infrastructure that is suitable for implementing knowledge cluster systems on the basis of this model. An application of the knowledge cluster systems, “the global risk management system,” is being developed as a joint research project between NICT and EEPIS-ITS. We also discussed building an evolving network of community knowledge by connecting heterogeneous knowledge bases. The knowledge cluster system project will continue until 2011. To ensure the success of the project, we are looking for international collaborations including but not limited to the following: • Research and development of: (1) the global knowledge grid, (2) knowledge discovery and data mining, (3) knowledge bases, (4) knowledge presentation media, and (5) applications of knowledge cluster systems including global risk management system. • Field experiments and technology demonstrations. • Exchange program for researchers and/or students. References [1]
[2]
[3] [4] [5]
Zettsu, K. and Kiyoki, Y.: Towards Knowledge Management based on Harnessing Collective Intelligence on the Web, Proceedings of the 15th International Conference of Knowledge Engineering and Knowledge Management – Managing Knowledge in a World of Networks – (EKAW2006), Lecture Notes in Computer Science Vol. 4248 pp.350–57 (2006). Cannataro, M. and Talia, D.: The Knowledge Grid: Designing, Building, and Implementing an Architecture for Distributed Knowledge Discovery, Communications of the ACM, Vol. 46, No. 1, pp.89–93 (2003). Papazoglou, M. P. and Georgakopoulos, D.: Service-Oriented Computing, Communications of the ACM, Vol. 46, No. 10, pp.24–28 (2003). Fu, X., Bultan, T. and Su, J.: Analysis of Interacting BPEL Web Services, Proceedings of the 13th international conference on World Wide Web, pp. 621 - 630 (2004). Kiyoki, Y. Kitagawa, T. and Hayama, T.: A Metadatabase System for Semantic Image Search by A Mathematical Model of Meaning, ACM SIGMOD Record Vol.23 No.4 pp.34– 41 (1994).
290
Information Modelling and Knowledge Bases XIX H. Jaakkola et al. (Eds.) IOS Press, 2008 © 2008 The authors and IOS Press. All rights reserved.
A Formal Ontology for Business Process Model TAP: Tasks-Agents-Products Souhei ITO, Shigeki HAGIHARA and Naoki YONEZAKI Department of Computer Science, Graduate School of Information Science and Engineering, Tokyo Institute of Technology Abstract. TAP is a general process modeling framework with which we can describe process specifications in terms of tasks, agents and products that are relevant to the concept of processes. However, its formal semantics and pragmatics have not been rigorously studied, i.e. the usage of vocabulary words provided by TAP framework and the pieces of reality which TAP framework captures has not been formally considered. In this paper, we clarify the pragmatics of TAP and the world structures of TAP process models by formal approach. We present the semantic structure for TAP process models and introduce a logical language to restrict world structures of TAP. A formal ontology for business process model TAP is the characterization described as a set of axioms in our logical language.
1 Introduction In the fields of business process modeling, there are several ontologies, e.g. the AIAI Enterprise Ontology [6], the Toronto Virtual Enterprise Ontology (TOVE) [1], the Resource Event Agent (REA) Enterprise Ontology [5, 2] and e3 -valueTM [3]. Generally, the term “ontology” is thought of as specifications of concepts and relationships between concepts. This definition of ontology permits several forms of ontologies. In fact, the above ontologies are not defined in the same way. TAP (Tasks-Agents-Products) [7] is also a business process modeling framework. Things which are relevant to processes are modeled as modeling objects in TAP. First class modeling objects in TAP are tasks, agents and products. Processes are modeled by describing relations between objects and behavioral specifications. Therefore, TAP can also be viewed as an business ontology. To precisely understand the business processes which each ontology captures and to compare these ontologies rationally (e.g. what is the common concept), we think the formal approach is essential. Therefore, we introduce the formal semantic structures corresponding to pieces of reality and a logical language to describe the ontologies. This type of ontology is sometimes referred to as formal ontology. In this paper, we use TAP as a general business process model to apply formal approach, since TAP has specific concepts such as an enaction of agents or tools, meta-tasks, abstractinstance level objects etc. which other business ontologies lack. To formalize these characteristics is a very challenging issue. First, we give the semantic structure of TAP. Then, the formal vocabulary to describe TAP specifications and the logical language to describe axioms are introduced. The semantics of this logical language is defined according to the semantic structure. Finally we present axioms to specify the characteristics of the intended world structures (concepts and relationships between them) of TAP in our logical language. It also constrains the consistent usage
S. Ito et al. / A Formal Ontology for Business Process Model TAP: Tasks-Agents-Products
291
of the vocabulary words in TAP and can be viewed as the pragmatics of TAP specifications. This set of axioms is a formal ontology for business process model TAP. This paper is organized as follows. Section 2 summarizes the business process model TAP. Section 3 introduces the semantic structure for TAP. Section 4 introduces the logical language LTAP for describing axioms which specify the characteristic of the world structures of TAP. In Section 5 we present the set of axioms. The final section is the conclusion.
2 Business process model TAP A process is modeled in terms of modeling objects, that are related to the important concepts in business processes. Modeling objects are either abstract level modeling objects or instance level modeling objects. The former are abstract description of something that may exist during an actual process performance and are used to describe a generic process model that covers various possible situations. The latter denote the actual observable things or phenomena that appear in the process and can be viewed as descriptions of their real counterparts. Figure 1 illustrates the modeling objects in TAP approach and the round boxes stand for them. A pair of modeling objects can be connected with a directed arc, which represents relationship between them.
Figure 1: Modeling objects and relationships in TAP approach
Each modeling object encapsulates attributes which are data elements, and has its own value for each attribute. For example, it can be considered that the modeling objects “task” has eight attributes - “name”, “definition”, “task category”, “duration”, “complexity”, “application domain”, “business domain”, and “size”. Moreover, TAP has five dimensions of process modeling - generalization, classification, aggregation, control, and behavior. We cannot give a detailed explanation for all of concepts and dimensions for lack of space so we only explain some of them. A task, one of the central concepts, is a general description of work and is independent of any specific situation in an actual process. Therefore the task definition does not contain information about when or by whom it is performed. Instead, there is a modeling object that denotes task instance, called task performance, to which such attributes belong. A task can have a substructure (subtasks). The temporal and causal relationships with each other are
292
S. Ito et al. / A Formal Ontology for Business Process Model TAP: Tasks-Agents-Products
Figure 2: Top level description of an example process Figure 3: Behavioral description of Tasks
described in conditional Petri net. It is the behavior description of tasks. We give an example of process model (Figure 2, 3). Figure 3 is a behavioral description of task “Develop Change & Test Unit” in Figure 2. Some task might produce or consume products which are also TAP specifications. These kinds of tasks are scheduling tasks, monitoring tasks and are referred to as meta-task or controlling task. A task can contain a meta-task as its subtask. In Figure 3, meta-tasks are represented as bold boxes.
3 The semantic structure for TAP In this section, we define the semantic structure for TAP. As mentioned in Section 2, TAP has several conceptual components of process modeling. The semantic structure for TAP contains such components. Definition 1 (Frame) A frame is a 7-tuple F = DO , DT , S, A, R, F, , where DO is a semantic domain of modeling objects tasks, agents and products etc. and DT is a semantic domain of time attribute values, S is a set of states, A ⊆ S × S is an accessibility relation, R is a set of intensional relations and F is a set of intensional functions. DT has the special element ε. DT − {ε}, is a totally ordered set and neither ε t nor t ε hold for all t ∈ DT . Let σi ∈ {O, T } for all i. An intensional relation of arity σ1 × · · · × σn on DO , DT , S is a total function p : S −→ P(Dσ1 × · · · × Dσn ), where P(D) is the power set of D. An intensional function of arity σ1 × · · · × σn −→ σn+1 on DO , DT , S is a total function f : S −→ (Dσ1 × · · · × Dσn −→ Dσn+1 ). In this paper, we only consider time attribute among many data elements because time attributes such as “start time” and “end time” are important to represent the enaction order of task performances. It is easy to extend the frame definition to the case where we consider other attributes by incorporating domains for them into the frame. S represents state of affairs of modeling objects. A is the state transition relation on S. s, t ∈ A means that a state s can become a state t. An intensional relation is a function from states to a mathematical (ordinary) relation. Therefore the extension of it varies by states. For example, the situation may happen that an extensional relation in a of a binary intensional relation p in a state s
S. Ito et al. / A Formal Ontology for Business Process Model TAP: Tasks-Agents-Products
293
holds between a and b but that of in a different state t does not. Intensional relations are semantic objects for relationships in TAP. Therefore relationships in TAP can be captured by intensional relations. Similarly, an intensional function is a function from states to a mathematical (ordinary) function. Intensional functions are semantic objects for attributes in TAP. Some attribute values may change along the proceeding of the enaction. Therefore an intensional function is used for a semantic object for attributes. ε is the value which indicates the value of a function ranging over DT undefined. For example, the situation that some task performance has not started yet can be represented in our structure as that the value mapped by the function corresponding to the attribute “start time” for the object corresponding to the task performance is ε.
4 The logical language LTAP In this section, we introduce the logical language LTAP with a formal vocabulary. The formal vocabulary consists of a set of predicates and attributes used in describing TAP specifications. The formal ontology is described in LTAP as a set of axioms.
4.1 Syntax Our logical language LTAP is many-sorted logic. Definition 2 (Sort) The set of sorts is Sort = {O, T }. Sort O represents modeling objects and T represents times. If we consider attributes other than time, we add sorts for them. Now, we introduce symbols of LTAP . Definition 3 (Symbol) The symbols of LTAP consists of the following: 1. Vocabulary V consisting of the following predicate symbols and function symbols. • Predicate symbols of arity O. Task , Product, AgentType, AgentRole, ToolType, ToolRole, TaskPerformance, ProductInstance, AgentInstance, AgentEnaction, ToolInstance, ToolEnaction, Edge, Bar , MetaTask . • Predicate symbols of arity O × O. is instance of , produces, is consumed by, performed by, supported by, plays role of , activates, of , constitutes. • Function symbols of arity O −→ O. source, destination. • Function symbols of arity O −→ T . start time, end time. 2. The set of constant symbols Con = Con O ∪Con T . Con O is the set of constant symbols of sort O and has the special symbols ⊥O and TAPspec. Con T is the set of constant symbols of sort T and has the special symbol ⊥T . 3. The set of variables Var = Var O ∪ Var T . Var O is the set of variables of sort O and Var T is the set of variables of sort T . 4. The set of connectives {∧, ∨, ¬, →, ↔, ∀O , ∀T , ∃O , ∃T , 2, 3}. 5. Equality symbol = of arity O × O and T × T .
294
S. Ito et al. / A Formal Ontology for Business Process Model TAP: Tasks-Agents-Products
6. A binary symbol ≤ of arity T × T . The vocabulary words for static specifications of process models are modeling object type names and relationship names appearing in Figure 1. constitutes(x, y) represents that x is a substructure of y. Edge, Bar , source and destination are used to describe conditional Petri net description. For example, a fragment of Figure 3 can be described as {Edge(approved), source(approved) = Modify Design, destination(approved) = b, Bar (b)}. start time and end time are attributes. If we consider other attributes, we add function symbols for them. Constant symbols ⊥O and ⊥T are used to indicate that the value of some function on some object is undefined. Definition 4 (Term, Atom) The sets Term O of terms of sort O, Term T of terms of sort T and Atom of atoms are the smallest X, Y and Z respectively satisfying the following properties: 1. 2. 3. 4. 5. 6. 7.
Con O ∪ Var O ⊆ X, Con T ∪ Var T ⊆ Y , If the arity of f ∈ V is O −→ O and t ∈ X then f (t) ∈ X, If the arity of f ∈ V is O −→ T and t ∈ X then f (t) ∈ Y , A1 , . . . , An ∈ Z ⇒ spec(A1 , . . . , An ) ∈ X, If t ∈ X and the arity of p ∈ V is O then p(t) ∈ Z, If t1 , t2 ∈ X and the arity of p ∈ V is O × O then p(t1 , t2 ) ∈ Z.
The term spec(A1 , . . . , An ) represents the TAP specification {A1 , . . . , An }. This construct is used to describe objects which are produced or consumed by meta-tasks. Definition 5 Let σ ∈ Sort. Formulas in LTAP are defined inductively as follows: 1. Atoms are formulas. 2. If ϕ and ψ are formulas then ϕ ∧ ψ, ϕ ∨ ψ, ¬ϕ, ϕ → ψ, ϕ ↔ ψ, 2ϕ and 3ϕ are also formulas. 3. If ϕ is a formula and x ∈ Var σ then ∀σ xϕ and ∃σ xϕ are also formulas. 4. If s, t ∈ Term σ then s = t is a formula. 5. If s, t ∈ Term T then s ≤ t is a formula. Notational Convention. The order of strength of connection between connectives is ¬, ∀, ∃, 2, 3, ∧, ∨, →, ↔. We write s = t instead of ¬(s = t). We simply write ∀xϕ and ∃xϕ instead of ∀O xϕ and ∃O xϕ respectively.
4.2 Semantics We define the semantics of LTAP with respect to frames (Definition 1) which is our semantic structures of TAP process models. Definition 6 (Model) Let F = DO , DT , S, A, R, F, be a frame. An interpretation I of symbols with respect to F is a function satisfying the following: 1. 2. 3. 4. 5. 6.
I(c) ∈ DO if c ∈ Con O , I(c) ∈ DT − {ε} if c ∈ Con T − {⊥T }, I(⊥T ) = ε, I(x) ∈ DO if x ∈ Var O , I(x) ∈ DT if x ∈ Var T , I(spec(A1 , . . . , An )) ∈ DO if A1 , . . . , An ∈ Atom,
S. Ito et al. / A Formal Ontology for Business Process Model TAP: Tasks-Agents-Products
295
7. I(p) ∈ R if p is a predicate symbol in V where the arity of p and I(p) are the same. 8. I(f ) ∈ F if f is a function symbol in V where the arity of f and I(f ) are the same. Let F be a frame and I be an interpretation with respect to F. A model is a pair M = F, I. Definition 7 (Interpretation of terms) Let M = DO , DT , S, A, R, F, , I be a model and s ∈ S. An interpretation (M, s) of terms in Term is a function satisfying the following: 1. 2. 3. 4.
(M, s)(c) = I(c) if c ∈ Con, (M, s)(x) = I(x) if x ∈ Var , (M, s)(spec(A1 , . . . , An )) = I(spec(A1 , . . . , An )), (M, s)(f (t1 , . . . , tn )) = I(f )(s)((M, s)(t1 ), . . . , (M, s)(tn )).
Definition 8 Let M = DO , DT , S, A, R, F, , I be a model and s ∈ S. The satisfaction relation |= is defined inductively as follows: M, s |= p(t1 , . . . , tn ) M, s |= t1 = t2 M, s |= ¬ϕ M, s |= ϕ ∧ ψ M, s |= ϕ ∨ ψ M, s |= ϕ → ψ M, s |= ϕ ↔ ψ M, s |= ∀σ xϕ M, s |= ∃σ xϕ M, s |= 2ϕ M, s |= 3ϕ
iff iff iff iff iff iff iff iff iff iff iff
(M, s)(t1 ), . . . , (M, s)(tn ) ∈ I(p)(s) (M, s)(t1 ) = (M, s)(t2 ) M, s |= ϕ M, s |= ϕ and M, s |= ψ M, s |= ϕ or M, s |= ψ M, s |= ϕ or M, s |= ψ M, s |= ϕ iff M, s |= ψ M[x → d], s |= ϕ for all d ∈ Dσ M[x → d], s |= ϕ for some d ∈ Dσ M, s |= ϕ for all s such that s, s ∈ A+ M, s |= ϕ for some s such that s, s ∈ A+
M[x → d] is the same as M except that it has I[x → d] as the interpretation of symbols. I[x → d] is the same as I except that it maps x to d. A+ is the transitive closure of A. Our modal logic holds transitivity so the system of our modal logic is K4.
5 Axioms In this section we describe axioms which characterize the intended world structures of TAP. Models of the axioms approximate the worlds which TAP approach intends to model. We pick several axioms from our formal ontology since we cannot include all of them for lack of space. In TAP approach, once objects or relations appeared at some state then they remain hereafter. In other words, histories must be hold in TAP approach. Axiom 1 and 2 state this. 1. ∀x(P (x) → 2P (x)), where P is a metavariable represent a unary predicate symbol in V . 2. ∀x∀y(P (x, y) → 2P (x, y)), where P is a metavariable represent a binary predicate symbol in V . Axiom 3 says that once the start time or the end time of some object are set at some state then they do not change hereafter. The start time or the end time are for task performances and are peculiar to them. 3. ∀x∀T y(f (x) = y ∧ y = ⊥T → 2f (x) = y), where f is a metavariable represents either start time or end time. We stipulate object types of binary relations, e.g. 4. ∀x∀y(produces(x, y) → (Task (x) ∧ Product(y)) ∨ (TaskPerformance(x) ∧ ProductInstance(y))).
296
S. Ito et al. / A Formal Ontology for Business Process Model TAP: Tasks-Agents-Products
Our ontology contains this kind of axioms for each binary relation in Figure 1. This type of axioms are for pragmatics of TAP. We have axioms for inter-relationships between abstrat level and istance level such as: 5. ∀x∀y∀z(activates(x, y) ∧ of (y, z) ∧ AgentEnaction(y) → ∃u∃v(is instance of (x, u) ∧ is instance of (z, v) ∧ performed by(u, v))). Agent enaction connects an agent instance to a task performance, describing e.g. when an agent is assigned to a task performance. In other words, agent enaction is a collection of information of an agent relevant to the task performance. Therefore, all agent enactions have a connection to some agent instance. We have the similar axiom for ToolEnaction. 6. ∀x(AgentEnaction(x) → ∃yof (x, y)). Moreover, each agent enaction or tool enaction is peculiar to some agent instance or tool instance. 7. ∀x∀y∀z(of (x, y) ∧ of (x, z) → y = z). Our ontology contains axioms of inter-relationship between is instance of relation and constitutes relation. This relationship is similar to the relationship between is-a relation and has-a relation [4], but there are some differences. We show such an axiom in the following: 8. ∀x∀y∀u∀v(constitutes(x, y) ∧ is instance of (x, u) ∧ is instance of (y, v) → constitutes(u, v)). In this axiom, u and v are tasks and x and y are task performances (this fact is derived from other axioms omitted in this paper). This do not hold for relationships between is-a and has-a. For example, let consider v as “car” and u as “air-conditioner”. y is an instance of car and x is an instance of air-conditioner. In this case the fact that y has x do not imply the fact that v has u, because some types of car (e.g. formula cars) do not have air-conditioners. However, TAP has this property since this axiom is a relationship of tasks and task performances. constitutes relations between task performances should conform to constitutes relations between tasks. Otherwise, the performance of some task may not be executed according to its substructure. def We define the macro formula time(x, y) = start time(x) = y ∨ end time(x) = y and def set time(x, y) = (start time(x, ⊥T ) ∧ 3start time(x, y)) ∨ (end time(x, ⊥T ) ∧ 3end time(x, y)). time(x, y) means that a time y is set to the start time or the end time of some object x. set time(x, y) means that a time y will be set to the start time or end time of some object x. 9. ∀x∀T y(set time(x, y) → ∀u∀T v(time(u, v) ∧ v = ⊥T → v ≤ y)). This axiom says that if a time y is set to some x in a state and a time v is set to some u in a future state reachable from the state then y ≤ v. One of conspicuous features of TAP is the notion of meta-task. Meta-tasks are tasks of planning, monitoring, and executing processes and are defined as tasks which produce or consume TAP specifications. The following are axioms for meta-tasks. 10. ∀x(MetaTask (x) → Task (x)). 11. Product(TAPspec). 12. is instance of (spec(A1 , . . . , An ), TAPspec). 13. ∀x(MetaTask (x) → ∃yconstitutes(x, y)). 14. ∀x∀y(MetaTask (x) ∧ (produces(x, y) ∨ is consumed by(y, x) → y = TAPspec). 15. ∀x∀y∀z((produces(x, y)∨is consumed by(y, x))∧is instance of (x, z)∧MetaTask (z) → is instance of (y, TAPspec)). In the set of axioms, meta-tasks are accounted as a task which produces and consumes TAP specifications. However, what is the semantics of producing or consuming TAP specifications was not specified. Such semantics is dependent on the actual work of a meta-task, but the kinds of such tasks are not many. For example, planning, evaluating, monitoring and executing are brief classification of meta-tasks. The semantics of producing and consuming TAP specifications are also classified according to these classes. We are now interested in
S. Ito et al. / A Formal Ontology for Business Process Model TAP: Tasks-Agents-Products
297
classifying meta-tasks and characterize them as formulas in our logical language. For this, we may have to augment the expressivity of our language. 6 Conclusion We introduced the semantic structure for TAP and the logical language to describe TAP specifications. Then we presented some axioms from our ontology which specifies the characteristic of the world structures of TAP in our logical language. There are several significance in this formal approach. From an ontological viewpoint, the axioms characterize the intended world structures which TAP can conceptualize and account for the intensional meanings of entities and relationships. Thus we can understand what is the world structure of business aspects of real worlds which TAP intends to model. This includes the understanding of the nature of entities occurring in business processes, e.g. what the task performance is, what the enaction of tasks is and what the meta-task is, etc. By axiomatizing formally, the ontology has the ability to deduce facts about entities and relationships in TAP business process models. We can evaluate the expressivity and adequacy of business process model TAP by the ability of deduction. From an engineering viewpoint, we can elucidate ambiguous points in syntax and semantics of TAP. We can automatically check the syntactic and semantic consistency of TAP specifications e.g. the consistency of start times and end times of task performances or the consistency of object types and relationships, etc. Therefore we can use business process model TAP in planning, controlling, improving and monitoring business processes with confidence. We can verify the properties of processes rigorously by checking whether the models of some TAP specification satisfy some property. Both the TAP specification and the property are expressed in our logical language. We want to extend our ontology to cover features such as generalization and classification which was not treated in this paper. Another important topic is to compare our ontology and other business ontologies in formal level. For this, we should formalize these ontologies. References [1] Mark S. Fox and Michael Gruninger. Enterprise modeling. AI Magazine, 19(3):109–121, 1998. [2] Guido L. Geerts and William E. McCarthy. An accounting object infrastructure for knowledge-based enterprise models. IEEE Intelligent Systems and Their Applications, 14(4):89–94, 1999. [3] Jaap Gordijn and Hans Akkermans. Value based requirements engineering: Exploring innovative ecommerce idea. Requirements Engineering Journal, 8(2):114–134, 2003. [4] Naoko Izumi and Naoki Yonezaki. A logic of ontology for object oriented software component. In Proceedings of the 11th European-Japanese Conference on Information Modeling and Knowledge Bases, pages 83–99. Amsterdam, IOS Press, 2001. [5] W. E. McCarthy. The REA accounting model: A generalized framework for accounting systems in a shared data environment. The Accounting Review, 57(3):554–78, 1982. [6] Mike Uschold, Martin King, Stuart Moralee, and Yannis Zorgios. The enterprise ontology. The Knowledge Engineering Review, 13(1):31–89, 1998. [7] Naoki Yonezaki, Tapani Kinnula, Motoshi Saeki, and Jan Ljunberg. TAP: A new model for software process: Tasks-Agents-Products. In Proceedings of the 5th International Conference on Software Engineering and Knowledge Engineering, pages 346–350, 1993.
298
Information Modelling and Knowledge Bases XIX H. Jaakkola et al. (Eds.) IOS Press, 2008 © 2008 The authors and IOS Press. All rights reserved.
A proposal for student modelling based on ontologies Angélica DE ANTONIO1, Jaime RAMÍREZ1, Julia CLEMENTE2 1 Facultad de Informática. Universidad Politécnica de Madrid. 28660 Boadilla del Monte, Madrid, Spain e-mail: {angelica, jramirez}@fi.upm.es 2
Universidad de Alcalá. Escuela Universitaria Politécnica Departamento de Automática Campus Universitario. Ctra. Madrid-Barcelona, Km. 33,600 28871 Alcalá de Henares, Madrid, Spain e-mail: [email protected]
Abstract. The advances in the educational field and the high complexity of student modelling has provoked it to be one of the more investigated aspects in Intelligent Tutoring Systems (ITSs). The Student Models (SM) should not only represent the student's knowledge, in a wide sense, but rather they should be, insofar as it is possible, a snapshot of the student's reasoning process. In this article, a new approach to student’s modelling is proposed that benefits of the Ontological Engineering advantages, so widely used at the present time, to advance in the pursue of a more granular and complete knowledge representation. The goal is to define an ontological basis for SMs characterized by a high flexibility for its integration in varied ITSs, a good adaptability to the student’s features, as well as to favor a rich diagnostic process with nonmonotonic reasoning capacities, allowing the treatment of the contradictions raised during the student's reasoning and diagnosis.
1. Introduction In spite of the tendencies in the educational field, in constant evolution, with new approaches to Intelligent Tutoring Systems research pushing this constant progress, the construction and maintenance of ITS’s modules is complex and it still presents many lacks; Artificial Intelligence and Software Engineering are bound to play a crucial role for their resolution and continuous improvement. Some authors like Mizoguchi & Bourdeau [1] have attributed the current limitations of these systems, primarily, to a lack of a explicit representation of the conceptualization on which each system is based. An extensive revision of the state of the art in student's modelling, a distinctive feature of ITSs, has leaded us to corroborate this statement. The purpose of this article is proposing a new approach to student’s modelling based on Ontological Engineering, following Mizoguchi and Bordeau [1], but, beyond the approach taken by these authors, introducing a new student's modelling taxonomy that has been built after a rigorous analysis of the types of knowledge about the student that can be represented in a SM. This generality will enable the adaptation of the student’s model to different types of ITS, and will facilitate the construction of ITS which are truly adaptive, with tutoring moulding to the student's individual features. Our approach also facilitates an appropriate and powerful cognitive diagnosis, with nonmonotonic reasoning capacities.
A. de Antonio et al. / A Proposal for Student Modelling Based on Ontologies
299
The present article starts with a brief description of the state of the art in Student Modelling, proceeds with an analysis of the motivation and general objectives of our work, and continues with a description of the adopted solution. We have centred our focus in pedagogic design, upon which our solution is sustained, and in the ontology proposed for the SM, with only a sketch of how the diagnostic process is approached. The conclusions and the current and future work lines related to the proposal put an end to the paper. 2. Previous Work in the Area So far, numerous approaches to SM have been proposed in the field of ITS, representing different information types and using different methods to infer the student's cognitive state [2], [3]. They can be classified as: x SMs that just represent the state of the student’s knowledge about the subject matter, including SMs that only represent correct knowledge (Overlay Models such as in [4], [5], etc., or Differential Models such as in [6], [7], etc., present the drawback that the student's knowledge is usually not strictly a subset of an expert’s knowledge) and SMs that also represent wrong knowledge with different approaches to the development of the error library [8], [9], etc. (the consideration of possible errors improves the understanding of the student). x SMs that also represent the student’s reasoning process: according to Clancey [10], these can be divided into Behavior simulation models, that only describe the actions the student is carrying out ([11], [12]), and Functional simulation models, that describe the student’s beliefs and goals, what the student knows and what he’s trying to do ([13], [14]), etc.). Some taxonomies for the student's knowledge modelling have been contributed in this field similar to the one used for the previous classification ([15], [16], etc.) and others deserve to be highlighted by their interesting contributions such as: a) the taxonomy in the De Koning and Bredeweg approach [17] based on the multi-stratified framework KADS [18] distinguishes the strategic knowledge (allowing the representation of the goals in problem solving, how to reach them, and the knowledge required for reasoning with them). In this respect, it is more and more recognized that the student's metacognitive process should be considered in the educational process (an example is the system TAPS [19]). b) the McCalla and Greer’s taxonomy [20], sustained in the idea of granularity based reasoning (level of detail in the vision of a concept). This feature, incorporated explicitly in an ITS, can facilitate the diagnosis of the student's behaviour. It is also important to point out there are not many works that consider the student's personal features to carry out an adaptive teaching-learning process. Some examples are [21], or the Chen and Mizoguchi’s proposal [22], where an ontology and an agent for the SM are defined. It is this latter work the one that has served as a starting point for the proposal presented in this article. However, their ontology suffers from important limitations, such as: a) lack of information related to the student's learning objectives; b) scarce information on most of the considered knowledge types and; c) in general, lack of clarity in the description of the concepts as well as in its organi