-

), then the value is the string between this title and the next title. If this is the last title in the HTML file, then the value is ended when (1) a blank line appears; (2) tag

---

or
appears. For example, in Fig. 1, the value for keyword History is the string after it until the next title Programs. – If the word is an item of a list(

or ), then the value is the string after it until the next

or or the end of the list; – If the word is a field in a table( or ), then the value of the field in the first column(except table head) of a 2-columns table is the field on the right. Otherwise, it is the field under it. For example, in the following table on the left, the value for Single room is 80. But the same field on the right is 50.

Price $

Single room

Single room

80

50

Double room

140

houses

Extra bed

Double room 90

Extra bed 20

30

Fig. 4. Two Tables In the Web

– If the word is at the beginning of a textual line, which itself consists of an independent paragraph(e.g. HTML tags separate it from the previous and

114

T. Guan, M. Liu, and L.V. Saxton

latter texts) and there are some HTML tags(e.g. and ) or more than two spaces to separate it from the latter words, then the value is the string after it until the end of the line. For example, in the following line, Office: CW308.2 The value for Office is CW308.2 since it is separated by tags and in the HTML page. If the keyword denotes a number(in a conditional expression as in Example 2.2), then the value is the first number occurring in the string obtained from the above rules. 3.2

String Pattern Mining

A string pattern contains a couple of constant words or variables(variables are indicated in select or where clauses) and is delimited by a pair of brackets. When it is given, the system first locates strings from pages to match the constant words in patterns. If successful, the noun phrase or number corresponding to variables are assigned to the variables. For example, a pattern [Dr. Name] can be matched against a string starting with “Dr.” and then the first noun phrase after “Dr.” is assigned to variable Name. For string “Dr. Jack Boan is ...” , then “Jack Boan” is assigned to Name. The reason why we focus on noun phrases and numbers is that we think most information involved in a query is represented in noun phrases or numbers(Verbs usually indicate an action or state). The definition for noun phrases is as follows, NP → N P2 | Det N P2 | NP Conj NP N P2 → Noun | Noun N P2 | Adj N P2 | N P2 PP PP → Prep NP where NP denotes a noun phrase, Det denotes a determiner, Conj denotes a conjunction, Adj denotes an adjective and Prep denotes a preposition. Two or more patterns may be linked using boolean operators. e.g. [Mr. Name] or [Ms Name]. The word matched against any one of the patterns can be set to variable Name. Of course, it is possible that more than one word is matched against a variable. If that happens, we count the number of matches in the page and choose the one with the highest frequency or the first if two or more words have the same maximum frequency count. In addition, wildcards can be used in string pattern, i.e. “∗” and “-”, where “-” denotes a word and “∗” represents any number of words. For example, the pattern [Dr. Name received Degree from ∗ in Year] is to extract information on variables Name, Degree and Year, while the university is ignored. 3.3

Structure Pattern Mining

Structure pattern mining means that a structure pattern is given to match a textual line or a row of tables(defined by tags and ) in web pages. NetQL treats each textual line or a row of tables as a set of fields and thus the syntax of a structure pattern is

Structure-Based Queries over the World Wide Web

115

{f1 , f2 , .... fn } where fi denotes a field, which may be: – a variable, e.g. X, Y or Z, – a constant which may contain wildcards as in string pattern, e.g. “Hong Kong”, 10, “University of ∗”; – a simple expression of form “variable operator constant”, where operators may be <, =, >, ≤, or ≥. For example, Z <100; – “-”, a field whose value can be ignored; or – “∗”, any number of fields whose values can be ignored; For example, structure pattern {X, “Canada”, ∗, Z <20 } is matched against a textual line or a row of a table, in which the second field is a string “Canada” and the last field is a number less than 20. The processing of a structure pattern is used to find the values of all fields in a textual line or a row and then match them against the pattern. For textual lines, fields are separated by delimiters, which are defined as two or more spaces, or a HTML tag, e.g. , etc. The value of a string field is the textual string(without HTML tags) between two delimiters and the value of a number field is the first number between two delimiters. For example, consider the following two lines, Order 4

Country Canada

Gold 6

Silver 5

Bronze 7

Total 18

There are 6 fields in each line. The second line matches with the pattern {X, “Canada”, ∗, Z <20 }; thus the value of variable X is 4 and for Z is 18. As for tables, the fields are delimited by tags and for each row (between and ). The value for a string field is the string eliminating all tags(included in <>). Similarly, if it is a number field, we take the first number in the string as the value. For example, the following table is a portion of a HTML file on a row of a table. Anne Black Guest House (YWCA) Special offer US$ 43 to 101

There are three fields. The value of the first field is “Anne Black Guest House” and the second is “Special offer” and the third is “US$43 to 101”. This example matches the pattern in Example 2.2.

116

T. Guan, M. Liu, and L.V. Saxton

We should note again that structure-based patterns cannot handle all structural information, e.g. complex tables. However, 2-dimensional relational tables, in which each row has the same number of fields, are the most popular in practice. Therefore, our method is feasible in many cases.

4

Complexity Control

Complexity is a big challenge for structure-based web queries since it is evaluated on original, distributed data. NetQL deals with complexity in two levels. Firstly, users are given various methods to control complexity. Secondly, an effective optimization technique is provided to guide the search to the closest or most promising path so that the expected results can be obtained as soon as possible. In this section, we discuss them respectively. 4.1

Users’ Role

If users have partial knowledge about the structure of the searched site, they can issue more specific path information to reduce a blind search. The more information users specify, the more efficient the query is. In example 2.1, if they know that all professors are listed under hyperlink faculty, then the path expression in from clause can be updated as http://www.cs.uregina.ca/→faculty.∗. The query only checks the pages which are under hyperlink faculty and contain the word professor. The run time is thus reduced significantly, especially when the query is issued at a remote site. Partial knowledge is possible for users since: (1) they have visited the site before; (2) web sites on a similar topic have similar structures, e.g. the structure of one university can be derived if other universities were visited before. Of course, if users know nothing about the site, they can limit queries in the following aspects: – Restrict the search to local data. The search only follows the links inside a web server where the starting page is located. – Restrict the search to a certain number of returned results. If the specified number of results is exceeded, the search stops. – Restrict the search to a certain amount of time. When the time reaches its limit, the search stops and returns results found. For example, select Publications from http://www.cs.uregina.ca/→faculty.∗ contain professor where Publications CONTAINS database restricted LOCAL and RESULTS < 10 This approach trades a portion of the results for fast response. It is useful in the cases where inaccurate and incomplete results are acceptable by users. However, if some users hope for an exhaustive search, internal optimization will be applied in this situation.

Structure-Based Queries over the World Wide Web

4.2

117

Optimization

Our idea is inspired by the heuristic search for problem-solving programs in AI. Rather than pruning the search at certain sub-graphs, our method attempts to guide the search to the closest or most promising path so that the expected results can be obtained as soon as possible. The queries restricted by time and the number of results will be benefited directly. Our algorithm uses semantic information on hyperlink labels for heuristic search. The semantic similarity between the current set of links and the goal can help the optimizer to decide which is preferred for the next step. Actually, humans employ this way to guide their navigation when they browse web pages manually. For example, if the starting point has the following structure,

http://www.cs.uregina.ca ...... Information

People

Research

Class Files

.........

Fig. 5. A Portion of web site

When we need to locate the pages containing professor, which hyperlink has the highest priority? Obviously, it should be People since this label is more similar to professor than any other. The key point for this method is how to compute the semantic distances between words or noun phrases. This problem has been widely studied in the fields of Natural language Processing(NLP) and Information Retrieval(IR). Various methods have been presented in [9,10,19]. In this paper, we follow the approach of word-word similarities based on WordNet presented in [9,19]. In WordNet, conceptual similarity is considered in terms of synset(a set of synonyms) similarity. The similarity between two synsets is approximated by the maximum information content of the super synsets in the hierarchy that subsumes both synsets. The information content of a synset is quantified as negative the log likelihood, -log P(s), and for our case P(s) is computed simply as a relative frequency: P P(s) = count(w) / N, w ∈ synset(s); where count(w) is the number of occurrence of the word w in the corpus(we use noun frequencies from the Brown Corpus of American English [5]), N is the total number of words observed. Thus the similarity of two synsets can be expressed as sim(s1 , s2 ) = maxs [IC(s)] = maxs [-log P(s)], where s ∈ Sup(s1 , s2 ) which is the set of super synsets that subsume both s1 and s2 . If there is no super synsets for s1 and s2 , then the similarity is 0.

118

T. Guan, M. Liu, and L.V. Saxton

Assume that a set of labels of candidate hyperlinks L = {l1 , l2 , ...., ln } is matched with a predicate p, the following algorithm is applied for optimization, – A stoplist is used to remove the common words (and, the, in etc) from each label li . The result is denoted as keywordset(li ); – The similarity of each li and p is computed as sim’(li , p) = max(sim(synset(w), synset(p))), where w ∈ keywordset(li ); Thus, the following heuristic rule is used, Rule: The link whose label has the maximum similarity with the predicate p is selected first for the next search. For example, in Fig. 5, we have sim’(“Information”, “Professor”) = 0; sim’(“People”, “Professor”) = 8.11; sim’(“Research”, “Professor”) = 0; sim’(“Class Files”, “Professor”) = 0; Therefore, the link with the label People is selected first for the next search.

5

Experiments

The most important facility in NetQL is information mining from multiple web pages. Our first attempt is to extract structural information from web pages based on the syntactic or semantic structures. The method is to transform a page into a labeled graph as in semistructured databases [2,6,7] and then obtain the desired information from the graph. However, this approach failed in our experiment since HTML provided many flexible constructors and thus most web pages were quite irregular. We therefore tried another information mining based on keywords, patterns and structure. Although this approach cannot guarantee 100% successes in mining the desired information, our initial experience showed that it was effective in practice. In Example 2.1, the average recall1 for keyword-based mining (i.e. the variable E-mail) was 93% and the average precision2 was 85%. However, the recall for pattern-based mining was a little bit low. It was only 21% for the pattern [Dr. Name]; nevertheless, the precision was 95%. From the experiments, the following observations are made: (1) There is no keyword before the desired information, such as names, title or addresses in personal homepages (humans can recognize them in the context or by semantic knowledge). This problem may be solved by pattern-matching to some extent. For example, some names always have a title before it, such as Dr., Mr., Ms etc. Of course, if there is no obvious pattern, it is hard to handle the problem. Also, simple concepts, e.g. name, address, place or prices, could be easily identified by NLP techniques. But complex concepts, e.g. bibliography, would be hard. (2) The information denoted by a keyword is a complex concept so that our program cannot mine it completely. For example, the rate of a hotel usually 1 2

Recall is the percentage of desired instances obtained by the system. Precision is the percentage of correct instances.

Structure-Based Queries over the World Wide Web

119

includes the rate for single-room, double-room, adult or children. The mining of these kinds of data also requires the semantic knowledge and NLP techniques. (3) Some information is represented as images and complex tables. These cannot be easily handled automatically (at least not by present NetQL). In reality, only a highly intelligent human could recognize the relationship between such data. In short, we find that mining pages from a local or remote site is easier than from global search since web pages in the former usually organized by the same institute. Thus the pages exhibits very similar stylistic properties. The irregular cases are web pages designed by different individuals. Such web pages can be presented in very different styles. Despite this, many common structures and styles do appear in web sites which share the same theme in practice. For example, most professors’ homepages have the following structure, name, title, address, biography, teaching, research interests, projects and publications. Therefore, information mining over the Internet is not impossible. Our second experiment focuses on the performance of local and remote search. The result shows that the optimizing technique is useful in improving the performance [14]. Although it is not effective in all cases, it out-performs exhaustive blind searching in almost all cases. Usually, a local search over 10,000 pages or more can be done in a couple of minutes. Under this time frame, the desired information can be found after the navigation of 5 pages from the starting point on average. If optimization is applied, the searching time is only a few minutes with an average navigation length of 8. For a remote search, it depends heavily on the speed and loads on the Internet. Our experience shows that about 1000 pages are accessed from the US and Canada within a couple of minutes without optimization. On average, only 3 pages are visited from the starting point for a medium-size site which contains around 10, 000 pages. However, if the optimizing technique is used, it is possible to obtain data at level 5 or 6 in a few minutes.

6

Conclusion and Further Work

An intelligent query language over WWW, NetQL and its implementation, is presented in this paper. Rather than developing a powerful language, we focus on the problems ignored by other languages. The main contributions of NetQL are: (1) it provides a novel approach to extract information from irregular textual web pages; (2) it supports various methods to control the complexity of queries. Future work will focus on the following questions: – Is it possible to extract information by the semantic knowledge? e.g. name, address or biography? – Are there other heuristic rules for web queries? – Which method is the best for semantic similarity in the context of WWW ? In short, structure-based web queries is a new area and current solutions are incomplete. There are lots of room for further research.

120

T. Guan, M. Liu, and L.V. Saxton

References 1. B. Adeberg: NoDOSE - A tool for semi-automatically extracting structured and semistructured data from text documents. In Proc. of the ACM SIGMOD International Conference on Management of Data, 1998 2. N. Ashish and C. Knoblock: Wrapper generation for semi-structured Internet sources. In 1st Workshop on Management of Semistructured Data, Arizona, 1997 3. P. Atzeni, G. Mecca and P. Merialdo: Semistructured and structured data in the Web: going back and forth. In 1st Workshop on Management of Semistructured Data, 1997 4. M. Costantino, R.G. Morgan, R. J. Collingham and R. Garigliano: Natural language processing and information extraction: Qualitative analysis of financial news articles. In Proc. of the Conf. on Computational Intelligence for Financial Engineering, 1997 5. W. N. Francis and H. Kucera: Frequency analysis of English usage: lexicon and grammar. Houghton Mifflin, 1982 6. M. Fernandez and D. Suciu: Query optimizations for semi-structured data using graph schema, In ICDE’98 7. R. Goldman and J. Widom: Interactive query and search in semistructured databases. Technical Report, Stanford University, 1998 8. J. Hammer, H.G. Molina, J. Cho, R. Aranha and A. Crespo.: Extracting semistructured information from the Web. In 1st Workshop on Management of Semistructured Data, Arizona, 1997. 9. J. Jiang and D. Conrath: Semantic similarity based on corpus statistics and lexical taxonomy. In Proc. of Int’l Conf. on Research on Computational Linguistics, Taiwan, 1997. 10. H. Kozima and T. Furugori: Similarity between words computed by spreading activation on an English dictionary. In Proc. of EACL-93(Utrecht), pp.232-239, 1993. 11. D. Konopnicki and O. Shmueli: W3QS: A query system for the world wide web. In VLDB’95, Zurich, 1995, pages. 54-65. 12. Z. Lacrox, A. Sahuguet, R. Chandrasekar and B. Srinivas: A novel approach to querying the Web: Integrating Retrieval and Browsing. ER97 Workshop on Conceptual Modeling for Multimedia Information Seeking, 1997 13. L.V.S. Lakshmanan, F. Sadri and I.N. Subramanian. A declarative language for querying and restructuring the Web. In Proc. of 6th. International Workshop on Research Issues in Data Engineering, RIDE’96, New Orleans, February 1996. 14. M. Liu: NetQL: an intelligent web query language. Master Thesis, University of Regina 15. G.A. Miller, R. Beckwith, C. Fellbaum, D. Gross and K. Miller: Introduction to WordNet: an on-line lexical database, International Journal of Lexicography, 1993. 16. A. Mendelzon, G. Mihaila and T. Milo: Querying the World Wide Web. In 1st Int. Conf. on Parallel and Distributed Information System, 1996. 17. D. Smith and M. Lopez: Information extraction for semi-structured documents. In 1st Workshop on Management of Semistructured Data, Arizona, 1997. 18. S. Soderland: Learning to extract text-based information from the world wide wed. In Proc. of 3rd International Conf. on Knowledge Discovery and Data Mining(KDD-97), 1997 19. A.F. Smeaton and I. Quigley: Experiments on using semantics distances between words in image caption retrieval. In SIGIR’96

Integrated Approach for Modelling of Semantic and Pragmatic Dependencies of Information Systems Remigijus Gustas

Department of Information Technology, University of Karlstad S-651 88 Karlstad, Sweden [email protected]

Abstract. Traditional semantic models are based on entity notations provided by several links. Links are established to capture semantic detail about relationships among concepts. The ability to describe a process in a clear and sufficiently rich way is acknowledged as crucial to conceptual modelling. Current workflow models used in business process re-engineering offer limited analytical capabilities. Entity-Relationship models and Data Flow Diagrams are closer to the technical system development stage and, therefore, they do not capture organisational aspects. Although object-oriented models are quite comprehensible for users, they are not provided by rules of reasoning and complete integration between static and dynamic diagrams. The ultimate objective of this paper is to introduce principles of integration for different classes of semantic and pragmatic representations.

1 Introduction Any information system activity needs to be defined in a context of organisational processes. Thus, two levels of information system models are necessary [5]. Organisational level model that defines an ideal system structure. Further it will be referred to as an enterprise model. The implementation level determines data processing needs for a specific application. Most conventional semantic data models are heavily centred around the implementation level. The way in which organisational activity is conceptualised will define what information system is appropriate. Activities in an organisational system can be expressed in terms of actions of communication and collaboration between actors of an information system. This kind of knowledge is crucial to reason about purposeful implications of organisational processes. Most semantic models that have been used in traditional information system modelling approaches neglect such essential aspects of communication. Enterprise engineering is a branch of requirements engineering which deals with an early phase of integrated information system development. At the same time it can be viewed as an extension and generalisation of system analysis activity. Enterprise T.W. Ling, S. Ram, and M.L. Lee (Eds.): ER’98, LNCS 1507, pp. 121−134, 1998.  Springer-Verlag Berlin Heidelberg 1998

122

R. Gustas

modelling takes place in the early and middle phases of information system development life cycle. The most difficult part of it is arriving at complete, integrated and consistent description of a new system that sometimes is known as conceptual, semantic or requirements model. Despite of apparent clarity of the semantic models used by various Object – Oriented methods and CASE tools, the research has shown that a large part of the maintenance costs can be attributed to improper enterprise modelling or to misconception of real requirements [15]. Various graphical diagrams [19] are used to define semantics of information systems. It is obvious that all notations have been designed to describe one or a few, but not all aspects of information systems. This means that information system models should comprise a combination of several notations, each for some particular aspect. This may lead to a difficult question: ‘how is it possible to use several notations in a complimentary way to develop clear models?’ [14]. More importantly, it is often possible to employ a notation to describe some other aspects than those it has been designed to use for. The solution could be found in the identification of a set of the basic semantic and pragmatic modelling primitives that are adequate to analyse static and dynamic aspects of processes. In this study we will present and analyse a set of abstractions that can be considered as a necessary basis to built an integrated enterprise model. The focus of this approach is on modelling primitives which are not only are taking into account semantic models of traditional approaches, but also put a communication aspect into the foreground of information system modelling. Integration of semantic, pragmatic and non-traditional communication dependencies is considered as a most important feature of the suggested framework. Such enterprise models can be useful for the purpose of understanding and reasoning that is critical to the success of conceptual engineering activity in many areas.

2 Pragmatic Dependencies Starting point in a business process re-engineering research, are initial requirement statements that express the wishes of stakeholders about new organisation of system. These initial requirements are usually presented as a natural language text that is often ambiguous, incomplete and inconsistent. Although processes of information system adaptation can be driven by these pragmatic statements, usually traditional semantic models are not taking into account dependencies between activities and goals. Moreover interdependencies between goal models and process models are usually defined in a very fuzzy way. Some requirement engineering methodologies have already identified the problem of making system requirements precise, unambiguous, complete and consistent. The process of bridging goals to information system specifications was entitled 'from fuzzy to formal' [7]. Predominance of fuzzy thinking in a goal modelling has led to serious lack of interaction between semantic and pragmatic descriptions of processes. More often, a process goal is merely postulated rather than expressed in terms of semantic dia-

Modelling of Semantic and Pragmatic Dependencies of Information Systems

123

grams. Such ignorance of a goal power was recognised in the area of cognitive modelling. Goals are usually understood as states of affairs or situations that should be reached or at least striven for. Situations are resulting from the actions [6]. Goals can be defined as desirable situations that are interpreted by an actor as final. Such pragmatic notions as objective, vision, goal, etc. express the wishes and desires of actors concerning system they design or manage. A goal hierarchy can be formed of interconnected goals on different levels of abstraction ranging from high-level business objectives to low-level operational goals. Usually, objectives at the bottom level are situations that can be defined in terms of various semantic dependencies. On neighbouring levels of decomposition, goals are related by the composition dependency. The opposite of a goal is a problem. A problem describes a situation which is not desirable. The notion of a problem is used to refer to a problematic situation of an actor. Semantic specification of the problem is regarded as a part of the actual specification. This means that the problem can not be identified without stating the goal. If the designer has no predefined goal then the problem does not make sense [9]. Usually problematic situations denote restrictions that actors try to avoid. A pragmatic link between an actor and desired situation is entitled to as a goal dependency ( g ). The problem link ( p ) are used to refer a problematic situation of an actor. The goal and problem dependencies can be used to refer desirable or not desirable states or situations. In the following chapters, two pragmatic dependencies of influence between goals will be formally defined. They are entitled to as negative influence dependency ( - ) and positive influence dependency ( + ). The negative influence dependency from A to B (A - B) indicates that the goal A hinders to the achievement of goal B. The positive influence dependencies ( + ) between two goals means that the achievement of the first goal, would contribute to the achievement of the second. The negative and positive influences between goals are imposed by the conflicting interests of actors.

3 Semantic Dependencies Most of the semantic modelling techniques are based on entity notations provided by several links. Links are established to capture semantic detail about static and dynamic relationships. Typically semantic constraints have to be general enough to specify dependencies of a system in the different perspectives such as the "why", "what", "who", "where", "when" and "how" [16]. Semantic constraints can be described by using intensional and extensional [13] dependencies of various kinds. Semantics of static intensional dependencies can be defined as cardinalities represented by minimum and maximum number of individuals of concepts. Extensional dependencies usually specify constrains between classes and instances. Static dependencies of concepts stem from various semantic data models. Graphical notations of several associations in Martin/Odell’s style [13] are represented in Fig. 2.1.

124

R. Gustas a)

A

B

(0 ,1 ;? ,? )

b)

A

B

(1 ,1 ;? ,? )

c)

A

B

(0 ,* ;? ,? )

d)

A

B

(1 ,* ;? ,? )

Fig. 2.1. Graphical notation of cardinality constraints. Note: the meaning of * is ‘many’ (i.e. more than one) and the meaning of ?is ‘not defined’.

Notations that are commonly used at the initial phase of concept modelling have to provide a clear understanding of cardinality constraints in both directions. The most common static dependencies that may be specified between any two concepts A and B are as follows: • (1,1;0,1) - Injection dependency which will be denoted by A ⇒ B , • (1,1;1,1) - Bijection dependency (A B), • (1,1;0,*) - Total Functional dependency (A B), • (1,*;1,1) - Surjection dependency (A B), • (1,*;0,1) - Surjective partial functional dependency (A ⇒> B ), • (1,*;1,*) - Mutual multivalued dependency (A B), • (1,*;0,*) - Total Multivalued dependency (A B), • (0,1;0,1) - Partial injectional dependency, (A |⇒ B ) , • (0,1;0,*) - Functional (partial) dependency (A B), • (0,*;0,*) - Multivalued (partial) dependency (A B). Many concepts have common constraints. The similarities can be shared between concepts by extracting and attaching them to a more general concept. In such a way, similar constraints can be inherited by several concepts. One way to represent generalisation hierarchies is by using of inheritance dependency. It will be denoted by a solid line arrow ( ). By means of inheritance similarities of concepts are shown. Aggregation is a conceptual operation which is useful for the formation of a concept interpreted as a whole from other concepts that may be viewed as component parts. Aggregation can be specified by a composition dependency ( ). In the area of artificial intelligence, sometimes, composition is referred to as a ‘part of’ relation [17]. Semantics of the composition link can be completed by cardinality constraints. Most behavioural diagrams put into foreground a dynamic link, which is very similar to the state transition in a finite state machine. Such transition link, constitute a modelling basis of various object-oriented diagrams that are used for specification of object behaviour. In our approach, a state transition is defined in terms of two states. If two states are connected by the transition dependency, then it means that by the action an object can be transferred from the actual state to the next state. Actual static constraints define a set of conditions for an object in the current state. The

Modelling of Semantic and Pragmatic Dependencies of Information Systems

125

expected state defines a set of conditions for an object in the desired state [11]. The graphical illustration of the transition dependency is presented in Fig. 2.2. ACTUAL STATE

ACTION

NEW STATE

Fig. 2.2. Transition dependency between actual and desired situation. Actions are represented by ellipse.

States are resulting from the actions [6]. Specification of actual and desired states is crucial to the understanding of action semantics. Any state transition dependency indicates a possibility to change a state, and visa versa, a possibility to accomplish the action can be specified by a state transition dependency, i.e. (ACTUAL STATE) (NEW STATE). Communication dependencies between two actors involved in a particular action, describe the "who" perspective. Such dependency link between two actors (agent and recipient) indicates that one actor depends on the other for some flow. The agent can be any actor who is able to send a flow, for example, individual, group, role, position, organisation, machine, information system, etc. Graphical notation of the flow dependency between an agent and recipient (AGENT FLOW RECIPIENT) is represented in Fig. 2.3. FLOW AGENT

R E C IP IE N T

Fig. 2.3. Flow dependency

The flow dependency represents a transfer of the ownership right for a particular object of FLOW. Before sending the flow, it is owned by an AGENT and later, dependent on whether a flow would be accepted or not, the ownership is transferred to a RECIPIENT. The flows can be decision, information or material. Recipients by depending on agents are able to achieve their goals.

4 Interaction between Static and Behavioural Dependencies Any flow dependency between two actors may imply a communicative action as well. It is then considered to be an action and a communication flow. It should be noted, that many approaches in the area of business process re-engineering do not view actions in two different perspectives [8]. Cohesion of action and flow results into a more complex abstraction. Therefore, the flow dependency link between two actors specifies that a recipient depends on an agent not just only by a specific flow, but also by action. Actors are specific sub-systems of the overall system. The semantic link ( ) from actor to action (ACTOR ACTION) indicates that the action can be initiated

126

R. Gustas

by any individual, which belongs to the class ACTOR. The presented dependency may define co-ordination, decision, control, etc. The dependency link from action to actor, i.e. ACTION ACTOR, means that an actor will be affected by the executed action. Often this dependency link is combined with the flow that is desired by a dependent actor. Graphical notation of the communicative action dependency is represented in Fig. 4.1. A CTIO N

AG EN T

FLOW RECIP IEN T

Fig. 4.1. Communicative action dependency between two actors

Underlying concepts and dependencies play an important role in various business modelling approaches which are based on communication [21]. A typical action workflow loop can be defined in terms of two communicative action dependencies. Sequences of communicative actions in workflow models [1] may serve as a basis to define obligations, authorisations and contracts [20]. Example of a typical workflow loop that is defined in terms of two communicative actions into opposite directions is represented in Fig. 4.2. ORDER

ORDER

CUSTOMER

SUPPLIER ITEM

SUPPLY

Fig. 4.2. Action workflow loop between Customer and Supplier

This graphical example shows that a customer is authorised to send an order to a supplier by using the predefined ordering action. If this order would be accepted then a supplier is obliged to supply an item. The supplier is also responsible to follow a contract, which can be defined in terms of relationships between incoming (ORDER) and outcoming flows (ITEM). A contract in the presented example can be defined as follows: If CUSTOMER ORDER SUPPLIER then SUPPLIER ITEM CUSTOMER. An agent carries out a specific action in order to achieve a predefined state. Existence of object x in some state would also impose the fulfilment of a set of static dependencies between states. Two kinds of changes such as disconnection and connection occur concerning the associations of an object during a transition of the object from one state to another. A disconnection removes an existing association from existence and connection adds a new association. The disconnected and connected associations are represented by entirely different relationships of two states. The definition of such noteworthy difference in a current and new state of action is very important to understand the nature of action. Only those processes, that conclude with a state change expressed by the disconnection or connection event, can

Modelling of Semantic and Pragmatic Dependencies of Information Systems

127

be interpreted as actions. For instance, a graphical example of the noteworthy difference among three different states, that are important to understand actions of customer and supplier, is illustrated by Fig. 4.3. Product Not O rdered Product Cust omer Product Needs to be O rdered

O rdered Product

O rder

Item

O rder

Sup plier

Product Needs not to be O rdered

Not Supplied Item

Supply

Supplied Item

Item

Fig. 4.3. Constraints to Order and to Supply. Note: inheritance ( ) in this paper is used in a non traditional way. Strict definition of the inheritance dependency is presented in chapter 7.

A transition from the state ‘Product Needs to be Ordered’ to the state ‘Ordered Product’ can be performed by the action of Ordering. Existence of an object x in the state ‘Ordered Product’ would imply that it may have one or many ‘Not Supplied Item’ associated to the x Product. The noteworthy changes performed by actions are important for actors, because they create a need to react. The reaction mechanism is represented in terms of communication flows. For instance, if ‘Product Needs to be Ordered’ then a Customer has to react appropriately. In this particular situation, a Customer is supposed to send an Order to a Supplier. If an item is ‘Not Supplied Item’ then the Supplier is supposed to Supply an Item to the Customer. The semantic difference between two states ‘Not Supplied Item’ and ‘Supplied Item’ is defined in terms of two semantic links (see ‘Ordered Product’ and ‘Product Needs not to be Ordered’).

5 Dependencies in an Extended Action Workflow Loop A typical action workflow loop include two communication flows sent into opposite directions. The customer is an actor who initiates the work flow loop to achieve his goal. The receiver of a flow is a performer. Flow dependencies in two opposite directions imply that certain relationships are established between two actors. In the

128

R. Gustas

reality it represents either a commitment or a contract [8] between customer and performer. Any business process defines a set of responsibilities as well as set of requests that a customer can ask a performer. Usually, the customer is an actor who initiates the action in order to achieve his goal that is referred by a desired situation (DS2). The goal corresponds to a final situation in a communicative action workflow loop. By using a flow in the forward direction, a customer is asking a performer for some action. If the request corresponds to a contract, it will always create a situation that is an opportunity for a performer. Pragmatic dependencies are represented graphically in Fig. 5.1. p

Cust omer

o Problematic Situation (PS1)

Action of Customer

Desired Situation (DS1)

g Flow 2

Flow 1

Problematic Situation (PS2)

p

Perfor mer

o Action of Supplier

Desired Situation (DS2)

Fig. 5.1. Graphical representation of pragmatic dependencies

A pragmatic link between an actor and his desired situation is entitled to as a goal dependency ( g ). The problem link ( p ) is used to refer a problematic situation of an actor. An opportunity link ( o ) is referring to an intermediate situation between a problematic and desired. If an actor has a social power to activate an action that changes a situation from the problematic to intermediate, then this intermediate situation for some other actor may help to create new desirable situations. According to the presented schema, a customer has a possibility to initiate the action by sending Flow 1 to a performer in order to avoid a problem denoted by a problematic situation (Customer p PS1). If a performer is satisfied by the flow (it will be accepted) then the problematic situation would be replaced by action of customer to a desired situation (DS1). In the next step, a performer by sending Flow 2 has a possibility to change his problematic situation (PS2) to the desired situation (DS2) that is regarded to as a goal of customer (Customer g DS2). Graphical example of the semantic and pragmatic dependencies in a typical action workflow loop is depicted in Fig. 5.2.

Modelling of Semantic and Pragmatic Dependencies of Information Systems

p

129

Cust omer

o Not O rdered Product

Item

O rder O rdered Product

g

O rder

Available Item

p

Sup plier

o Supplied Item Supply

Fig. 5.2. Example of dependencies in an action workflow loop

Satisfaction of actors is closely related to their goals and problems. In order to activate an action, an agent has to know about the opportunities available to a recipient. If a recipient is viewing the intermediate situation as an opportunity to achieve his goal or to avoid problem, then the flow, that has been sent by an agent, would be acceptable by a recipient.

6 Dependencies of Positive and Negative Influence Any two goals can be contradictory, if one of them is interpreted as a problem to reach another goal. Contradictory goal can influence negatively or hinder the achievement of a desirable situation. This means that interpretation of goal and problem is relative and dependent on the actor objectives. The same situation can be interpreted as a goal for one actor or as a problem for another. The negative influence dependency has been introduced in F3 [7] to specify contradictions between goals. The negative influence dependency from A to B (A B) indicates that goal A hinders to the achievement of goal B. Conditions for the existence of a negative influence dependency between two situations are as follows: if ACTOR p S1 and ACTOR g S4 then S1 - S4 . This axiom specifies that a problematic situation (S1) hinders to the achievement of a desired situation (S4). Moreover, in a context of an opportunity (S2), the following axiom is true: if ACTOR p S1 , ACTOR o S2 and ACTOR g S4 then S2 - S1, S2 + S4. According to this definition, an opportunity must influence negatively a problematic situation and influence positively a desired situation. The positive influence dependency between two goals means that the achievement of one goal would contribute to the achievement of the second. It should be noted that

130

R. Gustas

any problem that hinders to a problematic situation may be considered as an opportunity for some other actor, i.e. if ACTOR p S3 and S2 - S3 then ACTOR o S2. Any opportunity that influences positively a desired situation may be considered as an opportunity as well, i.e. if ACTOR g S5 and S4 + S5 then ACTOR o S4. The negative influence dependency is useful to express contradictions between goals of various actors. If one of the goals hinders to the achievement of the other, then these goals are in conflict. The positive influence dependency between two goals is indicating that the achievement of one goal helps to achieve the second. It should be noted that the additional pragmatic dependencies are derived according to the following inference rules: if A +

B , B - C then A - C ,

if A - B , B - C then A + C , if A +

B , B + C then A + C ,

if A B then A + B . is a goal composition dependency [12]. Here: Semantic and pragmatic dependencies in an action workflow loop are very important to analyse viability of business processes. It should be noted that viability of a single communicative action dependency between two actors guaranties that the desired situations create new possibilities for the recipients of flows. The customer tries to initiate an action, because he wants to avoid a problematic situation or to achieve a new desired situation. By depending on a performer, a customer is able to achieve a situation that can not be reached without involvement of a specific performer. At the same time, if the performer fails to deliver the flow to a customer, then customer becomes vulnerable to the failure. The negative and positive dependencies between various situations are imposed by different intentions of actors. If influences in the action workflow loop are incompatible, i.e. A + B and A - B, then this situation is referred to as a contradiction. By using a set of influences between situations from the point of view of different actors, the contradictory goals can be identified. It is not so difficult to see in the example of the previous chapter that pragmatic dependencies are not contradictory. An overall set of semantic and pragmatic dependencies of a particular process constitute a formal basis for inconsistency analysis. Inconsistencies may be eliminated either through negotiation among actors, and by disregarding some of the goals or by disregarding some of the actors. In a case of existence of a cost effective development action, inconsistency among goals may serve as a driving force for business process re-engineering. Inconsistencies between actor goals in a context of the same process would mean that one of the actors is vulnerable to the failure in the achievement of his goal. Consistency of pragmatic dependencies in the action workflow loop guaranties that interests of two actors are not conflicting. If interests of actors are in conflict, then the action workflow loop may be not viable. Viability of action workflow loops can be studied in terms of semantically complete diagrams.

Modelling of Semantic and Pragmatic Dependencies of Information Systems

131

7 Semantic Incompleteness of Diagrams The way normally people analyze systems is by reasoning on a basis of the model for a particular part of the system. Many analysts in the area of information system development define their systems in terms of initial conceptual models, and later extend them by making a whole lot of assumptions. Very often these conceptual models are quite vague, because of several reasons. Sometimes, dependencies of conceptual diagrams may not be defined strictly and can be interpreted ambiguously. Even if the diagrams developed by experts are presented in a formal way, the system model may be still not clear enough. This can take place for the reason that the description of system is incomplete. Elimination of semantic incompleteness in a diagram, by the refinement of relations among concepts is important, if system analyst wish to reason automatically about the expected scenarios, contingent actions and opportunities available in a particular business process. An applicable set of dependencies allows us to avoid semantic holes [10]. In this study about conceptual dependencies, we concentrate on a particular subset of semantic links, which is entitled to as totally applicable dependencies. Applicability of dependencies can be achieved through appropriate transformations of concept diagrams. Some methods call this process of change as strengthening or restricting [2], [3]. In object-oriented approach, the transformation process is entitled to as sharpening meaning of concepts [13]. It improves the ability to understand and communicate conceptual models. Transformations of diagrams have been mostly studied in the context of the static dependencies. In this approach, we have introduced the common basis to deal with both static and behavioural parts of representations. It means that semantic transformations can take into account both aspects of the information system description. Semantic relationships of information system can be specified by using two kinds of abstractions: aggregation and generalisation. Abstraction of aggregation is based on the presented set of the totally applicable binary dependencies. These links are as follows: ⇒ is the injection dependency, ⇒> is the surjective partial functional dependency and is the total functional dependency, is the composition dependency, is the communication dependency and is the transition dependency. These dependencies will be referred to as basic. The generalisation abstraction is based on the inheritance dependency ( ). Inheritance links can be of various kinds. Although the inheritance constructs are well-understood, there is no complete agreement of an interplay between inheritance dependency and other types of basic constraints of semantic models. For example, some of researchers understand the inheritance in the same way as the inclusion dependency. To eliminate ambiguity, we will define the inheritance dependency in terms of the presented basic constraints. Let Ad be a set of static and dynamic semantic links that are specified for the dependent from A concepts. The inheritance dependency from concept A to B is defined by A B if and only if A ⊆ B and Bd ⊆ Ad. The inheritance is characterised by the following axioms: B,B C then A C, 1) if A

132

R. Gustas

2) if A B , B ⇒ C then A ⇒ C , B,B C then A C, 3) if A 4) if A B,B C then A C, 5) if A B , B ⇒> C then A ⇒> C , 6) if A B, B C then A C, B, B C then A C. 7) if A Inheritance is defined in terms of two abstractions: intensional and extensional. If A inherits B, then the structure (intension) of concept B must be included as part of concept A intensional structure. Definition of inheritance in the extensional sense is defined as follows: if x ∈ B, B C then x ∈ C. It should be noted, that the presented definition is more general than that is assumed in object-oriented approach. For instance, concepts A and B can be interpreted as categories of states and actors. Any communicative action is defined unambiguously if and only if it is expressed in terms of applicable constraints. Two states of the semantically unambiguous action must be connected to other concepts by the total static dependencies. As far as actor communication links are concerned, the flow dependency from an agent to recipient is considered as an applicable constraint as well. The same condition holds for a current state that is linked by a transition dependency to the new state. It means that any object, which belongs to the current state, is applicable for a specified transition link. The presented set of totally applicable dependencies is useful to assess the semantic ambiguity [10] of specifications and to reason about a particular part of the system. Very often information system specifications are quite vague, because some of the semantic dependencies are optional. If the diagrams can be defined in terms of basic dependencies, then certain formal inference rules may be applied to derive additional semantic links that can be used to check semantic consistency of information system requirements at the enterprise level.

8 Conclusions Interplay between semantic and pragmatic dependencies lie in the foreground of the suggested framework. In this approach, we have shown how to bring various semantic diagrams together and to combine insights from the point of view of different goals. An attempt of the generic framework is to bridge goals of various actors and a way various business processes are described. Goals and desires of actors constitute an important part of knowledge about business processes. The intentional relationships among actors are viewed with potentially common and conflicting interests. The presented pragmatic dependencies can explain freedom of a specific actor and the extent to which actors are exposed to a danger. The usefulness of great number of semantic dependencies in the area of information system design is an open problem. For instance, many dependencies introduced in database theory encounter problems with missing attribute values. These problems result from the fact that either the instance of the attribute is

Modelling of Semantic and Pragmatic Dependencies of Information Systems

133

temporally unknown, but applicable, or that its value can never be known, because the attribute is not applicable for a specific instance [4]. For the unambiguous definition of concepts, only applicable dependencies can be used. Despite of strictness of this requirement, it allows us to discover contingent actions in the semantic diagrams and to introduce rules of reasoning [10] at the enterprise level. The presented actor dependencies constitute a unified basis for modelling of dynamic relationships and can be regarded as an extension and integration of the state transition and interaction diagrams. Actor goal dependencies constitute a unified basis for modelling of pragmatic relations that are able to define actor intentions. Goals justify and explain the presence of the semantic dependencies, which are used to specify components of information system requirements. Thus, such integrated approach offers a novel perspective on semantic analysis of information systems. The main difference between this framework and Yu & Mylopoulos approach [18] is that any actor dependency may be considered at the same time to be the action and the goal dependency. The action dependency is defined in terms of state transition and flow dependencies. It has also been shown how the negative and positive influence dependencies, introduced in F3 [7], can be formally defined in terms of the semantic and pragmatic dependencies. There is a growing interest to integrate information system development methodologies from different areas such as requirements engineering, method engineering, workflow management, business process modelling, object – oriented approach, etc. Many system analysts recognise that it is not enough to describe semantics of information system by concentrating distinctly on one of the methods. When re-engineering information systems most of models tend to neglect communication aspects among several actors of organisation. Dependencies of communication and co-operation between actors and their actions describe a very important part of knowledge about business processes. Unfortunately, communication approaches often neglect some behavioural aspects of system modelling that are basic in information system engineering. The presented set of totally applicable dependencies can be viewed as an integrated semantic modelling technique to specify the deeper structures of relationships among concepts. Suggested generic framework focuses on the modelling of static and dynamic constraints, where several actors co-operate to achieve new desired states. The basic dependencies provide a uniform formal basis in the area of concurrent business process modelling, analysis and integration. The purpose of such introduced basis is that eventually information system diagrams can be used as a tool to assist reasoning and validate enterprise models before they are implemented.

References 1. Action Technologies. Action Workflow Analysis Users Guide. Action Technologies, 1993. 2. A T Borgida. Generalisation/Specialisation as a Basis for Software Specification. On Conceptual Modelling, M Brodie, J Mylopoulos, J W Schmidt (eds.), Springer-Verlag, New York, 1984, pp.87-112.

134

R. Gustas

3. R Brachman, J G Schmolze. An Overview of the KLONE Knowledge Representation System. Cognitive science, 9(2), pp. 171-212, 1985. 4. E F Codd. The Relational Model for Database Management. Addison-Wesley Publ. Co., 1990. 5. G B Davis, M Olson. Management Information Systems. McGraw Hill, New York, 1985. 6. E D Falkenberg et al. A Framework of Information System Concepts. The Report of the IFIP WG8.1 Task Group FRISCO, 1996. 7. F3 Consortium. 'F3 Reference Manual (Esprit III Project 6612)', SISU, Kista, Sweden, 1994. 8. G Goldkuhl. Information as Action and Communication. The Infological Equation, Goteborg University, Sweden, pp. 63-79, 1995. 9. R Gustas, J Bubenko jr., B Wangler. Goal Driven Enterprise Modelling: Bridging Pragmatic and Semantic Descriptions of Information Systems. Information Modelling and Knowledge Bases VII, Y Tanaka, H Kangassalo, H Jaakola, A Yamamoto (eds.), IOS Press, 1996 , pp. 73 -91. 10. R Gustas. Towards Understanding and Formal Definition of Conceptual Constraints. Proc. of the European-Japanese seminar on Information Modelling and Knowledge Bases VI, 1994, IOS Press, pp. 381-399. 11. R Gustas. A Basis for Integration within Enterprise Modelling. Second Int. Conference on Concurrent Engineering: Research and Applications, Washington, DC Area, August 23-25, 1995, pp. 107-120. 12. R Gustas. A Framework for Description of Pragmatic Dependencies and Inconsistency of Goals. Proc. of the second Int. conference on the Design of Cooperative Systems, June 1214, 1996, Juan-Les-Pins, France, pp. 625-643. 13. J Martin, J J Odell. Object-Oriented Methods: Foundation. Prentice-Hall, New Jersey, 1995. 14. W E Riddle. Fundamental Process Modelling Concepts. NSF Workshop on Workflow and Process Automation in Information Systems, May 8-10, 1996. 15. K Siau, Y Wand, I Benbasat. The Relative Importance of Structural Constraints and Surface Semantics in Information Modelling. Information Systems, Vol. 22, No 2/3, pp 155-170, 1997. 16. J F Sowa, J A Zachman. Extending and Formalizing the Framework for Information Systems Architecture. IBM Systems Journal, 31(3), pp. 590 - 616, 1992. 17. V C Storey. Understanding Semantic Relationships. VLDB Journal, F Marianski (ed.), Vol.2, pp.455-487, 1993. 18. E Yu, J Mylopoulos. from E-R to 'A-R' - Modelling Strategic Actor Relationships for Business Process Reengineering. 13th Int. Conf. on the Entity - Relationship Approach, P Loucopoulos (ed.), Manchester, U.K., 1994. 19. E Yourdon. Modern Structured Analysis, Prentice-Hall, Englewood Cliffs, N.J., 1989. 20. H Weigand, E Verharen, F Dignum. Dynamic Business Models as a basis for Interoperable Transaction Design. Information Systems, Vol. 22, No 2/3, pp 139-154, 1997. 21. T Winograd, F Flores. Understanding Computers and Cognition: A New Foundation for Design. Ablex Norwood, NJ, 1986.

Inference of Aggregate Relationships through Database Reverse Engineering Christian SOUTOU Université de Toulouse II, IUT ‘B’ Groupe de Recherche ICARE 1 Place Georges Brassens, 31703 Blagnac, FRANCE

Abstract. This paper presents a process to improve the reverse engineering of relational databases. Our process extracts the current aggregate relationships from a relational database through a combination of data dictionary, data schema and data instance analysis. The process we propose can refine conceptual diagrams of commercial tools with reverse engineering options as Power AMC (Sybase), Designer (Oracle).

1 Introduction Reverse engineering is of increasing interest today in the context of migrating legacy database systems, in the development of multidatabase systems that integrate a variety of existing systems under a common data model interface, or in database evolution. The goal of a reverse engineering process is to produce a conceptual description of a given database where the input may consist of any combination of source code description, a data dictionary, a database instance, application programs. Algorithms for converting a relational schema into the original Entity-Relationship (ER) model [4] can be found in [7]. [15] proposes a more powerful approach that considers inheritance and provide a very detailed classification of relations and attributes. Other approaches [1,10,11,12] provide to define an Extended ER diagram from a relational schema. Other approaches for reverse engineering of relational database consider binary-relationship [18] or object-oriented as target semantic data models [3,9,14,17,27]. Some tools exist [6,8,17,18]. Recent relational database reverse engineering methodologies use instances [5,16,19,22]. The majority of these existing approaches do not take into account the extraction of n-ary relationships of degree higher than two. Aggregate relationships are not very well studied neither in commercial tools nor in reverse engineering methodologies. A reason can be that the aggregate relationship would complicate complex and big conceptual scheme. We think that is not so because we consider that aggregate relationships can refine n-ary relationships extracted in a reverse engineering process. So that the semantic of the T.W. Ling, S. Ram, and M.L. Lee (Eds.): ER’98, LNCS 1507, pp. 135−149, 1998.  Springer-Verlag Berlin Heidelberg 1998

136

C. Soutou

conceptual diagram resulting of our reverse engineering process is enhanced.[5] takes into account a kind of aggregate relationships in their process, we will see in Section 3 that it exists several kinds of aggregate relationships. If we assume that the starting point of a reverse engineering process can be any combination of data description and application programs, results may be inaccurate as the data themselves would not be taken into account. We believe that as the relational model is a data based model, the very thing to start with should be considering instances together with data definition statements. This work continues recent analysis completed with [23] dealing with the principles for reverse engineering n-ary relations (n≥2).

2. Aggregate Relationships The first paper dealing with aggregate relationships seems to be [20]. Aggregation is an abstraction which turns a relationship between objects into an aggregate abject. A relationship between objects is regarded as a higher level object. We consider here the aggregate relationships extracted from n-ary relationships.

2.1

Example

Let us consider the relational database Fig. 1. Foreign key attributes are into brackets, we also use an arrow from the foreign key attribute(s) to the primary key attribute(s). soft[softno#,softname]

dept[deptno#,deptname]

server[servno#,servtype]

owner[(softown)#,(deptown)#] installation[(softins#,deptins#),(servins),dateins]

Fig. 1. Relational schema example

Although Chen was the first to publish his contribution in the ACM TODS journal [4], another survey proposed also a conceptual model [13]. The fundamental differences of these two approaches is the interpretation of cardinality constraints particularly for nary relationships. The approach of Chen and its extensions [5,17,22,26] consider cardinality constraints based on the identifiers of relationships. The second approach [10,15,24] consider cardinality constraints which are based on participation constraints between entities and relationships. This approach is used by the majority of commercial tools and methodologies [2,25]. These two different approaches are neither complementary nor opposite [23]. We can note that the formalism of Chen [4] is more precise on the semantic of n-ary relationships.Indeed, each couple of cardinality constraints depends on the interaction with the other entities, the other approach does not. Though the cardinality constraints

Inference of Aggregate Relationships through Database Reverse Engineering

137

of binary relationships must be only inverted for the two different conceptual models, for n-ary relationships the semantic is not the same. See Figs. 2 and 3 the conceptual diagrams produced with our previous methodology [23] with the instances in the Appendix 1. Chen’s formalism of the relationship ‘Installation’ indicate that for a given pair of (department, soft) there is only one instance of server, for a given pair of (soft, server) there are one or many departments, for a given pair of (server, department) there are one or many softs. Participation constraints of the relationship ‘Installation’ indicate that a soft and a server can be implicated in one or many installations, a department can be implicated in many installations or cannot be implicated in any installation. Soft

1,N Installation dateinst

softno# softname

0,N

1,1 Server servno# servtype

1,N Dept

Owner

1,N

deptno# deptname

Fig. 2. Chen’s formalism

Soft

1,N Server

1,N Installation dateinst

softno# softname

1,N

servno# servtype

0,N Dept

Owner

0,N

deptno# deptname

Fig. 3. ParticipationConstraints

However these conceptual diagrams do not represent any semantic between the relationship ‘Installation’ and the relationship ‘Owner’. The binary relationship is semantically a subset of the ternary relationship. Song calls this type of binary relationship a Semantically Constraining Binary Relationships [21]. By looking at data appendix 1 we can infer an aggregate relationship between these two relationships see Fig. 4. We use the formalism based on participation constraints to describe aggregate relationships because this approach is used by the majority of commercial tools and methodologies. The aggregate relationship ‘Installation’ links the entity ‘Server’ to the relationship ‘Owner’ that we call aggregate. It express the fact that an installation of a soft in a server is valid if the department is the owner of the soft installed. This result improves the semantic of the conceptual scheme Fig. 2 and Fig. 3.

2.2

Taxonomy

We use the following notations to describe aggregate relationships (Fig. 5). The aggregate relationship (A2) links an entity (E3) to an aggregate (A1), the id of entities (ex : a1#,b1#,c1#), the properties of entities (ex : a2,...,b2...), the properties of relationships (ex :d1,..., p1,...). Minimum cardinality constraints (x,y,z,v) can be 0,1 or N. Maximum cardinality constraints between the aggregate relationship and the aggregate (Z, V) can be 1 or N.

138

C. Soutou

We can divide the aggregate relationships into four families according to the maximum cardinality constraint between the aggregate (A1) and the entity (E3). These families are (N-N, 1-N, N-1, 1-1). Indeed, the structure of relations which represent the aggregate and the aggregate relationship has no influence upon minimum cardinality constraints. Our process takes into account all these cases. Soft softno# softname

1,N

0,1

Owner

1,N Installation dateinst

Server servno# servtype

0,N Dept deptno# deptname

Fig. 4. Example of Aggregate Relationship and Aggregate

E1 a1# a2...

x,N

z,Z

A2

v,V E3 c1# c2...

[p1...]

A1 [d1...]

y,N E2 b1# b2...

Fig. 5. Aggregate Relationship and Aggregate

2.3

Inverse Reference

An important factor to take into account is inverse references. An inverse reference exist between two relations when the first includes a foreign key upon the second and vice-versa. We can note that the relational schemas which include inverse references represent in the majority of case two aggregate relationships instead of one. Let us consider the following relational schema including an inverse reference. mission[carnumber#,datem#, km, (driver)]

2

1

employee[empno#, empname, (caremp,datemp)]

Fig. 6. Relational schema with one inverse reference

Inference of Aggregate Relationships through Database Reverse Engineering

139

According to the instances appendix 2, this schema can represent either one aggregate relationship or two distinct aggregate relationships. The instances appendix 2 enable us to deduce two distinct aggregate relationships ‘AR1’ and ‘AR2’ Fig. 7. We could call ‘AR1’ as ‘occasionalpassenger’ (is a passenger which participate in only one mission), ‘AR2’ as ‘driver’. Car carno# totalkm

0,N

1,1

0,1 Employee AR1

Mission km

0,N Date

empno# empname

0,1 1,N

AR2

datem#

Fig. 7. Aggregate Relationships deduced from instances appendix 2

The instances appendix 3 enable us to deduce only one aggregate relationship ‘AR1’. We can see that data are required to infer a valid current conceptual schema. Car carno# totalkm

0,N

1,1

Mission km

0,1 Employee AR1

empno# empname

0,N Date datem#

Fig. 8. Aggregate Relationship deduced from instances appendix 3

3. Inferring Aggregate Relationships For a given relational schema we will see that it can exist many potential aggregate relationships. We call them potentials because only instances will provide to infer the correct cardinality constraints. On the other hand, for a given aggregate relationship different relational schema are possible.

140

C. Soutou

3.1

Maximum Cardinality Constraints

Table 1 describe maximum cardinality constraints of the aggregate relationships from two relations. Only the case (2) is taken into account in [5] as we said in the introduction. The case (3) can be illustrated Fig 6, see the aggregate relationships Fig. 7 and Fig. 8. The case (3) can lead to five sub-cases : four double aggregate relationships and one single aggregate relationship. We can note that the name of the aggregate relationships must be provided by human intervention. Table 1. Maximum cardinality constraints of aggregate relationships from two relations Relational schema

A1[a1#,b1#,d1...] E3[c1#,c2...,(a1,b1)]

Conditions on instances

Maximum Cardinality A1/E3 N-1

(a1,b1) non unique in E3

1-1

(a1,b1) unique in E3

1-N

c1 non unique in A1

1-1

c1 unique in A1

1-N N-1 1-1 N-1 1-N 1-1 1-1 1-1 1-1

c1 non unique in A1 (a1,b1) non unique in E3 (a1,b1) unique in E3 (a1,b1) non unique in E3 c1 non unique in A1 (a1,b1) unique in E3 c1 unique in A1 (a1,b1) unique in E3 (a1,b1,c1) are simultaneously in E3 et A1

(1) A1[a1#,b1#,d1...,(c1)] E3[c1#,c2...] (2) A1[a1#,b1#,d1...,(c1)]

2

1

E3[c1#,c2...,(a1,b1)] Two aggregate relationships

Single aggregate relationship (3)

Table 2 describe maximum cardinality constraints of the aggregate relationships from three relations. Due to space limitations we do not consider here the possible inverse references, but our process includes these cases. We can note that the name of the aggregate relationships is the name of A2. As an example of a combination of the cases (5) and (3) let us consider the relation ‘usualpassenger’ as follows : employee[empno#, empname, (caremp,datemp)] mission[carnumber#,datem#, km,(driver)] usualpassenger[(caruspg#,dateuspg#),(empuspg)#]

Fig. 9. Relational schema with inverse references

The instances of the relations ‘employee’ and ‘mission’ are given appendix 3, we consider an example of instances for the relation ‘usualpassenger’. We can deduce the aggregate relationships Fig.10.

Inference of Aggregate Relationships through Database Reverse Engineering

141

Table 2. Maximum cardinality constraints of aggregate relationships from three relations Relational schema

Conditions on instances

Maximum Cardinality A1/E3 N-1

(a1,b1) non unique in E3

1-1 1-N

(a1,b1) unique in E3 c1 non unique in A2

(5)

1-1 N-N

A2[(a1#,b1#),(c1#),p1..]

N-1

c1 unique in A2 c1 non unique in A2 for (a1,b1) given AND (a1,b1) non unique in A2 for c1 given c1 unique in A2 for (a1,b1) given AND (a1,b1) non unique in A2 for c1 given c1 non unique in A2 for (a1,b1) given AND (a1,b1) unique in A2 for c1 given c1 unique in A2 for (a1,b1) given AND (a1,b1) unique in A2 for c1 given

E3[c1#,c2...,(a1,b1)] A2[(a1#,b1#),p1...] A1[a1#,b1#,d1...] E3[c1#,c2...]

(4)

A2[(a1#,b1#),(c1),p1...] A1[a1#,b1#,d1...] E3[c1#,c2...]

1-N

A1[a1#,b1#,d1...] (6)

1-1

Car

1,N usualpassenger

carno# totalkm

0,N Employee

0,N 1,1 Mission km

0,N Date

AR1

0,1

empno# empname

0,1 1,N

AR2

datem#

Fig. 10. Aggregate Relationships deduced from instances appendix 3

3.2

Minimum Cardinality Constraints

Table 3 describe minimum cardinality constraints of the aggregate relationships from two relations. The case (3) where the aggregate relationships is deduced from the arrow n°2 can be illustrated with the instances appendix 2. Fig. 10 shows ‘AR2’ as ‘driver’. We can see according to the instances that the minimum cardinality constraint to the side of ‘E3’ must be 0 because it exists c1 (here ‘driver’) in ‘E3’ (here ‘employee’) and not in ‘A1’ (here ‘mission’) : for example the employee ‘A01’. According to the instances the minimum cardinality constraint to the side of ‘A1’ must be 1 because it is true that it doesn’t exist c1 (here ‘driver’) NULL in ‘E3’ (here ‘employee’).

142

C. Soutou Table 3. Minimum cardinality constraints of aggregate relationships from two relations Relational schema

A1[a1#,b1#,d1...]

E3

E3[c1#,c2...,(a1,b1)]

A1

(1) A1[a1#,b1#,d1...,(c1)] E3 A1

E3[c1#,c2...] (2) A1[a1#,b1#,d1..,(c1)]

E3

1 E3[c1#,c2...,(a1,b1)] A1 A1[a1#,b1#,d1..,(c1)]

2

E3

E3[c1#,c2...,(a1,b1)] A1 (3) Two aggregate relationships A1[a1#,b1#,d1...,(c1)]

E3

E3[c1#,c2...,(a1,b1)] A1 (3) Single aggregate relationship

Minimum Cardinality 0 1 0 1 N 0 1 N 0 1 0 1 0 1 N 0 1 N 0 1 0 1 0 1 N

Conditions on instances It exists (a1,b1) NULL in E3 It doesn’t exist (a1,b1) NULL in E3 It exists (a1,b1) in A1 which is not in E3 Every (a1,b1) in A1 is also in E3 (a1,b1) is neither NULL nor unique in E3 It exists c1 in E3 and not in A1 Every c1 in E3 is also in A1 c1 is neither NULL nor unique in A1 It exists c1 NULL in A1 It doesn’t exist c1 NULL in A1 Cf. case (1)

Cf. case (2)

It exists (a1,b1) NULL in E3 It doesn’t exist (a1,b1) NULL in E3 It exists c1 NULL in A1 It doesn’t exist c1 NULL in A1 Impossible: the maximum constraint is 1

Table 4 describe minimum cardinality constraints of the aggregate relationships from three relations. The case (6) can be illustrated with the instances appendix 2 and appendix 3. Fig. 10 shows the aggregate relationship ‘usualpassenger’. We can see according to the instances that the minimum cardinality constraint to the side of ‘E3’ must be 0 because it exists c1 (here ‘empuspg’) in ‘E3’ (here ‘employee’) and not in ‘A2’ (here ‘usualpassenger’) : for example the employee ‘A04’. According to the instances the minimum cardinality constraint to the side of ‘A2’ must be 1 because it is true that c1 (here ‘empuspg’) is unique in A2 for (a1,b1) (here ‘caruspg’,’empuspg’) given : the two last rows for example.

4. Process We suppose that there are no constraints on the uniqueness of the attribute names because we consider the constraint id in the data dictionary instead of the name of the attribute. Though we use Oracle, we can adopt this method for other relational database management systems having a data dictionary.

Inference of Aggregate Relationships through Database Reverse Engineering

143

Table 4. Minimum cardinality constraints of aggregate relationships from three relations Relational schema E3[c1#,c2...,(a1,b1)]

E3

A2[(a1#,b1#),p1...] A1 A1[a1#,b1#,d1...] E3[c1#,c2...]

(4) E3

A2[(a1#,b1#),(c1),p1..] A1 A1[a1#,b1#,d1...] E3[c1#,c2...]

(5) E3

A2[(a1#,b1#),(c1#),p1..] A1 A1[a1#,b1#,d1...]

4.1

(6)

Minimum Cardinality 0 1 0 1 N 0 1 N 0 1 0 1 N 0 1 N

Conditions on instances It exists (a1,b1) NULL in E3 It doesn’t exist (a1,b1) NULL in E3 It exists (a1,b1) in A2 which is not in E3 Every (a1,b1) in A2 is also in E3 (a1,b1) is neither NULL nor unique in E3 It exists c1 in E3 and not in A2 Every c1 in E3 is also in A2 c1 is neither NULL nor unique in A2 It exists c1 NULL in A2 It doesn’t exist c1 NULL in A2 It exists c1 in E3 and not in A2 Every c1 in E3 is also in A2 Every c1 in E3 is also in A2 AND (a1,b1) never unique in A2 for c1 given Impossible : c1 is primary key c1 unique in A2 for (a1,b1) given c1 never unique in A2 for (a1,b1) given

Data Dictionary in Input

The first step of our process consist in automatically selecting the aggregate relations from the data dictionary. We use a view of the data dictionary that we call ‘cross_references’ in order to take in input for extracting the relations which compose the aggregate relationships. This view enables us to join attributes and tables with foreign and primary key. For our running example appendix 4 the content of this view is described appendix 5.

4.2

Extraction of Relations Composing the Aggregate Relationships

The extraction of the relations which compose the aggregate relationships is made with a query for each of the six cases of aggregate relationships we take into account in this paper (see Section 3). These queries inspect the view ‘cross_references’. We can note that these queries are written once for every relational schema. Each query populate or not the table ‘aggregate_results’ which define the relations which compose the aggregate relationships. Due to space limitations we cannot show each of these queries. As an example let us consider the query appendix 6 which extracts the relations composing the case (6) of aggregate relationship. We detail the fact extracted for each predicate. For our running example this query insert only one row in the table ‘aggregate_results’. This row is written in bold Fig. 11.

144

C. Soutou

4.3

Extraction of the Aggregate Relationships

The following script describe the final step of the extraction of aggregate relationships. For our running example three queries have been successful. The following figure shows the final result of the extraction. select A2,A1,E3,aggregate from aggregate_results; A2 A1 E3 aggregate --------------- --------------- --------------- -----------------MISSION EMPLOYEE 3 INSTALLATION OWNER SERVER 5 USUALPASSENGER MISSION EMPLOYEE 6

Fig. 11. Contents of the table ‘aggregate_results’

5. Conclusion and Further Research An automatic process which improves relational database reverse engineering has been presented. This process is based on a combination of data dictionary, data schema and data instance analysis. For SQL92 relational databases (where foreign key and primary key clauses exist in the schema definition) this process is fully automated. For legacy systems (databases which do not support foreign key definitions), human intervention is required to propose potential foreign key attributes. We investigate a snapshot of the database extension that we find at the beginning of the reverse engineering process to extract six cases of aggregate relationships. As our process examine data, it is true that if additional data were given conclusions of cardinality constraints of n-ary relationships could be different. Thought the results of our process does not tell us anything definite about the equivalent conceptual schema it facilitate the meaning of the semantic expressed in n-ary relationships. Our process can be included in a complete database reverse engineering methodology or can refine results from commercial tools with reverse engineering options. Commercial tools produce a diagram taking in input either a file describing the relational schema or the database itself, data dictionary is taken into account but not data. The first step of our process consists in automatically extracting the n-ary relations from the data dictionary. The second step provides the inference of current minimum and maximum cardinality constraints of n-ary relationships. Our process generates a set of SQL queries for each n-ary relation of the database studied. The final step of our process will take in input the table ‘aggregate_results’ and in output the current cardinality constraints of each aggregate relationship extracted. The cardinality constraints will be inferred according to the conditions of instances described in detail table 1, 2, 3 and 4. A Pro*C program will generate adequate SQL query for each case of aggregate relationship.

Inference of Aggregate Relationships through Database Reverse Engineering

145

References [1]

[2] [3] [4] [5]

[6]

[7]

[8] [9] [10]

[11] [12]

[13]

[14]

[15]

[16]

[17]

Andersson, M. Extracting an Entity Relationship Schema from a Relational Database through Reverse Engineering, in Proceedings of the 13th Int. Conference on EntityRelationship Approach, (ed P. Loucopoulos), Springer Verlag, 881, (1994) 403-419. Batini, C., Ceri, S., Navathe, S.B. Conceptual Database Design : an Entity Relationship Approach, Benjamin Cummings, Redwood City (1992). Castellanos, M. A Methodology for Semantically Enriching Interoperable Databases, in Proceedings of the 11th British National Conference on Databases, (1993) 58-75. Chen, P.P. The Entity-Relationship Model : Towards a Unified View of Data, ACM Transactions on Database Systems, 1, 1, (Mar. 1976) 2-36. Chiang, R., Barron, T., Storey, V.C. Reverse engineering of relational databases : Extraction of an EER model from a relational database, Journal of Data and Knowledge Engineering, 12, 2, (1994) 107-142. Comyn-Wattiau, I, Akoka, J. Reverse Engineering of Relational Database Physical Schemas, in Proceedings of the 15th Int. Conference on Entity-Relationship Approach, (ed B. Thahleim), Springer Verlag, 1157, (Oct. 1996) 372-391. Davis, K.H., Arora, A.K. Converting a Relational Database Model into an Entity Relationship Model, in Proceedings of the 6th International Conference on EntityRelationship Approach, (Nov. 1987) 243-257. Englebert, V., Henrard, J., Hick, J.M., Roland, D. Hainaut, J.L. DB-MAIN : a database oriented CASE Tool, Engineering of Information Systems, 4, 1, (1996) 87-116. Gardarin, G. Translating relational to object databases, Engineering of Information Systems, 2, 3, (1994) 317-346. Hainaut, J.L., Tonneau, C., Joris, M., Chandelon, M. Transformation-based Database Reverse Engineering, in Proceedings of the 12th Int. Conference on Entity-Relationship Approach, Springer Verlag, 823, (1993) 364-375. Johanneson, P. A method for Translating Relational Schemas into Conceptual Schemas, in Proceedings of the 10th Int. Conference on Data Engineering, (1994) 190-201. Markowitz, K.M., Makowsky, J.A. Identifying Extended Entity-Relationship Object Structures in Relational Schemas, IEEE, Transactions on Software Engineering, 16, 8, (Aug. 1990) 777-790. Moulin P., Randon J., Savoysky S., Spaccapietra S., Tardieu H., Teboul M., « Conceptual model as database design tool », Proceedings of the IFIP Working conference on Modelling in Database Management Systems , G.M. Nijssen Ed., North-Holland, 1976. Narasimham, B. Navathe, S.B., Jayaraman, S. On Mapping ER and Relational Models into OO Schemas, in Proceedings of the 12th Int. Conference on Entity-Relationship Approach, Springer Verlag, 823, (1993) 402-413. Navathe, S.B., Awong, H. Abstracting Relational and Hierarchical Data with a Semantic Data Model, in Proceedings of the 6th International Conference on Entity-Relationship Approach, (Nov. 1987) 277-305. Petit, J.M., Kouloumdjian, J. Boulicaut, J.F., Toumani, F. Using Queries to Improve Database Reverse Engineering, Proceedings of the 13th Int. Conference on EntityRelationship Approach, Springer Verlag, 881, (1994) 369-386. Premerlani, W.J., Blaha, M.R. An Approach for Reverse Engineering of Relational Databases, in Proceedings of the IEEE Working Conference on Reverse Engineering, Baltimore, (Nov. 1993) 151-160.

146

C. Soutou

[18] Shoval, P., Shreiber, N. Database Reverse engineering : From the Relational to the Binary Relationship Model, Journal of Data and Knowledge Engineering, 10, (1993) 293-315. [19] Signore, O., Loffredo, M., Gregori, M. Cima, M. Reconstruction of ER Schema from Database Application : a Cognitive Approach, in Proceedings of the 13th Int. Conference on Entity-Relationship Approach, Springer Verlag, 881, (1994), 387-402. [20] H.A. Smith, D.C.P. Smith, Database Abstractions : Aggregation and Generalization, ACM Transactions on Database Systems, Vol 2, N°2, pp 105-133, 1977. [21] Song, Y.I., Jones, T.H. Analysis of Binary Relationships within Ternary Relationships in ER Modeling, in Proceedings of the 12th Int. Conference on Entity-Relationship Approach, Springer Verlag, 823, (1993) 271-282. [22] Soutou, C. Extracting N-ary Relationships through Database Reverse Engineering, in Proceedings of the 15th Int. Conference on Entity-Relationship Approach, (ed B. Thahleim), Springer Verlag, 1157, (Oct. 1996) 392-405. [23] Soutou, C. Relational Database Reverse Engineering : Extraction of Cardinality Constraints, to appear in Journal of Data and Knowledge Engineering. [24] Spaccapietra, S., Parent, C. ERC+ :an Object Based Entity Relationship Approach", in Conceptual Modeling, Databases and CASE : An Integrated View of Information Systems Development, (Ed P. Loucopoulos and R. Zicari), John Wiley (1993). [25] Tardieu., H., Rochfeld, A., Colleti, R. La méthode MERISE, Les Editions d’Organisation, Paris, (1986). [26] Teorey, T.J., Yang, D., Fry, J.P. A logical design methodology for relational databases using the extended entity-relationship model, ACM Computing Surveys, 18, 12, (June 1986), 197-222. [27] Vermeer, M.W.W., Apers, P.M.G. Reverse Engineering of Relational Database Applications, in Proceedings of the 14th Int. Conference on Entity-Relationship Approach, (ed M.P. Papazoglou), Springer Verlag, 1021, (1995) 89-100.

Appendix Appendix 1. Instances server servno# icare0 serviut Scoserv

servtype Sun Sparc 5 Pentium 100, NT 3.51 Pentium 200, Unix SCO

owner (softown)# OPNETU W7 W7 W7 ORA8NT ORA73U

(deptown)# GTR INFO GTR GIM GTR INFO

dept deptno# INFO GTR GIM GMP soft softno# W7 ORA73U ORA8NT OPNETU

deptname Computer Science Telecom. Network Industrial Maintenance Mechanics

softname Word7 Oracle V7.3, Unix Oracle 8, NT Opnet, Unix

Inference of Aggregate Relationships through Database Reverse Engineering installation (softins# OPNETU W7 W7 ORA8NT ORA73U

deptins#) GTR INFO GTR GTR INFO

(servins) icare0 serviut serviut serviut scoserv

dateins 25-JAN-97 13-MAR-96 14-MAY-96 12-MAY-96 20-OCT-97

Appendix 2. Instances mission carnumber# 7673-VW-31 6992-WP-31 7673-VW-31 employee empno# A01 A02 A03 A04 A05

datem# 25-JAN-97 13-MAR-96 20-OCT-97

empname E. Campo J.J. Mercier T. Val F. Peyrard C. Soutou

km 240 90 400

(driver) A04 A04 A03

(caremp 7673-VW-31 7673-VW-31 6992-WP-31 6992-WP-31

employee empno# A01 A02 A03 A04 A05

datem# 25-JAN-97 13-MAR-96 12-MAY-96 20-OCT-97

empname E. Campo J.J. Mercier T. Val F. Peyrard C. Soutou

usualpassenger (caruspg# dateuspg#) 7673-VW-31 25-JAN-97 7673-VW-31 25-JAN-97 7673-VW-31 25-JAN-97 6992-WP-31 13-MAR-96 7673-VW-31 20-OCT-97

km 240 90 460 400

(driver) A04 A02 A01 A03

(caremp 7673-VW-31 6992-WP-31 7673-VW-31 7673-VW-31

(empuspg)# A01 A02 A03 A05 A05

totalkm 140560 89600 167200

car carno# 7673-VW-31 6992-WP-31 1011-VF-31

totalkm 140560 89600 167200

datemp) 25-JAN-97 20-OCT-97 13-MAR-96 13-MAR-96

Appendix 3. Instances mission carnumber# 7673-VW-31 6992-WP-31 7673-VW-31 7673-VW-31

car carno# 7673-VW-31 6992-WP-31 1011-VF-31

datemp) 12-MAY-96 13-MAR-96 20-OCT-97 25-JAN-97

147

148

C. Soutou

Appendix 4. Tables create table car (carno char(10) not null primary key, totalkm number); create table employee (empno char(10) not null primary key, empname char(30), caremp char(10), datemp number); create table datemi (datemis number not null primary key); create table mission (carnumber char(10), datem number, km number, driver char(10), primary key(carnumber,datem), foreign key(datem) references datemi(datemis), foreign key(carnumber) references car(carno), foreign key(driver) references employee(empno)); create table usualpassenger (caruspg char(10), dateuspg number, empuspg char(10), primary key(caruspg,dateuspg,empuspg), foreign key(caruspg,dateuspg) references mission(carnumber,datem), foreign key(empuspg) references employee(empno)); alter table employee add constraint cond_occ foreign key(caremp,datemp) references mission(carnumber,datem); create table dept (deptno char(10) not null primary key, deptname char(30)); create table server (servno char(10) not null primary key, servtype char(30)); create table soft (softno char(10) not null primary key, softname char(20)); create table owner (softown char(10), deptown char(10), primary key(softown,deptown), foreign key(deptown) references dept(deptno), foreign key(softown) references soft(softno)); create table installation (softins char(10), deptins char(10), servins char(10), dateins date, primary key(softins,deptins), foreign key(softins,deptins) references owner(softown,deptown), foreign key(servins) references server(servno));

Appendix 5. View cross_references Constraint -------------COND_OCC COND_OCC SYS_C003024 SYS_C003025 SYS_C003026 SYS_C003027 SYS_C003028 SYS_C003029 SYS_C003030 SYS_C003030 SYS_C003031 SYS_C003032 SYS_C003033 SYS_C003034 SYS_C003034 SYS_C003034 SYS_C003035 SYS_C003035 SYS_C003036 SYS_C003038 SYS_C003039 SYS_C003040 SYS_C003041

Relation -------------------EMPLOYEE EMPLOYEE CAR CAR EMPLOYEE EMPLOYEE DATEMI DATEMI MISSION MISSION MISSION MISSION MISSION USUALPASSENGER USUALPASSENGER USUALPASSENGER USUALPASSENGER USUALPASSENGER USUALPASSENGER DEPT DEPT SERVER SERVER

Attribute -----------CAREMP DATEMP CARNO CARNO EMPNO EMPNO DATEMIS DATEMIS CARNUMBER DATEM DATEM CARNUMBER DRIVER CARUSPG DATEUSPG EMPUSPG CARUSPG DATEUSPG EMPUSPG DEPTNO DEPTNO SERVNO SERVNO

Type ---R R C P C P C P P P R R R P P P R R R C P C P

Refconstraint -------------SYS_C003030 SYS_C003030

SYS_C003029 SYS_C003025 SYS_C003027

SYS_C003030 SYS_C003030 SYS_C003027

Inference of Aggregate Relationships through Database Reverse Engineering SYS_C003042 SYS_C003043 SYS_C003044 SYS_C003044 SYS_C003045 SYS_C003046 SYS_C003047 SYS_C003047 SYS_C003048 SYS_C003048 SYS_C003049

SOFT SOFT OWNER OWNER OWNER OWNER INSTALLATION INSTALLATION INSTALLATION INSTALLATION INSTALLATION

SOFTNO SOFTNO SOFTOWN DEPTOWN DEPTOWN SOFTOWN SOFTINS DEPTINS SOFTINS DEPTINS SERVINS

C P P P R R P P R R R

149

SYS_C003039 SYS_C003043 SYS_C003044 SYS_C003044 SYS_C003041

Appendix 6 Query extracting the relations for aggregate case (6) insert into aggregate_results select distinct NULL,c0.relation,c2.relation,c4.relation,NULL,NULL,'6' from cross_references c0, cross_references c2, cross_references c3, cross_references c4 where c0.constraint in A2[a1#,b1#,c1#... (select c1.constraint from cross_references c1 where c1.type = 'P' group by c1.constraint having count(*)=3) and c0.relation=c3.relation and exists (select c1.constraint from cross_references c1 where c1.type = 'R' and c1.relation=c0.relation and c1.constraint=c3.constraint group by c1.ref_constraint having count(*)=2)

A2[(a1,b1),.. A1[...

and c3.ref_constraint=c2.constraint c2 refers to A1 and exists (select c1.constraint from cross_references c1 where c1.constraint=c2.constraint A1[a1#,b1#,... and c1.type='P' group by c1.constraint having count(*)=2) and c4.constraint = (select c1.ref_constraint from cross_references c1 where c1.relation=c0.relation and c1.type='R' group by c1.ref_constraint having count(*)=1)

c4 refers to E3

and not exists (select c1.constraint from cross_references c1 where c1.relation = c4.relation and c1.type = 'P' group by c1.constraint having count(*)>1)

E3[c1#,...

and exists (select c3.constraint from cross_references c3 where c3.relation = c0.relation and c3.type = 'R' and c3.ref_constraint= c4.constraint);

A2[...,

(c1)...

E3[c1#,...

On the Consistency of Int-cardinality Constraints Sven Hartmann FB Mathematik, Universit¨ at Rostock, 18051 Rostock, Germany

Abstract. In the entity-relationship model, cardinality constraints are frequently used to specify dependencies between entities and relationships. They impose lower and upper bounds on the cardinality of relationships an instance of a fixed type may participate in. However, for certain applications it is not enough to prescibe only bounds, but it is necessary to specify the exact set of permitted cardinalities. This leads to the concept of int-cardinality constraints as proposed by Thalheim [14]. Different from ordinary cardinality constraints this concept allows gaps in the sets of permitted cardinalities. Our objective is to investigate the consistency of a set of int-cardinality constraints for a database scheme, i.e. the question whether there exists a fully-populated database instance satisfying all the given int-cardinality constraints.

1

Introduction

In database design, great attention is devoted to the modeling of semantics. If we consider a database as a set of tuples over certain domain values, then semantics are usually given by integrity constraints. They specify the way by that data are associated to each other. Hence, integrity constraints help us to decide whether a database is meaningful for an application or not. During the last few decades, a plenty of different classes of integrity constraints have been discussed and actually used in database design. There are several books and monographs which give an overview on semantics in databases (cf. [10,11,13]). A general approach towards integrity constraints is developed in [14], and uses extensions from [1]. Within this paper, we use the entity-relationship model (ERM) to express database schemes. In this approach, cardinality constraints are among the most popular classes of integrity constraints. They impose lower and upper bounds on the number of relationships an entity of a given type may be involved in. Thus, cardinality constraints limit the possible structure of a database. This makes the cardinality constraints to be a very powerful class of constraints. However, for certain applications cardinality constraints are even not powerful enough to allow a straightforward specification of the desired semantics. To illustrate this, we present the following small example. T.W. Ling, S. Ram, and M.L. Lee (Eds.): ER’98, LNCS 1507, pp. 150–163, 1998. c Springer-Verlag Berlin Heidelberg 1998

On the Consistency of Int-cardinality Constraints

151

Example. A new travel agency is going to organize sight-seeing tours through Europe. Each of the offered tours visits a number of Europe’s most popular cities. Using the entity-relationship approach, the itineraries of the tours are planned according to the database scheme in Fig. 1 containing two entity types (tour and city) as well as two relationship types (visits and starts). Relationships of type starts determine in which city a certain tour starts, relationships of type visits specify that a certain city is visited during a given tour.

tour



visits

6 starts

-

? city

Fig. 1. Entity-relationship diagram for the travel agency in the example.

Obviously, each tour has exactly one starting point. Further, the management decided that every tour visits 3 or 4 cities. In addition, from time to time the organizers want to offer special tours visiting 7 cities to attract new customers. However, demand and financial limitations have to be taken into consideration. Hence, every week there shall start 2 or 3 tours in each of the cities in the catalogue. Only during the long vacations the management intends to offer 5 or 6 tours per week. Finally, every city shall be visited by 1 to 3 tours per week. But, the organizers would also accept 6 tours to visit certain cities. Then they are able to negotiate with the hotels for better rates. Modeling the specified restrictions with the help of ordinary cardinality constraints would obviously cause some trouble. For example, each city is allowed to be involved in 1, 2, 3 or 6 relationships of the type visits, but not in 4 or 5. Hence, it is not enough to give lower and upper bounds for the number of participations, but the complete list of permitted values has to be specified. This leads to a generalization of the ordinary concept of cardinality constraints, namely to int-cardinality constraints as proposed by Thalheim [12]. A formal definition of these constraints will be given in the sequel. Of course, our travel agency wants to know first of all, whether it is possible to construct a database instance, i.e. a catalogue of tours satisfying all the specified rules. Reasoning about such integrity constraints belongs to the fundamental tasks in database design. When reasoning about a set of constraints, one is frequently interested in whether this set is consistent and whether it implies further constraints. Given a database scheme, a set of integrity constraints defined on

152

S. Hartmann

the scheme is said to be consistent iff it admits at least one fully-populated database satisfying all these constraints. Obviously, consistency is a basic requirement for the correctness of the chosen scheme, i.e. representation of the modeled real world. The question whether a set of ordinary cardinality constraints is consistent has been considered e.g. in [9,14]. Our objective is to investigate this problem for the larger class of int-cardinality constraints. This paper is organized as follows. In Sect. 2, we briefly describe the data model to be used. All our considerations are carried out in the entity-relationship model (ERM) of Chen. In Sect. 3, we give a formal definition of int-cardinality constraints. How to check the consistency of a set of such constraints is studied in Sect. 4. Finally in Sects. 5 and 6, we will discuss two variations of the usual idea of consistency for int-cardinality constraints. Our results can be exploited to detect dummy values in int-cardinality constraints and for scheme correction as suggested by Thalheim [14].

2

The Data Model

In the sequel, we will use a particular data model, namely the entity-relationship model (ERM) which goes back to Chen [2]. This approach is based on a simple graphical representation. Using the diagram technique, even complex schemes can be understood and handled. The entity-relationship model has been so successful that it is used at present as a standard tool in conceptual database design and, in addition, several other branches of computer science (cf. [3]). Let us briefly introduce the basic concepts of the ERM. Let E be a non-empty, finite set. In the context of our approach, the elements of E are called entity types. With each entity type e we associate a finite set et called the domain or population of the type e (at moment t). The members of et are entity instances or entities, for short. Intuitively, entities can be seen as real-world objects which are of interest for some application or purpose. By classifying them and specifying their significant properties (attributes), we obtain entity types which are frequently used to model the objects in their domains. A relationship type r is a sequence (e1 , . . . , ek ) of elements from E. Relationship types are used to model associations between real-world objects, i.e. entities. A relationship or instance of type r is an element of the cartesian product et1 × . . . × etk where et1 , . . . , etk are the domains of e1 , . . . , ek , respectively, at moment t. A finite set rt of such relationships forms the population of r at moment t. The relationship types considered so far are often called relationship types of order 1 (cf. [14]). Analogously, relationship types of higher order may be defined hierarchically.

On the Consistency of Int-cardinality Constraints

153

Suppose now, we are given entity and/or relationship types of order less then i > 0. A sequence (q 1 , . . . , q k ) of them forms a relationship type r of order i. As above, we define relationships of type r as elements of the cartesian product q t1 × . . . × q tk for a given moment t. In a relationship type r = (q 1 , . . . , q k ) each of the entity or relationship types q 1 , . . . , q k is said to be a component type of r. Additionally, each of the pairs (r, q j ) is called a link. Let S = {r1 , . . . , rn } be a set of entity and relationship types where with each relationship type r all its component types belong to S, too. Then S is called a database scheme. Replacing each type q in S by its population q t at moment t, we obtain a database or instance S t of S. A database S t is fully-populated iff none of the sets q t is empty. In the sequel, we are only interested in fully-populated databases. From the graph-theoretical point of view, a database scheme S can be considered as a finite digraph ERD = (S, L) with vertex set S and a multiset L of arcs. In ERD there shall be an arc from r to e whenever e is a component type of the relationship type r. Hence, the arcs in ERD are just the links in the database scheme. The digraph ERD is also called the entity-relationship diagram of S. Usually, the entity types are represented graphically by rectangles, the relationship types by diamonds. Figure 1 shows the diagram of the database scheme used by our travel agency. It contains two entity types (tour, city) and two binary relationship types (starts, visits). However, in more complex applications one will probably find relationship types with more than two component types, too.

3

Int-cardinality Constraints

Now, we are ready to give a formal definition of int-cardinality constraints. For a relationship type r in S and a component type e of r, let D(r, e) be a given set of nonnegative integers. The int-cardinality constraint comp(r, e) = D(r, e) specifies that in each database state the number of instances of type r, an instance e of type e is involved in, belongs to D(r, e). Therefore, whenever an instance e of type e participates in exactly d relationships of type r, then d should be a member of the set D(r, e) of permitted cardinalities. Hence, comp(r, e) = D(r, e) holds iff for all database states S t and all e ∈ et we have |{r ∈ rt : r(e) = e}| ∈ D(r, e), where r(e) denotes the restriction of relationship r to the component type e. Note, that an int-cardinality constraint is an ordinary cardinality constraint, too, iff the set D(r, e) is an interval, i.e. a set of consecutive integers. The difference

154

S. Hartmann

of both concepts is that we now allow gaps in the set D(r, e) of permitted cardinalities. If no int-cardinality constraint is defined for a link (r, e), we may assume comp(r, e) = {0} ∪ N, where N denotes the set {1, 2, . . . } of positive integers. It is easy to see, that this does not represent a real constraint, but is more or less a technical agreement. In our example from Sect. 1, the specified requirements can be expressed with the help of the following int-cardinality constraints: comp(starts, tour) = {1}, comp(starts, city) = {2, 3, 5, 6}, comp(visits, tour) = {3, 4, 7}, comp(visits, city) = {1, 2, 3, 6}. Only the first constraint may be interpreted as an ordinary cardinality constraint. All the other sets on the right hand side have gaps. Let C be a set of int-cardinality constraints defined for each link in the given database scheme S. Every instance of S satisfying all the int-cardinality constraints in C is said to be legal. It is easy to check, that the empty database is always legal. However, the travel agency is of course not interested in catalogues without cities or tours. For the same reasons, we are looking only for fully-populated instances of S. By SAT (S, C) we denote the set of fully-populated legal database instances of S. The given set C of int-cardinality constraints is consistent iff it admits at least one fully-populated legal instance of S.

tour

 {3, 4, 7}

visits

6

{1}

starts

{1, 2, 3, 6}

-

{2, 3, 5, 6}

? city

Fig. 2. Entity-relationship diagram with labels for the int-cardinality constraints.

Ordinary cardinality constraints are often reflected graphically in the entityrelationship diagrams. For int-cardinality constraints this seems to be somehow more difficult, in particular, if the sets D(r, e) are large and have lots of gaps. Nevertheless, we propose to label the link (r, e) by the set D(r, e) when comp(r, q) = D(r, e) is given. For our example, the entity-relationship diagram together with these labels is shown in Fig. 2.

On the Consistency of Int-cardinality Constraints

4

155

Consistent Sets of Int-cardinality Constraints

To characterize consistent sets C of int-cardinality constraints we propose to use suitable systems of linear diophantine equations. These systems are chosen in such a way that the consistency of C is equivalent to the existence of an integral solution for the associated systems. Assume, C admits a legal database instance S t of S. The number of instances of a type q in the database scheme shall be denoted by g(q). Obviously, these numbers have to meet strict requirements: Fact 1. Let S be a database scheme and C be a set of int-cardinality constraints defined on S. Then C is consistent iff there exists a function g : S → N such that for every link (r, e) ∈ L there are nonnegative integers xd , d ∈ D(r, e), with P x = g(e) P d∈D(r,e) d (1) d∈D(r,e) dxd = g(r). Remark. For proofs of the results presented in this paper we refer to [8]. The question arises whether it will be possible to find a function g with the properties claimed in Fact 1. The following observation gives a first answer to this question. Fact 2. Let (r, e) ∈ L be a link, and g(e) as well as g(r) be positive integers. There exist nonnegative rational values xd , d ∈ D(r, e), satisfying (1) iff min D(r, e) ≤

g(r) ≤ max D(r, e) g(e)

(2)

holds. Although Fact 2 does not explicitely guarantee the existence of an integral solution to system (1), it can easily be exploited to ensure such a solution. Combining Facts 1 and 2 we finally obtain a new characterization of admissible functions g which is somehow easier to be handled. Fact 3. Let S be a database scheme and C a set of int-cardinality constraints defined on S. Then C is consistent iff there exists a function g : S → N such that (2) holds for every link (r, e) ∈ L. Of course, the main problem is to find a function g that satisfies the inequalities (2) simultanously for all the links (r, e). In the sequel we shall use shortest-path methods in suitable digraphs for this purpose. Let G = (S, L∪L−1 ) be the symmetric digraph which we obtain from the entityrelationship diagram ERD = (S, L) by adding to each link L = (r, e) its reverse

156

S. Hartmann

L−1 = (e, r). In the sequel, we use the term link only for the elements in L and arc for an element from L ∪ L−1 . On the arcs of G we define a weight function w : L ∪ L−1 → Q ∪ {∞} by ( ∞ if 0 ∈ D(r, e), w(L) = and w(L−1 ) = max D(r, e), (3) 1 otherwise, min D(r,e) where L is the original link (r, e) ∈ L and L−1 is its reverse. Special interest is devoted to directed cycles. A directed cycle Z is a sequence of consecutive arcs A1 , . . . , Ak in the digraph. It is said to be critical whenever its weight w(Z) = w(A1 ) · · · w(Ak ) is less than 1. 

tour

6

1 3

-

7

6

1 1

?

starts

visits 6 1



6 1 2

-

?

city

Fig. 3. The digraph G associated to the ERD from our example.

Figure 3 shows the digraph G obtained from the entity-relationship diagram ERD for our travel agency. Here, we labeled the arc by their weights according to (3). For an arbitrary link L = (r, e) ∈ L of the database scheme, let g(e) and g(r) be given positive integers. By the definition of the weight function w, inequality (2) holds iff we have both, g(e) g(r)

≤ w(L)

and

g(r) g(e)

≤ w(L−1 ).

Thus, to decide the consistency of a set of int-cardinality constraints, we are looking for a function g : S → N defined on the arc set of the digraph G such that g(v) ≤ w(A) g(u)

(4)

holds for every arc A = (u, v) in G. Note, that the arc A might be both, a link or its reverse. Functions satisfying (4) have already been used when reasoning about sets of ordinary cardinality constraints (cf. [6]). We shall call them feasible or admissible. Admissible functions and their relation to database design have been considered

On the Consistency of Int-cardinality Constraints

157

in [4,6]. In particular, admissible functions exist iff there is no critical cycle in G. For further properties of admissible functions, we refer to [6]. Theorem 4. Let S be a database scheme and C a set of int-cardinality constraints defined on S. Then C is consistent iff the digraph G has no critical cycle. Sketch of the proof. As pointed out above, a function g satisfies (2) for every link L = (r, e) of the database scheme iff it is admissible with respect to the weight function (3). On the other hand, as proved in [6], there exists such a function g iff the digraph G admits no critical cycle. Hence, the claim follows by Fact 3. u t In [6] a polynomial-time algorithm is proposed to construct admissible functions using shortest-path methods (namely a variation of the well-known Bellman-Ford algorithm). Therefore, the question whether a set C of int-cardinality constraints is consistent or not can be decided in polynomial time. The statement of Theorem 4 is especially remarkable, since exactly the same claim holds for ordinary cardinality constraints, too (see e.g. [9,6,14]). Obviously, a database instance satisfying an int-cardinality constraint comp(r, e) = D(r, e) also meets the relaxed cardinality constraint comp(r, e) = (a, b) = {a, a + 1, . . . , b}, where a and b denote min D(r, e) and max D(r, e), respectively. Of course, the converse is usually not true. Nevertheless, replacing the weaker cardinality constraint by the stronger int-cardinality constraint does not affect the consistency of the constraints. The set C of int-cardinality constraints in our example for the travel agency is consistent iff the relaxed set C 0 of ordinary cardinality constraints comp(starts, tour) = (1, 1) = {1}, comp(starts, city) = (2, 6) = {2, . . . , 6}, comp(visits, tour) = (3, 7) = {3, . . . , 7}, comp(visits, city) = (1, 6) = {1, . . . , 6} is consistent, too. This observation is rather astonishing, since SAT (S, C) is usually a proper subset of SAT (S, C 0 ). However, if SAT (S, C 0 ) is empty, then SAT (S, C) is as well. We record this result as Corollary 5. Let S be a database scheme. A set C of int-cardinality constraints defined on S is consistent iff the relaxed set C 0 of ordinary cardinality constraints is consistent, too.

158

5

S. Hartmann

Restricted Consistency

It is easy to check, that the digraph G in Fig. 3 contains no critical cycle. Hence, the set of int-cardinality constraints in our example is consistent. This enables us to construct a fully-populated legal database instance for the scheme in Fig. 2. However, for practical purposes it is often not enough to prove the mere existence of legal databases. The travel agency in our example would surely not be interested in databases with hundreds of thousands of cities or tours. Due to economic limitations the number of entity and relationship instances must be bounded from above. Thus, the question arises whether there exists a fullypopulated legal database of reasonable size. Suppose, we are given upper bounds N (q) for the numbers of instances of the types q in the database scheme. Again, an upper bound ∞ does not express a real constraint. We call a set C of int-cardinality constraints restricted consistent, if there exists a fully-populated legal database instance S t of S such that for every type q the number g(q) of its instances is bounded by N (q). Unfortunately, it happens to be difficult to decide this question, as we shall see in the sequel. Theorem 6. Let S be a database scheme and C a set of int-cardinality constraints defined on S. It is NP-complete to decide, whether C is restricted consistent (with respect to given bounds N (q) ∈ N ∪ {∞}). In order to see this one has to verify that the following decision problem is NP-complete: Restricted consistency. Does there exist a function g : S → N such that there is a nonnegative integral solution to the system (1) for every link (r, e) ∈ L and such that g(q) ≤ N (q) holds for every type q ∈ S, where N (q) is some integer (from the input)? Clearly, the problem belongs to NP, since for guessed function g and integers xd , d ∈ D(r, e) and (r, e) ∈ L, the equalities in the systems (1) can be tested in polynomial time. The proof of the NP-completeness uses reduction from Integer Knapsack, which is well-known to be NP-complete. Theorem 6 shows, that asking for restricted consistency may result in a considerable increase of the complexity of the appropriate decision problem. In general, solvers of integer linear programming problems seem to be the only way to tackle the Restricted consistency problem.

6

Strong Consistency

Figure 4 shows a catalogue of tours offered by our travel agency this week. It provides a fully-populated legal database instance to the scheme in Fig. 2.

On the Consistency of Int-cardinality Constraints tour 1 3 5 7 9 11

starts in Paris Vienna Prague Rome London Geneve

159

visits tour starts in visits Prague, Vienna, Rome 2 Paris Rome, London, Geneve Prague, Rome, Paris 4 Vienna London, Geneve, Prague Vienna, Rome, London 6 Prague Geneve, Vienna, Paris Prague, Vienna, Geneve 8 Rome London, Geneve, Paris Paris, Vienna, Rome 10 London Geneve, Prague, Paris Prague, Rome, London 12 Geneve Paris, London, Vienna

Fig. 4. A catalogue of tours offered by the travel agency in our example.

Clearly, this week every tour visits exactly 3 cities. But, a customer might wish to book a tour visiting 7 cities as promised by the travel agency. Should he wait with booking till next week? Will there ever be a tour visiting 7 cities? As we shall see shortly, the answer is no. The chosen int-cardinality constraints do not allow the construction of databases with tours visiting more than 3 cities. This is of course an unpleasant fact, in particular for the customer. However, there is no way out unless the management of the travel agency changes its policy, i.e. the chosen set of int-cardinality constraints. In our example, the permitted cardinalities 4 and 7 in the set D(visits, tour) are dummy values. They might be deleted without affecting the set SAT (S, C) of fully-populated legal databases. The problem is of course how to find those cardinalities which never will occur in a legal database instance. Let S be a database scheme and (r, e) a link of S. We call a consistent set C of int-cardinality constraints strongly consistent iff for every value d in the given set D(r, e) there exists a fully-populated legal database S t containing an instance e of type e which participates in exactly d relationships of type r, i.e. satisfies |r ∈ rt : r(e) = e| = d. We call this database S t a certificate for the value d in D(r, e) Hence, in a strongly consistent set C of int-cardinality constraints non of the sets D(r, e) contains dummy values. In this section we shall show how to detect such dummy cardinalities and, consequently, how to check the strong consistency of a given set C. It is easy to see, that the union of two legal database instances is again legal. Hence, if all the sets D(r, e) in C are finite, then C is strongly consistent iff there exists a fully-populated database S t which is a certificate for all values d ∈ D(r, e) and all links (r, e): just choose a certificate for each of these values d and join all them. This provides again a legal database, which is the claimed common certificate. If there is an infinite set D(r, e), too, the argument has to be slightly changed. Obviously, a database which forms a certificate for all values d in D(r, e) would

160

S. Hartmann

be infinite. However, for any choice of a finite subset D+ (r, e) ⊂ D(r, e) the last observation remains true. From Fact 1, we obtain Fact 7. Let S be a database scheme and C be a consistent set of int-cardinality constraints defined on S. Then C is strongly consistent iff there exists a function g : S → N such that for every link (r, e) ∈ L there are nonnegative integers xd , d ∈ D(r, e) satisfying (1) such that xd is positive for every ( if D(r, e) is finite, D(r, e) d∈ D+ (r, e) if D(r, e) is infinite, where D+ (r, e) is an arbitrary finite subset of D(r, e). As is Sect. 4, this result can be used to prove the following observation. Fact 8. Let (r, e) ∈ L be a link with a finite set D(r, e) of permitted cardinalities. Further, let g(e) and g(r) be positive integers. There exists a positive rational solution to system (1) iff min D(r, e) <

g(r) < max D(r, e) g(e)

holds, or D(r, e) is of size one and we have

g(r) g(e)

(5)

∈ D(r, e).

Combining Facts 7 and 8, we obtain a characterization of strong consistency. Fact 9. Let S be a database scheme and C a consistent set of int-cardinality constraints defined on S. Then C is strongly consistent iff there exists a function g : S → N such that (5) holds for every link (r, e) ∈ L whose set D(r, e) of permitted cardinalities is of size at least two. Recall the digraph G = (S, L ∪ L−1 ) introduced in Sect. 4. A directed cycle Z in G is said to be subcritical if its weight w(Z) equals 1. In the sequel, we will show how to use subcritical cycles to check strong consistency. An arc A = (u, v) lies on such a subcritical cycle Z iff w(A) =

g(v) g(u)

(6)

holds for every admissible function g. This helps us to derive the following consequences. Fact 10. Let S be a database scheme and C a consistent set of int-cardinality constraints defined on S. Further, let L = (r, e) ∈ L be a link of S. If the link L itself lies on a subcritical cycle in G, then all permitted cardinalities different from min D(r, e) are dummy values. If the reverse arc L−1 of the link lies on a subcritical cycle in G, then all permitted cardinalities different from max D(r, e) are dummy values.

On the Consistency of Int-cardinality Constraints

161

The digraph G obtained from the entity-relationship diagram of the database scheme for our travel agency contains two directed cycles. One of them is subcritical as Fig. 5 shows. On this cycle, we find for example the link (visits,tour). According to Fact 10, we may in particular conclude that every tour visits exactly 3 cities. The values 4 and 7 in comp(visits, city) are dummy values.

tour



1 3

visits

1

6

? starts

1 2

-

6

city

Fig. 5. A subcritical cycle in the digraph G from our example.

Theorem 11. Let S be a database scheme and C a consistent set of intcardinality constraints defined on S. Then C is strongly consistent iff the set D(r, e) of permitted cardinalities for a link (r, e) ∈ L is of size one whenever the link itself or its reverse arc lies on a subcritical cycle in G. Sketch of the proof. The necessity of the claim immediately follows from Fact 10. It remains to verify the sufficiency. As mentioned above, there exists an admissible function g such that (6) holds for an arc A = (u, v) iff A lies on a subcritical cycle in G. This function g satisfies the preconditions of Fact 9. u t Whenever a consistent set C of int-cardinality constraints is not strongly consistent, then there must be dummy values in at least one of the sets D(r, e) of permitted cardinalities. In [12], Thalheim suggests to delete these values from the sets D(r, e). This process is also called scheme correction. It reduces the amount of information necessary to describe the legal databases: There will never be a database state using any of the dummy values. Hence, it is unnecessary to store these cardinalities.

7

Algorithmic Aspects

Our investigations in Sects. 4 and 6 provide characterizations of consistent and strongly consistent sets of int-cardinality constraints, respectively. As claimed, both properties can be tested in polynomial time applying methods from combinatorial optimization. According to Theorem 4, C is consistent iff the digraph G contains no critical cycles with respect to the weight function (3). The existence of critical cycles,

162

S. Hartmann

i.e. directed cycles of weight smaller 1, can be tested using shortest-path methods. In [8], we present a variation of the well-known Floyd-Warshall algorithm (cf. [5]) to decide the existence of critical cycles. Its complexity is cubic in the size of the database scheme S. If a consistent set C of int-cardinality constraints is not strongly consistent, then a slight modification of this algorithm can be used to delete dummy values from the sets D(r, e) of permitted cardinalities. When deleting all dummy values in our example according to Fact 10, we obtain the int-cardinality constraints comp(starts, tour) = {1}, comp(starts, city) = {2}, . comp(visits, tour) = {3}, comp(visits, city) = {6} Hence, it is not longer surprising that in the database in Fig. 4 every tour visits exactly 3 cities. Due to Theorem 11 this is a consequence of the chosen set of constraints. Thus, scheme corrections result in a considerable decrease of the complexity of information.

8

Conclusions

In this paper, we have developed a theory for int-cardinality constraints which generalize the well-known concept of cardinality constraints. We have shown that significant properties of sets of int-cardinality constraints can be recognized with methods from integer and combinatorial optimization. In particular, we proved that the consistency of such sets can be checked by looking for critical cycles in an associated digraph. It is worth mentioning, that a set of int-cardinality constraints is consistent whenever the corresponding relaxed set of ordinary cardinality constraints is consistent, too. Hence, gaps in the sets of permitted cardinalities do not affect the consistency of those constraints. This is of special interest for database designers as int-cardinality constraints often allow a straightforward modeling of semantics, even in more involved database applications. In addition, we introduced two variations of the consistency problem, namely restricted consistency and strong consistency. For practical reasons is often necessary to ensure the existence of legal databases with a bounded number of entities and relationships. However, this small modification of the original question leads to a dramatic increase of the complexity of such a problem. Checking the strong consistency of int-cardinality constraints allows us to detect dummy values in the sets of permitted cardinalities. To give a realistic impression of the structure of legal databases, these values should be deleted via scheme corrections. Again, this can be managed within polynomial time.

On the Consistency of Int-cardinality Constraints

163

References 1. A.P. Buchmann, R.S. Carrera and M.A. Vazquez-Galindo, A generalized constraint and exception handler for an object-oriented CAD-DBMS, IEEE conf. (1986) 3849. 2. P. Chen, The Entity-Relationship Model: Towards a unified view of data, ACM TODS 1,1 (1984) 9-36. 3. P. Chen and H.-D. Knoell, Der Entity-Relationship-Ansatz zum logischen Systementwurf (BI-Wissenschaftsverlag, Mannheim, 1991). 4. K. Engel and S. Hartmann, Constructing realizers of semantic entity relationship schemes, Preprint 95/3, Universit¨ at Rostock (1995). 5. M. Gondran and N. Minoux, Graphs and algorithms (Wiley, New York, 1984). 6. S. Hartmann, Graph-theoretic methods to construct entity-relationship databases, in: M. Nagl (ed.), Graphtheoretic concepts in computer science, LNCS 1017 (Springer, Berlin, 1995) 131-145. ¨ 7. S. Hartmann, Uber die Charakterisierung und Konstruktion von EntityRelationship-Datenbanken mit Kardinalit¨ atsbedingungen, Ph.D. thesis, Universit¨ at Rostock (1996). 8. S. Hartmann, Int-cardinality constraints in data modeling, Preprint, Universit¨ at Rostock (1998). 9. M. Lenzerini and P. Nobili, On the satisfiability of dependency constraints in Entity-Relationship schemata, Information Systems 15 (1990) 453-461. 10. D. Maier, The theory of relational databases (Computer Science Press, Rockville/MD, 1983). 11. J. Paredaens, P. de Bra, M. Gyssens and D. van Gucht, The structure of the relational database model (Springer, Berlin, 1989). 12. B. Thalheim, Fundamentals of cardinality constraints, in: G. Pernul and A.M. Tjoa (eds.), Entity-relationship approach, LNCS 645 (Springer, Berlin, 1992) 7-23. 13. B. Thalheim, A survey on Database Constraints, Reihe Informatik I-8, Universit¨ at Cottbus (1994). 14. B. Thalheim, Fundamentals of Entity-Relationship Models (Springer, Berlin, 1997).

Realizing Next Generation Internet Applications: Are There Genuine Research Problems, or Is It Advanced Product Development? Chairpersons: Kamalakar Karlapalem (HKUST) and Qing Li (CUHK) Panelists: Dik Lee (HKUST), Mukesh Mohania (University of South Australia), and John Mylopoulos (University of Toronto/CUHK)

The aim of this panel is to be both educational in providing pointers to characteristics of the Next Generation Internet Applications (NGIA), and a forum for debating relevance of these characteristics in generating new research directions. Over last three years Internet driven applications industry has seen one of the highest growth rates. Almost every week new “start-up” companies are being setup, and new applications are released into the market. This growth rate is going to continue well into the next millennium to cater to NGIA. One of the critical aspects of these new applications is the development time from conceptualization to the release of the product. This could be as short as a week end. Under this high paced applications development environment the role of the academic researchers will be discussed by this panel. The NGIA range from content providers (multimedia, push/pull scenarios), electronic commerce (transactions, workflow), and internet-based virtual database systems. An overview of these applications will be the starting point of this debate. The debate will concentrate on distinguishing between research issues and development issues in each of the following aspects (though not limited to only these): – Role of data semantics in realizing NGIA. Do we need more modeling power than what we have now? Even if we do, will we use it? – Role of design methodologies in NGIA. We need efficient and fast design methodologies. And we need design methodologies that generate efficient and lean application programs. Are there any research issues here? – Role of component oriented application deployment. The end-users may just buy different highly efficient functional application components, and assemble them to deploy new applications. How does this change the way applications are designed and developed? – Role of integrating appliances into the realm of user applications. There is a need for external software that manages persistent data in the appliances and smart cards. What kind of system development issues arise? – Role of standard middle-ware in NGIA. Will this help? Are there any research issues (meta-data,?) that come up in developing this middle-ware? T.W. Ling, S. Ram, and M.L. Lee (Eds.): ER’98, LNCS 1507, p. 164, 1998. c Springer-Verlag Berlin Heidelberg 1998

Web Sites Need Models and Schemes Paolo Atzeni Dipartimento di Informatica e Automazione Universit` a di Roma Tre Via della Vasca Navale, 79 00146 Roma, Italy http://www.dia.uniroma3.it/˜atzeni/ [email protected]

The World Wide Web is likely to become the standard platform for future generation applications. Specifically, it will be the uniform interface for sharing data in networks, both Internet and intranet. Referring mainly to “data-intensive” Web sites, which have the publication of information and data as their main goal, we can say that, in most cases, they do not satisfy the users’ needs: the information kept is poorly organized and difficult to access; also it is often out-of-date, because of obsolete content and broken links. In general, this is a consequence of difficulties in the management of the site, both in terms of maintaining the structure and of updating the information. Many Web sites exist that have essentially been abandoned. We believe that this situation is caused by the absence of a sound methodological foundation, as opposed to what is now standard for traditional information systems. In fact, Web sites are complex systems, and, in the same way as every other complex system, they need to be developed in a disciplined way, organized in phases, each devoted to a specific aspect of the system. In a Web site, there are at least three components: the information to be published (possibly kept in a database); the hypertextual structure (describing pages and access paths); the presentation (the graphical layout of pages). It is widely accepted that data are described by means of models and schemes, at various levels, for example conceptual and logical. In a data-intensive Web site, it is common to have large sets of pages that contain data with the same structure (coming from tuples of the same relation, if there is a database): therefore, we argue for the relevance of the notions of model and scheme also for the description of hypertexts. Given the various facets that can arise, we also believe that hypertexts have, in the same way as data, both a conceptual and a logical level. These issues have led us to develop a methodology (Atzeni et al. [1]) that is based on a clear separation among three well distinguished and yet tightly interconnected design tasks: the database design, the hypertext design, and the presentation design. Both database and hypertext design have a conceptual phase followed by a logical one. Figure 1 shows the phases, the precedences among them, and their major products (schemes according to appropriate models). T.W. Ling, S. Ram, and M.L. Lee (Eds.): ER’98, LNCS 1507, pp. 165–167, 1998. c Springer-Verlag Berlin Heidelberg 1998

166

P. Atzeni

1. Database Conceptual Design: Database Conceptual Scheme HH (er Model) H

HH H

H H j H

3. Hypertext Conceptual Design: Hypertext Conceptual Scheme (ncm)

?

?

2. Database Logical Design: Database Logical Scheme (Relational Model)

4. Hypertext Logical Design: Hypertext Logical Scheme (adm)

? 5. Presentation Design: Page Templates (html)

Q

Q

Q

QQ s

  

   

6. Hypertext to DB Mapping and Page Generation: Web Site (html)

Fig. 1. The Araneus Design Methodology

The originality of the methodology is in the conceptual and logical design of hypertexts, which make use of specific models, developed in this framework: – ncm, the Navigation Conceptual Model , a conceptual model for hypertexts, which is essentially a variation of the er Model suitable to describe hypertextual features in an implementation independent way; – adm, the Araneus Data Model , a logical model for hypertexts (Atzeni et al. [2]), whose main construct is that of page scheme, used to describe the common features of similar pages. It is worth noting that other proposals have been recently published that present some similarities, though with a less detailed articulation of models (P. Fraternali and P. Paolini [4], Fernandez et al. [3]). The origins of the methodological aspects can be traced back to previous work on hypermedia design (Garzotto et al. [5], Isakowitz et al. [6], Schwabe and Rossi [7]).

Web Sites Need Models and Schemes

167

Acknowledgments. I would like to thank Gianni Mecca and Paolo Merialdo, together with whom most of the concepts mentioned here are being developed.

References 1. P. Atzeni, G. Mecca, and P. Merialdo. Design and Maintenance of Data-Intensive Web Sites. Advances in Database Technology—EDBT’98, Lecture Notes in Computer Science, Vol. 1377, Springer, 1998. 2. P. Atzeni, G. Mecca, and P. Merialdo. To Weave the Web. In International Conf. on Very Large Data Bases (VLDB’97), Athens, Greece, August 26-29, 1997. 3. M. F. Fernandez, D. Florescu, J. Kang, A. Y. Levy, D. Suciu. Catching the Boat with Strudel: Experiences with a Web-Site Management System. In Proc. of ACMSIGMOD Int’l Conference on Management of Data, 1998. 4. P. Fraternali and P. Paolini. A conceptual model and a tool environment for developing more scalable, dynamic, and customizable Web applications. Advances in Database Technology—EDBT’98, Lecture Notes in Computer Science, Vol. 1377, Springer, 1998. 5. F. Garzotto, P. Paolini, and D. Schwabe. HDM – a model-based approach to hypertext application design. ACM Transactions on Information Systems, 11(1):1–26, January 1993. 6. T. Isakowitz, E. A. Stohr, and P. Balasubramanian. RMM: A methodology for structured hypermedia design. Communications of the ACM, 58(8):34–44, August 1995. 7. D. Schwabe and G. Rossi. The Object-Oriented Hypermedia Design Model. Communications of the ACM, 58(8):45–46, August 1995.

ARTEMIS: A Process Modeling and Analysis Tool Environment S. Castano1 , V. De Antonellis2 , and M. Melchiori2 1 2

University of Milano, DSI - via Comelico, 39 - 20135 Milano Italy [email protected] University of Brescia, DEA - via Branze, 38 - 25123 Brescia - Italy {deantone,melchior}@ing.unibs.it

Abstract. To support business process understanding and reengineering, techniques and tools for process modeling and analysis are required. The paper presents the ARTEMIS tool environment for business process modeling and analysis. Process analysis is performed according to an organizational structure perspective and an operational structure perspective, to capture the degree of autonomy/dependency of organization units in terms of coupling, and the inter-process semantic correspondences, in terms of data and operation similarity, respectively. Processes are modeled as workflows and techniques developed for workflow analysis are presented in the context of a pilot application involving the Italian Ministry of Justice.

1

Introduction

Most private and public organizations have recently turned their attention to the process by which they operate, to improve service and product quality and customer satisfaction [13]. To support business process understanding and reengineering, techniques and tools for process modeling and analysis are studied [12, 9,14]. Moreover, reverse engineering techniques to reconstruct conceptual models of existing applications and databases are proposed for analysis purposes [1]. Process analysis is generally performed following an information processing viewpoint [10], focusing on input/output data and on the process structure and execution modalities [2,15]. In [6], we have presented a process analysis approach according to an inherently data-oriented perspective, that mainly focuses on characteristics of data manipulated and exchanged by processes and on related operations. In this paper, we extend the approach to the analysis of process structure and execution modalities described by workflow specifications, and we present the ARTEMIS (Analysis of Requirements: Tool Environment for Multiple Information Sources) tool environment for process modeling and analysis. The analysis techniques of ARTEMIS rely on workflow descriptions of processes and allow the analyst to discover and classify critical situations requiring reengineering interventions, according to operational structure and organizational structure perspectives. T.W. Ling, S. Ram, and M.L. Lee (Eds.): ER’98, LNCS 1507, pp. 168–182, 1998. c Springer-Verlag Berlin Heidelberg 1998

ARTEMIS: A Process Modeling and Analysis Tool Environment

169

Functionalities provided by ARTEMIS are illustrated together with results of their application to the international adoption processes of the Italian Juvenile Court of the Ministry of Justice, in the context of the PROGRESS (PROcess Guided REengineering Support System) project. PROGRESS is a research project, funded by the Italian National Research Council (CNR) and by the Italian National Consortium for Informatics (CINI), which aims at reengineering data and processes of Italian Public Administration information systems. The paper is organized as follows. In Sect. 2, we present an overview of the ARTEMIS functionalities. In Sect. 3, we describe process workflow modeling, while, in Sect. 4, we describe the ARTEMIS analysis functionalities, with application to selected processes of the Juvenile Court. Finally, Sect. 5 draws some conclusions and describes future work.

2

Functionalities of the ARTEMIS Tool Environment

ARTEMIS provides the following analysis functionalities to support reengineering activities: – Process form cataloging. Process forms in PROGRESS provide textual description of organization units and related processes, giving preliminary information on their input/output and composing tasks. Once such forms are filled-in with information on processes to be analyzed and modeled, they are stored and properly classified by means of keywords to facilitate their subsequent retrieval through ARTEMIS. – Process workflow modeling. In ARTEMIS, processes to be analyzed are modeled as workflows, according to the WIDE workflow model, presented in Sect. 3. The WIDE model has been developed in the framework of the Esprit Project WIDE (Workflow on Intelligent Distributed database Environment) [3], and allows the representation of operational and organizational aspects relevant for workflow analysis. Process workflow modeling is accomplished by interfacing the WIDE workflow specification tool, called FORO designer. – Process workflow analysis. These are the core functionalities of ARTEMIS, which operate according to the following analysis perspectives. Analysis of the operational structure. Processes are analyzed with respect to their input/output information entities and their functionality, in order to identify situations of replication/redundancy/overlapping of activities and to evaluate the relevance and repetitiveness of processes within a given unit. The analysis is based on the following similarity coefficients: i) entity-based similarity coefficient, to evaluate the degree of similarity of two processes with respect to their input/output information entities. It can be a point of reference to evaluate the adequacy of data usage in the operational structure; ii) functionality-based similarity coefficient, to analyze process relationships due to operation commonality. It can be a point of reference to check the adequacy of production/manipulation of information in the operational structure.

170

S. Castano, V. De Antonellis, and M. Melchiori

Analysis of the organizational structure. Processes are analyzed with respect to their exchanged information flows in order to evaluate the degree of interdependency between them. On the basis of the number of exchanged information flows, we can measure the degree of coupling between different processes. The analysis is based on the following coupling coefficients: i) actual coupling coefficient, to evaluate the degree of coupling of processes in the same or different organization units, based on the analysis of information flows involving identical entities; ii) potential coupling coefficient, to evaluate the degree of coupling, based on the analysis of information flows involving entities that are similar according to the semantic dictionary contents. Coupling coefficients can be a point of reference to understand the information flow network and its implications to determine, at an aggregated level, the information flows among the separate involved processes and envisage possible regrouping of processes to simplify or expedite the flow and improve global effectiveness. To support the process analysis, two further functionalities are provided in ARTEMIS: – Interactive construction of a semantic dictionary. The evaluation of similarity and coupling coefficients requires the comparison of information entities and of operations of different processes. A semantic dictionary is exploited to handle possible synonyms and other terminological relationships between entity and operation names. The semantic dictionary is semi-automatically built before starting the analysis, where information entity names and operation names are stored as terms and organized by generalization and aggregation mechanisms. Issues related to the construction of the semantic dictionary are presented in [6]. – Interactive extraction of process descriptors. Process descriptors provide summary information on processes to be analyzed, for the evaluation of similarity and coupling coefficients. They are interactively extracted from workflow specifications produced with the modeling functionality. In the following, we will focus on functionalities for process workflow analysis, with application to the International Adoption processes of the Italian Juvenile Court. In particular, after a general description of the process workflow model, we discuss the analysis perspectives and the associated coefficients and present results of their application to selected processes.

3

Process Workflow Modeling

In ARTEMIS, processes are modeled as workflows, using the WIDE workflow model [3]. In WIDE, a workflow schema is the model of a (business) process. It is a collection of tasks, which are the elementary work units that collectively achieve the workflow goal. Tasks are organized into a flow structure defining the

ARTEMIS: A Process Modeling and Analysis Tool Environment

171

execution dependencies among tasks, i.e., the order in which tasks are executed. Sequential, parallel and conditional executions can be specified. A workflow case is an execution of a workflow schema, i.e., an instance of the process. Multiple executions of the same process may be active at the same time, and are denoted by a different activation number. Each task within a workflow schema has associated a set of characteristics, among which: the name; a textual description; the actions, either manual or a sequence of statements of the description language defining how both temporary and persistent workflow data are manipulated by the task; roles which may perform the task, and a set of constraints concerning task assignment to agents; the task information (documents, forms, dossier, accessed databases), classified as input and output to be used/produced when achieving the task; exceptions, that can be specified to handle abnormal situations that can occur during the execution of the task and need to be managed properly. The flow structure is specified by means of a set of constructs allowing sequence, alternative, and parallelism. Two tasks may be directly connected by an edge to denote sequence: as soon as the first one ends, the second one is scheduled for execution. More complex execution dependencies are specified by means of the fork connectors (for initiating concurrent execution) and join connectors (for synchronizing after concurrent execution) [3]. A case is executed by scheduling tasks (as defined by the flow structure) and by assigning them for execution to a human or an automated agent. As a case is started, the first task (the successor of the start symbol) is activated. As a task connected to the stop symbol is completed, the case is also completed. The WIDE workflow model includes also an advanced construct of supertask to enable modularization in a workflow specification. It is a composite task, composed of elementary (atomic) tasks or of other supertasks. As the predecessor of the supertask is completed, the first task in the supertask is activated. As the last task in the supertask is completed, the successor of the supertask is scheduled for execution. For example, the International Adoption process is modeled in WIDE as a set of supertasks (shadowed boxes in Fig. 1), describing the composite activities involved in the process. The process starts when a couple submits an application to be declared eligible for an international adoption. After receiving the submitted application, international qualifying is attested for the couple, waiting for a foreign judge action (this condition is represented by means of the oval box preceding Foreign Action Procedure supertask in the figure). Depending on the validity of the action, the Revoke Fostering or the Adoption task is executed. If the action is valid, after one year and if no other events occur in the meantime, the Adoption supertask can start. If the foreign action is invalid, the Revoke Fostering activity is started, which is also executed when a notification arrives stating that the fostering is not valid. This situation is modeled using a conditional fork (diamond symbol after Foreign Action Procedure in the figure). The workflow can terminate in two different cases. In the first case, termination occurs if the fostering is not valid, or no adoption is allowed (this

172

S. Castano, V. De Antonellis, and M. Melchiori

Application Couples applying for the int. adoption are recorded and evaluated

Intern. Qualifying The international qualifying is attested

arrival of a foreign judge action

Foreign Action Procedure The action of a foreign judge is given legal effectiveness Exceptions Notification from social welfare that fostering is not valid; --> end ST

Valid foreign action

Invalid foreign action

Revoke Fostering A revoke fostering action is issued revoked fostering

validated fostering 1 year or notification from social welfare that fostering is not valid or the couple gives back the children

notification arrival

end WF no notification

(starting WF National Adoption)

Adoption The adoption action is issued

Exceptions Notification from social welfare on incorrect fostering; the couple gives back the children --> end ST, starting ST Revoke Fostering

no adoption adoption

end WF

Fig. 1. The International Adoption process in WIDE

ARTEMIS: A Process Modeling and Analysis Tool Environment

173

... PUBLIC PROSECUTOR Input International adoption dossier Output: International adoption dossier + Opinion of the Public Prosecutor

Attorney Consultation Issue Opinion of the Public Prosecutor EDP

Input: International adoption dossier Output : International adoption dossier

Update EDP Update DataBase

I,S PROCEDURE (Date of submitting acts to the Public Prosecutor)

OFFICIAL RECEIVER Input : International adoption dossier, possible appeal Output : International adoption dossier + action request

Action Requirement Issue a requirement on the effectiveness of the adoption or fostering foreign act CHAMBER OF COUNCIL

Input : International adoption dossier Output : International adoption dossier + result of trial

Trial and Decisions

Issue result of trial

CHANCELLERY

Input: International adoption dossier Output International adoption dossier + Action of adoption validation or fostering validation + communication of the result of trial + registers

Getting Results Update Registers I,S,U A CTION (all the attributes) EDP

Input: International adoption dossier Output: International adoption dossier + validation action of international fostering or international adoption

Insert Results Update Database and Print Results

Wait for appeal

Appeal

No Appeal

Fig. 2. The Foreign Action Procedure activity in WIDE

is modeled by using a conditional join -circle symbol- before the end WF symbol on the leftside), and the National Adoption workflow can be started. In the second case, the workflow terminates with successful adoption. Each supertask of the International Adoption is in turn expanded into a workflow, to model its corresponding component tasks and executing agents. In Fig. 2, part of the expansion of the supertask Foreign Action Procedure into elementary tasks is shown. For each task, the organization unit responsible for task execution is reported, together with the input and output documents.

174

S. Castano, V. De Antonellis, and M. Melchiori

Moreover, if database operations are performed from within the task, these are listed, by specifying the type (Select, Update, Insert, Delete) and the involved attributes of database entities and/or relationships.

4

Process Workflow Analysis

In ARTEMIS, process workflow analysis is performed using descriptors. A process descriptor gives a summary, structured representation of the features of a process that are relevant for the application of the analysis coefficients. A descriptor provides the following information for a process: i) the name of organizational unit responsible for process execution; ii) the set IN of input information entities; iii) the set OU T of output information entities; iv) the set OP of operations performed by the process, formally described as triplets h action, constitutive entities (CST ), circumstantial entities (CSM )i. In ARTEMIS, descriptors are interactively extracted from WIDE workflow specifications, by analyzing the internal representation generated by the FORO designer tool. Since workflows describe complex processes (i.e., business processes), descriptors are extracted from workflow tasks, to allow for a fine-grained analysis of performed activities. In particular, the organization unit field of the descriptor corresponds to the agent that performs the task (graphically shown in the upper right hand corner of the task diagram, see Fig. 1); IN and OU T sets of information entities correspond to input and output documents associated with the task (graphically they are labeled as INPUT and OUTPUT in the figure); the set OP of operations that the task performs are derived from task description, by recognizing the action (i.e., a verb), the set CST of constitutive entities required by the operation, and the set CSM of circumstantial entities involved in the operation. The operations are interactively extracted, starting from task description, with tool assistance. As an example, in Fig 3, the descriptor for the task Getting Results within the Foreign Action Procedure workflow is shown.

Process descriptor TASK: Getting Results ORGANIZATION UNIT: Chancellery INPUT: {international adoption application, code of the minor, code of the couple, role number, dossier of the couple, dossier of the minor, action of delegation assignment, authorization of the Ministry of the Internal and Foreign Affairs, action request, result of trial } OUTPUT: { communication to the Ministry of the Internal and Foreign Affairs, registers } OPERATIONS: { }

Fig. 3. An example of descriptor for the task Getting Results

ARTEMIS: A Process Modeling and Analysis Tool Environment

4.1

175

Analysis of the Operational Structure with Experimentation in PROGRESS

Following this perspective, processes are analyzed according to similarity criteria and classified into families to evaluate possible situations of redundancy, replication, or inconsistency with respect to manipulated information entities and performed operations. Process analysis and classification is based on the following similarity coefficients. Entity-based similarity coefficient. The Entity-based similarity coefficient of two processes Pi and Pj , denoted by ESim(Pi , Pj ), is evaluated by comparing the input/output information entities in their corresponding descriptors, that is,

ESim(Pi , Pj ) =

2 · Atot (IN (Pi ), IN (Pj )) 2 · Atot (OU T (Pi ), OU T (Pj )) + | IN (Pi ) | + | IN (Pj ) | | OU T (Pi ) | + | OU T (Pj ) |

where Atot (IN (Pi ), IN (Pj )) (respectively, Atot (OU T (Pi ), OU T (Pj ))) denotes the total value of “affinity” between the pairs of input (respectively, output) information entities in Pi and Pj , and | | denotes the cardinality of a given set. Atot is obtained by summing up the affinity values of all the pairs of input/output information entities that have affinity in the semantic dictionary. Proper “affinity functions” are available on the dictionary which, given two names, determine their affinity value A() ∈ [0, 1] based on the existence of a path of terminological relationships (e.g., synonymy, hypernymy) between them, on its length and on the strength of the involved relationships. High affinity values (i.e., A() ∈ [0.7, 1]) denote that the corresponding information entities can be considered for the evaluation of the ESim coefficient. ESim can assume values in the range [0, 2]. It is 0 when no pairs of entities with affinity are found in IN and OU T , while has value 2 when each input and output entities of Pi has affinity with an input and output entity of Pj and viceversa. Intermediate values are proportional to the number of pairs of information entities with affinity in Pi and Pj and to their affinity value. Functionality-based similarity coefficient. The Functionality-based similarity coefficient of two processes Pi and Pj , denoted by FSim(Pi , Pj ), is evaluated by comparing the operations in their corresponding descriptors. Also in this case, the comparison is based on the semantic dictionary, that is, F Sim(Pi , Pj ) =

2 · Atot (OP (Pi ), OP (Pj )) | OP (Pi ) | + | OP (Pj ) |

where Atot (OP (Pi ), OP (Pj )) denotes the total value of affinity of the pairs of operations that are similar in Pi and Pj . Two operations are similar if their actions, their constitutive information entities and, if defined, their circumstantial information entities have affinity in the dictionary. The similarity value of two

176

S. Castano, V. De Antonellis, and M. Melchiori

operations is obtained by summing up the affinity values of their corresponding elements. F Sim assumes values in the range [0, 3]. It is 0 when no similar operations are found in Pi and Pj , while has value 3 when each operation of Pi is similar to an operation of Pj and vice versa, and elements in each operation pair have the greatest affinity value. Intermediate values are proportional to the number of pairs of similar operations in Pi and Pj and to the affinity of their elements. Once selected processes for the analysis, the user can interactively set similarity thresholds to filter out process pairs based on computed ESim and F Sim values. Similarity reports are produced by ARTEMIS in form of tables, listing process pairs according to different criteria (e.g., decreasing order of similarity, alphabetical order). Two kind of semantic correspondences are identified between two processes Pi , Pj : i) Semantic equivalence, (Pi ≡ Pj ), denoting processes whose ESim and F Sim coefficients have the maximum value. This means that they perform same real-world activity (activity replication) and for them the unification of the involved activity should be evaluated. ii) Semantic relationship, (Pi ∼ Pj ), denoting processes with T1 ≤ ESim(Pi , Pj ) < max and T2 ≤ F Sim(Pi , Pj ) < max, where T1 and T2 are similarity thresholds specified by the user. This means that processes execute partially overlapping real-world activities, and this situation should be analyzed to evaluate the unification or the standardization of the involved activities. Let us now discuss the results of performing the operational structure analysis on the adoption processes in PROGRESS. Four processes related to the national and international adoption procedures have been selected for the analysis, namely, National Evaluation, Art. 144 Law 189, International Adoption, and National Adoption. The analysis has been performed separately for each process, since the four selected processes represent different procedures with different objectives. Two organization units play an important role in all examined processes, namely the EDP and the Chancellery units, which have been identified as crucial units for reengineering activities. We report the results of the analysis of the tasks performed by the EDP and the Chancellery organization units within the International Adoption process. Analyzing the International Adoption workflow means analyzing all the workflows corresponding to its supertasks, that is, the Application, International Qualifying, Foreign Action Procedure, Revoke Fostering, and Adoption workflows (see Fig. 1). We first performed the operational structure analysis of all International Adoption tasks of each organization unit separately, to classify the activities performed by the unit and reason about their relevance and repetitiveness. Then, we performed the analysis of all International Adoption tasks of both the EDP and Chancellery together, to recognize activity replication/overlapping and evaluate restructuring interventions. This way, ARTEMIS allowed us to reason about different aspects and point out critical situations of different nature, within the same organization unit or involving the two units together. Analysis of all tasks of a given organization unit. Tasks performed by the Chancellery and the EDP in the International Adoption process have been

ARTEMIS: A Process Modeling and Analysis Tool Environment

177

analyzed separately for each unit. On the basis of obtained similarity values, tasks of each unit were classified into families. Families obtained with ARTEMIS were used to classify the activities performed by the two considered organizational units. For example, for the Chancellery, we recognized four main categories of activities performed within the International Adoption, namely Update Registers, Getting Application, Registration, and Request of Documents. Each category groups tasks executed in different points of the examined workflows, which are characterized by similar operations on similar documents. Identified activity categories are the basis for evaluating the following parameters: i) replication of a given task in a workflow; ii) repetitiveness of a given category within a workflow; iii) relevance of a given category in terms of other categories which depend on its accomplishment in order to start their execution. Based on these parameters, it is possible to establish if a category has to be considered as “crucial” and if its task occurrences refer to similar operational conditions. In this case, the category of tasks is candidate to automation, if the activities are (partially) manually performed. By analyzing task categories for the considered units, we found that the Chancellery performs mainly administration activities while the EDP unit is characterized by several database update activities. By evaluating the repetitiveness of each category within the International Adoption process, we discovered that crucial tasks are those concerning register update for the Chancellery and database update for the EDP, respectively. Moreover, by examining the execution flow in correspondence of tasks of a given category, we observed that, in the present way of working, tasks concerning register update are mandatory for the continuation of the activities in the workflow, while corresponding database updates are not. Consequently, tasks performed by the Chancellery are to be considered as relevant for the International Adoption process. On the contrary, tasks performed by the EDP unit are non relevant in the present configuration, being executed time after the necessary information is produced. As a consequence, the database does not reflect in real-time the situation contained in paper-based registers. It would be desirable to have real-time updates of the database, as soon as the necessary information is produced. This would be possible if the Chancellery were enabled to perform on-line updates on the database. To formulate possible solutions of workflow reorganization to meet such a requirement, we need to exploit also the results of the analysis of the tasks performed by both EDP and Chancellery units in the International Adoption. Analysis of all tasks of different organization units. EDP and Chancellery tasks related for the International Adoption process were analyzed for evaluating their similarity; in Table 1, we report the obtained ESim and F Sim coefficients. As we can see from these values, only tasks with a semantic relationship characterize these two units in the considered process. By analyzing the tasks with the highest similarity values (i.e., the four tasks characterized by ESim = 1 and F Sim = 2, 8 and the pair with ESim = 2 and F Sim = 1, 2 in Table 1), together

178

S. Castano, V. De Antonellis, and M. Melchiori

Table 1. ESim and F Sim coefficients for the Chancellery and the EDP tasks in the International Adoption process Chancellery Application Submitting(75) Document Request(78) Getting Results(98) Getting Results(98) Getting Results(98) Getting Results(98) Getting Results(102) Getting Results(107) Int. Adoption Application(88) Int. Adoption Application(88) Int. Adoption Application(88) Int. Adoption Application(88) Int. Adoption Application(88) Registration of Decrees(86) Registration of Decrees(86) Registration (77) Registration (91) registration (91) Registration (91) Registration (91)

EDP Insert (76) Insert (76) EDP Update(95) Insert Results(99) Insert (89) Registration (90) Insert Results(103) Inserting Results(108) EDP Update(95) Inserting Results(87) Insert Results(99) Insert (89) Registration (90) EDP Update(83) Inserting Results(87) Insert (76) EDP Update(95) Inserting Results(99) Insert (89) Registration (90)

ESim

FSim

0,66666670,9777778 0,8 1,2 0,98181821,466667 1 2,8 0,15384621,2 0,4 0,8685715 1 2,8 1 2,8 0,18181820 0,12307690,8 0,15384620,8 1 1,2 0,5 0,65 0,93333331,3 1 2,8 0,8 1,066667 0,75 2,066667 0,66666671,3 0,28571430,8 0,66666670,5428572

with the involved workflows, we discovered the repetition of a recurring “pattern” in task execution, that is, in correspondence of a task of register update by the Chancellery, a “dual” task performing the database update is performed by the EDP. For example, let us consider the tasks Getting Results and Insert Results (fourth row in Tab. 1) performed by the Chancellery and the EDP units, respectively. In the current Foreign Action Procedure workflow, these two tasks are performed in sequence (see Fig. 2, boxed area), with disadvantages related to periodical update overhead for the EDP and inconsistent state of the database with respect to information in the registers. According to obtained similarity values and by taking into account current execution modalities, we envisaged two possible solutions: Solution a) concurrent tasks (see Fig. 4): the EDP and the Chancellery can execute the updates concurrently. To make this possible, the Chamber of Council has to send a copy of the documents to be updated to both EDP and Chancellery. This solution puts a (limited) overhead to the activity of Chamber of Council, to make an additional copy of produced documents to be sent to the EDP. This solution can be useful in a transition phase, from the current workflow to the target workflow, defined according to Solution b).

ARTEMIS: A Process Modeling and Analysis Tool Environment

179

... Input : International adoption dossier Output: registers, communication of the result of trial

Input: International adoption dossier Output: International adoption dossier + validation action of international fostering or international adoption CHANCELLERY

Getting Results

Update Registers

EDP

Insert Results I,S ACTION (all the attributes) Update Database and Print Results

2

...

Fig. 4. Solution a): Concurrent tasks ...

CHANCELLERY Input : International adoption dossier Output : International adoption dossier + validation action of international fostering or international adoption registers , communication of the result of trial

Getting Results Update Registers Update Database and Print Results

I,S ACTION

(all the attributes)

...

Fig. 5. Solution b): Unified tasks

Solution b) unified tasks (see Fig. 5): with this solution, the activities actually demanded to the EDP and the Chancellery are unified into a unique task under responsibility of the Chancellery. This way, the Chancellery is enabled to perform directly necessary database updates, as soon as modifications are introduced in the registers. This solution, whose benefits are evident, requires an underlying distributed architecture to be implemented. 4.2

Analysis of the Organizational Structure with Experimentation in PROGRESS

The effectiveness of an organization unit depends on its level of coupling with outside (i.e., other organization units). Following this analysis perspective, we analyze the interactions between different units, to understand the information flow network, and its implications. The analysis is based on input and output information entities to determine, at an aggregate level, the information flow among the processes. When analyzing processes of different organization units, two kinds of flows are relevant: “actual flows”, denoted by 7→ek ,eh , originated by the exchange of the same information entity; “potential flows”, denoted by

180

S. Castano, V. De Antonellis, and M. Melchiori Table 2. Actual Coupling Coefficients for Chancellery and EDP processes CHANCELLERY Getting Results(98) Getting Results(102) Getting Results(102)

EDP Inserting Results(99) Inserting Results(99) Inserting Results(103) Getting Results(107) Inserting Results(99) Getting Results(107) Inserting Results(103) Getting Results(107) Inserting Results(108) Registration of Decrees(86) Inserting Results(87)

AC 12 15 15 17 18 19 12

;ek ,eh , originated by the exchange of information entities with affinity. To evaluate process coupling in a precise way allowing to point out the nature of involved entities, the following coupling coefficients are introduced. Actual Coupling coefficient. The Actual Coupling coefficient of two processes Pi and Pj , denoted by AC(Pi , Pj ) measures the amount of relationships between the processes due to actual information flows, that is, AC(Pi , Pj ) =| {hek , eh i | Pi 7→ek ,eh Pj } | where {hek , eh i | Pi 7→ek ,eh Pj } is the set composed of information entities that originate actual flows between Pi and Pj . Information flows are determined by comparing the input (respectively, output) information entities of one process with the output (respectively, input) information entities of the other one. Potential Coupling coefficient. The Potential Coupling coefficient of two processes Pi and Pj , denoted by P C(Pi , Pj ), measures the amount of relationships between the processes due to the actual and potential flows between them, that is, P C(Pi , Pj ) =| {hek , eh i | Pi 7→ek ,eh Pj ∨ Pi ;ek ,eh Pj } | The Potential coupling coefficient is evaluated by exploiting the semantic dictionary, to recognize information entities with affinity. Analogously to the operational structure, summary reports for both the actual and potential coupling coefficients can be produced with ARTEMIS, with different presentation options. The user can decide which kind of coupling coefficient to compute, and on which processes. In particular, it is possible to compute coupling coefficients on families of similar processes obtained as a result of the operational structure analysis, if processes of different organization

ARTEMIS: A Process Modeling and Analysis Tool Environment

181

units are involved, to provide complementary insights of process families for a comprehensive analysis. In our experimentation, coupling coefficients have been computed for the tasks of the Chancellery and EDP units. In Table 2, we report the actual coupling values for the International Adoption tasks that have a semantic relationship in the two organization units. Since the examined tasks involve the same information entities, the potential coupling is not relevant. The higher the coupling values, the higher their level of dependency, which suggests unification interventions. On the basis of the analysis of the volumes and type of the exchanges, it is possible to envisage possible regrouping of processes to simplify or expedite this flow. By analyzing the information flows between the Chancellery and EDP units, we found that they exchange the same documents several times, but not all exchanged documentation is necessary for the accomplishment of the tasks. Moreover, such additional documentation exchanges contribute to slow the overall process. A solution based on the task unification would contribute also to reduce the number of exchanged documents. Another solution could be the reorganization of the documentation flow, to reduce and optimize the number of document exchanges required to accomplish the goal. In general, modifications to existing workflow schemas can be incorporated in ARTEMIS, by producing new workflows, and new analysis coefficients evaluated and compared with previous ones, to facilitate the analysis. For this purpose, reporting functionalities offered by the ARTEMIS tool environment can be exploited.

5

Concluding Remarks

In this paper, we have presented the ARTEMIS tool environment for process modeling and analysis. Based on workflow specification of processes, the analysis is performed according to an organizational structure perspective and an operational structure perspective, to capture the degree of autonomy/dependency of organization units in terms of coupling and the inter-process semantic correspondences, in terms of data and operation similarity, respectively. The analysis tool environment described in this paper is intended to be a support to facilitate process reengineering activities, by providing similarity-based techniques for the systematic analysis of processes for aspects related to information and operation similarity, and to exchanged information flows. Results of performing the analysis on the adoption processes of the Juvenile Court of the Ministry of Justice have been discussed. Actually, such results and proposed solutions are under examination of the Juvenile Court. Goal of future work is the analysis of execution modalities of single processes and communication protocols between them, to discover possible inefficiency and failures and enrich tool capabilities for reengineering.

182

S. Castano, V. De Antonellis, and M. Melchiori

References 1. Aiken, P.: Data Reverse Engineering, McGraw-Hill 1996. 2. Barros, A.P., ter Hofstede, A.H.M., Proper, H.A.: Towards Real-Scale Business Transaction Workflow Modelling. In Proc. of CAiSE’97 - Int. Conf. on Advanced Information Systems Engineering, Barcelona, Spain (1997). 3. Casati, F., Ceri, S., Pernici, B., Pozzi, G.: Conceptual Modeling of Workflows. In Proc. of OO-ER’95, Int. Conf. on the Object-Oriented and Entity-Relationship Modelling, Gold Coast, Australia (1995). 4. Castano, S., De Antonellis, V., Fugini, M.G., Pernici, B.: Conceptual Schema Analysis: Techniques and Applications. ACM Transactions on Database Systems (to appear). 5. Castano, S., De Antonellis, V.: Semantic Dictionary Design for Database Interoperability. Proc. of ICDE’97, IEEE Int. Conf. on Data Engineering, Birmingham (1997). 6. Castano, S., De Antonellis, V.: A Framework for Expressing Semantic Relationships Between Multiple Information Systems for Cooperation. Information Systems, Special Issue on CAiSE’97, 27(3/4) (1998). 7. Castano, S., De Antonellis, V.: Reference Conceptual Architectures for Reengineering Information Systems. International Journal of Cooperative Information Systems 4(2 & 3) (1995). 8. Castano, S., De Antonellis, V.: Reengineering Processes in Public Administrations. Proc. of OO-ER’95, Int. Conf. on the Object-Oriented and Entity-Relationship Modeling, Gold Coast, Australia, (1995). 9. Fong, J.S.P., Huang, S.M.: Information Systems Reengineering, Springer-Verlag (1997). 10. Galbraith, J.R.: Designing Complex Organizations, Addison-Wesley Publishing Company (1973). 11. Hammer, M.J.: Reengineering Work: Don’t Automate. Obliterate, Harvard Business Review, July/August (1990). 12. Georgakopoulos, D., Hornik, M., Sheth, A.: An Overview of Workflow Management: From Process Modeling to Workflow Automation Infrastructure. Distributed and Parallel Databases, Kluwer Academic Publishers, 3 (1995). 13. Karagiannis, D. (Ed.): Special Issue on Business Process Reengineering, SIGOIS Bulletin, 16(1) (1995). 14. Nurcan, S., Grosz, G., Souveyet, C.: Describing Business Processes with a Guided Use Case Approach. Proc. of CAiSE*98, - Int. Conf. on Advanced Information Systems Engineering, Pisa, Italy, (1998). 15. Workflow Management Coalition: The wfmc specification - terminology & glossary. Doc. WFMC-TC00-1011 (1996).

From Object Oriented Conceptual Modeling to Automated Programming in Java* 1

1

1

Oscar Pastor , Vicente Pelechano , Emilio Insfrán , Jaime Gómez

2

1

Department of Information Systems and Computation Valencia University of Technology Camí de Vera s/n 46071 Valencia (Spain) { opastor | pele | einsfran }@dsic.upv.es 2 Department of Languages and Information Systems Alicante University C/ San Vicente s/n 03690 San Vicente del Raspeig. Alicante (Spain) {[email protected]}

Abstract. The development of Internet commercial applications and corporate Intranets around the world, that often use Java as programming language, is a significant topic in modern Software Engineering. In this context, more than ever, well-defined methodologies and high-level tools are essential for developing quality software in a way that should be as independent as possible of the changes in technology. In this article, we present an OO method based on a formal object-oriented model. The main feature of this method is that developers' efforts are focused on the conceptual modeling step, where analysts capture system requirements, and the full implementation can automatically be obtained following an execution model (including structure and behaviour). The final result is a web application with a three-tiered architecture, which is implemented in Java with a relational DBMS as object repository.

1 Introduction The boom of “web computing” environments, has originated the development of Internet commercial applications and the creation of corporate Intranets around the world. The use of the Java language [1] in these environments has opened significant research related to the proper implementation of final correct software products. In this context, where new technologies are continuously emerging, the software development companies must make suitable methods, languages, techniques and tools * This work has been supported by the CICYT under MENHIR/ESPILL TIC 97-0593-C05-01 Project and by DGEUI under IRGAS GV97-TI-05-34 Project. T.W. Ling, S. Ram, and M.L. Lee (Eds.): ER’98, LNCS 1507, pp. 183−196, 1998.  Springer-Verlag Berlin Heidelberg 1998

184

O. Pastor et al.

for dealing with the new market requirements. More than ever, well-defined methodologies and the high-level tools which support them are essential for developing quality software in a way that should be as independent as possible of the changes in technology. The idea of clearly separating the conceptual model level, centered in what the system is, and the execution model, intended to give an implementation in terms of how the system is to be implemented, provides a solid basis for operational solutions to the problem. If we have rich-enough conceptual modeling environments to capture the relevant system properties in the problem space, a correct software representation at the solution space can be easily generated. Related work in this area has been developed based on the idea that we can significantly reduce the complexity of advanced-application specification and implementation by using a model-equivalent language (a language with a one-to-one correspondence to an underlying, executable model [3]). Nowadays OO methodologies like OMT [2], OOSE [4] or Booch [5] are widely used in industrial software production environments. Industry trends attempt to provide unified notations such as the UML proposal ([6]) which was developed to standardize the set of notations used by the most well-known existing methods. Although the attempt is commendable, this approach has the implicit danger of providing users with an excessive set of models that have overlapping semantics without a methodological approach. Following this approach we have CASE tools such as Rational ROSE/Java [7], FrameWork [8] or Paradigm Plus [9] which include Java code generation from the analysis models. However if we go into depth with this proposed code generation feature, we find that it is not at all clear how to produce a final software product in Java that is functionally equivalent to the system description collected in the conceptual model. This is a common weak point of these approaches. Far from what is required, what we have after completing the conceptual model is nothing more than a template for the declaration of classes where no method is implemented and where no related architectural issues are taken into account. In order to provide an operational solution to the above problem, in this paper we present a method that is based on a formal object-oriented model. The main feature of this method is that developers' efforts are focused on the conceptual modeling step, where analysts capture system requirements, and the full implementation is automatically obtained following an execution model (including structure and behaviour). The final result is a web application with a three-tiered architecture, which is implemented in Java with a relational DBMS as object repository. The main contribution of this approach with respect to the topic of model-equivalent programming languages is that we start from a graphical representation of a formal object-oriented specification language for conceptual modeling purposes. After having defined a finite set of behavioural patterns, a mapping from these patterns to software components in a given software development environment (Java in this paper) is defined. In consequence, our methodological approach is not attached to any particular programming language. A CASE Tool gives support to this method. It constitutes an operational approach to the ideas of the automated programming paradigm [10]: a collection of system information properties in a graphical environment (conceptual modeling step),

From Object Oriented Conceptual Modeling to Automated Programming in Java

185

automated generation of a formal OO system specification and of a complete software prototype (including static and dynamic) obtained from the conceptual model and which is functionally equivalent to the quoted system specification (execution model step).

2 The OO-Method: An Object-Oriented Method Following the OO-Method strategy, the software production process starts with the conceptual modeling step where we have to collect the relevant system properties. Once we have an appropriate system description, a formal OO specification is automatically obtained. This specification is the source of a well-defined execution model which determines all the implementation-dependent features in terms of user interface, access control, service activation, etc. This execution model provides a well-structured framework that enables the building of an automatic code generation tool. It is important to note that the formal specification is hidden from the OOMethod user: the relevant system information is introduced in a graphical way, which is syntactically compliant with the conventional OO models, but which is semantically designed to fill the class definition templates according to the formal OO basis.

2.1 OASIS: an Object-Oriented Formal Model The OO-Method was created on the formal basis of OASIS, an OO formal specification language for Information Systems [11]. In fact, we can see the OOMethod as a graphical OASIS editor, that provides the conventional approach of using an object, dynamic and functional model to make designers think that they are using a conventional OO method. The formalism is in this way hidden to them, avoiding the controversial attached to the use of formalisms in software development environments. Previous works on this idea can be found in the tunable formalism in object-oriented system analysis presented in [12]. Our approach provides a different kind of tunable formalism: the OASIS expressiveness is fully preserved, but it is presented to analysts according to a well-known conventional graphical notation (UML compliant). Below, we give a quick overview of the characteristics of OASIS. From an intuitive point of view, an object can be viewed as a cell or capsule with a state and a set of services. The state is hidden to other objects and can be handled only by means of services. The set of services is the object's interface, which allows other objects to access the state. Object evolution is characterized in terms of changes of 1 states. Events represent atomic changes of state and can be grouped into transactions . When we build a system specification, we specify classes. Classes represent a collection of objects sharing the same template. The template must allow for the 1

Molecular units of processing composed of object services that have the properties of nonobservability of intermediate states and the all-or-nothing policy during execution.

186

O. Pastor et al.

declaration of an identification mechanism, the signature of the class including attributes and methods, and finally a set of formulae of different kinds to cover the rest of the class properties: • integrity constraints (static and dynamic) which state conditions that must be satisfied. • valuations which state how attributes are changed by event occurrences. • derivations which relate some attribute’s values to others. • preconditions which determine when an event can be activated. • triggers which introduce internal system activity. Finally, as an object can be defined as an observable process, a class definition should be enriched with the specification of the process attached to the class. This process will allow us to declare possible object lives as terms whose elements are events and transactions. OASIS deals with complexity by introducing aggregation and inheritance operators. A complete description of the OASIS language can be found in [13].

2.2

Conceptual Modeling

Conceptual modeling in OO-Method collects the Information System relevant properties using three complementary models: Object Model: a graphical model where system classes including attributes, services and relationships (aggregation and inheritance) are defined. Additionally, agent relationships are introduced to specify who can activate each class service (client/server relationship). Dynamic Model: another graphical model to specify valid object life cycles and interobject interaction. We use two kinds of diagrams: • State Transition Diagrams to describe correct behaviour by establishing valid object life cycles for every class. By valid life, we mean a right sequence of states that characterizes the correct behaviour of the objects. • Object Interaction Diagram: represents interobject interactions. In this diagram we define two basic interactions: triggers, which are object services that are activated in an automated way when a condition is satisfied, and global interactions, which are transactions involving services of different objects. Functional Model: is used to capture semantics attached to any change of an object state as a consequence of an event occurrence. We specify declaratively how every event changes the object state depending on the involved event arguments (if any) and the object’s current state. We give a clear and simple strategy for dealing with the introduction of the necessary information. This is a contribution of this method. It allows us to generate a complete OASIS specification in an automated way. More detailed information can be found in [14]. From these three models, a corresponding formal and OO OASIS specification is obtained using a well-defined translation strategy. The resultant OASIS specification

From Object Oriented Conceptual Modeling to Automated Programming in Java

187

acts as a complete high-level system repository, where the relevant system information, coming from the conceptual modeling step, is captured.

2.3

Execution Model

Once all the relevant system information has been specified, we use an execution model to accurately state the implementation-dependent features associated with the selected object society machine representation. More precisely, we have to explain the pattern to be used to implement all the system properties in a logical three-tiered architecture for any target software development environment: • interface tier: classes that implement the interaction with end users presenting a visual representation of the application and giving users a way to access and control the object's data and services. • application tier: classes that fully implement the behaviour of the business classes specified in the conceptual modeling step enforcing the semantics of our underlying object model • persistence tier: classes that provide services allowing the business objects to interact with their specified permanent object repository. In order to easily implement and animate the specified system, we predefine a way in which users interact with system objects. We introduce a new way of interaction, close to what we could label as an OO virtual reality, in the sense that an active object immerses in the object society as a member and interacts with the other society objects. To achieve this behaviour the system has to: 1. identify the user (an access control): logging the user into the system and providing an object system view determining the set of object attributes and services that it can see or activate. 2. allow service activation: finally, after the user is connected and has a clear object system view, the users can activate any available service in their worldview. Among these services, we will have system observations (object queries) or events or transactions served by other objects. The process of access control and the building of the system view (visible classes, services and attributes to the user) are implemented in the interface tier. The information to properly configure the system view is included in the system specification obtained in the conceptual modeling step. Any service activation has two steps: build the message and execute it (if possible). In order to build the message the user has to provide information to: 1. identify the object server: The server object existence is an implicit condition for executing any service, unless we are dealing with a new event 2. At this point, the persistence tier retrieves the object server from the database. 2

Formally, a new event is a service of a metaobject that represents the class. The metaobject acts as object factory for creating individual class instances. This metaobject (one for each class) has the class population attribute as a main property, the next oid and the quoted new event.

188

O. Pastor et al.

2. introduce event arguments: The interface tier asks for the arguments of the event being activated (if necessary). Once the message is sent, the service execution is characterized by the occurrence of the following sequence of actions in the server object (the application tier): 1. check state transition: verification in the object State Transition Diagram (STD) that a valid transition exists for the selected service in the current object state. 2. precondition satisfaction: the precondition associated with the service must hold If 1 and 2 don't hold, an exception will arise and the message is ignored. 3. valuation fulfillment: the induced event modifications (specified in the Functional Model) take place in the involved object state. 4. integrity constraint checking in the new state: to assure that the service execution leads the object to a valid state, the integrity constraints (static and dynamic) are verified in the final state. If the constraint does not hold, an exception will arise and the previous change of state is ignored. 5. trigger relationships test: after a valid change of state, the set of condition-action rules that represents the internal system activity is verified. If any of them hold, the specified service will be triggered. The previous steps guide the implementation of any program to assure the functional equivalence between the object system specification collected in the conceptual model and its reification in a programming environment. Some interesting related work can be found in [15], where a tool (IPOST) that automatically generates a prototype from an object-oriented analysis model is introduced. This permits users to refine the model to generate a requirement specification. The contribution of OOMethod comes from the fact that the resultant prototype is not only a requirements specification. It is much closer to the final software product, because what is generated uses the solution space notation (Java, as we are going to show).

3 An Architecture for Implementing the Execution Model in Java The abstract execution model shown above is based on a generic three-tiered architecture. Below, we will introduce a concrete implementation using web technology and Java as the programming language. This will provide a methodological framework to deal with Java implementations, which are functionally equivalent to the source conceptual model.

3.1 Translating a Conceptual Model into Java Classes using the Execution Model Starting from the proposed Execution Model, we want to design the architecture of classes needed to implement the three logic levels: interface, application and persistence. Following, we specify the most relevant features of the Java classes needed to support the intended architecture:

From Object Oriented Conceptual Modeling to Automated Programming in Java

189

At the interface level we have complementary classes, which are not explicitly used in the conceptual model but which help to implement the interaction between the user and the application, following the underlying object model semantics. • Access_control class. This class extends a panel with the typical widgets to allow users to be identified as a member of the object society (by providing the object identifier, password and the class to which the user belongs). One access control object is created every time that a user wants to be connected to the system as an active object sending and receiving messages. This class implements the first step of our execution model. import java.awt.*; import java.lang.*; import excepciones.*; //exceptions defined for //the application. public class Access_control extends Panel { ... } • System_view class: Once an active object (user) is connected to the system, a system_view object (instance of the system_view class) will show a page with the same number of items as classes the user is allowed to interact with. These items are clickable regions that the user can activate in order to see the services (also displayed as clickable items) that he/she can use. The declaration of the class is the following: import java.applet.*; import java.awt.*; import excepciones.*; //exceptions defined for //the application. public class System_view extends Applet implements Runnable { ... } • Service_activation class: This class will define a typical web interface for data entry, where the relevant service arguments are requested. The service_activation object will be a generic one for all the services of all the classes. Depending on the service activated, it will show the corresponding edit boxes for the identification of the object and the parameters of the service. Once they are fulfilled, the user can send the message to the destination (Ok) or ignore the data request action (Cancel). import java.awt.*; public class Service_activation extends Panel {...} At the application level we have the classes that implement the behaviour of the business classes specified in the conceptual model. In order to ensure that the implementation of the application classes will follow our underlying object model semantic and have persistence facilities, we will define our business classes as an implementation of an OASIS interface and an extension of an Object_mediator class [16]. The OASIS interface specifies the necessary services to support the execution model structure, as shown in the following paragraph:

190

O. Pastor et al.

import java.awt.*; interface Oasis { void check_preconditions (string event) void check_state_transition (string event) void check_integrity_constraints () void check_triggers () ... } At the persistence level a Java class called object_mediator must be created. It implements the methods for saving, deleting and retrieving system domain objects that are stored in a persistent secondary memory (object repository). The object_mediator class has the following general structure: class Object_mediator { void delete (); void save (); void retrieve (); } JDBC classes will be used for proper interaction with the involved RDBMS servers in the implementation of these methods. Even if we focus on Java in this paper, this design could be translated to any other OO programming language by properly distributing the application components depending on the target environment characteristics and necessities.

3.2 Distributing Java Classes in a Web Architecture Once the previous components have been created, they must be properly distributed in a web architecture. Many proposals to distribute Inter/Intranet application components exist. We are going to present a three-tiered web architecture (see Fig. 1) which fits very well with the OO-Method Execution Model features presented above. A client (a web browser) unloads the relevant HTML pages from a web server, together with the applets that conform the application interface. The user will interact with the Java system objects through the web client. These Java system objects will be stored in a web application server that will query or update the object state stored in a Data Server, through the services provided by the JDBC objects. It is important to remark again that the components distributed in this architecture are generated in an automated way using the OO-Method CASE tool. This is done by defining a precise mapping between the finite set of behavioural patterns that constitutes the Conceptual Model in OO-Method and their corresponding software representation in the target software development environment, Java in this work. Preconditions, state transitions, integrity constraints and triggers are declared in the graphical OO-Method models, and implemented in each one of the Java Business Classes as was presented before. As they finally are well-formed formulas in OASIS, these formulas are translated into the Java syntax. This is how the problem space

From Object Oriented Conceptual Modeling to Automated Programming in Java

191

concepts are converted into their corresponding software representation. Let’s more clearly show these concepts through an example.

Fig. 1. Three-tiered web architecture

4 Automated Programming in Java. A Case Study A CASE Tool supporting the OO-Method allows us to model and automatically generate fully functional prototypes in Java. In order to better understand the architecture and behaviour of the previous component classes generated using the Java Execution Model, we introduce a Rent-a-Car case study as a brief example: “A company rents vehicles without drivers. These vehicles are bought at the beginning of the season and usually sold when the season is over. When a customer rents a vehicle, a contract is generated and it remains open until the customer returns the vehicle. At that time, the total amount to be paid is calculated. After this step, the Vehicle is ready to be rented again”. First, we construct the Conceptual Model (object, dynamic and functional models) by identifying the classes and specifying their relationships and all the static and dynamic properties. Due to space limitations, we cannot present all the three OOMethod models (object, dynamic and functional) of this simple example, but let’s assume that the classes identified in this problem domain are the following: Contract,

192

O. Pastor et al.

Customer, Vehicle and Company. Every class will have a set of attributes, services, preconditions, integrity constraints, valid transitions, evaluations and triggers. Based on the Execution Model proposed above, we obtain the architecture of the web application in an automated way. This architecture includes Java classes and relational tables attached to the conceptual model in Table 1. Table 1. Rent-a-Car System implementation architecture. Persistence Tier Customer table Vehicle table Contract table ...

Application Tier Customer class Vehicle class Contract class ...

Interface Tier Access_control class System_view class Service_Activation class ...

The Java code that implements a business class in the application tier following the execution model strategy can be seen in the code example for the Vehicle class in the following paragraph (comments have been introduced with the aim of making it selfexplanatory): package application_tier; import excepciones.*; import object_mediator.*; import oasis.*; public class Vehicle extends Object_mediator implements Oasis { // Attributes specified in the conceptual model class Vehicle private string state; ... // Events specified in the conceptual model // The following method implements the change of object's state

protected boolean rent() throws EX_Check_Error { state="rented"; ... } ... // The following method implements the precondition checking

protected void check_precondition(string event) throws EX_Check_Error {...}

From Object Oriented Conceptual Modeling to Automated Programming in Java

193

// The following method implements the execution of the rent event

public void eval_rent() { try { retrieve( ); // retrieves the object from the Database check_precondition('rent'); check_state_transition('rent'); rent(); check_integrity_constraint(); check_triggers(); save( ); // saves the object in the Database } catch(EX_Check_Error e) {...} } ... }

Next, we are going to describe an illustrative scenario for the Rent-a-Car prototype generated. This scenario will show the interaction between Java objects and their behaviour when a user enters into the object system. When a client loads the main HTML page that calls the Java applet, an instance of the Access Control class is created and User identification is required (see Fig. 2). After a User connects to the Rent-a-Car system, a menu page with an option for every class will appear (see Fig. 2). If the user clicks in a class option, a new menu page associated with the selected class will be generated (including one option for every class event or transaction).

Fig. 2. A User Access Control Page and the Rent-a-Car System View page.

In our example, when the user selects the Vehicle class, the service list offered by this class is shown, as can be seen in Fig. 3.

194

O. Pastor et al.

Every service option activation will generate a new request parameter page, as can be seen Fig. 3. This page will ask the user for the arguments needed to execute the service. The Ok control button has a code associated with it that will call to a class method that implements the effect of the service on the object state. This method will check the state transition correctness and method preconditions. If this checking process succeeds, the object change of state is carried out according to the functional model specification. We finish the method execution by verifying the integrity constraints and the trigger condition satisfaction in the new state. Object state updates in the selected persistent object system become valid through the services inherited from the object_mediator class.

Fig. 3. Vehicle Class Services Menu Page and Parameter Request Page.

5 Conclusions and Further Work Advanced web applications will depend on object technology to make them feasible, reliable and secure. To achieve this goal, well-defined methodological frameworks, which properly connect OO conceptual modeling and OO software development environments, must be introduced. The OO-Method provides such an environment. The most relevant features are the following: • an operational implementation of an automated programming paradigm where a concrete execution model is obtained from a process of conceptual model translation • a precise object-oriented model, where the use of a formal specification language as a high-level data dictionary is a basic characteristic

From Object Oriented Conceptual Modeling to Automated Programming in Java

195

All of this is done within the next-generation web development environments, making use of the Inter/Intranet architectures and using Java as a software development language. Research work is still being undertaken to improve the quality of the final software product that is generated including advanced features such as user-defined interface, schema evolution, optimized database access mechanisms or how the structure of the generated code impacts the performance of the system.

Acknowledgments. We wish to thank the anonymous referees for their valuable comments and suggestions.

References 1. Arnold K., Gosling J. The Java Programming Language. Sun MicroSystems. AddisonWesley, 1996. 2. Rumbaugh J. et al. W. Object Oriented Modeling and Design. Englewood Cliffs, Nj. Prentice-Hall. 1991. 3. Liddle S.W., Embley D.W., Woodfield S.N. Unifying Modeling and Programming Through an Active, Object-Oriented, Model-Equivalent Programming Language . In Proceedings th of 14 International Conference on Object-Oriented and Entity-Relationship Modeling (OOER’95). 13-15 Dec 1995. Gold Coast, Australia. Lecture Notes in Computer Science v. 1021; p. 55-64. 1995. 4. Jacobson I. et al. G. OO Software Engineering , a Use Case Driven Approach. Reading, Massachusetts. Addison -Wesley. 1992. 5. Booch,G. OO Analysis and Design with Applications. Addison-Wesley, 1994. 6. Booch G., Rumbaugh J., Jacobson I. UML. v1. Rational Software Co., 1997. 7. Rational Software Corporation. Rational Rose User's Manual. 1997. 8. Ptech FrameWork. Ptech Inc., Boston, MA, USA, Web Site: http://www.ptechinc.com/. 1998. 9. Platinum Technology, Inc., Paradigm Plus: Round-Trip Engineering for JAVA, White Paper. Platinum Web Site: http://www.platinum.com/. 1997. 10. Balzer R. et al. Software Technology in the 1990s: Using a New Paradigm. IEEE Computer, Nov. 1983. 11. Pastor O., Hayes F., Bear S. OASIS: An object-oriented specification language. In P. Loucopoulos, editor, Proceedings of the CAiSE'92 conference, pp. 348-363, Berlin, Springer, LNCS 593 (1992). 12. Clyde S. W., Embley D.W., Woodfield S.N. Tunable Formalism in Object-Oriented System Analysis: Meeting the Needs of Both Theoreticians and Practitioners. In Proceedings of OOPSLA’92 Conference. Vancouver, Canada; p. 452-465. 1992. 13. Pastor O., Ramos I. Oasis 2.1.1: A Class-Definition Language to Model Information rd Systems Using an Object-Oriented Approach, October 95 (3 edition).

196

O. Pastor et al.

14. Pastor O. et al. OO-METHOD: An OO Software Production Environment Combining Conventional and Formal Methods. In Antoni Olivé and Joan Antoni Pastor editors, Proceedings of CAiSE97 conference, pp. 145-158, Berlin, Springer-Verlag, LNCS 1250. June 1997. 15. Jackson R.B., Embley D.W., Woodfield S.N. Automated Support for the Development of Formal Object-Oriented Requirements Specification. In Proceedings of CAiSE-94 Conference. Utrecht, The Netherlands. Lecture Notes in Computer Science; v. 811; p.135148. 1994. 16. Argawal S., Jensen R., and Keller A. M. Architecting object applications for high performance with relational databases. In OOPSLA Workshop on Object Database Behaviour, Benchmarks, and Performance, Austin, 1995.

An Evaluation of Two Approaches to Exploiting RealWorld Knowledge by Intelligent Database Design Tools 1

2

Shahrul Azman Noah and Michael Lloyd-Williams 1

Department of Information Studies, University of Sheffield, Western Bank, Sheffield S10 2TN, UK [email protected] 2 School of Information Systems and Computing, University of Wales Institute Cardiff, Colchester Avenue, Cardiff CF3 7XR, UK [email protected]

Abstract. Recent years have seen the development of a number of expert system type tools who’s primary objective is to provide support to a human during the process of database analysis and design. However, whereas human designers are able to draw upon their experience and knowledge of the real world when performing such a task, knowledge-based database design tools are generally unable to do so. This has resulted in numerous calls for the development of tools that are capable of exploiting real-world knowledge during a design session. It has been claimed that the use of such knowledge has the potential to increase the appearance of intelligence of the tools, to improve the quality of the designs produced, and to increase processing efficiency. However to date, little if any formal evaluation of these claims has taken place. This paper presents such an evaluation of two of the approaches proposed to facilitate system-storage and exploitation of real-world knowledge; the thesaurus approach and the knowledge reconciliation approach. Results obtained have demonstrated that certain aspects of the claimed benefits associated with the use of real-world knowledge have been achieved. However, the extent to which these benefits have been attained and subsequently statistically validated varies.

1 Introduction Recent years have seen the development of a number of intelligent database design tools that employ expert system technology in order to provide support to a human designer during the process of database analysis and design [3, 12, 24]. Such tools are generally intended to act as assistants to human designers [26], being capable of providing guidance, advice, proposing alternate solutions, and helping to investigate the consequences of design decisions [11]. The effectiveness of existing tools has demonstrated the viability of representing database design expertise in a computer program, however observing such systems in T.W. Ling, S. Ram, and M.L. Lee (Eds.): ER’98, LNCS 1507, pp. 197−210, 1998.  Springer-Verlag Berlin Heidelberg 1998

198

S.A. Noah and M. Lloyd-Williams

use makes it clear that human designers contribute far more than database design expertise to the design process [25]. Human designers, even when working in an unfamiliar domain, are able make use of their knowledge of the real world in order to interact with users, make helpful suggestions and inferences, and identify potential errors and inconsistencies [20, 24]. Conversely, the majority of existing intelligent database design tools do not possess such real-world knowledge, and are therefore required to ask many questions during a design session that may be viewed as being trivial [10, 20]. A human designer for instance, would recognize terms such as “client”, “customer” and “patron” as being potentially synonymous, regardless of the application domain. Existing intelligent database design tools are unable to identify such situations. This situation has resulted in numerous calls for the representation of real-world knowledge within such tools, coupled with the ability to reason with and make use of this knowledge. A number of approaches to representing and exploiting such real-world knowledge have been proposed, including the thesaurus approach [10, 11], and the knowledge reconciliation approach [21, 25]. These approaches have been accompanied by various claims [10, 20] that the use of such knowledge has the potential to increase the appearance of intelligence of the tools, to improve the quality of the designs produced, and to increase processing efficiency. However to date, little if any formal evaluation of these claims has taken place. This paper presents an evaluation of the thesaurus and knowledge reconciliation approaches, as originally employed by the Object Design Assistant [9, 10] and the View Creation System [22, 23] respectively, the intention being to initiate the gathering of evidence to support or otherwise the claims previously stated.

2 Method of Investigation In order to conduct evaluative experiments on the use of the thesaurus and knowledge reconciliation approaches, a prototype intelligent database design tool, the Intelligent Object Analyzer (IOA), was developed. IOA provides support for the design of the structural (data) aspects of object oriented databases. The intended user is a database designer or systems analyst who is familiar with systems modeling concepts and the domain to be modeled. Knowledge of objectoriented databases or of object-oriented analysis and design techniques is not a requirement. It is not the purpose of this paper to discuss IOA in depth, however, a brief outline of the structure and method of operation is required in order to illustrate how the real-world knowledge may be represented and exploited during design processing. The current version of the IOA system runs in a PC environment, and was developed using Common LISP (Allegro CL\PC). The IOA knowledge-base contains a mixture of rules and facts. Rules correspond to knowledge of how to perform the design task (the order in which design activities take place), detecting and resolving ambiguities, redundancies and inconsistencies within an evolving design, and handling the gradual augmentation of an evolving design as a design session

An Evaluation of Two Approaches to Exploiting Real-World Knowledge

199

progresses. Facts are used to represent two views of the application domain; an initial representation (the problem domain model) as provided by the user, and the objectoriented design generated from this initial representation. During a design session, IOA follows a two-step procedure. • The first step involves creating an initial representation of the application domain (known as the problem domain model) and the subsequent refinement of this model. • The second step involves the refinement of the problem domain model by detecting and resolving any inconsistencies that may exist, and the transformation of the model into object-oriented form. The first stage of processing requires a set of declarative statements that describe the application domain to be submitted to IOA. These statements are a variation of the method of interactive schema specification described by Baldiserra et al [1], being based upon the binary model described by Bracchi et al [5]. Each statement links together two concepts (taking the form A verb-phrase B), and falls into one of three classes of construct, corresponding directly to the structural abstractions of association, generalization, and aggregation. The statements are used to construct a problem domain model representing the application domain. Once constructed, IOA attempts to confirm it’s understanding of the semantic aspects of the problem domain model; that is, whether each structure within the model represents generalization, aggregation or association. Once constructed, the problem domain model is submitted to a series of refinement procedures in order to detect and resolve any inconsistencies (such as redundancies that may be present within generalization hierarchies) that may exist. These procedures are performed both with and without the requirement of user input (sometimes referred to as external and internal validation respectively). Once such inconsistencies have been resolved, IOA makes use of the problem domain model in order to generate a conceptual model (in object-oriented form). As previously discussed, the IOA system has been developed in order to assist with a series of experiments designed to evaluate the contribution of real-world knowledge to the activities of intelligent database design tools. In order to facilitate this aim, IOA is capable of conducting design sessions both with and without making use of realworld knowledge.

2.1

Representation of Real-World Knowledge

The following text provides a brief overview of the methods of knowledge representation employed by the thesaurus and knowledge reconciliation approaches. Those interested in further details of each approach, along with the claimed benefits associated with their use, are referred to the relevant source literature (see for instance [10, 11, 21, 25]). Figs. 1 and 2 present illustrative fragments of real-world knowledge (a university domain) using the thesaurus and knowledge reconciliation approaches respectively. The knowledge presented in Figs. 1 and 2 is not claimed to be statistically repre-

200

S.A. Noah and M. Lloyd-Williams

sentative, but is seen as a reasonable representation of certain aspects of a university domain. The main purpose is to provide an illustration of the thesaurus and knowledge reconciliation approaches, not to produce the definitive structures for the domain concerned. Programme Faculty

Course 1 opt

aggr mand

Lecturer Academicstaff

mand

V

opt

opt

Department

assoc N

assoc

1

N mand

mand

Student

assoc 1

N

gen Postgraduatestudent Graduatestudent

Undergraduatestudent

Fig. 1. Fragment of knowledge (university) represented using the thesaurus approach

Faculty

Course

aggr Lecturer

Attachedto

Department

Enrolled

Allocatedto Postgraduatestudent

Student gen Undergraduatestudent

Fig. 2. Fragment of knowledge (university) represented using the knowledge reconciliation approach

It can be seen that although the thesaurus and knowledge reconciliation approaches share similar semantic constructs (in representing domains using a series of concepts linked together via abstraction mechanisms), there are a number of differences apparent. The domain concepts represented by the thesaurus approach may be referred to by any number of associated synonyms where appropriate [11]. This is not the case with the knowledge reconciliation approach. Both approaches employ abstraction mechanisms (generalization, aggregation and association) to link concepts together, however, the knowledge reconciliation approach requires association links to be explicitly named. All abstraction mechanisms represented by the thesaurus

An Evaluation of Two Approaches to Exploiting Real-World Knowledge

201

approach are categorized rather than being named, thus, allowing the links between any pair of concepts to take any name provided by the user. Integrity constraints are not represented by the knowledge reconciliation approach, as are indeed membership requirements (mandatory or optional) for links between pairs of concepts. Both forms of constraint, however, are represented by the thesaurus approach. The IOA system is capable of processing in three modes, without the use of realworld knowledge (basic mode), using real-world knowledge provided by the thesaurus approach, and using real-world knowledge provided by the knowledge reconciliation approach. The basic mode of processing has been previously described in this paper. At various stages during basic mode processing, the IOA conducts a dialogue with the user in order to confirm its understanding of the application domain or to obtain additional information. When making use of real-world knowledge provided by the thesaurus approach, the tool refers to this knowledge wherever possible, only resorting to questioning the user if the real-world knowledge cannot provide the required information. This procedure is also followed when making use of real-world knowledge provided by the knowledge reconciliation approach. However, in addition to this procedure, IOA also attempts to conduct a process of reconciliation of knowledge, where the description of the application domain submitted by the user is compared and matched with the system-held real-world knowledge in order to assist the IOA in identifying potential missing elements (concepts and relationships) within the user’s domain description.

2.2

Testing and Evaluation Strategy

In order to evaluate the use of real-world knowledge by the IOA, a number of domains were modeled using both thesaurus and knowledge reconciliation approaches. In each case, the real-world knowledge structures were developed independently of the example scenarios used during testing. This was a deliberate attempt to minimize any bias that might be introduced by taking the content of the test material into account. In normal circumstances, it would be a logical procedure to use each example application domain encountered to augment the real-world knowledge held, thus, increasing the knowledge of the system in the way a human designer would automatically update his/her knowledge when working within a new domain. However, for purposes of testing the effectiveness of the use of real-world knowledge, it was decided to develop the knowledge structures completely independently of the examples encountered. For each example domain, the evaluation process involved the execution of a set of benchmarks (test-cases) in basic mode, and by exploiting knowledge provided by the thesaurus and knowledge reconciliation approaches. Thus, for each example domain, three sets of results were obtained and compared, following an approach recommended by O'Keefe & Preece [16]. The test-cases used were generated from a set of design problems which were primarily extracted from the available literature, the advantage being that the accompanying solution could be used as a benchmark and compared with the IOA-suggested solution in order to confirm the appropriateness or

202

S.A. Noah and M. Lloyd-Williams

otherwise of the designs produced. Each of the example design problems was systematically altered by dividing them into multiple test-cases with varying degrees of complexity. Within the scope of the testing, the complexity of a design test-case is defined as the number of concepts, and the relationships between the concepts [8, 17]. For instance, the university design problem found in Rob & Rob [19] was systematically fragmented to generate a total of five test-cases, having complexity degrees of 3, 7, 10, 13 and 17 respectively, as illustrated by Fig. 3.

Runs

Dean

Employs Professor

Chairs

Teaches

School Operates Department

Has

Offers Course Contains

Student

Section Advises Example of a design problem

Test-case 1

Test-case 5

Test-case 2

Test-case 4

Test-case 3

Fig. 3. Generation of test-cases with varying degrees of complexity

The number and the quality of test-cases employed has a direct and significant impact on the reliability of the results produced. Exhaustive testing, although

An Evaluation of Two Approaches to Exploiting Real-World Knowledge

203

generally desirable, is impractical, since a large number of test-cases must be executed and evaluated even in the simplest of design problems [7]. The results presented in this paper emanate from a series of tests performed on university domain problems found in the general literature [2, 4, 6, 19]. A total of 24 test-cases were generated from these initial problems, thereby providing the observed results with statistical validity, the required number of 15 observations [18] for this form of experiment being exceeded. The main criteria of interest used during the evaluation are as follows: • Processing time. Processing time refers to the CPU time required to perform a single design action (such as resolving an inconsistency). Processing time is not influenced by human factors as it is measured from the point at which the tool commences an action until that action is complete. Processing time is however influenced by the complexity of the design input; the complexity of the systemheld domain knowledge and the reasoning associated with it; and the specification of the processor of the PC in use. • User/tool interaction. User/tool interaction refers to the number of interactions required between the tool and the user in order for the tool to confirm its understanding of some aspect of the application domain or to acquire additional information should it be required. • Suggestion of missing design elements. This criterion measures whether the elements (within the generated design) are based entirely upon user-provided information, or are included as a direct result of the system consulting with its realworld knowledge. • Completeness of the resulting design. Completeness is defined as the ability of a data model to meet all the user information requirements [13]. Within the scope of the testing performed, completeness is measured in terms of the number of missing classes and relationships associated with the design example used. In order to prevent bias during testing, processing time and the number of user/tool interactions measured did not include processing arising as a direct result of suggestions made by the tool (for instance relating to potential missing elements within the evolving design) as a result of consulting its encapsulated real-world knowledge. The assumption underpinning this decision is that increased processing time and number of user/tool interactions involved in such processing are beneficial to the design process, and should not be viewed as being detrimental to performance efficiency. The research hypothesis of this investigation is that the use of real-world knowledge by an intelligent database design tool has the capability of increasing the efficiency of the tool (by reducing the number of processing time and user/tool interactions required); increase the completeness of the resulting design output (by minimizing the number of missing elements); and increase the appearance of tool intelligence (by providing suggestions for missing information and minimizing the number of interactions required).

204

S.A. Noah and M. Lloyd-Williams

3 Analysis of Results As previously discussed, a total of 24 test-cases were generated and executed within each of the three available processing modes. Results obtained from processing the test-cases using the real-world knowledge (the thesaurus and knowledge reconciliation approaches) were compared with the results obtained when no such knowledge was in use (the basic approach). Table 1 provides a preliminary overview of the results. Table 1. Preliminary overview of results Criteria

Basic

Thesaurus

Mean CPU time per complexity (sec) Mean user/tool interactions per complexity Mean suggested elements per test Mean missing elements per test

3.86 3 0 9

3.22 2 0 9

Knowledge Reconciliation 6.94 7 4 5

Table 1 illustrates that the thesaurus approach required a lower mean of CPU time and number of user/tool interactions per complexity compared with the basic approach. In contrast, the knowledge reconciliation approach required a higher mean of CPU time and number of user/tool interactions per complexity. These results are supported by the linear regression results of the CPU time (in seconds) and the number of interaction required by the three approaches as illustrated in Fig. 4 and Fig. 5. These figures illustrate that compared to the basic approach, the thesaurus approach resulted in a reduction of approximately 6.7% of the CPU time required, and of approximately 14.3% of the number of user/tool interactions required for each increase in complexity. The knowledge reconciliation approach, however, resulted in an increase of CPU time required and of user/tool interactions required by approximately 54.1% and 21.4% respectively.

Time (sec.)

200.0

199.47 132.98

150.0 100.0

86.31

66.49 43.15 40.25

50.0 0.0 0 Basic

10 Thesaurus

129.46 120.76

80.51

Complexity

20

Knowledge Reconciliation

Fig. 4. CPU time required for each increase in complexity

30

An Evaluation of Two Approaches to Exploiting Real-World Knowledge

Interaction

120 80

67

101 84 72

56

34

40

205

48

28 24

0

0 0

Basic

10 Thesaurus

Complexity

20

30

Knowledge Reconciliation

Fig. 5. User/tool interaction required for each increase in complexity

Table 1 also illustrates the potential of the knowledge reconciliation approach to provide suggestions for the inclusion of (required) elements within the resulting designs. Such elements would not be identified or included when using the other approaches. Thus the knowledge reconciliation approach has the capacity to facilitate greater level of completeness of the designs produced. These preliminary findings provide a general overview of the results obtained. In order to validate the effectiveness of each of the approaches, a statistical hypothesis test was conducted in order to assess the significance of the differences between the results observed from the execution of test-cases both with and without the use of real-world knowledge at the 5% of significance level. Although there are several recommended statistical methods available to test such hypotheses, the paired t-test method is highly appropriate in such circumstance as those prevailing in this study [14, 15]. A discussion of the statistical analysis of the observed results follows.

3.1

The Thesaurus Approach

Based upon the paired t-test results presented in Table 2 it is apparent that there are significant differences between the thesaurus approach and the basic approach in terms of the number of user/tool interactions and the CPU time required per complexity. The observed significance level and the negative t-Value1 suggest that the null hypothesis should be rejected for both criteria. It may, therefore, be stated that

1

As the objective of the statistical analysis was to validate whether the approaches taken to representing real-world knowledge significantly reduced or increased any of the evaluation criteria, referring to the P value alone will not provide a sufficient result as it only informs as to whether there is any significant different between the observed results. In this case however, the t-Value can be used [18], where a negative t-Value implies that the observed criterion is significantly reduced by the use of real-world knowledge and a positive t-Value implies otherwise.

206

S.A. Noah and M. Lloyd-Williams

the thesaurus approach increased the overall processing efficiency by reducing the number of user/tool interactions and the CPU time required per complexity. Table 2. Paired t-test reault- the thesaurus and basic approaches Criteria Interaction per complexity CPU time per complexity Suggested elements per test Completeness per test

t-Value -3.30 -3.39 N/A N/A

df 23 23 23 23

Sig. T (P) 0.003 0.002 N/A N/A

However, Table 2 also indicates that the aspect of completeness and the suggesting elements per test were not significantly different between the two approaches (the statistical test was invalid as the thesaurus and the basic approaches do not provide suggestions for missing information, therefore, both approaches result in similar numbers of missing elements within the resulting designs). Accordingly, the use of thesaurus approach has not resulted in an improvement in the quality of the resulting design output (measured in terms of increasing the completeness of the designs produced). The significant reduction in the number of user/tool interactions required suggests an increase in the appearance of intelligence of the tool. However, the (statistical) non-significance of the suggested elements per test criterion may be viewed as jeopardizing this claim.

3.2

The Knowledge Reconciliation Approach

The capability of the knowledge reconciliation approach to increase the appearance of tool intelligence (by facilitating the suggestion of potential missing design elements) is evidenced by the results illustrated in Table 3. In this case, P < 0.05 with a positive t-Value indicates that the criterion of suggesting related elements was significantly increased with the use of the knowledge reconciliation approach. However, the reduction of the number of user/tool interactions was non-significant (although P < 0.05, the t-Value is positive indicating that the knowledge reconciliation approach significantly increased the number of interaction per complexity), resulting in the conclusion that the claim for an overall increase in the appearance of tool intelligence is unjustifiable. Table 3. Paired t-test result - the knowledge reconciliation and basic approaches Criteria Interaction per complexity CPU time per complexity Suggested elements per test Completeness per test

df 23 23 23 23

t-Value 8.11 9.26 6.09 -6.09

Sig. t (P) 0.000 0.000 0.000 0.000

An Evaluation of Two Approaches to Exploiting Real-World Knowledge

207

The claim for an increase in overall processing efficiency of the tool as a result of the use of the knowledge reconciliation approach is not supported by the results obtained. This is evidenced by the non-significant reduction in the number of user/tool interactions and CPU time required. Although P < 0.05 in both cases, the tValues are positive, indicating that required CPU time and user/tool interactions per complexity increased overall when using the knowledge reconciliation approach. As the capability of suggesting missing design elements was proven to be significantly increased, it may be argued that the completeness of the designs produced should be corresponding increased (by minimizing the number of missing elements). This aspect is illustrated in Table 3, where P < 0.05 and is accompanied by a negative t-Value. Therefore, the improvement in the completeness of the design suggests that the claim of increasing the quality of the designs produced has been met in terms of this criterion for the knowledge reconciliation approach.

4 Discussion and Conclusion Tables 4 and 5 present a summary of the conclusions reached for both thesaurus and knowledge reconciliation approaches.

Table 4. Summary of conclusions Criteria

Reduces the number of user/tool interactions Reduces the CPU time required Increases the no. of missing elements suggested Increases the completeness of designs produced

Approaches Thesaurus Knowledge Reconciliation Yes No Yes No No Yes No Yes

Table 5. Associated claimed benefits - conclusions Claimed Benefits

Increases overall tool processing efficiency Improves quality designs produced Increases overall appearance of tool intelligence

Approaches Thesaurus Knowledge Reconciliation Yes No No Yes Unjustifiable Unjustifiable

Both tables provide a conclusive evidence that certain aspects of the claimed benefits have generally been achieved by the use and exploitation of real-world knowledge represented by the thesaurus and knowledge reconciliation approaches.

208

S.A. Noah and M. Lloyd-Williams

The claimed of increasing the overall processing efficiency in intelligent database design tools by the use of real-world knowledge has been met by the thesaurus approach. The conclusion ensued as a result of the significant reduction in the number of user/tool interactions and CPU time required for each increase in complexity. These reductions occurred to a certain extent due to the additional information held by thesaurus approach as opposed to the knowledge reconciliation approach. For instance, constraint and membership requirements related information which are not represented by the knowledge reconciliation approach, yet both have the potential to impact upon the performance-related criteria. The claimed for an improve in the quality of the designs produced, on the other hand, has only been met by the knowledge reconciliation approach as a result of the significant increased of completeness in the overall designs produced. The increased in completeness was due to the fact that the approach is capable of identifying which elements are though to be missing, and of suggesting the inclusion of these elements to the user. These results suggest that the system-held real-world knowledge has the potential to guide the tool in playing an active part during the design process, and at the same time increase aspects of the appearance of intelligence of the tool [20]. The question of whether the use of real-world knowledge could increase the overall appearance of intelligence of an intelligent database design tool, therefore, remains largely unresolved, as both approaches did not produce any conclusive results. The thesaurus approach, although, significantly reduced the number of user/tool interactions required, was augmented by a corresponding non-significant increase in the number of suggestions made regarding possible missing design elements within the evolving design. The knowledge reconciliation approach on the other hand, although, significantly increased the number of suggestions made for missing design elements, was found incapable to significantly reduce the number of user/tool interactions required. Although encouraging results have been obtained from the testing and evaluation work, it is recognized that consideration must be given to a number of practical issues. The effectiveness of the tool depends greatly on the accuracy and completeness of the system-held real-world knowledge, and the results obtained from the tests may be influenced to certain extent by the variety and coverage of the generated test-cases.

5 Summary and Future Work This paper has presented the findings of an assessment of the thesaurus and knowledge reconciliation approaches to representing and exploiting real-world knowledge by an intelligent database design tool (IOA). The intention of this experiment has been to evaluate the claims made regarding the use of domain knowledge (represented as the thesaurus and knowledge reconciliation approaches) by intelligent database design tools, and not to compare the effectiveness or the efficiency between these representative approaches. The results obtained have demonstrated that certain aspects of the claimed benefits associated with the use of such real-world knowledge

An Evaluation of Two Approaches to Exploiting Real-World Knowledge

209

(increased processing efficiency, improved quality designs produced, and increased the appearance of intelligence) have been achieved. However, the extent to which these benefits have been attained and subsequently statistically validated varies. Similar experiments were conducted to the clinical/hospital and library domains, and consistent findings as presented in this paper were obtained. Although, there are a number of methodologies proposed for the testing and evaluation of expert systems [14, 15], there are only a handful of papers that describe actual experiences related to the testing and performance evaluation of operational systems. This paper has presented an approach and methodology for performing such a task. The methodology employed involved: the generation of a series of test-cases, the processing of the test-cases, and the use of statistical analysis to evaluate the observed results. Ongoing work includes extending the variety of the evaluative experiments by subjecting the tool to a range of test-cases including a series of intentionally-generated errors. Such test-cases contain a combination of different types and numbers of synthesized errors, including synonymous class(es), synonymous relationship(s), and combinations of both. Acknowledgments. The authors wish to thank the anonymous referees for their helpful and constructive comments on a previous version of this paper.

References 1. Baldiserra, C., Ceri, S., Pelagatti, G. & Bracchi, G. (1979) “Interactive specification and formal verification of user's views in database design”, In: Proceedings of the 5th International Conference on Very Large Databases, Rio de Janeiro, Brazil. 262-272. 2. Batini, C., Ceri, S. & Navathe, S. (1992) Conceptual Database Design: An Entity Relationship Approach. Redwood City, CA: Benjamin-Cummings. 3. Bouzeghoub, M. (1992). “Using expert systems in schema design”. In Loucopoulos, P. & Zicari, R. (eds.) Conceptual Modeling, Databases, and CASE: an Integrated View of Information Systems Development. New York: Wiley, 465-487. 4. Bowers, D. S. (1993) From Data to Database. London: Chapman Hall. 5. Bracchi, G., Paolini, P. & Pelagatti, G. (1976) “Binary logical associations in data modeling”. In: Nijsen, G. M. (ed.) Modeling in Data Base Management Systems. Amsterdam: North-Holland, 125-148. 6. Elmasri, R. & Navathe, S. B. (1989) Fundamentals of Database Systems. Redwood City, CA: Benjamin Cummings. 7. Gonzalez, A. J., Gupta, U. G. & Chianese, R. B. (1996). “Performance evaluation of a large diagnostic expert system using a heuristic test case generator”. Engineering Application of Artificial Intelligence, 9(3), 275-284. 8. Kesh, S. (1995) “Evaluating the quality of entity relationship models”. Information and Software Technology, 37(12), 681-689. 9. Lloyd-Williams, M. (1993). “Expert system support for object-oriented database design”. International Journal of Applied Expert Systems, 1(3), 197-212. 10. Lloyd-Williams, M. (1994). “Knowledge-based CASE tools: improving performance using domain specific knowledge”. Software Engineering Journal, 9(4), 167-173.

210

S.A. Noah and M. Lloyd-Williams

11. Lloyd-Williams, M. (1997). “Exploiting domain knowledge during the automated design of object-oriented databases”, In: Embley, D. W. & Goldstein, R. C. (eds.), Proceedings of the 16th International Conference on Conceptual Modeling, Berlin: Spinger-Verlag, 16-29. 12. Lloyd-Williams, M. & Beynon-Davies, P. (1992). “Expert system for database design: a comparative review”. Artificial Intelligence Review, 6, 263-283. 13. Moody, D. L. & Shanks, G. G. (1994). “What makes a good data model? Evaluating the quality of entity-relationship models”, In: Loucopoulos, P. (eds.), Proceedings of the 13th International Conference on the Entity-Relationship Approach, Berlin: Springer-Verlag, 94-101. 14. O'Keefe, R. M., Balci, O. & Smith, E. P. (1987). “Validating expert system performance”. IEEE Expert, Winter, 81-90. 15. O'Keefe, R. M. & O'Leary, D. E. (1993). “Expert system verification and validation: a survey and tutorial”. Artificial Intelligence Review, 7(1), 3-42. 16. O'Keefe, R. M. & Preece, A. D. (1996) “The development, validation and implementation of knowledge-based systems”. European Journal of Operational Research, 92(3), 458-473. 17. Pippenger, N. (1978) “Complexity theory”. Scientific American, 238(6), 90-102. 18. Rees, D. G. (1995) Essential Statistics (3rd Edition). London: Chapman & Hall. 19. Rob, P. & Rob, C. C. (1993) Database Systems: Design, Implementation and Management. Belmont, CA: Wadsworth Publishing. 20. Storey, V. C. (1992). “Real world knowledge for databases”. Journal of Database Administration, 3(1), 1-19. 21. Storey, V. C., Chiang, R. H. L., Dey, D., Goldstein, R. C., Sundararajan, A. & Sundaresan, S. (1994) “Knowledge reconciliation for common sense reasoning”. In: De, P. & Woo, C. (eds.) Proceeding of the 4th Annual Workshop on Information Technologies and Systems. Vancouver: Univ. British Columbia, 87-96. 22. Storey, V. C. & Goldstein, R. C. (1990a). “Design and development of an expert database design system”. International Journal of Expert Systems Research and Applications, 3(1), 31-63. 23. Storey, V. C. & Goldstein, R. C. (1990b). “An expert view creation system for database design”. Expert Systems Review, 2(3), 19-45. 24. Storey, V. C. & Goldstein, R. C. (1993). “Knowledge-based approach to database design”. Management Information Systems Quarterly, 17(1), 25-46. 25. Storey, V. C., Goldstein, R. C., Chiang, R. H. L. & Dey, D. (1993). “A common-sense reasoning facility based on the entity-relationship model”, In: Elmasri, R. A., Kouramajian, V. & Thalheim, B. (eds.) Proceedings of the 12th International Conference on the Entity Relationship Approach, Berlin: Springer-Verlag, 218-229. 26. Vessey, I. & Sravanapudi, A. P. (1995) “CASE tools as collaborative support technologies”. Communications of the ACM, 37(1), 83-102.

Metrics for Evaluating the Quality of Entity Relationship Models Daniel L. Moody Simsion Bowles and Associates, 1 Collins St., Melbourne, Australia 3000. email: [email protected] Abstract. This paper defines a comprehensive set of metrics for evaluating the quality of Entity Relationship models. This is an extension of previous research which developed a conceptual framework and identified stakeholders and quality factors for evaluating data models. However quality factors are not enough to ensure quality in practice, because different people will have different interpretations of the same concept. The objective of this paper is to refine these quality factors into quantitative measures to reduce subjectivity and bias in the evaluation process. A total of twenty five candidate metrics are proposed in this paper, each of which measures one of the quality factors previously defined. The metrics may be used to evaluate the quality of data models, choose between alternatives and identify areas for improvement.

1

Introduction

The choice of an appropriate representation of data is one of the most crucial tasks in the entire systems development process. Although the data modelling phase represents only a small proportion of the total systems development effort, its impact on the final result is probably greater than any other phase (Simsion, 1994). The data model is a major determinant of system development costs (ASMA, 1996), system flexibility (Gartner, 1992), integration with other systems (Moody and Simsion, 1995) and the ability of the system to meet user requirements (Batini et al., 1992). For this reason, effort expended on improving the quality of data models is likely to pay off many times over in later phases. Previous Research Evaluating the quality of data models is a discipline which is only just beginning to emerge. Quantitative measurement of quality is almost non-existent. A number of frameworks for evaluating the quality of data models have now been proposed in the literature (Roman, 1985; Mayer, 1989; von Halle, 1991; Batini et al., 1992; Levitin and Redman, 1994; Simsion, 1994; Moody and Shanks, 1994; Krogstie, Lindland and Sindre, 1995; Lindland, Sindre and Solveberg, 1994; Kesh, 1995; Moody and Shanks, 1998). Most of these frameworks suggest criteria that may be used to evaluate the quality of data models. However quality criteria are not enough on their own to ensure quality in practice, because different people will generally have different interpretations of what they mean. According to the Total Quality Management (TQM) literature, measurable criteria for assessing quality are necessary to avoid “arguments of style” (Zultner, 1992). The objective should be to replace intuitive notions of design “quality” with T.W. Ling, S. Ram, and M.L. Lee (Eds.): ER’98, LNCS 1507, pp. 211−225, 1998.  Springer-Verlag Berlin Heidelberg 1998

212

D.L. Moody

formal, quantitative measures to reduce subjectivity and bias in the evaluation process. However developing reliable and objective measures of quality in software development is a difficult task. As Van Vliet (1993) says: “The various factors that relate to software quality are hard to define. It is even harder to measure them quantitatively. There are very few quality factors or criteria for which sufficiently sound numeric measures exist.”

Of the frameworks that have been proposed, only two address the issue of quality measurement. Moody and Shanks (1994) suggest a number of evaluation methods, which in some cases are measures (eg. data model complexity) and in other cases are processes for carrying out the evaluation (eg. user reviews). Kesh (1995) defines a number of metrics for evaluating data models but these are theoretically based, and of limited use in practice. Most of the other frameworks rely on experts giving overall subjective ratings of the quality of a data model with respect to the criteria proposed.

2

A Framework for Evaluating and Improving the Quality of Data Models

This paper uses the framework for data model evaluation and improvement proposed by Moody and Shanks (1998) as a basis for developing quality metrics. An earlier version of the framework was published in Moody and Shanks (1994). This framework was developed in practice, and has now been applied in a wide range of organisations around the world (Moody, Shanks and Darke, 1998). The framework is summarised by the Entity Relationship model shown below. Stakeholder

interacts with

concerned with

Weighting defines importance of

Quality Factor

Quality Metric evaluated by

used to improve

Improvement Strategy

Fig. 1. Data Model Quality Evaluation Framework

• Quality factors are the properties of a data model that contribute to its quality. These answer the question: “What makes a good data model?”. A particular quality factor may have positive or negative interactions with other quality factors— these represent the trade-offs implicit in the modelling process.

Metrics for Evaluating the Quality of Entity Relationship Models

213

• Stakeholders are people who are involved in building or using the data model, and therefore have an interest in its quality. Different stakeholders will generally be interested in different quality factors. • Quality metrics are define ways of evaluating particular quality factors. There may be multiple measures for each quality factor. • Weightings define the relative importance of different quality factors in a problem situation. These are used to make trade-offs between different quality factors. • Improvement strategies are techniques for improving the quality of data models with respect to one or more quality factors. A previous paper (Moody and Shanks, 1994) defined the stakeholders and quality factors relevant to data modelling, as well as methods for evaluating the quality of data models. This paper defines metrics for each quality factor. Stakeholders The key stakeholders in the data modelling process are: • The business user, whose requirements are defined by the data model • The analyst, who is responsible for developing the data model • The data administrator, who is responsible for ensuring that the data model is consistent with the rest of the organisation’s data • The application developer, who is responsible for implementing the data model (translating it into a physical database schema) Quality Factors The proposed quality factors and the primary stakeholders involved in evaluating them are shown in Fig. 2 below. Business User

Completeness

Business User

Integrity

Business User

Flexibility

Business User

Understandability

DATA MODEL QUALITY

Correctness

Data Analyst

Simplicity

Integration

Data Analyst

Data Administrator

Implementability

Application Developer

Fig. 2. Data Model Quality Factors

These quality factors may be used as criteria for evaluating the quality of individual data models or comparing alternative representations of requirements. Together they incorporate the needs of all stakeholders, and represent a complete picture of data model quality. The following sections define quality measures for each quality factor.

214

3

D.L. Moody

Completeness

Completeness relates to whether the data model contains all information required to meet user requirements. This corresponds to one half of the 100% principle—that the conceptual schema should define all static aspects of the Universe of Discourse (ISO, 1987). Completeness is the most important requirement of all because if it is not satisfied, none of the other quality factors matter. If the requirements as expressed in the data model are inaccurate or incomplete, the system which results will not satisfy users, no matter how well designed or implemented it is. Evaluating Completeness In principle, completeness can be checked by checking that each user requirement is represented somewhere in the model, and that each element of the model corresponds to a user requirement (Batini et al, 1992). However the practical difficulty with this is that there is no external source of user requirements—they exist only in people’s minds. Completeness can therefore only be evaluated with close participation of business users. The result of completeness reviews will be a list of elements (entities, relationships, attributes, business rules) that do not match user requirements. Fig. 3 illustrates the different types of completeness mismatches:

Data Model



 



User Requirements

Fig. 3. Types of Completeness Errors

• Area 1 represents elements included in the data model that do not correspond to any user requirement or are out of scope of the system—these represent unnecessary elements. We call these Type 1 errors. • Area 2 represents user requirements which are not represented anywhere in the data model—these represent gaps or omissions in the model. We call these Type 2 errors. • Area 3 represents items included in the data model that correspond to user requirements but have been inaccurately defined. We call these Type 3 errors. • Area 4 represents elements in the data model that accurately correspond to user requirements. The objective of completeness reviews is to eliminate all items of type 1, 2 and 3.

Metrics for Evaluating the Quality of Entity Relationship Models

215

Proposed Completeness Metrics The proposed quality measures for correctness all take the form of mismatches with respect to user requirements. The purpose of the review process will be to eliminate all such defects, so that the model exactly matches user requirements:

✎ ✎

✎

✎

4

Metric 1. Number of items in the data model that do not correspond to user requirements (Type 1 errors). Inclusion of such items will lead to unnecessary development effort and added cost. Metric 2. Number of user requirements which are not represented in the data model (Type 2 errors). These represent missing requirements, and will need to be added later in the development lifecycle, leading to increased costs, or if they go undetected, will result in users not being satisfied with the system Metric 3. Number of items in the data model that correspond to user requirements but are inaccurately defined (Type 3 errors). Such items will need to be changed later on the development lifecycle, leading to rework and added cost, or if they go undetected, will result in users being unsatisfied with the system. Metric 4. Number of inconsistencies with process model. A critical task in verifying the completeness of the data model is to map it against the business processes which the system needs to support. This ensures that all functional requirements can be met by the model. The result of this analysis can be presented in the form of a CRUD (Create, Read, Update, Delete) matrix. Analysis of the CRUD matrix can be used to identify gaps in the data model as well as to "prune away" unnecessary data from the model (Martin, 1989).

Integrity

Integrity is defined as the extent to which the business rules (or integrity constraints) which apply to the data are enforced by the data model1. Integrity corresponds to the other half of the 100% principle—that the conceptual schema should define all dynamic aspects of the Universe of Discourse (ISO, 1987). Business rules define what can and can’t happen to the data. Business rules are necessary to maintain the consistency and integrity of data stored, as well as to enforce business policies (Date, 1989; Loffman and Rush, 1991). All rules which apply to the data should be documented in the data model to ensure they are enforced consistently across all application programs (ISO, 1987). Evaluating Integrity Like completeness, integrity can only really be evaluated with close participation of business users. The rules represented by the data model may be verified by translating them into natural language sentences. Users can then verify whether each rule is true 1

In the original version of the evaluation framework (Moody and Shanks, 1994) integrity was included as part of completeness, but has since been separated out as a quality factor in its own right.

216

D.L. Moody

or false. This is useful as a check on the integrity of the data model because business users often have difficulty understanding the constraints defined in data models, particularly cardinality rules on relationships (Batini et al., 1992). Many CASE tools can automatically translate relationship cardinality rules into natural language sentences, provided relationships have been named correctly. Proposed Integrity Metrics The proposed quality measures for integrity take the form of mismatches between the data model and business policies. The purpose of the review process will be to eliminate all such defects:

✎ ✎

5

Metric 5. Number of business rules which are not enforced by the data model. Non-enforcement of these rules will result in data integrity problems and/or operational errors. Metric 6. Number of integrity constraints included in the data model that do not accurately correspond to business policies (i.e. which are false). Incorrect integrity constraints may be further classified as: • too weak: the rule allows invalid data to be stored • too strong: the rule does not allow valid data to be stored and will lead to constraints on business operations and the need for user “workarounds”.

Flexibility

Flexibility is defined as the ease with which the data model can cope with business change. The objective is for additions and/or changes in requirements to be handled with the minimum possible change to the data model. The data model is a key contributor to the flexibility of the system as a whole (Gartner, 1992; Simsion, 1994). Lack of flexibility in the data model can lead to: • Maintenance costs: of all types of maintenance changes, changes to data structures and formats are the most expensive. This is because each such change has a “ripple effect” on all the programs that use it. • Reduced organisational responsiveness: inflexible systems inhibit changes to business practices, organisational growth and the ability to respond quickly to business or regulatory change. Often the major constraint on introducing business change—for example, bringing a new product to market—is the need to modify the computer systems that support it (Simsion, 1988). Evaluating Flexibility Flexibility is a particularly difficult quality factor to assess because of the inherent difficulty of predicting what might happen in the future. Evaluation of flexibility requires identifying what requirements might change in the future, their probability of occurrence and their impact on the data model. However no matter how much time spent thinking about what might happen in the future, such changes remain hard to anticipate. In this respect, evaluating flexibility has much in common with weather forecasting—there is a limit to how far and how accurately the future can be predicted.

Metrics for Evaluating the Quality of Entity Relationship Models

217

Proposed Flexibility Metrics The proposed measures for evaluating flexibility focus on areas where the model is potentially unstable—where changes to the model might be required in the future as a result of changes in the business environment. The purpose of the review process will be to look at ways of minimising impact of change on the model, taking into account the probability of change, strategic impact and likely cost of change. A particular focus of flexibility reviews is identifying business rules which might change.

✎ ✎

✎ 6

Metric 7. Number of elements in the model which are subject to change in the future. This includes changes in definitions or business rules as a result of business or regulatory change. Metric 8. Estimated cost of changes. For each possible change, the probability of change occurring and the estimated cost of making the change postimplementation should be used to calculate the probability-adjusted cost of the change. Metric 9. Strategic importance of changes. For each possible change, the strategic impact of the change should be defined, expressed as a rating by business users of the need to respond quickly to the change.

Understandability

Understandability is defined as the ease with which the data model can be understood. Business users must be able to understand the model in order to verify that it meets their requirements. Similarly, application developers need to be able to understand the model to implement it correctly. Understandability is also important in terms of the useability of the system. If users have trouble understanding the concepts in the data model, they are also likely to have difficulty understanding the system which is produced as a result. The communication properties of the data model are critical to the success of the modelling effort. However empirical studies show that in practice data models are poorly understood by users, and in most cases are not developed with direct user involvement (Hitchman, 1995). While data modelling has proven very effective as a technique for database design, it has been far less effective for communication with users (Moody, 1996a). Evaluating Understandability Understandability can only be evaluated with close participation with the users of the model—business users and application developers. In principle, understandability can be checked by checking that each element of the model is understandable. However the practical difficulty with this is that users may think they understand the model while not understanding its full implications and possible limitations from a business perspective.

218

D.L. Moody

Proposed Understandability Metrics The proposed measures for understandability take the form of ratings by different stakeholders and tests of understanding. The purpose of the review process will be to maximise these ratings.

✎

✎

✎

7

Metric 10. User rating of understandability of model: user ratings of understandability will be largely based on the concepts, names and definitions used, as well as how the model is presented. A danger with this metric is that it is common for users to grasp familiar business terms without appreciating the meaning represented in the model. As a result, they may think they understand the model while not really understanding its full implications for the business. Metric 11. Ability of users to interpret the model correctly. This can be measured by getting users to instantiate the model using actual business examples (scenarios). Their level of understanding can then be measured by the number of errors in populating the model. This is a better operational test of understanding than the previous metric—it measures whether the model is actually understood rather than whether it is understandable (Lindland et al, 1994). This is much more important from the point of view of verifying the accuracy of the model. Metric 12. Application developer rating of understandability. It is essential that the application developer understands the model fully so that they can implement it correctly. Getting the application developer to review the model for understandability is particularly useful for identifying where the model is unclear or ambiguous because they will be less familiar with the model and the business domain than either the analyst or business user. Many things that seem obvious to those involved in developing the model may not be to someone seeing it for the first time.

Correctness

Correctness refers to whether the model conforms to the rules of the data modelling technique being used. Rules of correctness include diagramming conventions, naming rules, definition rules and rules of composition (for example, each entity must have a primary key). Correctness is concerned only with whether the data modelling technique has been used correctly (syntactic or grammatical correctness). It answers the question: “Is this a valid model?”. Another important aspect of correctness, and a major focus of data modelling in practice, is to ensure that the model contains no redundancy—that each fact is represented in only one place (Simsion, 1994). Evaluating Correctness Correctness is the easiest of all the quality factors to evaluate, because there is very little subjectivity involved, and no degrees of quality—the model either obeys the rules or it does not. Also, the model can be evaluated in isolation, without reference to user requirements. The result of correctness reviews will be a list of defects, defining where the data model does not conform to the rules of the data modelling technique. Many of these checks can be carried out automatically using CASE tools.

Metrics for Evaluating the Quality of Entity Relationship Models

219

Proposed Correctness Metrics The proposed quality measures for correctness all take the form of defects with respect to data modelling standards (syntactic rules). We break down correctness errors into different types or defect classes to assist in identifying patterns of errors or problem areas which may be addressed by training or other process measures. The purpose of the review process will be to eliminate all such defects:

✎

✎

✎

Metric 13. Number of violations to data modelling conventions. These can be further broken down into the following defect classes: • Diagramming standards violations (eg. relationships not named) • Naming standards violations (eg. use of plural nouns as entity names) • Invalid primary keys (non unique, incomplete or non-singular) • Invalid use of constructs (eg. entities without attributes, overlapping subtypes, many to many relationships) • Incomplete definition of constructs (e.g. data type and format not defined for an attribute; missing or inadequate entity definition) Metric 14. Number of normal form violations. Second and higher normal form violations identify redundancy among attributes within an entity (intra-entity redundancy). Normal form violations may be further classified into: • First normal form (1NF) violations • Second normal form (2NF) violations • Third normal form (3NF) violations • Higher normal form (4NF+) violations Metric 15. Number of instances of redundancy between entities—for example, where two entity definitions overlap or where redundant relationships are included. This is called inter-entity redundancy, to distinguish this from redundancy within an entity (intra-entity redundancy—Metric 14) and redundancy of data with other systems (external redundancy—Metric 21). Fig. 4 summarises the types of redundancy and corresponding metrics. Redundancy Internal Redundancy

External Redundancy

Intra-Entity Redundancy

Metric14

Inter-Entity Redundancy

Metric 15

Metric 21

Fig. 4. Classification of Redundancy

220

8

D.L. Moody

Simplicity

Simplicity means that the data model contains the minimum possible constructs. Simpler models are more flexible (Meyer, 1988), easier to implement (Simsion, 1991), and easier to understand (Moody, 1997). The choice of simplicity as a quality factor is based on the principle of Ockham’s Razor, which has become one of the cornerstones of the scientific method. This says that if there are two theories which explain the same observations, the one with the fewer constructs should be preferred (Dubin, 1979). The extension of this to data modelling is if there are two data models which meet the same requirements, the simpler one should be preferred. Evaluating Simplicity Simplicity is the easiest of all quality factors to evaluate, because it only requires a simple count of data model elements. This can be done automatically by CASE tools, or carried out manually. It takes no skill (apart from the ability to count!) and is totally objective. Simplicity metrics are particularly useful in comparing alternative data models—all other things being equal, the simpler one should be preferred. Proposed Simplicity Metrics Metrics for evaluating simplicity take the form of complexity measures. The purpose of the review process will be to minimise the complexity of the model while still satisfying user requirements. The following metrics represent alternative ways of measuring complexity of a data model—Metric 17 is recommended as the most useful of the measures proposed:

✎

✎

✎

Metric 16. Number of entities (E). This is the simplest measure of the complexity of a data model. The justification for this is that the number of entities in the logical data model is a surrogate measure for system complexity and development effort. Symons (1988, 1991) found that in sizing of business (“data rich”) applications, the major determinant of software size (and development effort) was the number of entities. Metric 17. Number of entities and relationships (E+R). This is a finer resolution complexity measure which is calculated as the number of entities (E) plus the number of relationships (R) in the data model. This derives from complexity theory, which asserts that the complexity of any system is defined by the number of components in the system and the number of relationships between them (Klir, 1985; Pippenger, 1978). Subtypes should not be included in the calculation of the number of entities because these represent subcategories within a single construct and generally do not translate into separate database tables. In addition, many to many relationships should be counted as three constructs, since when they are resolved, they will form one entity and two relationships (Shanks, 1997). This helps to standardise differences between different modelling styles. Metric 18. Number of constructs (E+R+A). This is the finest resolution complexity measure, and includes the number of attributes in the calculation of data

Metrics for Evaluating the Quality of Entity Relationship Models

221

model complexity. Such a metric could be calculated as a weighted sum of the form aNE + bNR + cNA where NE is the number of entities, NR is the number of relationships and NA is the number of attributes. In practice however, such a measure does not provide any better information than Metric 17.

9

Integration

Integration is defined as the level of consistency of the data model with the rest of the organisation’s data. In practice, application systems are often built in relative isolation of each other, leading to the same data being implemented over and over again in different ways. This leads to duplication of data, interface problems and difficulties consolidating data from different systems for management reporting (Moody and Simsion, 1995). The primary mechanism for achieving corporate-wide data integration is a corporate data model (Goodhue et al., 1992). The corporate data model provides a common set of data definitions which is used to co-ordinate the activities of application development teams so that separately developed systems work together. The corporate data model allows opportunities for sharing of data to be identified, and ensures that different systems use consistent data naming and formats (Martin, 1989). Evaluating Integration Integration is assessed by comparing the application data model with the corporate data model (Batini, Lenzerini and Navathe, 1986). The result of this will be a list of conflicts between the project data model and the corporate data model. This is usually the responsibility of the data administrator (also called information architect, data architect, data manager), who has responsibility for corporate-wide sharing and integration of data. It is their role to maintain the corporate data model and review application data models for conformance to the corporate model. Proposed Integration Metrics Most of the proposed measures for integration are in the form of conflicts with the corporate data model or with existing systems. The purpose of the review process will be to resolve these inconsistencies.

✎

✎

Metric 19. Number of data conflicts with the Corporate Data Model. These can be further classified into: • Entity conflicts: number of entities whose definitions are inconsistent with the definition entities in the corporate data model. • Data element conflicts: number of attributes with different definitions or domains to corresponding attributes defined in the corporate data model. • Naming conflicts: number of entities or attributes with the same business meaning but different names to concepts in the corporate data model (synonyms). Also entities or attributes with the same name but different meaning to concepts in the corporate data model (homonyms). Metric 20. Number of data conflicts with existing systems. These can be further classified into:

222

D.L. Moody

•

•

•

✎

✎

Number of data elements whose definitions conflict with those in existing systems e.g. different data formats or definitions. Inconsistent data item definitions will lead to interface problems, the need for data translation and difficulties comparing and consolidating data across systems. Number of key conflicts with existing systems or other projects. Key conflicts occur when different identifiers are assigned to the same object (eg. a particular customer) by different systems. This leads to fragmentation of data across systems and the inability to link or consolidate data about a particular entity across systems. Number of naming conflicts with other systems (synonyms and/or homonyms). These are less of a problem in practice than other data conflicts, but are a frequent source of confusion in system maintenance and interpretation of data.

Metric 21. Number of data elements which duplicate data elements stored in existing systems or other projects. This is called external redundancy to distinguish it from redundancy within the model itself (Metrics 14 and 15). This form of redundancy is a serious problem in most organisations—empirical studies show that there are an average of ten physical copies of each primary data item in medium to large organisations (O’Brien and O’Brien, 1994). Metric 22. Rating by representatives of other business areas as to whether the data has been defined in a way which meets corporate needs rather than the requirements of the application being developed. Because all data is potentially shareable, all views of the data should be considered when the data is first defined (Thompson, 1993). In practice, this can be done by a high level committee which reviews all application development projects for data sharing, consistency and integration (Moody, 1996b).

10 Implementability Implementability is defined as the ease with which the data model can be implemented within the time, budget and technology constraints of the project. While it is important that a data model does not contain any assumptions about the implementation (ISO, 1987), it is also important that it does not ignore all practical considerations. After all, there is little point developing a model which cannot be implemented or that the user cannot afford. Evaluating Implementability The implementability of the data model is assessed by the application developer, who is responsible for implementing the data model once it has been completed. The application developer provides an important “reality check” on what is technically possible and/or economically feasible. The process of reviewing the model also allows the application developer to gain familiarity with the model prior to the design stage to ensure a smooth transition.

Metrics for Evaluating the Quality of Entity Relationship Models

223

Proposed Implementability Metrics Proposed measures of implementability all take the form of ratings by the application developer. The purpose of the review process will be to minimise these ratings:

✎ ✎ ✎

Metric 23. Technical risk rating: estimate of the probability that the system can meet performance requirements based on the proposed data model and the technological platform (particularly the target DBMS) being used Metric 24. Schedule risk rating: estimate of the probability that the system can be implemented on time, based on the proposed data model Metric 25. Development cost estimate: this is an estimate of the development cost of the system, based on the data model. Such an estimate will necessarily be approximate but will be useful as a guide for making cost/quality trade-offs between different models proposed. If the quote is too high (exceeds available budget), the model may need to be simplified, reduced in scope or the budget increased.

11 Conclusion This paper has proposed a comprehensive set of metrics for evaluating the quality of data models based on the set of quality factors proposed by Moody and Shanks (1998). A total of twenty five candidate metrics are identified, with eighteen secondary metrics which may be used to classify defects in more detail. It is not expected that all of these metrics would be used in evaluating the quality of a particular data model. Our aim in this paper has been to be as complete as possible—to suggest as many metrics as possible as a starting point for analysis. Selection of the most appropriate metrics should be made based on their perceived usefulness and ease of calculation. Further Research The next step in this research is to validate and refine these metrics in practice. This will help to identify which metrics are most useful. It is proposed to use action research as the research paradigm for doing this. Action research (Checkland and Scholes, 1990) is a research method in which practitioners and researchers work together to test and refine principles, tools, techniques and methodologies that have been developed to address real world problems. It provides the ability to test out new methods in practice for mutual benefit of researchers and practitioners. Moody, Shanks and Darke, 1998 (in this conference) describe how action research has already been used to validate the framework and the quality factors proposed. Further research is also required to develop strategies for improving the quality of data models once a quality problem has been identified. Definition of improvement strategies would complete the specification of the framework.

224

D.L. Moody

References 1. 2. 3.

4. 5. 6. 7. 8.

9. 10.

11. 12. 13.

14. 15. 16. 17. 18. 19. 20.

AUSTRALIAN SOFTWARE METRICS ASSOCIATION (ASMA) (1996): ASMA Project Database, Release 7, November, P.O. Box 1287, Box Hill, Victoria, Australia, 3128. BATINI, C., CERI, S. AND NAVATHE, S.B. (1992): Conceptual Database Design: An Entity Relationship Approach, Benjamin Cummings, Redwood City, California. BATINI, C., LENZERINI, M. AND NAVATHE, S. (1986): A Comparative Analysis of Methodologies for Database Schema Integration, ACM Computing Surveys, 18(4), December: 323-364. CHECKLAND, P.B. and SCHOLES, J., Soft Systems Methodology in Action, Wiley, Chichester, 1990. DATE, C.J. (1989):, Introduction to Database Systems (4th Edition), Addison Wesley. DUBIN, R. (1978): Theory Building, The Free Press, New York. GARTNER RESEARCH GROUP (1992): "Sometimes You Gotta Break the Rules", Gartner Group Strategic Management Series Key Issues, November 23. GOODHUE, D.L., KIRSCH, L.J., AND WYBO, M.D. (1992): The Impact of Data Integration on the Costs and Benefits of Information Systems, MIS Quarterly, 16(3), September: 293-311. HITCHMAN, S. (1995) Practitioner Perceptions On The Use Of Some Semantic Concepts In The Entity Relationship Model, European Journal of Information Systems, 4, 31-40. INTERNATIONAL STANDARDS ORGANISATION (ISO) (1987), Information Processing Systems - Concepts and Terminology for the Conceptual Schema and the Information Base, ISO Technical Report 9007. KESH, S. (1995): Evaluating the Quality of Entity Relationship Models, Information and Software Technology, 37 (12). KLIR, G.J. (1985): Architecture of Systems Problem Solving, Plenum Press, New York. KROGSTIE, J., LINDLAND, O.I. and SINDRE, G. (1995): Towards a Deeper Understanding of Quality in Requirements Engineering, Proceedings of the 7th International Conference on Advanced Information Systems Engineering (CAISE), Jyvaskyla, Finland, June. LEVITIN, A. and REDMAN, T. (1994): Quality Dimensions of a Conceptual View, Information Processing and Management, Volume 31. LINDLAND, O.I, SINDRE, G. and SOLVEBERG, A. (1994): Understanding Quality in Conceptual Modelling, IEEE Software, March. LOFFMAN, R.S. AND RUSH, R.M (1991): Improving Data Quality, Database Programming and Design, 4(4), April, 17-19. MARTIN, J. (1989): Strategic Data Planning Methodologies, Prentice Hall, New Jersey. MAYER, R.E. (1989): Models for Understanding, Review of Educational Research, Spring. MEYER, B. (1988): Object Oriented Software Construction, Prentice Hall, New York. MOODY, D.L. AND SHANKS, G.G. (1994): What Makes A Good Data Model? Evaluating the Quality of Entity Relationship Models, in P. LOUCOPOLIS (ed.) Proceedings of the Thirteenth International Conference on the Entity Relationship Approach, Manchester, December 14-17, 94-111.

Metrics for Evaluating the Quality of Entity Relationship Models

225

21. MOODY, D.L. AND SIMSION, G.C. (1995): Justifying Investment in Information Resource Management, Australian Journal of Information Systems, 3(1), September: 25-37. 22. MOODY, D.L. (1996a) “Graphical Entity Relationship Models: Towards A More User Understandable Representation of Data”, in B. THALHEIM (ed.) Proceedings of the Fourteenth International Conference on the Entity Relationship Approach, Cottbus, Germany, October 7-9, 227-244. 23. MOODY, D.L. (1996b) Critical Success Factors for Information Resource Management, Proc. 7th Australasian Conference on Information Systems, Hobart, Australia, December. 24. MOODY, D.L. (1997): “A Multi-Level Architecture for Representing Enterprise Data Models”, Proceedings of the Sixteenth International Conference on the Entity Relationship Approach, Los Angeles, November 1-3. 25. MOODY, D.L. and SHANKS, G.G. (1998): What Makes A Good Data Model? A Framework for Evaluating and Improving the Quality of Entity Relationship Models, Australian Computer Journal (forthcoming). 26. MOODY, D.L., SHANKS, G.G. and DARKE, P. (1998): Improving the Quality of Entity Relationship Models—Experience in Research and Practice, in Proceedings of the Seventeenth International Conference on Conceptual Modelling (ER ’98), Singapore, November 16—19. 27. O’BRIEN, C. AND O’BRIEN, S. (1994), “Mining Your Legacy Systems: A Data-Based Approach”, Asia Pacific DB2 User Group Conference, Melbourne, Australia, November 21-23. 28. PIPPENGER, N. (1978): Complexity Theory, Scientific American, 238(6): 1-15. 29. ROMAN, G. (1985): A Taxonomy of Current Issues in Requirements Engineering, IEEE Computer, April. 30. SHANKS, G.G. (1997) Conceptual Data Modelling: An Empirical Study of Expert and Novice Data Modellers, Australian Journal of Information Systems, 4:2, 63-73 31. SIMSION, G.C. (1988): Data Planning in a Volatile Business Environment, Australian Computer Society Conference on Strategic Planning for Information Technology, Ballarat, March: 88-92. 32. SIMSION, G.C. (1991): Creative Data Modelling, Proceedings of the Tenth International Entity Relationship Conference, San Francisco, 112-123. 33. SIMSION, G.C. (1994): Data Modelling Essentials, Van Nostrand Reinhold, New York. 34. SYMONS, C.R. (1988): Function Point Analysis: Difficulties and Improvements, IEEE Transactions on Software Engineering, 14(1), January. 35. SYMONS, C.R. (1991): Software Sizing and Estimating: MkII Function Point Analysis, J. Wiley and Sons. 36. THOMPSON, C. (1993): "Living with an Enterprise Model", Database Programming and Design, 6(12), March: 32-38. 37. VAN VLIET, J.C. (1993): Software Engineering: Principles and Practice, John Wiley and Sons, Chichester, England. 38. VON HALLE, B. (1991): Data: Asset or Liability?, Database Programming and Design, 4(7), July: 13-15. 39. ZULTNER, R.E. (1992): “The Deming Way: Total Quality Management for Software”, Proceedings of Total Quality Management for Software Conference, April, Washington, DC, April, 134-145.

A Transformational Approach to Correct Schema Refinements Donatella Castelli and Serena Pisani Istituto di Elaborazione dell’Informazione Consiglio Nazionale delle Ricerche Via S. Maria, 46 Pisa, Italy e-mail: {castelli,serena}@iei.pi.cnr.it Abstract. This paper extends a database schema transformation language, called Schema Refinement Language, with a composition operator and a rule for deriving the conditions under which a composed transformation is guaranteed to produce a correct schema refinement. The framework that results from this extension can be exploited for improving the reliability of the database schema design also when other design frameworks are used.

1

Introduction

The reliability of a schema design is usually obtained by reducing the set of operators that can be used to carry out the design from the conceptual to the logical schema to a fixed set[1], [3], [4], [5], [6], [7], [9], [10], [11]. Each operator is provided with the conditions under which it is guaranteed to produce a correct schema refinement. Usually, these conditions can be proved by simply checking the schema structure. A drawback of this approach is that the set of given operators are often insufficient to cover the specific needs that occur in everyday practice. For some applications, the set of chosen transformations may be too low-level, whereas, for others, this set may be too specialised. This paper proposes a novel approach for supporting a correct schema refinement which overcomes the above drawback. In this case, the transformational operators can be built dynamically following the designers’ needs. The approach proposed relies upon a design language called Schema Refinement Language (SRL)[13]. SRL consists of a set of schema transformation primitives with the associated set of their applicability conditions. The proof of these conditions ensures the correctness of the design step. A composition operator for this language that permits the definition of a personalised set of schema refinement operators is proposed. Moreover, a rule for automatically deriving the applicability conditions of a composed transformation from the applicability conditions of the component transformations is introduced. Because of its generality, the framework presented can also be exploited to automatically derive the correctness conditions of schema transformations that are specified in other languages. The SRL framework is described in the next section. Section 3 introduces the composition operator. Section 4 presents the rule for deriving the applicability T.W. Ling, S. Ram, and M.L. Lee (Eds.): ER’98, LNCS 1507, pp. 226–240, 1998. c Springer-Verlag Berlin Heidelberg 1998

A Transformational Approach to Correct Schema Refinements

227

conditions of a composed schema transformation. In particular, it discusses how, by exploiting this rule, it is possible also to discover situations in which the definition of a transformation is incorrect. Section 5 shows how the results presented can be exploited to derive the applicability conditions in different transformational frameworks. To illustrate this point, a few examples are presented, taken from well known transformational frameworks. Section 6 contains concluding remarks. The algorithm for the generation of the applicability conditions is given in the Appendix.

2

Schema Refinement Language

The Schema Refinement Language (SRL) assumes that the whole design relies on a single notation able to represents semantic models. This notation, illustrated briefly through the example in Fig. 11 , allows to model the database structure and behavior into a single module, called Database Schema (DBS)[8]. This module encloses classes, attributes, is-a relationships, integrity constraints and operations. A graphical representation of the structural part of the schema on the example is given in Fig. 4(a). The DBS notation is formalised in terms of the formal model introduced within the B-Method[2]. This formalisation allows to exploit the B theory and tools for proving expected properties of the DBS schemas. The SRL primitive operators implement DBS schema transformations. They are given in Table 1. The equality conditions that appear as a parameter in the add/rem transformations specify how the new/removed element can be derived from the already existing/remaining ones. These conditions are required since only redundant components can be added and removed in a refinement step. SRL does not permit to add or remove schema operations. It only permits to change the way in which an operation is defined. Note that the operation definitions are also automatically modified as a side effect of the transformations that add and remove schema components. In particular, these automatic modifications add appropriate updates for each of the new schema components, cancel the occurrences of the removed components and apply the proper variable substitutions. A transformation can be applied when its applicability conditions are verified. These are sufficient conditions, to be checked before the execution of the transformation, that prevent from applying meaningless and correctness breaking schema design. Each applicability condition is composed by the conjunction of simple conditions, that in the rest of the paper will be called applicability predicates. The criterion for the correctness of schema design is based on the following definition (for a formal definition see[13]): Definition (DBS schema refinement relation) A DBS schema S1 refines a DBS schema S2 if: 1

In the figure, ; is the relation composition operator.

228

D. Castelli and S. Pisani

database schema Materials class vein of VEIN with (type:string) class aspect of ASPECT with (colour:string, has vein:vein) class material of MATERIAL with (name:string, has aspect:aspect) class marble is-a material with () class stone is-a material with () constraints ran(has aspect)=dom(has vein) initialization material,name,has aspect,aspect,colour,has vein,vein,type,marble,stone:=Ø operations vt←marble veins types=vt:={t| ∃m∈marble·has aspect;has vein(m)=v∧type(v)=t} Fig. 1. A Database Schema. Table 1. SRL language. add.class (class.name, class.name =expr ) rem.class (class.name, class.name =expr) add.attr(attr.name, class.name, attr.name =expr) rem.attr (attr.name, class.name, attr.name =expr) add.isa(class.name1, class.name2) rem.isa (class.name1, class.name2) mod.op (op.name, body)

(a) S1 and S2 have the same signature2 ; (b) there exists a 1:1 correspondence between the states modelled by S1 and S2 ; (c) the database B1 and B2 , modelled by S1 and S2 , when initialised and submitted to the same sequence of updates, are such that each possible query on B1 returns one of the results expected by evaluating the same query on B2 .2 This notion of correctness is a restricted version those used within the B refinement theory. Let us outline that the main concerns in defining the SRL framework have been simplicity and generality. These qualities are achieved defining both the model and the schema refinement language provided with very primitive mechanisms. SRL, as presented above, however, is not sufficiently general to be used to interpret other schema design frameworks. The schema transformations are usually more complex of those listed above. In order to overcome this limitation, a composition operator for SRL is introduced in the next section.

2

With “same signature” we indicate that S1 and S2 have corresponding operations with the same names and the same input and results parameters.

A Transformational Approach to Correct Schema Refinements

3

229

Composition Operator

The composition operator permits to defines complex transformations from simpler ones. Before introducing it, the following preliminary definition is needed. Definition (Consistent operation modification) A set of SRL schema transformations t1 , t2 , . . ., tn specifies consistent operation modifications if, for each pair of transformations (tk , tj ), with 1≤ k,j ≤ n, and k 6= j, that modify the same operation op, at last one of the conditions below holds: 2) bodyj v bodyk 1) bodyk v bodyj ; where v is the algorithmic refinement as defined in[2] and bodyk and bodyj are the new behaviour of op, specified by, respectively, tk and tj .2 Intuitively, this definition means that all the bodies that are specified for the same operations by different transformations must describe the same general behaviour. They can only differ for being more or less refined. The SRL composition operator can now be defined as follows. Definition (Composition operator “◦”) Let t1 , t2 , . . . , tn be a set of SRL schema transformations that specify consistent operation modifications. Let be a DBS schema where: Cl, Attr, IsA, Constr and Op are sets of respectively classes, attributes, is-a relationships, integrity constraints, and schema operations. Op always contains an operation Init that specifies the schema initialisation. The SRL schema transformation composition operator is defined as follows: t1 ◦ t2 ◦ . . . ◦ tn () = where ACl/RCl, AAttr/RAttr and AIsA/RIsA are sets formed, respectively, by the set of classes, attributes and is-a relationships that are added/removed by t1 , t2 , . . . , tn and are extracted from the component transformation parameters. RemSubst* is the transitive closure of the variable substitutions x:=E dictated by the conditions that are specified when an element is removed. If we have, for example, rem.class(c, c=E) ◦ rem.class(d, d=f(c)) ◦ rem.class(e, e=F) then RemSubst* is the parallel composition of the substitutions c:=E, d:=f(E) and e:=F. [RemSubst*]X is the expression that is obtained by applying the substitution RemSubst* to X. For example, [x:=E]R(x) is R(E). This substitution permits to rephrase integrity constraints and operation definitions by removing the cancelled schema components. AConstr are the conjunction of the inherent constraints associated with the new schema components and the conditions that specify how an added element relates to the remaining ones. Finally, Op0 is the new set of operation definitions. These result from the modifications that are required explicitly and from the automatic adjustments caused by the addition and removal of schema components. When more than one of the component transformations modifies an operation, the more specialised behavior is selected.2 Note that the result of a composition depends from the component transformations and not from the order in which they appear. Differently from other proposals[9], [10], [11], SRL so extended is a complete DBS schema refinement language. This property ensures that SRL is powerful

230

D. Castelli and S. Pisani

enough to express every DBS schema transformation. As a consequence of this property, the designer can progressively enrich the set of schema refinement transformations following his needs. The following example illustrates how new transformations can be build. Example. Let us suppose that the transformation illustrated in Fig. 2 has to be defined. This transformation adds a direct relationship a1 between two classes that were related by an indirect link. This new link is defined as composition of a2 and a3 . Moreover, it removes the relationship a3 . This transformation can be built as composition of simple SRL transformations in the following way: path replacement(C1 , a1 , C2 , a2 , a3 ) = add.attr(a1 , C1 , a1 =a2 ;a3 ) ◦ rem.attr(a3 , C2 , a3 =a−1 2 ;a1 ) C1

a2- C 2 a3 ? C3

⇒

C1

a2- C 2 a1 - C 3

Fig. 2. Path replacement.

The transformation path replacement can be used as any other SRL transformation. For example, it can be applied to the database schema Materials of Fig. 1 as follows: path replacement(marble,has vein marble,aspect,has aspect,has vein) obtaining the DBS schema presented in Fig. 3. Fig. 4 illustrates graphically the effect of the transformation on the static part of the schema. database schema Materials1 class vein of VEIN with (type:string) class aspect of ASPECT with (colour:string) class material of MATERIAL with (name:string, has aspect:aspect) class marble is-a material with (has vein marble:vein) class stone is-a material with () constraints ran(has aspect)=dom(has aspect−1 ;has vein marble) initialization material,name,has aspect,aspect,colour,has vein marble,vein,type,marble,stone:= Ø operations vt←marble veins types=vt:={t| ∃m∈marble·has vein marble(m)=v∧type(v)=t} Fig. 3. DBS Materials1 .

This section has illustrated as the DBS schema refinement transformations can be dynamically built. The next section shows as, in this dynamic context, it is still possible to support the designer in carrying out a correct design process.

A Transformational Approach to Correct Schema Refinements name material has aspect - aspect colour

6 marble

stone

has vein vein type

?

(a)

231

name material has aspect - aspect colour

6 stone marble has vein marble (b)

vein

type

6

Fig. 4. Path replacement.

4

Applicability Conditions

The generation of the applicability conditions of a composed transformation has a double purpose. First, it permits to highlight mistakes in the definition of the transformation and suggests to the designer how remove them. Second, it provides a set of sufficient conditions for ensuring that the application of the transformation results in a correct design. The applicability conditions are generated constructively[13] by the Applicability Condition Generating Algorithm (ACGA), given in Appendix. ACGA generates the applicability conditions by considering the schema structure and the modifications brought by the component transformations. As far as the applicability conditions of composed transformations, the following property holds[13]: Definition (SRL is a refinement language) Let t1 , t2 , . . ., tn be SRL schema transformations and S be a DBS schema. The application of the transformation t1 ◦ t2 ◦ . . . ◦ tn (S), when its applicability conditions are verified, produces a refinement of S.2 This property ensures the correctness of any SRL database design. The following two sections describe in details the two uses of the applicability conditions that have been mentioned above. 4.1

Applicability of a Transformation

As there are no constraints on how the transformations should be composed, it may happen that a newly defined transformation results to be never applicable, i.e. its applicability conditions are never verified. If t is a composed transformation, with n parameters and applicability conditions applt , the proof of the following condition permits to exclude such wrong definition: ∃p1 , · · · , pn , S · applt(p1 ,···,pn )(S) where p1 , · · ·, pn are the parameters of t and S is a DBS schema. This condition expresses that the transformation is applicable if there is at least one instance of the parameters and a schema that verify the applicability conditions. On the contrary, there is some mistake in the definition of t. To illustrate the last point, let us examine the following example3 : 3

In that follows,  stands for the range restriction, C2 is-a-reach C1 is verified if there exists an is-a path between C1 and C2 , and C3 is-aC2 stands for the inherent constraint “C3 is a subclass of C2 ”.

232

D. Castelli and S. Pisani

specialisation(C1 ,C2 ,C3 ,a,v)= add.is-a(C3 ,C2 ) ◦ rem.is-a(C2 ,C1 ) ◦ rem.class(C2 ,C2 =dom(a{v})) The applicability conditions of this transformation are given in Table 2: Table 2. Applicability conditions of specialisation. C3 ∈ Cl ∧ C2 ∈ Cl (C3 , C2 ) 6∈IsA (C2 , C1 ) ∈ (IsA ∪ {(C3 , C2 )}) C2 6∈ {a, v} (Constr ∧ AConstr) ⇒ ¬(C2 is-a-reach C3 ) (Constr ∧ (AConstr - {C3 is-aC2 })) ⇒ C3 ⊆C2 (Constr ∧ AConstr) ⇒ C2 =dom(a{v}) ¬∃a ∈Attr(C2 ) ¬∃C∈Cl·((C,C2 )∈((IsA-{(C2 , C1 )})∪{(C3 ,C2 )}) ∨ (C2 ,C)∈((IsA-{(C2 ,C1 )})∪{(C3 ,C2 )}))

Carrying out the verification of the above applicability conditions it turns out that the last condition will be never verified. Actually, C can be instantiated with “C3 ”. The failure of the proof gives us an indication of what is wrong in the definition of the transformation. The transformation removes the class C2 that is is-a related to C3 . Independently from the values that will be given to C2 and C3 , when the transformation will be instantiated it will always produce a dangling is-a relationship, as illustrated in Fig. 5. a C  C 1 2 (a)

a C 1

C3

?i

C3

(b) Fig. 5. Specialisation.

4.2

Correctness of a Design Step

The applicability conditions of a composed transformation are parametric with respect to the parameters of the composed transformation. By reasoning on these conditions, it turns out that some of them can be solved without instantiating the parameters; others can be discharged by simply comparing the values of the parameters. This suggest us to automatically prune these predicates and associate to the instance of a transformation only the so simplified set of applicability predicates. The pruning is done at different stages. When a transformation, with parameters p1 , . . . , pn , is defined, the set of applicability predicates is scanned and, for each predicate Pij of the set, the proof of ∀ p1 . . . pn , S · Pij , where S is a DBS schema, is attempted. If the proof is successful, Pij is inserted in the set of the applicability predicates that have not to be proved anymore. The

A Transformational Approach to Correct Schema Refinements

233

second kind of pruning is executed when the transformation is instantiated. By reasoning on the structure of the component transformations and the values of the parameters, several applicability predicates are discharged. The ACGA algorithm, reported in the Appendix, actually implements a mix between the generation of the applicability conditions and the second pruning, The result is the set of applicability predicates that the designer has to prove for a particular application of the transformation. Notice that this set is often very small. Moreover, since the SRL framework and its application conditions are formalised, an automatic, or at least guided, discharge of the applicability conditions that are generated is possible. As an example of dynamic generation of the applicability conditions of a composed transformation, let us see which are the applicability conditions of the transformation path replacement, as invoked in the example of Sect. 3. In these conditions, the following abbreviations are used: • Constr stands for the constraints of the initial schema; • Inh stands for the inherent constraints that are implicitly added by the transformation; • NewConstr1 = Constr ∧ Inh ∧ has vein=has aspect−1 ;has vein marble • NewConstr2 = Constr ∧ Inh ∧ has vein marble=has aspect;has vein The applicability conditions of path replacement are: (a) NewConstr1 ⇒ dom(has aspect;has vein) ⊆ marble (b) NewConstr2 ⇒ has vein = has aspect−1 ;has vein marble The first condition requires that the added relationship, defined as composition of has aspect and has vein, be defined on the class marble. The second condition requires that the removed relationship be derivable as the sequential composition of the remaining ones. Those listed above are the only applicability conditions that are returned to the designer. Several others are checked and discharged by the ACGA algorithm automatically.

5

Exploiting SRL in Other Design Frameworks

The described approach can be used for achieving a more reliable design also in other frameworks. In all the design frameworks that can be interpreted as a special case of the one that has been described, the applicability conditions of any transformation can be generated by exploiting th given framework. In order to generate these conditions, it is sufficient, first, to define the refinement transformation as the difference between the initial and the final schemas and, then, to express this difference as composition of SRL primitives. At this point, the applicability conditions follow automatically. This approach to the derivation of the applicability conditions of a refinement transformation can be useful in all those design contexts in which the preconditions for a correct design are not given. These include also those contexts in which there are no established transformations but the design is done by writing down the logical schema directly. In this case, the transformation is implicit

234

D. Castelli and S. Pisani

and, of course, there are no preconditions for guaranteeing its correctness. This approach may be useful also when there are applicability conditions, but they are only given informally. In these cases, assistance tools for the verification of these preconditions cannot be built. Using the suggested approach we can take advantage of those provided for SRL. The approach illustrated above has an inherent limitation: it can be applied only if its premises agree with those of SRL. In particular, the employed model must be a submodel of DBS and the DBS schema refinement relation must conform to the one that has been given in Sect. 2. In order to illustrate how the approach proposed can be useful in other frameworks, we present two examples of derivation of applicability conditions. The examples consider two refinement transformations taken from two different well known languages. Example 1. The first example considers one of the schema transformations proposed in[3]. The transformations within this set do not change the information content of the schema. Moreover, the assumed data model conforms to DBS. The transformation chosen is: elimination of dangling subentities in generalisation hierarchies. For brevity, below it will be named elimination. The transformation elimination removes n non overlapping subclasses and reduces them to a superclass. The elements of the superclass are partitioned in n groups by the value of added attribute. Fig. 6 shows this transformation. C C1

6

···

Cn

⇒

(a)

C a (b)

Fig. 6. elimination.

The schema in Fig. 6(b) differs from the schema in Fig. 6(a) since it has a new attribute a and the classes C1 ,· · ·, Cn and their is-a relationships are missing. This difference can be easily expressed as composition of SRL primitives: elimination(a,C,(v1 ,· · ·,vn ),(C1 ,· · ·,Cn ))= add.attr(a,C,a={(x,y)|x∈(C1 ∪· · · ∪Cn )∧(x∈C1 →y=v1 )∧ · · · ∧(x∈Cn →y=vn )})◦ rem.class(C1 ,C1 =dom(a{v1 }))◦ · · · ◦rem.class(Cn ,Cn =dom(a{vn }))◦ rem.isa(C1 ,C)◦ · · · ◦rem.isa(Cn ,C) The transformation elimination, for example, can be applied to the schema S in Fig. 7(a) to obtain the schema of Fig. 7(b): elimination(type,Material,(marble,stone),(Marble,Stone))(S) The applicability conditions associated to the above instantiation of the elimination are the following4 : 4

NewConstr stands for the conjunction of the constraints of the initial schema and a subset of those added. This subset consists of all the constraints added by the component transformations that differ from the transformation that has generated the condition.

A Transformational Approach to Correct Schema Refinements Material Marble

6

⇒

Stone

235

Material type (b)

(a) Fig. 7. elimination.

• NewConstr ⇒ (dom({(m,t) | m∈(Marble∪Stone) ∧ (m∈Marble → t=marble ∧ m∈Stone → t=stone)}) ⊆ Material) • NewConstr ⇒ Marble=dom(type{marble}) • NewConstr ⇒ Stone=dom(type{stone}) Example 2. The second example is taken from[10]. Here, a set of ER schema transformations to support the designer during the schema development is proposed. One of these transformations is disaggregation a compound attribute. For brevity, it will be called disaggregation. This transformation is semanticspreserving, i.e. it does not change the information content of the schema. The transformation disaggregation replaces the compound attribute with its component fields as shown in Fig. 8. C a< a1 ,· · ·,an >

⇒

a. a1 . C a. an (b)

(a) Fig. 8. disaggregation.

The schema in Fig. 8(a) differs from the schema in Fig. 8(b) since in the second one there are n new attributes, a a1 ,· · ·, a an , and the attribute a is missing. This difference can be expressed as composition of SRL primitives as follows5 : disaggregation((a1 ,· · ·,an ),C,a) = add.attr(a a1 ,C,a a1 ={(x,y) |x∈C ∧ y=prj1 (a(x))}) ◦ · · · ◦ add.attr(a an ,C,a an ={(x,y) | x∈C ∧ y=prjn (a(x))}) ◦ rem.attr(a,C,a={(x,y) | x∈C ∧ y=}) Let us, for example, apply the transformation disaggregation to the schema S in Fig. 9(a) to produce the schema in Fig. 9(b): disaggregation((colour,vein),Material,aspect)(S). Material aspect (a)

⇒

aspect colour Material aspect vein (b)

Fig. 9. disaggregation.

5

In that follows, prji (z) stands for the projection of z on the i-th element.

236

D. Castelli and S. Pisani

The only applicability condition of disaggregation is6 : NewConstr⇒aspect={(m,a)|m∈Material ∧a=}

6

Conclusions

This paper has proposed a solution to the problem of offering a flexible instrument for supporting a correct database design. Flexibility is achieved by employing a general model and by providing a means for defining refinement transformations dynamically. Flexibility, in this case, does not degrade correctness since the applicability conditions can still be associated to the dynamically built transformations. The paper has shown also as, by exploiting the flexibility of SRL, it is possible to employ SRL for generating the applicability conditions of refinement transformations proposed in other contexts. In particular, this use of the given framework permits to improve the reliability of the database design process also in frameworks where no specific instrument for such purpose exists. We have experimented this particular application of the framework proposed in carrying out the design of two databases. The first is the multimedia database for supporting the MIAOW system[14]. This database, designed as part of the Marble Industry Advertising over the World ESPRIT Project (n. 3990), maintains information about stones and stone actors. The second is a database that maintains multimedia data about the history of Computer Science in Italy[15]. It was implemented as part of the Italian National Project Museo Virtuale della Storia dell’Informatica. The design of these two database was carried out by generating a sequence of OMT-like schemas[12]. Each schema in the sequence was produced by ad-hoc transformations. For each of such transformations, the corresponding applicability conditions were generated. These design experiences, on the one hand, confirmed the utility of the approach illustrated in discovering specific design mistakes and, on the other, suggested us improvements to our framework. In particular, it highlighted situations in which an automatic discharging was preferable. Let us conclude by outlining that the particular type of formalisation that has been chosen for the presented framework allows to built tools that assist in the generation and discharge of the applicability conditions. In particular, the same B tools that can be used for proving properties of DBS schema can also be exploited for supporting the discharge of the SRL applicability preconditions. Moreover, they can also be used to implement the more sophisticated version of the ACGA algorithm.

References 1. P. Assenova, P. Johannssen. Improving Quality in Conceptual Modelling by the Use of Schema Transformation. In Lecture Notes in Computer Science, n. 1157, pp. 277-291, Springer-Verlag, 1996. 6

NewConstr are defined as in the previous example.

A Transformational Approach to Correct Schema Refinements

237

2. J.R. Abrial. The B-Book. Cambridge University Press, 1996. 3. C. Batini, S .Ceri, S.B. Navathe. Conceptual Database Design. Redwood City, CA, The Benjamin/Cummings Publishing Company, Inc., 1992. 4. C. Batini, G. Di Battista, G. Santucci. Structuring Primitives for a Dictionary of Entity Relationship Data Schemas. IEEE Transactions on Software Engineering, 19(4), April 1993. 5. P.L. Bergstein, W.L. H¨ ursch. Maintaining Behavioral Consistency during Schema Evolution. In First JSSST International Symposium on Object Technologies for Advanced Software, Kanazawa, Japan, Lecture Notes in Computer Science, SpringerVerlag, pp.176-193, November 1993. 6. P. Mc.Brien, A. Poulovassilis. A Formal Framework for ER Schema Transformation. In Lecture Notes in Computer Science, n.1331, pp.408-421, Springer-Verlag, 1997. 7. P. van Bommel. Database design by computer-aided schema transformations. Software Engineering Journal, pp. 125-132, July 1995. 8. D. Castelli, E. Locuratolo. ASSO: A Formal Database Design Methodology, in Information Modelling and Knowledge Bases VI, H. Jaakkaola et al. eds., IOSPress, 1994. 9. A. D’Atri, D. Sacc` a. Equivalence and Mapping of Database Schemas. In Proceedings of the 10th International Conference on Very Large Data Bases, pp. 187-195, Singapore, August 1984. 10. J.L. Hainaut. Transformation-based Database Engineering, Tutorial of the Very Large Data Bases Conference, Zurigh, Switzerland, September 1995. 11. I. Kobayashi. Losslessness and Semantic Correctness of Database Schema Transformations: Another Look of Schema Equivalence. Information Systems, 11(1), pp. 41-59, 1986. 12. J. Rumbaugh, M. Blaha, W. Premerlani, F. Eddy, W. Lorensen. Object-Oriented Modeling and Design, Prentice Hall, Englewood Cliffs, New Jersey 07632, 1991. 13. D. Castelli, S. Pisani. A Transformational Approach to Database Design, IEI-CNR Technical Report, 1998. 14. B. Biagi, D. Castelli, F. Niccolini, S. Pisani. MIAOW: An object oriented Multimedia DB application on the WWW for the Stone Market, IEI: B4-37-12-97, 1997. 15. G. De Marco, S. Pisani. Disegno e Realizzazione della Base di Dati Multimediale di Supporto al Progetto “Museo Virtuale della Storia dell’Informatica in Italia”, IEI-CNR Internal Note B4-25, 1997.

Appendix: The ACGA Algorithm The Applicability Condition Generating Algorithm (ACGA) is described below. This algorithm takes as input a set of SRL transformations {t1 , t2 , . . ., tn }, their applicability conditions, the set ver appl, specified below, and a DBS schema . It returns a set of applicability predicates. The description of the algorithm is given using an informal notation and makes use of the following abbreviations: • ACl/Attr and RCl/Attr indicate, respectively, the set of classes and attributes that are added and removed; • Attr(C), AAttr(C) and RAttr(C) indicate, respectively, the set of attributes that are defined on the class C in the initial schema, added to C and removed from C by the composed transformation;

238

D. Castelli and S. Pisani

• AConstr are the constraints that are added by the component transformations; • RemSubst* are the variable substitutions induced by all the removals of schema components; • C1 is-a C2 stands for the inherent constraint “C1 is a subclass of C2 ”; • a attribute-of C is the inherent constraint “a is an attribute of the class C”; • ver appl is the set of applicability conditions that are proved to be verified when the composed transformation is defined. The algorithm consists of four steps. For sake of brevity, only the step 4 is reported explicitly. The first step initialises the set appl◦ which maintains the applicability predicates returned by the algorithm. The second step generates a first group of applicability predicates. These require that the component transformations specify consistent modifications. The third step generates a temporary set of applicability predicates to be proved. This set is scanned in the fourth step. If a predicated of this set is found to be false, then appl◦ is set to “false” and the algorithm is terminated. Each predicate of the set that cannot be discharged by the checks operated by the algorithm is inserted in the set appl◦ . Step 4 of the Applicability Condition Generating Algorithm appl := appl - ver appl; % This assignment removes the applicability conditions that have been verified when the composed transformation has been defined. repeat p := extract(appl); appl := appl - p; case type(p) of x∈Cl then if not(x∈(ACl ∪ Cl)) then appl◦ := false % This predicate is false if x is not in the initial schema and there is no transformation in the composition that adds the class x. It is true otherwise. or x6∈Cl then if not(x6∈Cl) then appl◦ := false % This predicate is false if the class x is in the initial schema. It is true otherwise. or x∈Attr(C) then if not(x∈Attr(C)) then appl◦ := false % This predicate is false if the x is not an attribute of C in the initial schema. It is true otherwise. or x6∈Attr then if not(x6∈Attr) then appl◦ := false % This condition is false if the attribute x is in the initial schema. It is true otherwise. or x∈IsA then if not(x∈IsA) then appl◦ := false % This predicate is false if the is-a relationship x is not in the initial schema. It is true otherwise. or x6∈IsA then if not(x6∈IsA) then appl◦ := false % This predicate is false if the is-a relationship x is in the initial schema. It is true otherwise. or Free(E)⊆(Cl ∪ Attr) then if not(Free(E)-(ACl ∪ AAttr)) ⊆ Cl ∪ Attr) then appl◦ := false % The predicate is false if the free variables in E are not added variables or they do not belong to the initial schema. It is true otherwise. or x6∈Free(E) then if not(x6∈Free(E)) then appl◦ := false % The predicate is false if x is a free variable of E. It is true otherwise. or ¬∃C∈Cl·(C,C1 )∈IsA ∨ (C1 ,C)∈IsA then if C1 ∈(Cl∩ACl∩RCl) then do nothing else if (∃C∈(Cl∪ACl)·((C1 ,C)∈(AIsA-RIsA)∨(C,C1 )∈(AIsA-RIsA))) ∨

A Transformational Approach to Correct Schema Refinements

239

(∃C∈Cl·((C,C1 )∈(IsA-RIsA∨(C1 ,C)∈(IsA-RIsA)))∨ (∃ C∈Cl·((C,C1 )∈(IsA∩AIsA∩RIsA)∨(C1 ,C)∈(IsA∩AIsA∩RIsA))) then appl◦ :=false % The predicate has not to be checked if the class C1 , that is removed, belongs to the initial schema and it is added by a transformation in the composition. It is false, if there are not removed is-a relationships that involve C1 . or ¬∃a∈Attr(C1 ) then if C1 ∈(Cl∩ACl∩RCl) then do nothing else if (∃a∈(AAttr(C1 ) - RAttr(C1 ))) ∨ (∃a∈(Attr(C1 ) - RAttr(C1 ))) ∨ (∃a∈(Attr(C1 ) ∩ AAttr(C1 ) ∩ RAttr(C1 ))) then appl◦ := false % The predicate has not to be checked if the class C1 , that is removed, belongs to the initial schema and it is added by a transformation in the composition. It is false if there are not removed attributes defined on C1 . or Constr⇒C2 ⊆C1 then if ∃ C3 ,· · ·,Cn ∈(Cl∪ACl) · (C1 =C2 ∪C3 ∪ · · · ∪Cn )∈AConstr then do nothing else appl◦ := appl◦ ∪ {(Constr∧(AConstr - {C2 is-a C1 })) ⇒ C2 ⊆C1 }; % The predicate is true if there is a transformation in the composition that adds the class C1 and defines it as the union C2 and other classes. or Constr⇒ ¬(C1 is-a-reach C2 ) then appl◦ :=appl◦ ∪{(Constr∧AConstr)⇒ ¬(C1 is-a-reach C2 )}; or Constr⇒x=E then if x=E∈AConstr then do nothing else appl◦ := appl◦ ∪ {(Constr∧AConstr) ⇒ x=E}; or Constr⇒dom(F)⊆C then appl◦ :=appl◦ ∪ {(Constr∧(AConstr {a attribute-of C | ∃a· a=F∈AConstr})) ⇒ dom(F)⊆C}; orbody1 vbody2 then appl◦ :=appl◦ ∪{[RemSubst*]body1 v[RemSubst*]body2 } % The variable substitutions must be taken into account when evaluating the refinement relation. until appl = ∅ ∨ appl◦ = false; return appl◦ ;

The Guidelines of Modeling - An Approach to Enhance the Quality in Information Models Reinhard Schuette1 and Thomas Rotthowe2 * 1

Department of Business Administration, especially Production and Information Management, University of Essen, Universitaetsstr. 9 D-45141 Essen, Germany [email protected]

http://www.pim.uni-essen.de/ 2

Department of Information Systems, University of Muenster, Steinfurter Str. 107 D-48149 Muenster, Germany [email protected] http://www-wi.uni-muenster.de/is

Abstract. Although the necessity of building models seems obvious, it is astonishing that only few works deal with the fundamental problems of modeling - how models need to be understood from an epistemological point of view (e.g.: how does a modeler come to a model; why and in which ways do models built by different designers differ from another). The paper bases on the assumption that the subjective position of the modeler is the characterizing issue for the result of the modeling process – and that this subjectivity needs to be managed. We derive from this assumption the need for a sound structural framework in order to deal with the subjectivism of building models. With the Guidelines of Modeling (GoM) we present a framework of principles that improve the quality of information models by reducing the subjectivism in the information modeling process. The quality of modeling is supported by recommendations for an efficient, comprehensive, and correct design of information models. In the first part of the paper the guidelines are derived from the specific problems that stem from the subjective process of the system design. The Guidelines of Modeling contain six principles to ameliorate the quality of information modeling which are described in detail. Subsequently, the basic guidelines are placed in a structural framework, the GoMst Architecture, which consists of two dimensions: 1 the range of model-use (reference models for a class of organizations or an industry, and companynd specific models) and 2 the degree of precision or concretion (general, system views, and description language).

*

We would like to thank the anonymous reviewers for their helpful comments to improve this paper.

T.W. Ling, S. Ram, and M.L. Lee (Eds.): ER’98, LNCS 1507, pp. 240−254, 1998.  Springer-Verlag Berlin Heidelberg 1998

The Guidelines of Modeling - An Approach to Enhance the Quality

241

1 Introduction: Theoretical Aspects in Information Modeling The starting point of information modeling is the question of how to explicate the knowledge generated in a problem area in models (documents). It seems that the appropriate conceptualization of the knowledge and its representation in information models is not a problem for itself. In the research community a pragmatic approach is dominating, that means, technical questions are predominate, and fundamental considerations and reflections beyond current trends are missing. Therefore, notations and problem solving techniques are presented, but the underlying assumptions are rarely examined. It is not the intention of this paper to analyze the necessity of discussing the underlying assumptions in the disciplines of computer science or information systems science. We think that in the empirically oriented sciences, where methods for the analysis of the reality are used and sets of arguments are formulated about the reality, the need for discussing philosophical problems is very high (for example questions and controversies about the reality itself, about the perception of the reality, about the distortion of the perception, about the construct achievement of the perceiving subject, about methods for formulating statements about the reality). From the general meaning of the philosophy of science to the specific area of information modeling we draw the conclusion that, at first, philosophical comments about the ontological and epistemological positions, about the way, designers and users can recognize the world, need to be carried out. By following this approach we are able to derive a well-founded - even if not provable - starting position for the problems of information modeling. This seems to be even more significant when one looks at the research findings in systems design and systems engineering. In these disciplines, one assumes objectivity in systems design and development, which in our opinion contradicts several basic positions. As an example we quote the analysis conducted by KROGSTIE and S‡LVBERG, who examined sixteen procedure models for the development of business application systems, from which only three assumed a non-trivial relationship between the model and the modeler [25]. We propose the thesis that scientific examinations of information modeling without paying attention to the designer are inaccurate. On the other hand, questions about semantics are disregarded, which we attach high importance for the quality of the system to be designed. In this paper, we first examine the basic understanding of a model (Sect. 2). It is shown, that the relation between the designers, the users of the model (the subjects) and the real world situation (the object) lead to philosophical problems. We abandon to take an unambiguous philosophical position. Instead, we argue from a modern epistemological perspective. From the result of the definition of the term model the importance of the modeling subject (the designer) can be derived. By following certain rules in the modeling process it will be possible to build inter-subjectively comparable models. An architecture for the derivation of these rules is presented with the Guidelines of Modeling (GoM) (Sect. 3). Especially in multi-designer projects

242

R. Schuette and T. Rotthowe

these rules are absolutely necessary. Moreover, a model user, who was not involved in the modeling process, would be only able to understand the models, if he knew the underlying guidelines of the model design. In the area of software engineering, for example, one can observe increasing instances where different software components need to be combined (e.g., component ware). For the implementation of the combination process the range of the different tasks need to be reconciled, which leads to several problems on the semantic level. By obeying guidelines in the modeling process, this reconciliation will be much easier. The utilization of the GoM is examined in empirical studies in a joint research program funded by the German Federal Ministry of Education, Science, Research and Technology [42, 8].

2

The Basic Understanding of a Model

Traditionally, information models are defined as a mapping or representation of an area of the real world for the use of a subject [20]. From this definition one can derive the conclusion that models are somewhat similar to an entity, a system or any other area of the real world [19]. In accordance with TARSKIS semantic model theory many researchers use the logical model term. This definition talks about a model, if the interpretation of a mathematical structure were true for all axioms and derivation rules of the structure ([see also [14, 16]). This understanding of model does not assume a relation to the reality but analyzes the interrelation of structures. The model definition of mapping or representation - which often means that there is a similarity between the real system and the model system - is dominant in the research field of information modeling, since the objective of building information models is to draw conclusions about the reality. Therefore, we use the mapping-driven model definition as a point of demarcation for the following argumentation in order to analyze the profound problems of such a model understanding. In this way, a suitable basis can be built that is of high importance for any research that deals with modeling. If a model had some similarity with the reality, one would have to assume that the reality were not only independent from its observer (as it is perceived in the ontological objectivity or ontological realism, resp.), but were also assuming an epistemic objectivity. Only if the subjective observation of the reality corresponded exactly to the reality itself (assumption of a naive realism), a mapping-driven definition of the term model would be acceptable. The proponents of this paradigm can not necessarily be called naive realists, but the proposed model definition has to assume, in consequence, a naive realism, provided the mapping is not understood as a declaration of the modeler, but still claims for a structural and behavioral accordance with the reality. Since the observation of the reality is not independent from the observing subject (the observer or designer), the mapping-oriented approaches can not be brought into accord with modern epistemological positions. Nevertheless, the objective in the research literature often is to strive for the most possible homomorphy between reality and model system. This goal does not take into account that at most the homomorphy between the picture perceived by the human brain and

The Guidelines of Modeling - An Approach to Enhance the Quality

243

the model system can be reached because of the subjective observation of the reality. The brain has an active character and the observation of the reality – according to newest biological research – can be understood as an interpretation and meaning allocation of for itself meaningless neural processes [44]. Therefore, any number of realities do exist, since the subjective perception of the reality takes the role of the actual original (the reality), which implies many different homomorph mappings. In this case, the model can not be proven right, since any statement of a modeler is necessarily perceived as a valid conclusion about the structure and behavior of the reality. In opposition to the definitions given above, we define a model not as an formal mapping of an area of the real world but as the result of a construct done by a modeler, who examines the elements of a system for a specific purpose such as the redesign of an organization or the development of an information system at a given point of time with a specific language [42, see also 9, 50]. In order to avoid the misleading paradigm of the existing definitions, the structure-giving achievement of the modeler is stressed, as it is outlined in modern insights such as the critical realism by POPPER [35] or ALBERT [1, 2] (there exists a world independent from us but we cannot claim that we recognize the world as it is), the evolutionary epistemological theory [48] (all sets about the reality are hypothetical and we take the assumption, that the reality has structures) or the moderate constructivist philosophy (see as one kind of this epistemological position [38]; a reality outside of us does exist, but the assumption that our knowledge says something about the structure of reality, is too optimistic). The term construct has its origin in the active character of human perception. The emphasizing of the subjectivity in the modeling process demands for specific measures for the management of the subjectivity in order to develop intersubjectively comparable and provable models. The highlighting of the subjectivity stresses the importance of the designer. If a representation- or mapping-oriented definition of the term model were chosen, the modeling as a transformation of given structures would happen ”quasi on its own”. This contradicts to the actual procedure, in which the designer defines the structure of the modeled problem area. Modeling is more a definition of structure than a transformation of given structural complexes. In an information systems modeling project the question of how to derive the hierarchical decomposition is not answered through observations but through constructional activities. Moreover, the structure giving process is not independent from theory but is very much influenced by theory. The starting point of modeling, the system theory divided in structural, functional and hierarchical aspects, already forms a theory in which the real world is discussed. Based on this assumption, the modeling moves away from an observational language towards a theory language. The experiences of the logical empiricism have shown that a theory language can not be reduced to an observation language [2]. Since the subjectivity of modeling can not be eliminated but only managed, a demand for rules that have to be carried out in the modeling process can be derived. With the Guidelines of Modeling a framework is presented, which includes objectives of modeling that can be used for the evaluation of measures for increasing the model quality.

244

R. Schuette and T. Rotthowe

3 Standardization of Information Modeling through the Guidelines of Modeling 3.1

The Intention of the Guidelines of Modeling (GoM)

Inter-subjectively comparable models require the obedience of modeling rules, since ”the design process is not deterministic: different designers can produce different enterprise models of the same enterprise.” [18]. Only through the formulation of modeling conventions the design process becomes comprehensible and the integration of different models into an enterprise model is simplified. This way the modeling process can not only be shortened, but also the danger of a defective integration can be reduced. With the GoM we have the objective to develop specific and definite design recommendations which lead to a higher quality of models through the execution of syntactic rules. The guidelines serve as an evaluation framework for the development and assessment of information models on the requirements definition level with consideration of the designer and the model user. Therefore, aspects relevant for the assessment on the implementation description level are not considered because of the fundamental importance of the model quality on the requirements definition level. Errors and quality defects on this level may cause high expenditures and efforts on the following phases: An error is ”typically 100 times more expensive to correct in the maintenance phase on large projects than in the requirements phase” [12]. The term GoM was chosen intentionally in analogy to the German GoB, the generally accepted accounting principles (GAAP), in order to characterize through the wording the intention of the ordering framework [41]. Furthermore, the analogy with the GAAP is also represented by the intentional restrictions of the modeling freedom in the preparation of financial statements on the one hand and the information models on the other hand. Besides the analogy between the GoM and the GAAP, other existing frameworks for the evaluation of information model quality were taken into account. One can differentiate view-specific and view-independent (general, comprehensive) approaches. View-specific approaches can be found expecially for data models, as the works of [36, 3, 30] show. Other researchers also define criteria for data model quality [17, 27, 49]. For process models, reflections can be found in [15], who concentrates on criteria for an ex-post assessment with a focus on objectively measurable figures. Considerations on general frameworks are, for instance, presented by [26, 10, 22, 23, 24, 34, 33] and in the alternative first - afflicted with theoretical problems - version of the GoM [see for example 10, 11]. The GoM constitute a comprehensive approach which extends existing frameworks with regard to the formulated rules and the differentiation between situational and general modeling. The here presented version of the GoM were developed by [42, 43].

The Guidelines of Modeling - An Approach to Enhance the Quality

3.2

245

The General Guidelines of Modeling

The GoM consist of six general principles, the principle of construction adequacy, the principle of language adequacy, the principle of economic efficiency, the principle of clarity, the principle of systematic design, and the principle of comparability. 3.2.1 The Principle of Construction Adequacy Because of the refusal of the traditional definition of the term model which focuses on the similarity between a model and the real world, the requirement of homomorphy is untenable, that means nobody is capable of judging whether a model is correct in comparison to the reality. Instead, we propose the principle of construction adequacy, which, under the assumption of a critical realism, also can be called reality adequacy. From the viewpoint of the application system designer the construction quality of a model requires that a consensus can be found according to the represented problem. The results of this consensus can only be proven by questioning the persons affected (e.g., the designer, the user). In the case of an enterprise modeling project which focuses on the current organizational setting, organization handbooks can be utilized to support the evaluation. Usually, the facts of the case are not fully depicted in a way that sufficiently reflects the situation. In the case of an as-is modeling task for business application projects program documentation or program code is available. Again, in this situation, a definite representation of the real situation (the program code) is not possible, as has been shown in numerous reengineering projects. Besides the consensus about the problem to be constructed, a consensus about the type of construction needs to be defined, i. e., an agreement about the model representation needs to be settled. The explicit definition of information objects is required, which leads to multiple uses of these objects in different models and situations. Another aspect of the explicit definition is the context-invariant modeling of problems. In integration projects, the examined conflicts with differing model representations of a real world situation verify the importance of intra- and intermodel relations that aim at the uniform application of elements within a model for the same facts of the case. Intra-model relations aim at the uniform application of construction patterns within one model, such as naming conventions and the standardized use of the modeling language. Inter-model relations aim at the standardized representation of the case facts in different models. For example, a function called ”order entry” should have the same naming throughout all process models, as long as they have the same content (for problems of constructing the ”right” words see [32]). The construction adequacy does not cover the interrelation between models of different types in the sense of the separation of structural and behavioral models. The latter is covered by the principle of systematic design (see 3.2.5).

246

R. Schuette and T. Rotthowe

The appropriateness of a model implies the consideration, which specific information objects need to be included in the model. The pursued objective of the model addressee has to be taken into account during the definition procedure of the problem area relevant for the analysis. For instance, from the viewpoint of the decision instance of the organization, the problem area is fairly broad because of the many decision parameters that are relevant for this target group. The appropriateness of the model is highly affected by the selection of the relevant information objects in the design process, since information objects determine the degree of abstraction of the model, and, therefore, influence the usability of the model from the viewpoint of the model addressees. A criteria very closely related to the type of representation is the criteria of minimalism. A model is minimal when no information objects can be removed from the model without a loss of information for the potential model user [29, 3]. Thereby, we define minimalism as an endogenous characteristic of a model, i.e. the relevance of information is defined during the modeling process. It can not be proven as an exogenous model standard, because the criteria of relevance requires always a model in the background (the meta model), which itself again requires a meta model for itself, etc. (infinite regress) (see for example the problems to construct decision models with ”perfect” complexity, [51]). The major problem of the principle of construction adequacy is to get a consensus between analysts and users ([31, 13], and as an critical reaction against the demand of consensus in philosophy [37]). 3.2.2 The Principle of Language Adequacy The construction of a model is executed by using an (artificial) language. Whereas the principle of construction adequacy represents a criteria for the evaluation of the problem representation in the model, the principle of language adequacy focuses on the interrelation between the model system and the used language. The principle is differentiated in language suitability and language correctness. The language suitability includes especially the problem-related selection of modeling techniques and the selection of relevant model constructs (elements) [45, 22]. The problem area and the modeling purpose determine the necessary semantic powerfulness of a modeling method. For the purpose of the organizational design (e.g., restructuring, business reengineering) the method of the event-driven process chains (EPC) by [21, 40] can be suitable, where the EPC might not be as sufficient as petri nets for the purpose of simulation. The purpose of the model requires in some cases a specific degree of formalism of the language in order to carry out simulations. Therefore, semi-formal (artificial) languages are used in information modeling. They build the starting point of a technological implementation. The knowledge of a model addressee, which includes his experiences with modeling techniques, determines the language comprehensibility, the subjective suitability of a language, which is also dependent on its technical support. Therefore, the tool support is also an evaluation criteria for the language comprehensibility. Language correctness as an highly objective goal focuses on the correct application of the language syntax, i.e., its grammar. The use of a language meets the require-

The Guidelines of Modeling - An Approach to Enhance the Quality

247

ments of correctness, if it follows the rules of the meta model, that means whether the model is complete and consistent with the meta model [3, 26, 22] (see Fig. 1 for an example). A model is complete according to the meta model, if the relationships between the information objects described in the meta model are applied in the model itself in the same way. A model is consistent with the meta model, if the included information objects in the model are completely defined in the meta model. For example, the modeling of an attribute in the model requires the specification of it in the meta model, otherwise the model would be inconsistent. The language correctness is based on general formal requirements, therefore, a model can also be called correct or incorrect in terms of language.

entity type

(0,m )

(2,m )

violation of co nsiste ncy

violation of co m pleteness

cus tom er

M eta m odel system

relationship type

cus tom er

produc t

M odel system

A N r, A N a m e

Fig. 1. Violation of the completeness and consistency

3.2.3 The Principle of Economic Efficiency With the principle of economic efficiency we formulate an economic restriction. Every activity in economic institutions is subject to its economic efficiency. This general maxim, which stems from the basic phenomena of economic problems, the scarcity of resources, is also applicable to the process of information modeling. The produced cost of the design process need at least to be covered by the proposed utilization of cost cuttings and revenue increases. Since the purpose of organizations in most cases is the maximization of profit, the principle defines the borderline for the modeling intensity. Therefore, the principle in most cases causes conflicts with the other principles. The demand for economic efficiency can easily be put in relation to the above mentioned principles and is of specific importance for generating a consensus. The consensus about the model can be in danger because of two reasons. First, a changing conceptualization of the modeling subjects (the designer) may require an adaptation of the problem definition, which may lead to a different model. Secondly, modifications of the modeling objectives may lead to model adjustments. The cost of using a language correlates to the complexity of its application. Comprehensive languages allow a faster modeling of case facts. Opposite economic consequences are possible, if a used language does not have the necessary semantic

248

R. Schuette and T. Rotthowe

powerfulness which may lead to additional efforts at a later point of time. The same is applicable to the degree of formalization of the language which eases the modeling construction but requires additional information for a later system specification. It is very difficult to operationalize this objective of economical efficient model design and use. With regard to the amount of time and money spent on investments in modeling projects the demand for efficient measures is evident. Further research needs to be conducted to find means to operationalize this objective. 3.2.4 The Principle of Clarity The principle of clarity deals with the comprehensibility and explicitness of model systems. Under the objectives of clarity we subsume an addressee-oriented hierarchical decomposition, layout design (arrangement of the elements) and filtering of information. The explicitness of the hierarchical decomposition of systems was already discussed as an aspect of the system description. For bigger information modeling projects, where some hundred different models are designed, the designer needs structural guidelines for arranging the models on different levels of abstraction in order to manage the comprehensibility of the model system. Criteria for the development of decomposition need to be defined, such as organizational or information technology focus, structural or behavioral analysis, concentration on material or information flow, etc. Through the uniform application of these criteria, the comprehensibility of broader model systems can be increased. The comprehensibility of the layout design concerns the graphical arrangement of the information objects, and, therefore, supports the lucidity of a model. Rules are demanded for the arrangement and interrelation description of information objects, e.g., the placing of entity types in a data model from left to right representing their importance (see Fig. 2) ([6, 4, 5], see also [28]). Criteria

Objectives/Goals

Angle

Angles between edges should not be too small.

Area

Minimize the area occupied by the drawing.

Balance

Balance the diagramm with respect to the axis.

Convex

Minimize the number of bends along edges.

Crossing

Maximize the number of faces drawn as convex polygons.

Degree

Place nodes with high degree in the center of the drawing.

DIM

Minimize differences among nodes’ dimension.

Length

Minimize the global length of the edges.

Maxcon

Minimize length of the longest edge.

Symmetry

Have symmetry of sons in hierarchies.

Uniden

Have uniform density of nodes in the drawing.

Vert

Have verticality of hierarchical structures.

Fig. 2. Layout design [46]

The Guidelines of Modeling - An Approach to Enhance the Quality

249

The limitation of the number of different information objects serves the simplicity, since the model becomes less complicated. From the model users´ point of view, the broadening of modeling techniques leads to a lower comprehensibility of models. On the other hand, a more detailed and extended set of elements might be more suitable for the application system developer, since he might need very detailed information in order to develop the actual system. The comprehensibility of filtering focuses on the addressee-oriented preparation of the model. The model is presented according to the information needs of the model users. One can differentiate content-specific filters and methodical filters. Contentspecific filters have the objective to depict different detail levels of case facts in one model. The model user has to decide which level of detail he needs for his purpose. Methodical filters allow the model user to configure the meta model. 3.2.5 The Principle of Systematic Design The principle of systematic design concentrates on the well accepted differentiation between diverse views within modeling. Information models describe the logical composition of structure and behavior of information systems. If one wants to describe or predict the behavior of a system, he needs to know its structure. If one wants to attain a specific system behavior, he has to give it a specific structure [47]. Therefore, the principle of systematic design wants to put the demand for inter-model consistency between structural and behavioral models in concrete forms. This necessarily requires a comprehensive meta model which includes both types of models. For instance, the principle enforces to use the same information objects in the structural and the behavioral models. 3.2.6 The Principle of Comparability The principle of comparability aims at the semantic comparison of two models, i.e., the content of two models shall be compared regarding its correspondence and similarity. The comparison can be carried out on the level of the model system and on the level of the meta model system. A comparison of two model systems requires the comparison of the problems to be modeled. A comparison on the level of the meta models is only possible, if the different languages and grammars, respectively, are compatible. Therefore, the formulation of this principle demands the possibility to transfer one meta model in a different meta model. For this transfer process, a relationship model for the integration of different meta models needs to be built. The transfer is possible, if each of the model elements can be expressed in the other language. As long as the powerfulness of both methods is similar, they can be transferred completely. The comparison of the semantics in models is much more difficult, since the number of objects to be compared is much higher and the elements used in the meta models obey higher standards than the semantic elements in the model system.

250

R. Schuette and T. Rotthowe

3.3

The GoM-Architecture

The outlined principles are presented as the Guidelines of Modeling. They form the basis of the GoM-Architecture. These general principles are specified in two further dimensions, the structure and the behavior aspect [43]. On this level measures are defined, which - independent from the modeling language – are valid for both types of model systems. The highest degree of specification can be obtained by formulating methodspecific guidelines in which concrete recommendations are given for the application of each method. Therefore, these general and view-specific guidelines can be understood as generic modeling conventions that need to be specified in accordance to the modeling method. In this paper we intentionally described the architecture of the GoM, especially the goals, which should be achieved in modeling projects. In the research project, explicit measures for the modeling in business practice were developed for specific methods, e.g., entity relationship models (ERM), event-driven process chains (EPC), and petri nets. For instance, naming conventions, suffix and standardized patterns were developed in order to improve the clarity and the construction adequacy, business pattern (accounts receivable, supplier) were defined in order to support the comparability. Table 1 shows an example of suffix patterns of the SAP R/3-System [39]. Table 1. Examples for suffix patterns Domain independent suffix ~ division (subdivision by set criteria) ~ hierarchy (combination of objects, where each object has only one superordinate object) ~ classification (combination of objects in a class following a specific characteristic) ~ type (combination of objects in a type following more than one characteristic)

Domain dependent suffix ~ order (request for doing an specific activity) ~ ledger (rule for the depiction of value or quantity flows) ~ account (structure for the depiction of value or quantity flows) ~ ratio (description of a quantitative number

Moreover, measures for different model purposes (multi-objective nodels (workflow management system, information system development, enterprise resource planning software configuration etc.) were built, in order to be able to depict different purposes consistently in one model. Currently, we develop guidelines of modeling for the Unified Modeling Language (UML) which is more and more used in modeling projects. Furthermore, the guidelines can be differentiated for the application on companyspecific models and reference models [41]. For example, reference models exist for industrial corporations [40] and for retail corporations [7]. The structure of the general guidelines is independent from this differentiation, that means that these six

The Guidelines of Modeling - An Approach to Enhance the Quality

251

principles can be utilized in all modeling areas. An overview about the guidelines and their objectives is depicted in Fig. 3.

Principle of Economic Efficiency

„ „ Principle of Construction Adequacy

„ „

„ „

Consensus Language application and comprehensibility Comparability Systematic structure

Principle of Systematic Design

„

Consensus about the problem definition Consensus about the model representation „ Intra-model consistency „ Inter-model consistency within one view „ Minimality

„

Inter-model consistency between structure and behavior models Information systems architectures

Supplementary Principles Necessary Principles

R e fe ren ce m o d e ll (g enera lly ap plicab le ) Principle of Comparability

„

C o m pa n y-sp e cific m o d e l (s itua tio nal)

„ Principle of Language Adequacy

„ „

Language correctness Language suitability „ Semantic powerfulness „ Formalization „ Language comprehensibility (incl. tool support)

Comparability on the meta model level „ complete transfer/translation „ consistent translation Comparability on the model level

Principle of Clarity

„ „ „

Hierarchy Layout design Filtering „ methodical filter „ content filter

Fig. 3. The Guidelines of Modeling – overview

The guidelines can be utilized independently from each other. For instance, the principle of economic efficiency restricts in many cases the principle of construction adequacy. Since the guidelines aim to serve the evaluation of models, first one needs to select the perspectives on the problem area in order to derive the objectives of the model construction. Therefore, differing concentrations of the six guidelines can be carried out in a specific modeling project.

4 Conclusions The construction of models on the requirements definition level builds the foundation for the system design and development. Furthermore, the use of models is extended to the purpose of business process reengineering, customizing of packaged software,

252

R. Schuette and T. Rotthowe

activity based costing, ISO certification and the selection of software. Therefore, the demand for a high model quality is evident. The determining factor for the quality are the modeling subjects (the designers) themselves. We outlined in this paper, how the subjectivity of modeling can be managed through the standardization of the modeling process. Hereby one has to consider the subjectivity of the model designer and the model user. Subjectivity means, that the model designer and the model user have differing thought patterns, which not only determine what one perceives, but as what something should be thought of [9]. Only through thought patterns perceptions are combined to a meaningful whole. Because of the variety of potential thought patterns one can not assume that conceptualizations of different persons are identical (especially if one takes into account differing “Weltanschauungen” (world view) of model designer and model user). The guidelines of modeling support the traceability of the conceptualizations and support their representations in the information model. The GoM go beyond aspects of language applications such as the discussions about the Unified Modeling Language (UML). Whereas UML also covers standards for naming conventions, attempts for the uniformation and definition of modeling do not exist in UML. The Guidelines of modeling – in addition to the standardization efforts – take into account the interrelation between the problem to be modeled and its representation, the economic efficiency of modeling or measures for the improvement of the model clarity.

References 1. Albert, H.: Traktat ueber rationale Praxis. Tuebingen 1978. 2. Albert, H.: Kritische Vernunft und menschliche Praxis. Stuttgart 1977. 3. Batini, C.; Ceri, S.; Navathe, S. B.: Conceptual Database Design. An Entity-RelationshipApproach. Redwood City et al. 1992 4. Batini, C.; Furlani, L.; Nardelli, E.: What is a good diagramm? A pragmatic approach. In: th Proceedings of the 4 International Conference on the Entity-Relationship Approach: The Use of ER Concepts in Knowledge Representation. Berlin et al. 1985, pp. 312-319. 5. Batini, C.; Nardelli, E.; Tamassia, R.: A Layout Algorithm for Data Flow Diagrams. IEEE Transactions on Software Engineering, 12 (1986) 4, pp. 538-546. 6. Batini, C.; Talamo, M.; Tamassia, R.: Computer Aided Layout of Entity Relationship Diagrams. The Journal of Systems and Software, 4 (1984), pp. 163-173. 7. Becker, J.: The Architecture of retail information systems. In: J. Becker, H. L. Grob, U. Müller-Funk, G. Vossen, (Ed.), Working papers of the Department of Information Systems. No. 46. Muenster 1996. 8. Becker, J.; Ehlers, L.; Schuette, R.: Grundsaetze ordnungsmaessiger Modellierung. Konzeption, Vorgehensmodell, informationstechnische Realisierung, Nutzen. In: Statusseminar des BMBF Softwaretechnologie. 23.-24. Maerz 1998. Projekttraeger Informationstechnik des BMBF bei der DLR e.V. im Auftrag des BMBF. U. Grote, G. Wolf (Eds.). Berlin 1998. 9. Bretzke, W.-R.: Der Problembezug von Entscheidungsmodellen. Tuebingen 1980.

The Guidelines of Modeling - An Approach to Enhance the Quality

253

10. Becker, J.; Rosemann, M.; Schuette, R.: Grundsaetze ordnungsmaessiger Modellierung. Wirtschaftsinformatik, 37 (1995) 5, pp. 435-445.11. Becker, J.; Rosemann, M.; Schuette, R.; Rotthowe, T.: A Framework for Efficient Information Modeling - Guidelines for Retail Enterprises. Paper accepted at INFORMS Conference on Information Systems and Technology, Montreal, April 1998. 12. Boehm, B. W.: Software Engineering Economics. Englewood Cliffs 1981. 13. Bostrom, R.P.: Successful Application of Communication Techniques to Improve the Systems Development Process. Information & Management, 16 (1989), pp. 279-295. 14. Bunge, M.: Scientific Research I. The Search for Systems. Berlin et al. 1967. 15. Daneva, M.; Heib, R., Scheer, A.-W.: Benchmarking Business Process Models. In: A.-W. Scheer (Ed.), Working papers of the Institut fuer Wirtschaftsinformatik. Heft 136. Saarbruecken 1996. 16. Elmasri, R.; Navathe, S. B.: Fundamentals of Database Systems. 2nd Ed., Redwood City et al. 1994. 17. Hars, A.: Referenzdatenmodelle. Grundlagen effizienter Datenmodellierung. Wiesbaden 1994. 18. Harwryszkiewycz, T.H.: Introduction to systems analysis and design. New York et al. 1991. 19. Hesse, W.; Barkow, G.; Braun, H.v.; Kittlaus, H.-B.; Scheschonk, G.: Terminologie der Softwaretechnik. Ein Begriffssystem fuer die Analyse und Modellierung von Anwendungssystemen. Teil 2. Informatik Spektrum, 17 (1994) 2, pp. 96-105. 20. Hudson, D. L.: Practical Model Management Using CASE Tools. QED Publishing Group Boston, London, Toronto 1993. 21. Keller, G.; Nuettgens, M.; Scheer, A.-W.: Semantische Prozessmodellierung auf der Basis "Ereignisgesteuerter Prozessketten (EPK)". In: A.-W. Scheer (Ed.), Working papers of the Institut fuer Wirtschaftsinformatik, No. 89. Saarbruecken 1992. 22. Krogstie, J.: Conceptual Modeling for Computerized Information Systems Support in Organizations. PhD Thesis, Trondheim 1995. 23. Krogstie, J.; Lindland, O. I.; Sindre, G.: Defining Quality Aspects for Conceptual Models. In: Proceedings of the International Conference on Information System Concepts (ISCO3). Towards a Consolidation of Views. Marburg 1995. Preprint. 24. Krogstie, J.; Lindland, O. I.; Sindre, G.: Towards a Deeper Understanding of Quality in Requirements Engineering. In: CAiSE `95. J. Iivari, K. Lyytinen, M. Rossi (Eds.). Berlin 1995, pp. 82-95. 25. Krogstie, J.; S lvberg, A.: A Classification of Methodological Frameworks for Computerized Information Systems in Organizations. In: Method Engineering. S. Brinkkemper, K. Lyytinen, R. J. Welke (Eds.). London et al. 1996, pp. 278-295. 26. Lindland, O. I.; Sindre, G.; S lvberg, A.: Understanding Quality in Conceptual Modeling. IEEE Software, 11 (1994) 2, pp. 42-49. 27. Maier, R.: Qualitaet von Datenmodellen. Wiesbaden 1996. 28. Moody, D.: Graphical Entity Relationship Models. Towards a More User Understandable Representation of Data. In: Conceptual Modeling. ER ’96. 15th International Conference on Conceptual Modeling. Cottbus, Germany, October 1996. B. Thalheim (Ed.). Berlin et al. 1996, pp. 227-244. 29. McMenamim, S. M.; Palmer, J. F.: Essential Systems Analysis. New York 1984. 30. Moody, D. L.; Shanks, S.: What Makes a Good Data Model? Evaluating the Quality of Entity Relationship Models. In: Entity-Relationship Approach - ER `94. Business Modelling and Re-Engineering. P. Loucopoulos (Ed.). Berlin et al.1994, pp. 94-111.

‡

‡

254

R. Schuette and T. Rotthowe

31. Oliver, I.; Langford, H.: Myths of Demons and Users. Evidence and Analysis of Negative Perceptions of Users. In: Information Analysis. Selected Readings. Ed. by R. Galliers. Addison-Wessly Publishing, Sydney et al. 1987, S. 113-123. 32. Ortner, E.; Schienmann, B.: Normative Language Approach. A Framework for Understanding. In: Conceptual Modeling. ER ’96. 15th International Conference on Conceptual Modeling. Cottbus, Germany, October 1996. B. Thalheim (Ed.). Berlin et al. 1996, pp. 261-276. 33. Pohl, K.: Process-Centered Requirements Engineering. Taunton, Somerset 1996. th 34. Pohl, K.: The Three Dimension of Requirements Engineering. In: Proceedings of the 5 International Conference on Advanced Information Systems Engineering-CAiSE `93. C. Rolland, F. Bodart, C. Cauvet (Eds.). Berlin et al. 1993, pp. 275-292. th 35. Popper, K. R.: Objective Knowledge. 4 Ed., Oxford 1984. 36. Rauh, O.: Guetekriterien fuer die semantische Datenmodellierung. HMD, 28 (1991) 158, pp. 91-110. 37. Rescher, N.: Pluralism. Against the Demand for Consensus. Clarendon Press, Oxford 1993. 38. Rescher, N.: Objectivity. The Obligations of Impersonal Reason. Notre Dame, London 1997. 39. SAP: R/3 System. (http:///www.sap-ag.de/products/index.htm, 14. July 1998) 40. Scheer, A.-W.: Business Process Reengineering. Reference Models for Industrial Enterprises. 2nd ed. Berlin et al. 1994. 41. Schuette, R.: Handelsinformationssysteme. Unpublished presentation, March 14, 1994. Muenster 1994. 42. Schuette, R.: Grundsaetze ordnungsmaessiger Referenzmodellierung. Dissertation, Universitaet Muenster. Muenster 1997 (to be published in Gabler Verlag, Reihe neue betriebswirtschaftliche forschung (nbf), Wiesbaden 1998). 43. Schuette, R.: Grundsaetze ordnungsmäßiger Modellierung. Reformulierung eines Ansatzes zur Informationsmodellqualität. Paper presented at the research colloquium at the Institute for Computer Sciences of the University of Koblenz. Koblenz, 04. Dec 1997. (http://www.wi.uni-muenster.de/is/mitarbeiter/isresc). nd 44. Shepherd, G. M.: Neurobiology. 2 Ed. Oxford University Press 1989. 45. Seltveit, A. H.: Complexity Reduction in Information Systems Modeling. PhD thesis University of Trondheim. Trondheim 1994. 46. Tamassia, D.; Di Battisti, G.; Batini, C.: Automatic graph drawing and readibility of diagrams. IEEE Transactions on Systems, Man and Cybernetics, 18 (1988) 1, S. 61-79. 47. Ulrich, H.: Die Unternehmung als produktives und soziales System. Bern 1968. 48. Vollmer, G.: Evolutionaere Erkenntnistheorie. 6. Ed., Stuttgart 1994. 49. Zamperoni, A.; Loehr-Richter, P.: Enhancing the Quality of Conceptual Database Specifications through Validation. In: Entity-Relationship-Approach - ER´93. R. Elmasri, V. Kouramajian, B. Thalheim (Eds.). Berlin et al. 1994, pp. 85-98. 50. Zelewski, S.: Petrinetzbasierte Modellierung komplexer Produktionssysteme. Band 2. Bezugsrahmen. Working paper no. 6 of Institut für Produktionswirtschaft und Industrielle Informationswirtschaft. Leipzig 1995. 51. Zentes, J.: Die Optimalkomplexion von Entscheidungsmodellen. Ein Beitrag zur betriebswirtschaftlichen Meta-Entscheidungstheorie. Köln et al. 1976.

Improving the Quality of Entity Relationship Models—Experience in Research and Practice Daniel L. Moody1, Graeme G. Shanks2, and Peta Darke2 1 Simsion Bowles and Associates, 1 Collins St., Melbourne, Australia 3000. email: [email protected] 2

School of Information Management and Systems, Monash University, P.O. Box 197, Caulfield East, Melbourne, Australia 3145. email: [email protected] [email protected]

Abstract. This paper is an extension of previous research which developed a framework for evaluating and improving the quality of Entity Relationship models. The framework has now been used extensively in research and practice, including application in two of the largest commercial organisations in Australia. The experiences gained have been used to further develop and refine the framework. This paper describes how the framework has been used to: (a) quality assure data models as part of application development projects (product quality); (b) reengineer application development procedures to build quality into the data modelling process (process quality); (c) provide automated support for the evaluation process (Data Model Quality Advisor); (d) investigate the differences between data models produced by expert and novice data modellers. The results show that use of the framework has the potential to significantly improve research, practice and teaching of data modelling.

1.

Introduction

Difficulties and inadequacies in requirements analysis have been identified as a major factor in information systems failures (Boehm, 1981). Industry studies show that more than half the errors that occur during application development are the result of incomplete or inaccurate requirements analysis (Martin, 1989). The traditional thrust of software quality assurance has been to use “brute force” testing at the end of development (van Vliet, 1993). However Total Quality Management (TQM) approaches suggest that it is faster and cheaper to concentrate effort during the early development phases of a product, in order to detect and correct defects as early as possible (Deming, 1986). Studies in practice have shown that quality assurance during the early phases of systems development can be 33 times more cost effective than testing done at the end (Walrad and Moss, 1993). A key activity during requirements analysis is the development of the conceptual or logical data model. Although data modelling represents only a very small proportion of the overall development effort, it has a profound impact on the final result. The

T.W. Ling, S. Ram, and M.L. Lee (Eds.): ER’98, LNCS 1507, pp. 255−276, 1998.  Springer-Verlag Berlin Heidelberg 1998

256

D.L. Moody, G.G. Shanks, and P. Darke

data model is a major determinant of system development costs (ASMA, 1996), flexibility (Gartner, 1992), integration with other systems (Moody and Simsion, 1995) and ability to meet user requirements (Batini et al., 1992). Improving the quality of data models will therefore be a major step towards improving the quality of systems development. In practice, the quality of data models is evaluated in an ad hoc way, based on common sense and experience (Moody and Shanks, 1994). There are few guidelines for evaluating the quality of data models, and little agreement even among experts as to what makes a “good” data model. As a result, the quality of data models produced in practice is almost entirely dependent on the competence of the data modeller.

Product vs. Process Quality In the Total Quality Management (TQM) literature, the distinction is frequently made between product and process quality: • Product quality focuses on the characteristics of the product itself. The approach is to carry out inspections of the finished product and look for defects, and then to correct them. • Process quality focuses on the characteristics of the process used to build the product. The focus of process quality is on defect prevention rather than detection, and aims to reduce reliance on mass inspections as a way of achieving quality (Deming, 1986). In the context of data modelling, product quality relates to the characteristics of the data model itself (the product), while process quality relates to how data models are developed (see Fig. 1).

D a t a M o d e llin g ( p r o d u c t io n p ro c e s s )

P ro c e s s Q u a lit y

r e s u lt s in S eg m en t P ro d u c t

D a ta M o d e l ( p ro d u c t)

O r g a n i s a ti o n a l U nit

M a r ke t S eg m en t

M a r ke t S eg m en t V alu e

M a r ke t S eg m en t V ar ia b le

b e l o n g s to

S ta f f M e m b e r

C u s to m e r

C u s to m e r A c c ou n t R e l a ti o n s h i p

A c c ou n t

C u s to m C on d it ion s

P ro d u c t

P ro d u c t Q u a lit y

Fig. 1. Product vs Process Quality

Outline of the Paper The structure of the paper is: • Section 2 reviews previous research into data model quality • Section 3 describes the quality evaluation framework used as the theoretical basis for this research • Section 4 describes how the framework has been used to evaluate the quality of an application data model as part of an action research study (product quality) • Section 5 describes how the framework has been used to re-engineer the process of data modelling in an organisation as part of an ongoing action research study (process quality) • Section 6 describes how the framework has been used to provide automated support for the evaluation process (the Data Model Quality Advisor), and analyses its effectiveness using a laboratory experiment • Section 7 describes how the framework has been used to analyse differences between expert and novice data modellers as part of a laboratory experiment.

Improving the Quality of Entity Relationship Models

2.

257

Previous Research

Most of the research on data model quality to date has focused on product quality. Quality is frequently defined as a list of desirable properties of a data model. These properties are usually developed based on experience in practice, intuitive analysis and reviews of relevant literature. Although these lists provide a useful starting point for understanding and improving quality in data modelling, they are mostly unstructured, use imprecise definitions, often overlap, and properties of models are often confused with language and method properties (Lindland, Sindre and Solveberg, 1994). In many cases, they are simply the expert opinion of the author. A more comprehensive approach to quality is to define frameworks that organise and structure the key concepts and features in data modelling. Moody and Shanks (1994) propose a framework comprising a number of interrelated concepts for the evaluation and improvement of data models: quality factors, quality measures, stakeholders, weightings and improvement strategies. This framework is readily usable by practitioners (see section 3 for a detailed discussion of this framework). Krogstie, Lindland and Sindre (1995) and Lindland, Sindre and Solveberg (1994) define a framework based in semiotic theory which defines a conceptual model as a set of statements made in a language. For each semiotic level (syntactic, semantic, pragmatic) the framework defines quality goals and the means to achieve them. Finally, Kesh (1995) develops a framework for evaluating data models based on ontology. This framework defines quality criteria and metrics for evaluating the quality of data models. Although these last three frameworks are soundly based in theory, they are at a high level of abstraction and not readily applicable in practice (Shanks and Darke, 1997). A summary of approaches to quality in data modelling is shown in Table 1 below. Table 1. Approaches to Quality in Data Modelling APPROACH

FEATURES

TYPE

von Halle (1991)

Features of data models which maximise value to the organisation

List

Batini et al. (1992)

Quality features of a good schema, schema transformations

List

Simsion (1994)

Design and evaluation of alternative ER models

List

Levitin and Redman (1994)

Quality dimensions, reinforcements and tradeoffs

List

Moody and Shanks (1994)

Defines a comprehensive framework for evaluating and improving the quality of data models

Framework

Lindland et al. (1994)

Based in semiotic theory, separation of quality goals from means

Framework (semiotics)

Kesh (1995)

Separates ontology from behaviour. Defines metrics for evaluating quality.

Framework (ontology)

Krogstie et al. (1995)

Extends Lindland et al. Framework with agreement goal and social construction theory

Framework (semiotics)

258

D.L. Moody, G.G. Shanks, and P. Darke

The two most serious deficiencies in the existing literature are: • None of the approaches has been applied systematically in practice • None of the approaches adequately address the question of process quality Both of these issues are addressed in this paper.

3.

A Framework for Evaluating and Improving the Quality of Data Models

The framework for data model quality used in this research is defined by the Entity Relationship model shown in Fig. 2 (Moody and Shanks, 1998). An earlier version of the framework was published in Moody and Shanks, 1994. The framework consists of five major constructs (each shown as entities in Fig. 2):

Stakeholder

interacts with

concerned with

Weighting defines importance of

Quality Factor

Quality Metric evaluated by

used to improve

Improvement Strategy

Fig. 2. Data Model Quality Evaluation Framework

• Quality factors are the characteristics of a data model that determine its quality. These answer the question: “What makes a good data model?”. A particular quality factor may have positive or negative interactions with other quality factors—these represent the trade-offs implicit in the modelling process. • Stakeholders are people who are involved in building or using the data model, and therefore have an interest in its quality. Different stakeholders will generally be interested in different quality factors. • Quality metrics define ways of evaluating particular quality factors. There may be multiple measures for each quality factor. • Weightings define the relative importance of different quality factors in a problem situation. These are used to make trade-offs between different quality factors. • Improvement strategies are techniques for improving the quality of data models with respect to one or more quality factors.

Improving the Quality of Entity Relationship Models

259

Stakeholders The key stakeholders in the data modelling process are: • The business user, whose information requirements are represented by the data model • The analyst, who is responsible for developing the data model • The data administrator, who is responsible for ensuring that the data model is consistent with the rest of the organisation’s data • The application developer, who is responsible for implementing the data model (translating it into a physical database schema)

Quality Factors The proposed quality factors and the primary stakeholders involved in evaluating them are shown in Fig. 3. These quality factors may be used as criteria for evaluating the quality of individual data models or comparing alternative representations of requirements. Together these quality factors incorporate the needs of all stakeholders, and represent a complete picture of data model quality. Business User

Completeness

Data Analyst

Simplicity

Business User

Flexibility

DATA MODEL QUALITY

Integration

Data Administrator

Understandability

Business User

Implementability

Application Developer

Fig. 3. Data Model Quality Factors

A range of evaluation methods and metrics have been identified for evaluating each of the quality factors (Moody and Shanks, 1998; Moody, 1998).

4.

Action Research Study: Improving Product Quality

This section describes the use of the quality framework in evaluating a data model developed as part of an application development project. In this case, the framework is used to improve product quality.

Action Research It is essential for ideas, principles and methods developed in a research environment to be tested in practice. Ultimately, the scientific merit of any method is an empirical

260

D.L. Moody, G.G. Shanks, and P. Darke

rather than a theoretical question (Ivari, 1986). As Rescher (1977) says, “there is no better way of justifying a method than by establishing that it works”. A major barrier to empirical validation of methods is that it is very difficult to get new approaches, especially those developed in academic environments, accepted and used in practice. Practitioners who have developed familiarity and expertise with existing techniques are reluctant to adopt academic approaches which are theoretically sound but unproven in practice. As Bubenko (1986) says: "Among the large number of methodologies published, the great majority are developed in academic environments, but only a very small fraction of these have ever been applied to cases of realistic size and complexity. The acceptance of academic methods in practice is low, and in general, the rate of transfer of research results and know how from research to industry is embarrassingly slow."

This section and the next describe how action research has been used to bridge the gap between theory and practice in our research into data model quality. Action research (Checkland and Scholes, 1990) is a research method in which practitioners and researchers work together to test and refine principles, tools, techniques and methodologies that have been developed to address real world problems. It provides the ability to test out new methods in practice for mutual benefit of researchers and practitioners. Action research is often confused with case study research. However case study research examines phenomena in its natural setting, with the researcher an independent observer (Benbasat, Goldstein and Mead, 1987). In action research, the researcher is an active participant. The action research cycle is illustrated in Fig. 4.

1. Develop method

3. Refine method

2. Apply method in practice

A major advantage of action research is that it helps overcome the problem of persuading practitioners to adopt new techniques, and overcome the cultural Fig. 4. The Action Research Cycle divide that exists between information systems academics and practitioners (Avison, 1991). Using this approach, the quality evaluation framework proposed has now been applied extensively in practice. This paper describes its use in two of the largest commercial organisations in Australia (Sects. 4 and 5). Each study is structured in the following way: • Background to the situation: organisational context/problem situation • The Intervention: how the framework was applied • Benefits in Practice: how use of the framework improved performance of the task • Learning about the Method: what was learned about the method and how it was refined as a result

Improving the Quality of Entity Relationship Models

261

Background to the Situation The organisation involved in this study was a telecommunications company and one of the largest commercial organisations in Australia. The organisation had embarked on a major application development project to replace the core system in their national telemarketing centre. This project was regarded as mission critical, and as a result, an independent quality review was commissioned at the end of the requirements analysis stage to confirm that the data model would meet current and future business requirements. The consultancy was offered as part of a competitive quotation process.

The Intervention The framework described in Sect. 3 was used to conduct a quality review of the data model. The review was conducted over ten working days (elapsed time one month), and involved review sessions with the project team, inspection of the model, interviews with business representatives, application development staff and Corporate Data Management staff. A report was produced with separate sections discussing findings and recommendations for each quality factor. Over a hundred individual quality issues were identified as a result of the review process. A number of recommendations and an action plan was developed for improving the model prior to proceeding to the design phase.

Benefits in Practice The major benefits found as a result of using the evaluation framework were:

Systematic Approach The framework was found to be an excellent tool for organising and structuring the review process. Use of formal evaluation criteria, stakeholders and evaluation methods helped the reviewer to focus the task and carry out a comprehensive analysis in a very limited timeframe. Use of a pre-defined and published framework also gave the review more credibility in the client’s eyes—it provided evidence of a “scientific” approach rather than relying on experience of reviewers alone. This proved to be the deciding factor in winning the contract in the first place—none of the other bidders had a formally defined quality assurance framework.

Global Perspective of Quality The framework forced the discipline of considering the full range of quality factors. In the absence of the framework, the review would have been much more ad hoc, and many issues would have been overlooked. Interestingly, the quality factor which turned out to be the most critical in the context of this study, integration, is not included in any other framework. Most approaches to data model quality consider data models as if they were standalone products, whereas in practice, they generally form a small part of a much bigger picture—the organisation’s total information resource.

262

D.L. Moody, G.G. Shanks, and P. Darke

Development Cost Savings A number of major flaws in the data model were identified by the review which resulted in significant cost savings for the project (and the organisation): • A number of important user requirements were identified that were missing from the model (completeness). The cost of discovering and adding these in later stages would have been many times higher—according to Boehm (1981), the cost of adding to or changing requirements after the analysis stage is on average 3.5 times more during design, 50 times more at the testing stage, and 170 times more after delivery. • A number of requirements were identified in the model that were not required by users (completeness). Removing these requirements reduced the development effort required. • A high level of overlap was found between the data model and existing systems (integration)—removal of this data reduced the size of the model by almost half. This dramatically reduced development effort and avoided ongoing business costs of data duplication (data entry, storage, maintenance and reconciliation). • A number of inconsistencies were identified between data definitions in the data model and in source systems (integration). Resolving these differences significantly reduced data translation effort and the cost of building interfaces.

Learning About the Method Validation of Quality Factors There are two questions that need to be addressed in validating the proposed set of quality factors: 1. Completeness: how can we be sure that we haven’t left something out of our list of quality factors? Is the set of quality factors sufficient for evaluating the quality of data models? 2. Relevance: how do we know that the quality factors are relevant to the problem? Are all the quality factors necessary for evaluating the quality of data models? In terms of completeness, the review process revealed that the framework was missing an important quality factor, because a number of issues were identified which could not be classified under any of the existing quality factors. As a result, a new quality factor, correctness, was incorporated in the framework. Correctness was defined as whether the model conforms to the rules of the data modelling technique (i.e. whether it is a valid data model). This includes diagramming conventions, naming rules, definition rules, rules of composition and non-redundancy. The set of quality factors passed the relevance test because at least one quality issue was identified for each quality factor. There was also general agreement among the project team that all of the quality factors were important.

Improving the Quality of Entity Relationship Models

263

Importance of Process Quality Interestingly, while the review was focused on issues of product quality, most of the flaws in the model could be traced back to problems in the process used to develop it rather than skill deficiencies in the project team. The data modellers involved were very technically competent (the model scored a perfect “5” on correctness), however their major failing was in not involving the other stakeholders in the process. This was the primary cause of the problems in the areas where the model was found to be most deficient: completeness, flexibility, integration and implementability. For this reason, a major recommendation of the review was to change the data modelling process to explicitly involve all stakeholders—this was identified as the single most critical success factor in the project. This underlined the importance of stakeholders as a component of the framework. This study identified a major weakness in the framework—while it addresses product quality (how to evaluate the quality of a completed data model or compare alternatives), it says very little about process quality (how to develop high quality data models). The next study addresses this issue.

5.

Action Research Study: Improving Process Quality

This section describes how the evaluation framework has been used to improve the process of building data models as part of a longitudinal action research study in an organisation. One of the major precepts of Total Quality Management (TQM) is that the effective way to improve the quality of a product is to improve the process by which it is developed (Deming, 1986). The objective should be to build quality into the process rather than trying to add it in at the end through inspections of the finished product.

Background to the Situation The organisation involved in this study was a large Australian financial institution, with over 25,000 staff. A central Data Administration group was responsible for developing and/or reviewing (depending on availability of resources) data models produced as part of application development projects. The group was also responsible for overall data coordination, the corporate data repository and data modelling standards and data naming standards. The organisation had a standard systems development methodology consisting of five major stages: Business Case Analysis, Requirements Definition, Logical Design, Physical Design and Implementation. The central Data Administration group was required to “sign off” all data models at the end of the Requirements Definition stage (see Fig. 5). Signoff was required before application development projects could progress to the Logical Design stage. This is a fairly typical arrangement in practice (Horrocks and Moss, 1994; Moody, 1996). A number of problems were found with this process:

264

D.L. Moody, G.G. Shanks, and P. Darke

Duplicate Definitions of Data By far the most serious problem identified was that different project teams were defining the same data over and over again in different ways, resulting in data redundancy and unnecessary development costs. This was happening despite the fact that the organisation had implemented a central corporate data repository and developed data naming standards specifically to address this issue. An audit of the corporate repository found that a single attribute (Branch Code) had been defined over sixty times.

Inconsistent Quality of Models

Business Case Analysis

Requirements Definition Data Model Signoff

Logical Design

Database Design Signoff

Independent inspections of a number of data models produced as part of application Physical Design development projects found huge variations in quality. The reason for this was that the central Data Administration group had a small number of expert data modellers and a large number of relatively inexperienced ones Implementation (less than two years experience). This is quite a common situation in practice. While the policy was to have experienced data Fig. 5. Existing Quality Assurance modellers paired with inexperienced ones, Process lack of resources often prevented this. Where data models were developed by nonData Administration staff, the situation was even worse—sometimes analysts who developed data models had no prior data modelling experience at all.

Need for Rework In a number of cases, project teams submitted data models for Data Administration signoff at the end of the Requirements Definition stage, only for major quality problems to be found. This led to significant rework, missed deadlines, added cost for projects and friction between the Data Administration group and project teams. A major part of the problem was that project teams did not realise what the signoff involved—most thought that it was simply a check for conformance to data modelling standards.

The Intervention The data model evaluation framework described in Sect. 3 was used as the basis for developing quality assurance procedures for data models and incorporated into the systems development methodology. The objective was to build quality into the requirements definition process (process quality), rather than just using the framework

Improving the Quality of Entity Relationship Models

265

to evaluate quality of data models at the end (product quality). The major components of the new quality assurance process were:

Preventative Reviews Traditional quality systems take a “big bang” approach to quality assurance, with an inspection carried out at the end of each phase to ensure conformance to specifications. This is the situation in the current quality assurance process (Fig. 5), and resulted in a “no win” situation for both parties: projects were often delayed and the Data Administration group was blamed for it. The concept of preventative reviews in TQM suggests that reviews should be carried out iteratively at specific checkpoints in the development process. By having regular reviews, the model never gets too far away from where it should be, thus minimising rework at the end. It also allows the reviewer to have input into the model while it is being developed, rather than simply pointing out errors after the process is complete. Three review checkpoints were defined as part of the requirements definition phase (see Fig. 6): 1. Preview: This review takes place Business Case before any detailed analysis has Analysis been carried out, but after the scope of the project has been dePreview (integration) fined. The objective is to identify opportunities for reuse and sharing Midpoint Requirements Review of data. The primary quality factor Definition (alternatives) considered at this stage is inteSignoff gration. This is also when weight(correctness) ings are defined for each quality factor. Approval Logical Design 2. Mid-point review: this review is of variations carried out at the midpoint of the Signoff requirements definition phase, (defects) when all requirements gathering activities (interviews, workshops Physical Design etc.) have been completed and a first cut model has been produced. The objective of this review is to explore alternative solutions, using the first cut model as a starting Implementation point. The quality factors considered in this review are completeness, flexibility, underFig. 6. Revised Quality Assurance Process standability, simplicity and implementability. Errors of detail (correctness) are ignored at this stage.

266

D.L. Moody, G.G. Shanks, and P. Darke

3. Final review (signoff): this is the stage of final approval, and if the earlier reviews have been carried out, should be more or less a formality. The major focus of this review is correctness. The participants involved in each review were the project manager, the data analyst who developed the model, business representative(s), the information architect, application developer(s) and a data analyst from outside the project. The additional data analyst was chosen on the basis of specialist knowledge in the area, level of experience or because they were working on a related project. Each review participant was required to independently inspect the data model prior to the review and score it according to the quality criteria. The review session then served as a forum for reaching agreement on evaluations, looking for ways to improve the model and exploring alternatives. In this way, all stakeholders were actively involved in developing the data model (as opposed to a passive review role).

Information Architect Role A major problem in the existing situation was the lack of coordination between separately developed data models. Even though the organisation had a corporate-wide repository, this was not enough to ensure consistency of data definitions in practice— this underlines the fact that data sharing is a people issue rather than a technical issue (Davenport, 1994). Reuse of data took place largely as a matter of chance because individual data analysts were not aware that data had been modelled before unless they had worked on that project themselves. To address this problem, a new position was created within the Data Administration group (Information Architect), with responsibility for overall coordination of all data models developed across the organisation. The Information Architect was not involved in developing data models, only in reviewing them for consistency and overlap (integration).

Data Model Quality Guidelines A set of data model quality guidelines were developed based on the quality factors defined in Sect. 3. The quality factors and review process were summarised on one page to form a “Quick Reference Sheet”, and were given out to project teams in advance, so they were aware of what would be covered in each review.

Benefits in Practice The quality assurance process described above was used to quality assure over forty data models over a two year period. The major benefits identified were:

Improved Reuse of Data By far the most useful change to the quality assurance process was the “preview” review. While it seems unusual (even contradictory) to have a quality review before any work has been done, this was where most benefits were found. By meeting with project teams before they had begun analysis, the Data Administration group was able to save them work (rather than creating extra work as in the existing process) by providing corporate data standards and identifying opportunities for reuse of data.

Improving the Quality of Entity Relationship Models

267

Reduced Requirements Errors Errors or omissions in the data model lead to change requests in later development stages. Change requests result in rework and added cost for projects and were a major source of project overruns and missed deadlines in the organisation. Data model change requests submitted after the end of the Requirements Definition stage were reduced by almost 30% over the two year period. The reduction in change requests translated into millions of dollars in development cost savings for individual projects and significant development productivity improvement across the organisation.

Exploration of Alternatives Use of the framework helped to change the focus of reviews from just looking for errors (negative quality) to improving models (positive quality) and exploring alternative solutions. In the existing process, it was too late to explore alternatives by the time the model was reviewed. The involvement of review participants external to the project team also helped to introduce different points of views and in some cases resulted in innovative solutions to problems.

Improved Practices The framework led to improved work practices within the Data Administration group: • The quality and consistency of data models produced was significantly improved. Instead of the quality of models being dependent entirely on the expertise of the data analyst responsible, at least two experienced data modellers (the Information Architect and one other) were involved in developing each model. This resulted in better leverage of more experienced resources. • The review process allowed inexperienced data modellers to learn from the more experienced members of the group (skills transfer) • Use of a common evaluation framework led to better consensus within the group as what constituted “good practice” rather than everyone having their own modelling style.

Lessons Learned A great deal was learned about the application of the framework as part of this study. Some of the more important lessons learned were:

Extension of the Framework The most important result of this study was the extension of the framework to include the process dimension. Previously, the framework focused exclusively on product quality—how to evaluate the quality of a completed model. As a result of this study, the framework was augmented to incorporate processes for building quality data models. This required the addition of a new construct, Review Checkpoint, which defines: • Who (stakeholders involved in the review) • What (quality factors considered in the review) • When (where in the development methodology each review takes place).

268

D.L. Moody, G.G. Shanks, and P. Darke

The revised evaluation framework is shown in Fig. 7: involves

Review Checkpoint

Stakeholder

considers

concerned with interacts with

Weighting defines importance of

Quality Factor

evaluated by

Quality Metric

used to improve

Improvement Strategy

Fig. 7. Revised Data Model Quality Evaluation Framework

Validation of Quality Factors Two new quality factors were suggested for inclusion in the framework as part of this study. Both of these involved splitting existing quality factors into two: • Data redundancy, while included as part of correctness in the framework, could be considered as a factor in its own right because it is such a fundamental requirement of data modelling. A great deal of effort is spent in practice ensuring that the model contains no redundancy (eg. using normalisation). However it was decided to keep this as part of correctness, because it can be considered as part of the “rules” of data modelling (whether a model is valid or not). • Definition of business rules (or integrity constraints), while included as part of completeness, could be considered as a separate factor, integrity. Business rules are particularly important in a banking environment, because of the need to guarantee data integrity and enforce procedural rules. Also, there is increasing emphasis in practice on the use of business rules as a requirements gathering technique in its own right (von Halle and Conkey, 1997; Ross, 1996). As a result, it was decided to extend the framework to include integrity as a separate factor. In terms of the 100% principle (ISO, 1987), this separates static aspects (completeness) and dynamic aspects (integrity). The final set of quality factors, together with the primary stakeholder responsible for each, is shown in Fig. 8. The number of quality factors increased from six to eight as a result of the two action research studies (Correctness as part of the first study and Integrity as part of the second study), which is still conceptually manageable. The resulting quality factors can be divided into “business dimensions” or “technical dimensions” based on whether they are evaluated by business or technical experts.

Improving the Quality of Entity Relationship Models

269

Business Dimensions of Quality Business User

Completeness

Business User

Business User

Integrity

Flexibility

Business User

Understandability

DATA MODEL QUALITY

Correctness

Data Analyst

Simplicity

Integration

Data Analyst

Data Administrator

Implementability

Application Developer

Technical Dimensions of Quality Fig. 8. Revised Quality Factors

How Much Quality Is Enough? Since all projects have time and budgetary constraints, an important issue is knowing when to stop. Previous studies on the economics of quality assurance in the software development process have found diminishing returns as quality assurance expenditures extend beyond 10—15% of the overall development effort (Boehm, 1981; Abdel-Hamid, 1988). However Total Quality Management recommends that substantially more quality assurance effort should be spent in early development phases to detect and correct defects as early as possible in the product lifecycle (Deming, 1986). For this reason, 20—25% was used as the level for quality assuring data models. This was originally perceived by many people to be excessive, but the additional review costs were justified many times over by the reduction in requirements errors and change requests over the development lifecycle.

Evaluation of Completeness In practice, user signoff is often considered to be the de facto measure for completeness. That is, if users agree that the data model meets their requirements, then it is, by definition, complete. This was found to be almost universally untrue in practice. One problem is that users generally do not understand data models (Hitchman, 1995; Shanks, 1997). Another problem is that they are often not sure what their requirements are. The only way to ensure completeness in practice is by rigorous testing—mapping against functional requirements, use of test cases or scenarios and use of prototypes. A key requirement is to get business users to test out the model in practice rather than passively reviewing it—this tests whether the model is understood rather than whether it is understandable (Lindland et al., 1994).

270

6.

D.L. Moody, G.G. Shanks, and P. Darke

Automated Support for the Evaluation Framework

This section describes how the evaluation framework has been used as the basis for developing an automated tool to support the process of data model quality evaluation.

The Data Model Quality Advisor The framework described in Sect. 3 has been incorporated into an expert assistant tool, the Data Model Quality Advisor (DMQA), which is currently implemented as a prototype in Visual Basic and Microsoft Access (Shanks and Darke, 1997). The DMQA acts as an advisor in the evaluation of conceptual data models, and has been designed to assist both novice and expert data modellers. It provides a hypertext explanation facility for the constructs of the quality evaluation framework, and supports the evaluation and comparison of up to three data models. The hypertext explanation facility provides a graphical view of the framework as a user interface. Explanations and examples of any of the framework constructs can be viewed by selecting the appropriate icon within the graphical model. The user is able to navigate amongst constructs of the framework using the hypertext links provided. Users can also access the framework via the meta-model of the quality evaluation framework (shown earlier in Fig. 2) as an alternative user interface. The evaluation and comparison facility of the DMQA supports the allocation of weightings for each quality factor by different stakeholders. Ratings for each quality factor for up to three alternative data models may then be entered by each stakeholder. These are stored for subsequent comparison. The DMQA also provides advice on alternative ways of evaluating the quality factors. After all stakeholders' ratings have been entered, a summary of their evaluations with rankings for the alternative models can be displayed. The user may seek explanation of any component of the framework during evaluation and comparison using the explanation facility.

Experimental Evaluation A laboratory experiment was conducted to examine the usefulness and useability of the quality evaluation framework and the DMQA in providing support for the evaluation of data models. The study involved twenty experienced data modelling practitioners and academics. The participants in the study were each asked to use the DMQA to learn about the quality evaluation framework and to assist them in evaluating three alternative data models for a small case example. Each participant then completed a questionnaire about the evaluation framework and about their use of the DMQA to evaluate the three data models. Participants were asked to indicate the extent to which they agreed or disagreed with each statement using a 5-point Likert scale (1 "strongly agree" to 5 "strongly disagree"). The results obtained in the study are summarised in Table 2 below. Most of the responses were positive, falling in the “agree” range (1.5—2.5), with question 7 falling in the “disagree” range (3.5—4.5) and question 3 undecided.

Improving the Quality of Entity Relationship Models

271

Table 2. Summary of Questionnaire Statements and Participant Responses Statement

Rating

T-value

P-value

Overall

1.

Evaluating the quality of a conceptual data model is critical to the successful development of an information system

1.55 (0.60)

t(18)=-10.808

<0.01

Agree

2.

It is important to consider alternative conceptual data models within systems development

1.70 (0.73)

t(18)=-7.964

<0.01

Agree

3.

A framework for evaluating the quality of conceptual data models would constrain the way practitioners prefer to work

3.15 (0.81)

t(18)=0.828

not significant

Undecided

4.

In practice, conceptual data models are evaluated in an ad hoc way based on personal opinion

1.80 (0.70)

t(18)=-76.665

<0.01

Agree

5.

The tool is useful when evaluating and comparing conceptual data models

2.25 (0.78)

t(18)=--4.300

<0.01

Agree

6.

The ratings summary screen helps to compare conceptual data models

1.80 (0.83)

t(18)=--6.466

<0.01

Agree

7.

The tool is irrelevant to understanding and using the quality framework

3.55 (.083)

t(18)=-2.963

<0.05

Disagree

The empirical study indicated strong support for the need to evaluate the quality of conceptual data models and for the use of a quality framework to support the evaluation of conceptual data models in practice. The DMQA was considered to be useful in explaining the quality evaluation framework and its constructs, and in the evaluation and comparison of conceptual data models.

7.

Use of the Framework as a Research Tool

This section describes how the framework has been used as an experimental tool for investigating the differences between expert and novice data modellers.

Previous Research A number of laboratory based empirical studies of conceptual modelling have compared data modelling formalisms and contrasted expert and novice performance (see for example Shoval and Even-Chaime, 1987; Batra et al. 1990; Batra and Davis, 1992; Maiden and Sutcliffe, 1992; Kim and March, 1995). Although these studies use the quality of the model as a basis for comparison, most of them define quality in terms of one dimension only, i.e. completeness. This is not meant as a criticism of these studies, but reflects the lack of a comprehensive quality framework for evaluating data models.

272

D.L. Moody, G.G. Shanks, and P. Darke

Shanks (1997) used five of the seven quality factors from the Moody and Shanks (1994) framework to contrast expert and novice performance in an experimental study of data modelling practitioners. Each participant in the experiment developed an Entity Relationship model using a narrative case study transcribed from an interview with a domain expert. The models were evaluated by three independent raters using the following quality factors: • Correctness was evaluated in terms of the number of errors in the use of the Entity Relationship technique • Completeness was evaluated in terms of number of requirements missing from the model. This was expressed as a percentage of the total user requirements. • Simplicity was evaluated in terms of the number of entities and relationships in the model • Flexibility was evaluated using a 7 point Likert scale • Understandability was evaluated using a 7 point Likert scale • Overall quality was evaluated using a 7 point Likert scale

Experimental Results Significant differences were found between the models designed by experts from those designed by novices. Table 3 below shows a summary of the results (integration and implementability were not evaluated as part of the study). Table 3. Summary of Results Quality Factor

Expert (mean/SD)

Novice (mean/SD)

Inter-rater Reliability (α)

T-value

P-value

84.33 (17.24)

71.52 (17.47)

0.97

t(37)=2.29

0.0137

Completeness

72.89 (14.04)

56.00 (13.52)

0.92

t(37)=3.88

0.0002

Simplicity

32.06 (18.43)

23.52 (7.80)

0.98

t(37)=1.93

0.0305

Flexibility

4.18 (0.81)

3.32 (0.75)

0.82

t(37)=3.44

0.0007

Correctness

Understandability Overall Quality

3.96 (1.03)

3.53 (0.77)

0.73

t(37)=1.50

0.0705

58.28 (13.97)

47.33 (1.177)

0.87

t(37)=2.66

0.0058

Differences in Product Quality The study found that data models designed by experts were more correct, complete, complex, flexible, and better understood than models built by novices. The greatest differences were found in completeness and flexibility which is consistent with the findings of the previous studies in data modelling process. A surprising finding was that experts also produced models that were more complex, which was contrary to expectations. However this can be explained by the fact that data models produced by novices were much less complete—completeness thus acts as a confounding variable

Improving the Quality of Entity Relationship Models

273

in this result. It is expected that given the same level of completeness, experts would produce simpler models.

Interactions between Quality Factors A correlation analysis was conducted to determine any interrelationship between the quality factors using the Pearson product moment correlation. Strong positive correlations were found between: • Correctness and completeness (.643, p<.001) • Correctness and understandability (.657, p<.001) • Correctness and flexibility (.650, p<.001) • Completeness and understandability (.442, p<.01) • Completeness and flexibility (.690, p<.001) A strong negative correlation was found between completeness and simplicity (.532, p<.01). This confirms the earlier hypothesis that the simplicity of novice models was due to their incompleteness. Strong positive correlations were found between overall perceived quality of the model and: • Understandability (.757, P<.001) • Completeness (.643, P<.001) • Correctness (.711, P<.001) • Flexibility (.532, P<.01). Understandability

Flexibility

Simplicity

Completeness

Correctness

The interactions found between the quality factors are summarised in Fig. 9 below:

Correctness Completeness Simplicity Flexibility Understandability

+ + + + +

Fig. 9. Interactions between Quality Factors

Validation of Quality Factors The high levels of inter-rater reliability on all quality factors indicate that the quality factors defined provide a consistent basis for evaluating data models. All of the factors except for simplicity had strong correlations with overall quality, which confirmed their relevance. The non-result for simplicity can be explained by the fact that the models which were simpler were also less complete. The observed correlations between the various quality factors are substantively less than 1,

274

D.L. Moody, G.G. Shanks, and P. Darke

suggesting that the quality factors are separate but correlated measures of quality (Gable, 1996). This confirms the independence of the quality factors.

Benefits for Research Use of the framework in this study gave a more comprehensive view of the differences between experts and novices than previous studies, and provided the basis for reliable evaluation of the quality of data models. Understanding the quality differences between models produced by experts and novices can provide useful knowledge for improving the teaching of data modelling in universities.

8.

Conclusion

This paper has described how the data model quality evaluation framework proposed by Moody and Shanks (1994) has been applied in research and in practice. The experiences gained have been used to further develop and refine the framework. In particular, the paper describes how the framework has been used to: a) Quality assure individual data models as part of application development projects (product quality); b) Reengineer application development procedures to build quality into the data modelling process (process quality) c) Build an expert assistant to support the evaluation process (the Data Model Quality Advisor); d) Investigate differences between data models produced by expert and novice data modellers. The results show that use of the framework has the potential to improve practice, research and teaching in data modelling. A major finding of the research is that the most significant gains in quality are made as a result of improving the process of data modelling (process quality) rather than quality assuring the final result (product quality). This supports the findings of the Total Quality Management literature, which argues that sustainable improvements in quality can only be achieved by modifying processes to eliminate defects. However the previous research in this area has concentrated almost exclusively on product quality.

References 1. 2. 3. 4. 5.

ABDEL-HAMID, T.K. (1988) The Economics of Software Quality Assurance: A Simulation Based Study, MIS Quarterly, 12:3. AUSTRALIAN SOFTWARE METRICS ASSOCIATION (ASMA) (1996) ASMA Project Database, Release 7, November, P.O. Box 1287, Box Hill, Victoria, Australia, 3128. AVISON, D. (1991) Action Programmes for Teaching and Researching in Information Systems, Australian Computer Journal, 23:2, 66-72. BATINI, C., CERI, S. AND NAVATHE, S.B. (1992) Conceptual Database Design: An Entity Relationship Approach, Benjamin Cummings, Redwood City, California. BATRA, D. and DAVIS, J. (1992) Conceptual Data Modelling in Database Design: Similarities and Differences between Expert and Novice Designers, International Journal of Man-Machine Studies, 37, 83-101.

Improving the Quality of Entity Relationship Models 6. 7. 8. 9.

10. 11. 12. 13. 14. 15. 16.

17. 18. 19. 20.

21. 22. 23. 24. 25.

275

BATRA, D., HOFFER, J.A. and BOSTROM, R.P. (1990) Comparing Representations with Relational and EER Models, Communications of the ACM, 33:2, 126-139 BENBASAT, I., GOLDSTEIN, D.K. and MEAD, M. (1987) The Case Research Strategy in Studies of Information Systems, MIS Quarterly, 11:3, 369-386 BOEHM, B.W. (1981) Software Engineering Economics, Prentice-Hall Inc., Englewood Cliffs, New Jersey. BUBENKO, J.A. (1986) Information Systems Methodologies: A Research View, in OLLE, T.W., SOL, H.G., VERRIJN-STUART, A.A. (eds.), Information Systems Design Methodologies: Improving the Practice, North-Holland, Amsterdam, 289-318. CHECKLAND, P.B. and SCHOLES, J. (1990) Soft Systems Methodology in Action, Wiley, Chichester. DAVENPORT, T.H. (1994) Saving IT’s Soul: Human Centred Information Management, Harvard Business Review, March-April. DEMING, W.E. (1986) Out of the Crisis, MIT Center for Advanced Engineering, Cambridge, MA. GABLE, G. (1996) Consultant Engagement Success: Client versus Consultant Views, Proc. 7th Australian Conference on Information Systems, 253-263 GARTNER RESEARCH GROUP (1992): Sometimes You Gotta Break the Rules, Gartner Group Strategic Management Series Key Issues, November, 23. HORROCKS, B. and MOSS, J. (1994) Practical Data Administration, Prentice Hall International, BCS Practitioner Series, London. INTERNATIONAL STANDARDS ORGANISATION (ISO) (1987), Information Processing Systems - Concepts and Terminology for the Conceptual Schema and the Information Base, ISO Technical Report 9007. IVARI, J. (1986) Dimensions Of Information Systems Design: A Framework For A Long Range Research Program. Information Systems, June, 39-42. KESH, S. (1995) Evaluating the Quality of Entity Relationship Models, Information and Software Technology, 37 (12). KIM, Y-G. and MARCH, S.T. (1995) Comparing Data Modelling Formalisms, Communications of the ACM, 38:6, 103-113. KROGSTIE, J., LINDLAND, O.I. and SINDRE, G. (1995) Towards a Deeper Understanding of Quality in Requirements Engineering, Proceedings of the 7th International Conference on Advanced Information Systems Engineering (CAISE), Jyvaskyla, Finland, June. LEVITIN, A. and REDMAN, T. (1994) Quality Dimensions of a Conceptual View, Information Processing and Management, 31,:1, 81-88. LINDLAND, O.I, SINDRE, G. and SOLVEBERG, A. (1994) Understanding Quality in Conceptual Modelling, IEEE Software, March, 42-49. MAIDEN, N. and SUTCLIFFE, A. (1992) Analysing the Novice Analyst: Cognitive Models in Software Engineering, Intl. Journal Man Machine Studies, 36:7, 719-740 MARTIN, J. (1989): Strategic Data Planning Methodologies, Prentice Hall, New Jersey. MOODY, D.L. (1996) Critical Success Factors for Information Resource Management, Proc. 7th Australasian Conference on Information Systems, Hobart, Australia, December.

276

D.L. Moody, G.G. Shanks, and P. Darke

26. MOODY, D.L. (1998): Metrics for Evaluating the Quality of Entity Relationship Models, in Proceedings of the Seventeenth International Conference on Conceptual Modelling (ER ’98), Singapore, November 16—19. 27. MOODY, D.L. AND SHANKS, G.G. (1994): What Makes A Good Data Model? Evaluating the Quality of Entity Relationship Models, in P. LOUCOPOULOS (ed.) Proceedings of the Thirteenth International Conference on the Entity Relationship Approach, Manchester, December 14-17, 94-111. 28. MOODY, D.L. AND SHANKS, G.G. (1998): What Makes A Good Data Model? A Framework for Evaluating and Improving the Quality of Entity Relationship Models, Australian Computer Journal (forthcoming). 29. MOODY, D.L. AND SIMSION, G.C. (1995) Justifying Investment in Information Resource Management, Australian Journal of Information Systems, 3(1), September: 2537. 30. RESCHER, N. (1977) Methodological Pragmatism: Systems-Theoretic Approach to the Theory of Knowledge, New York University Press. 31. ROSS, R.G. (1996) The Business Rule Book, Database Research Group, Boston, MA, 2nd edition. 32. SHANKS, G. (1997) Conceptual Data Modelling: An Empirical Study of Expert and Novice Data Modellers, Australian Journal of Information Systems, 4:2, 63-73 33. SHANKS, G. and DARKE, P. (1997) Quality in Conceptual Modelling: Linking Theory and Practice, Proceedings of the Pacific Asia Conference on Information Systems (PACIS), Brisbane, (May), Queensland University of Technology, 805-814 34. SHOVAL, P. and EVEN-CHAIME, M. (1987) Database Schema Design: An Experimental Comparison Between Normalisation and Information Analysis, Database, 18:3, 30-39 35. SIMSION, G.C. (1994): Data Modelling Essentials, Van Nostrand Reinhold, New York. 36. VAN VLIET, J.C. (1993) Software Engineering: Principles and Practice, John Wiley and Sons, Chichester, England. 37. VON HALLE, B. (1991): Data: Asset or Liability?, Database Programming and Design, 4(7), July: 13-15. 38. VON HALLE, B. AND CONKEY, J. (1997) Rewiring the Business, Database Programming and Design, 10(1), January, 11-15 39. WALRAD, C. AND MOSS, E. (1993) Measurement: The Key to Application Development Quality, IBM Systems Journal, 32(3), 445-460. 40. ZULTNER, R.E. (1992) The Deming Way: Total Quality Management for Software, Proceedings of Total Quality Management

The Troll Approach to Conceptual Modelling: Syntax, Semantics, and Tools? Antonio Grau, Juliana K¨ uster Filipe, Mojgan Kowsari, Silke Eckstein, Ralf Pinger, and Hans-Dieter Ehrich Technische Universit¨ at Braunschweig, Informatik, Abt. Datenbanken, Postfach 3329, D–38023 Braunschweig, Germany {grau,filipe,kowsari,eckstein,pinger,ehrich}@idb.cs.tu-bs.de http://www.cs.tu-bs.de/idb/welcome_e.html

Abstract. In this paper, we present the use of Troll for the conceptual modelling of distributed information systems. Troll offers both textual and graphical notations. Troll has been used in practice to model an industrial information system. We use an extract of this case study to describe briefly the syntax and underlying semantics of the language. We also show a set of software tools that are being developed to support the modelling with Troll. These tools include editors, checkers as well as an animator for validating Troll specifications. We report on the experiences we gained by applying the language to the industrial project. Finally, a short description on further work is given.

1

Introduction

One main issue in the conceptual modelling of information systems is that a model has to describe both structural and behavioural aspects of a system. Traditionally, the modelling of both aspects has been treated separately. In the last years the application of the object–oriented approach to this area has contributed enormously to integrate these aspects in only one formalism. The object–oriented approach includes within an object both structural and behavioural properties. Numerous object-oriented techniques for conceptual modelling have emerged. Informal object-oriented methodologies like UML [10] are very popular. Regarding formal methods either new languages like LCM [8], ALBERT [3] and OASIS [19] or extensions to well-known languages as in VDM++ [4], Z++ [16], and LOTOS [1] have been developed [20].Troll, the language dealt with in this paper, belongs to the group of new formal languages developed for supporting the object-oriented paradigm. Troll is based on OBLOG [22] and takes ideas and concepts from abstract data types, behaviour modelling, specification of reactive systems and concurrency theory. In this paper we want to present Troll in its third version [2]. Previous versions have been published in [13,12] among ?

Work reported here was partially supported by the EU under ESPRIT-IV WG 22704 ASPIRE, the DFG under Eh 75/11-1, the Spanish Ministry of Education and Culture and the Technical Federal Board, lab 3.41, project CATC.

T.W. Ling, S. Ram, and M.L. Lee (Eds.): ER’98, LNCS 1507, pp. 277–290, 1998. c Springer-Verlag Berlin Heidelberg 1998

278

A. Grau et al.

others. In contrast to older versions, Troll version 3.0 is devoted to modelling distribution issues and has also been designed with the aim of executability. In this paper we do not want to compare Troll with other approaches, but give a description of its concepts and features. Our approach combines informal and formal techniques taking advantages from both of them. For giving a first overview of the system and as a means of communication between users and developers, a graphical notation called OmTroll is used. From OmTroll a frame of a textual Troll specification can be derived. The textual specification has to be refined to a complete system specification. The formal nature of Troll helps to achieve concise and unambiguous system specifications as well as to develop better CASE tools. We are currently developing a CASE environment for modelling and validating Troll specifications. For giving Troll a formal semantics, a special kind of temporal logic was needed to describe properties of distributed objects. This logic is called Distributed Temporal Logic (Dtl) [6] and is based on n-agent logics [17]. Each object has a local logic allowing it to make assertions about system properties through communication with other objects. Objects are represented by a set of Dtl formulae interpreted over labelled prime event structures [18]. Interaction between concurrent objects is done by synchronous message passing. The system model is obtained from its object models, whereby object interaction is represented by shared events. We are using Troll to develop an information system in an industrial environment. Our experiences with the use of Troll in the project are very positive. The objective of the project is to develop a large information system supporting the activities of different user groups. The complexity of the organisation and the system that is supposed to support this organisation is rather high. Besides, the system has to integrate already existing applications and respecified ones. In order to achieve a deep understanding of the organisational structures we have to build an abstract model of the organisation which has to cover all structural aspects and organisational activities. The paper is organised as follows. In Sect. 2 we introduce the case study. In Sects. 3 and 4 we make use of a simplified extract of the case study for illustrating the syntax and semantics of Troll, respectively. Section 5 gives a short description of the CASE environment. We report on the experiences we gained by applying Troll to the case study in Sect. 6. Finally, Sect. 7 summarises the paper and outlines further work.

2

Case Study

Our case study describes the activity of Group 3.4 settled in the PTB, which stands for Physikalisch-Technische Bundesanstalt1 . The PTB is a government institute that cooperates in national and international committees, establishing rules for the fields of safety technology and standardisation among others. The group 3.4 is concerned with testing and certifying electrical equipment, e.g., 1

Federal institute of weights and measures.

The Troll Approach to Conceptual Modelling

279

motors, switches, heaters, which are to be used in explosion-hazardous areas. The focus is, for example, to avoid causing an explosion by turning on the lights with a switch in a chemical factory. Therefore, the switch has to be tested before it may be used in such areas. The testing procedure takes between 3-6 months and is worked out by approximately 100 certifying officers individually and manually according to the European Standards. On average, 1000 certificates per year are issued. It is important that all informations in connection with a certificate are available and reusable at any time. In 1994, the PTB started a cooperation with the database group of the Technical University Braunschweig to computerise the testing procedure. The system is called CATC (Computer Aided Testing and Certifying) and includes different subsystems based on the certifying process [11]. The certifying process for the electrical equipments is composed of the following parts: – administration management includes the preliminary screening of the documents for the company producing or providing the electrical equipment. Such documents contain, e.g., name and address(es) of the company, name of the company responsable as well as the technical description of the equipment. This information is essential to setup the application form and has to be permanently available. Later, the staff prepares the settlement of the account and certifies documents depending on the test results. – design approval includes the assessment of design papers for the equipment based on descriptions and its accordance with the European Standards. Here the staff creates a check list including the necessary experiments that have to be done by the operators. – experimental tests are performed by the operators in the test lab and store all relevant data. There are different tests depending on the kind of the electrical equipment (for example temperature tests for heaters, pressure test for motors). The tests will be done on samples sent by the company. After the tests are done, the staff reads the results and decides whether more tests are necessary. Otherwise, the certification may be issued. As mentioned above the testing and certifying process is a cooperation between staff and operators. The communication is an important factor to run the process correctly and more quickly. Both of them have to access the test results. The operators produce and store the test results while the staff reads and interprets them. The test results belonging to a certain experiment can be preselected and accessed by the staff. The operators start the experiment under a certain pressure and during a valid time. Due to European Standards, every application should be tested by more than one experiment to avoid any mistake. While the experiments for one application are running, the staff may set up the next experiment. In the next section we will illustrate Troll and OmTroll using the communication depicted above. Because it would go beyond the scope of this paper, we exclude details from our system description and just consider a simplified extract of it.

280

3

A. Grau et al.

Troll

Troll is a formal object–oriented language for the specification of distributed information systems at a high level of abstraction [2]. The textual language Troll has a graphical variant called OmTroll which borrows concepts from OMT [21]. In the following we first give a brief survey of the underlying ideas and then explain the main features of Troll and OmTroll by means of a simplified part of the case study from the previous section. In our approach, an object–oriented system is considered to be a concurrent community of interacting objects. Objects are units of structure and behaviour. Similar objects are described in terms of object classes. They may be composed to complex objects through aggregation and arranged in inheritance hierarchies through specialisation. Objects defined separately may communicate through global interactions. Figure 1 shows the so called community diagram of our ex-

Fig. 1. OmTroll Community Diagram

ample. Some of the object classes in there are described not only by their name but also by their signature (attributes and action declarations). We will explain this later in conjunction with the textual part of the language. The community diagram contains the aggregated object class Group, which represents group 3.4 of the PTB. It has object classes Application, Company and Standard as components, whereby Application itself has Experiment as such. Aggregation is represented by a diamond and the dot at the top of some of the object classes denotes multiple components. For example, each application may own a number of accompanying experiments, locally identified by their experiment number (expNr). The object class Group is called a node of the system which is concurrent to the

The Troll Approach to Conceptual Modelling

281

other node shown in Fig. 1, namely User. Object class User is specialised in Staff and Operator. Specialisation is graphically presented by a triangle. Besides the community diagram OMTroll provides three more kinds of diagrams: data type diagrams for the declarations of user defined complex data types, behaviour diagrams to specify the allowed behaviour for each object class and communication diagrams to show interaction between object classes. An example for the last one is shown in Fig. 4. From the OmTroll diagrams a textual specification fragment can be derived. It contains all object classes with their signature, their component, specialisation and communication relationships as well as all user defined data types. The behaviour parts of the object classes, i.e., the operation definitions and local constraints, have to be extended. For example the object class Application as object class Application components Experiments(expNr:nat):Experiment; attributes company : |Company| constant; labor : labors; appl_date : date constant; nextExpNr : nat initialized 1, hidden; actions *createAppl(comp:|Company|, dat: date, lab:labors); newExperiment(nam: string, st:msset, !expNr: nat); behavior createAppl(comp, dat, lab) do company:=comp, appl_date:=dat, labor:=lab od; newExperiment(nam, st, expNr) do Experiments(expNr).createExp(nam, st), expNr:=nextExpNr, nextExpNr:=nextExpNr+1 od; constraints nextExpNr<10; end;

Fig. 2. Object Class Application

shown in Fig. 2 has several Experiments as components, each one identified by a unique experiment number. It contains four attributes: company is an object valued attribute, i.e., a link to object class Company. In contrast to components, objects referenced by attributes are not embedded in the classes where the attributes are declared. The second attribute labor has a user defined data type labors, which we do not specify here. appl date is a constant attribute, that means the date is once set and can never be changed. The attribute nextExpNr is initialised with one and it is hidden, i.e., it is only visible inside of object class Application. It stores the next number used to identify the experiments

282

A. Grau et al.

belonging to this application. Furthermore, there are two actions: createAppl is a birth action that creates an object of class Application. It has all attributes of the class as parameters excluding nextExpNr, which is automatically initialised. newExperiment calls the birth action of the component class and increases nextExpNr. Finally, a constraint assures that there can not be more than nine experiments for each application. After we explained the main language constructs to specify object classes we will now take a look at the specification of a whole object system (see Fig. 3). The keyword object system followed by the name of the system is the opening bracket. The specification itself consists of four parts: data type and object class specifications, object declarations and a behaviour part where the global interactions are described. In our case the name of the system is CATC. We object system CATC data type ... object class Application ... end; object class ... objects IG34:Group; objects Users(userId:nat):User; behavior Staff(Users(userId)).createExperiment(appNr, nam, st, expNr) do IG34.Applications(appNr).newExperiment(nam, st, expNr) od; end. Fig. 3. Object System CATC

do not consider any user defined data types, but we already took a close look to object class Application. Besides this all other object classes shown in the community diagram (Fig. 1) have to be specified here. We declare two kinds of objects: IG34 is the name of an instance of object class Group. After the occurrence of the accompanying birth actions this object will also contain objects of class Application as well as of class Experiment as components. The declaration of several objects of class User is done by parameterisation, i.e., each instance is identified by an userId of type nat. As User is the base class of an specialisation hierarchy, the objects will either have a special aspect Staff or Operator. In the behaviour part we give an example for a calling from an object of class Staff (identified by the userId of the base class) to a certain object of class Application, which can be reached via the aggregating object IG34 through ”dot notation”. Whenever the action createExperiment occurs in Staff, the action newExperiment in Application is called, which in turn calls the birth action of class Experiment (see Fig. 2). The other object interactions of our example are shown in Fig. 4, where one can see the following: The communication between Application and Experiment has already been described just as the communication between Staff and Application. The four remaining relationships between Staff and Experiment and between Operator and Experiment concern the testing and certifying process. First the Operator has

The Troll Approach to Conceptual Modelling

283

Fig. 4. OmTroll Communication Diagram

to know under which pressure the test has to be done, he asks for the setup values and gets the answer from action giveSetup in Experiment. After carrying out the tests he stores the results in Experiment. After this the Staff can ask for the results, check them and store an assessment.

4

Troll Semantics

In this section we explain by means of the previously introduced example some aspects of the semantics underlying the Troll language. Due to space limitations, we focus on inter-object communication. For more details on other semantical features of Troll, like object relationships, see for instance [7]. From a system specification in Troll we can derive both information on its structural and dynamic parts. The system structure is given by the system object instances together with their attributes and actions as described in the corresponding object classes. The structural part of the system is captured by a system signature. The system dynamics is given by the global interactions among the objects, constraints, initialisation declarations, action effects on the attributes, action preconditions, etcetera. The dynamic aspect of a system can be expressed in logical formulae. We use a distributed temporal logic called Dtl [6]. Formulae written in Dtl are interpreted in labelled prime event structures. A system specification is given by a system signature together with a set of Dtl formulae, the axioms of the system. In what follows we consider Σ = (S, Ω) a signature where S is a set of data and object sorts, i.e., S = SD ∪ SO , and Ω is an S ∗ × S-indexed family of sets of operation symbols. A system signature is given by ΣI = (Id, At, Ac) where Id is a set of object identifiers (the system object instances), At is an Id × S-indexed family of sets At = {Ati,s }i∈Id,s∈S of attribute symbols, and Ac is an Id × S ∗ -indexed family

284

A. Grau et al.

of sets Ac = {Aci,x }i∈Id,x∈S ∗ of action symbols. For an arbitrary a ∈ Ati,s , a is an attribute of object i and type s. For an arbitrary b ∈ Aci,s1 ...sn , b is an action of object i with n parameters of sorts s1 . . . sn . We illustrate these concepts with our example. In what follows, let 0, 1, 2, . . . be user numbers, 100, 200, . . . be application numbers and j an arbitrary user number. The CATC system signature is given by ΣCAT C = (Id, At, Ac) where for instance Id = {IG34, Users(0), Users(1), Users(2), . . .} AtIG34,nat = {IG34.Applications(100).nextExpNr, . . .} AtIG34,msset = {IG34.Applications(100).Experiments(7).setup, . . .} AcIG34,string msset nat = {IG34.Applications(100).newExperiment, . . .} AcUsers(j),nat nat msset = {Users(j).giveResults}

Before showing some CATC system axioms, we ought to give a brief understanding on Dtl. Dtl is an extension of linear temporal logic based on n-agent logics [17]. Each object i ∈ Id has a local logic Dtli . A local logic Dtli allows the object i to make assertions about system properties from its local viewpoint. Such system properties can be made based on the communication with other objects. Dtl may be defined as follows: Dtl::= {Dtli }i∈Id Dtli ::= i.Hi | i.Ci Hi ::= Atom | (Hi ⇒ Hi ) | (Hi U Hi ) | (Hi S Hi ) Atom::= f alse | TΣ θ TΣ | ATΣ θ TΣ | ACΣ | ACΣ Ci ::= Ci ⇒ j.Cj for some j ∈ Id, i 6= j

The local logic of object i is split into a local home logic Hi and a communication logic Ci . Hi is a first order linear temporal logic. TΣ , ATΣ and ACΣ represent data, attribute and action terms respectively. An atomic formula of Hi can be the boolean constant f alse; the predicate θ applied to two data terms or to an attribute and data term, where θ is a comparison predicate (e.g., =, ≤, . . .); the predicate  (enabling) applied to an action term; or the predicate (occurrence) applied to an action term. A formula in Hi can be obtained by applying successively the connective ⇒ and the temporal operators U (until) and S (since) to atomic formulae. The other temporal operators like next X, sometime in the future F and always G can be derived from these (cf. [6] for details). The communication logic Ci allows to express communication among several objects from the local viewpoint of object i. Formulae in Ci contain occurrences of actions for object i like, e.g., i.a ∧ i.b. We now give some examples of CATC system axioms. To increase their readability we will omit the local object identity. The next formula denotes the global interaction among a staff instance and the instance IG34, and it corresponds mainly to a one to one translation of the system specification (cf. Fig. 3). It is a communication formula from object Staff(Users(0)). Staff(Users(0)).createExperiment(100,nam,st,expNr) ⇒ IG34.Applications(100).newExperiment(nam,st,expNr) If the above global interaction occurs it causes a further communication within the node IG34. This is captured by the next formula (cf. Fig. 2).

The Troll Approach to Conceptual Modelling

285

IG34.Applications(100).newExperiment(nam,st,expNr) ⇒ IG34.Applications(100).Experiments(expNr).createExp(nam,st) ∧ expNr = IG34.Applications(100).nextExpNr ∧ X (IG34.Applications(100).nextExpNr = expNr + 1) Dtl is interpreted over labelled prime event structures. Prime event structures were first introduced by Winskel (for more details cf., e.g., [18]) and allow a nice and simple description of distributed computations as event occurrences together with a causal and a conflict relation between them. The causal relation implies a (partial) order among event occurrences, while the conflict relation allows to express choice. Events in conflict cannot belong to the same system or object run. Formally, a prime event structure is a triple E = (Ev, →∗ , #), where Ev is a set of events, →∗ and # are binary relations on events called causality (a partial order) and conflict (irreflexive and symmetric) respectively. Two events which are neither in conflict nor related by causality are said to be concurrent. If there are no concurrent events the structure is called sequential. In order to be able to use event structures as a model for our objects, we have to attach to them some more information in the form of event labels. We introduce a labelling function µ : Ev → P(Ac), i.e., a total function attaching to each event the actions which occurrence it denotes. A prime event structure enriched by a labelling function is called a labelled prime event structure. Labelled prime event structures constitute a non-interleaving model of concurrency and are a fair choice when a denotational semantics is needed. Objects in Troll do not have intra-object concurrency, and are therefore modelled by sequential labelled event structures. Let (Ei , µi ) be a model for the object instance i ∈ Id. A system model is obtained by composing the corresponding instance models. The idea of the composition construction is to identify shared communication events and leave all the other events concurrent. Figure 5 shows part of the CATC system model focusing on the communication between a staff (Users(0)) and an operator (Users(1)) instances. We represent events by boxes with the corresponding label (set of actions) written inside. Arrows between boxes denote the causality relation among events. The communication, as presented already in the OMTroll communication diagram of Fig. 4, is done indirectly via experiments.

5

Troll-WorkBench

The architecture and aims of the Troll-WorkBench were described in [9]. In this paper we only want to give a brief description of its functionalities. – OmTroll Editor: This editor provides developers with different diagrams for modelling the system in OmTroll (community, communication and data type declaration diagrams). Figures 1 and 4 showed the specification of our example using the OmTroll editor. An experience we gained from projects where we used OmTroll and Troll, is that specifiers tend to work switching between both notations. For this reason, an automatic translation can be

286

A. Grau et al. ...

IG34 Users(0) Users(0).login(Carol,15-1-98,staff)

Users(1)

IG34.Applications(100)

Users(1).login(Emil,3-1-98,operator)

IG34.Applications(100).createAppl(...) Users(0).createExperiment(100,...,7) IG34.Applications(100).newExperiment(...,7) IG34.Applications(100).Experiments(7).createExp(...)

...

# ...

Experiments(7)

Users(1).askSetup(100,7,...) IG34.Applications(100).Experiments(7).giveSetup(...)

#

Users(1).startExperiment(100,7,...) Users(1).giveResults(100,7,...) IG34.Applications(100).Experiments(7).storeResults(...) ...

Users(0).askResults(100,7,...) IG34.Applications(100).Experiments(7).giveResults(...) ... ...

Fig. 5. Modelling communication between a staff and an operator instances.

–

– –

– –

done between both notations, obtaining not only the freedom in the method, but also the consistency in the notations. At the moment the method does not allow a complete translation because there is not a direct correspondence between the behaviour part of Troll and the object behaviour diagram of OmTroll. Investigations towards a complete translation are future work. Troll Editor: For textual specifications a Troll language mode has been implemented in the (X)Emacs editors. It constitutes the front end of the system. Here other system tools can be invoked providing a common interface. Some of the new capabilities added to the editor are: management of specification projects, automatic generation of patterns, search for keywords through the project files, cross-referencing, automatic generation of documentation files, different font styles for reserved words and automatic indentation. The textual editor is depicted in Fig. 6. Syntax Checker: This tool checks whether the syntax of the specification meets the Troll syntax. The syntax checker creates a syntax graph from the specification which is used as internal structure for all remaining tools. Semantic Checker: The semantic checker analyses the specification looking for possible type errors and Troll specific semantics inconsistencies. The last task is not an easy one, because of the complex Troll rules regarding attributes and actions visibility in order to maintain object encapsulation principles. Report Generator: The report generator, including a pretty printer, allows to export the specification to a document. In this way the model and the documentation of the model can be contained in one document. Animator: For validation purposes, we are implementing an animator for Troll specifications. By animating specifications we mean the process of identifying a set of scenarios, where a scenario is a sequence of events which

The Troll Approach to Conceptual Modelling

287

Fig. 6. Troll language mode for XEmacs. The syntax and semantics checkers are included in the editor.

may occur in the domain of the system, and testing them against the specification. By this way, we can see whether the specification meets the intended user requirements involved in these scenarios. In order to test a specification it is necessary to execute it. There are different approaches for executing a specification. We have chosen code generation rather than a Troll interpretation because of two principal reasons. On the one hand, we have gained positive experiences from the CATC project, about how Troll objects can be implemented using an object-oriented programming language and on the other hand, generating code could later make possible to use some parts of the animator for evolutionary prototyping purposes. Implementation Aspects: For the implementation we are only using well– known public domain tools. A prototype of the Troll-WorkBench including editors and checkers can be downloaded from our ftp site.

6

Experiences

The conceptual modelling, design and implementation of the CATC system for one of the three laboratories of the group 3.4 in the PTB has been done and we are reusing this model for the other ones [11]. We gained positive experiences

288

A. Grau et al.

with using object-oriented formal techniques in our project especially in describing our universe of discourse. During this time, we tried to get an overview in specific sub-problem domains and developed a first rough global architecture. One advantage of using Troll was to achieve first a rather global view before considering details. Changes to the finer grained specification documents did not affect the global view. OmTroll community diagrams help us to improve the communication skills between developers and federal board employees in their understanding of technical and administrative processes. In the analysis phase, the focus is on providing a structured intuitive representation of the aspects that are relevant for the planned information system. This represention has to be unambiguous and precise, structured to support the management of the model complexity and independent of concrete implementations. Without tool support, respecifying and redoing already developed system parts was over and over disappointing. Tools support turned out to be one key factor in order to avoid frustration when having to change a model again. We had regular meetings with all team members to propagate important design decisions concerning interfaces, but we spent a lot of time answering syntactical and semantical questions by using Troll. The fact that specification is the most important part of the work is also reflected by the amount of time used for modelling and implementation, respectively. On average 70% of the time was spent on specification purposes whereas the rest of the time was spent on implementation including integration. During analysis and design, we spent more than 30% of the time checking manually syntax and semantic mistakes in our specification. This was not always easy within 17000 lines of Troll code. By refining the specification we also used Troll in the design phase. Our implementation environment was a relational database and had C++ as target language. Therefore, we defined general rules to transform structural aspects of Troll specifications to relational database schemes and to translate information about attributes, and actions into C++. The translation from compact Troll notation into C++ was straightforward. The factor between Troll and C++ lines of code was of 1:6. Because we have not developed the different subsystems of CATC at the same time, we have started to integrate them using the Troll-WorkBench. Since we are now using tools for modelling CATC in other laboratories of the PTB, we have the possibility to let the developers share the diagrams. This helps them to discover ambiguities by defining interfaces. Overall, we strongly believe that the formal object oriented method has paid off.

7

Summary and Further Work

In this paper we have presented the Troll approach to conceptual modelling of information systems. Troll has been used in modelling an industrial application. A simplified extract of this application has been used herein to describe briefly the syntax and semantics of Troll. We have also shown the functionalities of a CASE environment for Troll. Very positive experiences have been

The Troll Approach to Conceptual Modelling

289

gained using Troll in the industrial setting. We have shortly reported on these experiences. Further research focuses on modularisation concepts. It became obvious that a higher level structuring mechanism than object classes is needed. This is mainly a result of the case study presented above, where 17000 lines of Troll code were produced. Besides this we are going to provide reusable modules in a catalogue which will be integrated in the Troll-WorkBench. First investigations on the language level are presented in [5]. Work towards a logical and semantical foundation of such concepts has been covered among others in [14,15]. Moreover, we are also considering the support of other kinds of communication mechanisms for Troll. At the specification level it is convenient to be able to abstract from too many details, therefore in some cases asynchronous communication could be preferable to synchronous one. An advantage of asynchronous communication is also given by the higher level of concurrency that can be achieved and is especially important for distributed systems. Considerations on the logic and denotational semantics for asynchronous communication can be found among others in [15]. A further research direction we are currently considering is the development of a Troll tool for model checking. Usually verification is done at the end phases of the development process verifying a program or a hardware description. With our tool, we aim at checking the global system behaviour in earlier phases. The expected behaviour of our system can be expressed as a set of Dtl formulae. The intention is to check a system model, represented by a state transition system, against this set of formulae. Finally, since some parts of the CATC project include the specification of real time aspects, we are also considering the extension of Troll to cope with them.

References 1. E. Cusack, S. Rudkin, and C. Smith. An Object-Oriented Interpretation of LOTOS. In S. Vuong, editor, Formal Description Techniques II, FORTE’89, pages 211–226. North–Holland, Amsterdam, 1990. 2. G. Denker and P. Hartel. Troll – An Object Oriented Formal Method for Distributed Information System Design: Syntax and Pragmatics. Informatik-Bericht 97–03, Technische Universit¨ at Braunschweig, 1997. 3. P. Du Bois. The Albert-II Language – On the specification and the Use of a Formal Specification Language for Requirements Analysis. PhD thesis, Facultes Universitaires Notre–Dame de la Paix, Namur, Belgium, 1995. 4. E.H. D¨ urr and J.v. Katwijk. VDM++, A Formal Specification Language for Object–Oriented Design. In Proceedings of TOOLS7 (Technology of ObjectOriented Languages and Systems). Prentice Hall, 1992. 5. S. Eckstein. Towards a module concept for object oriented specification languages. In J. B¯ arzdi¸ nˇs, editor, Proc. 3rd Int. Baltic Workshop on Databases and Information Systems, Riga, Latvia, April 15-17, 1998, pages 180–188, 1998. 6. H.-D. Ehrich, C. Caleiro, A Sernadas, and G. Denker. Logics for Specifying Concurrent Information Systems. In J. Chomicki and G. Saake, editors, Logics for Databases and Information Systems, pages 167–198. Kluwer Academic, 1998.

290

A. Grau et al.

7. H.-D. Ehrich and P. Hartel. Temporal Specification of Information Systems. In A. Pnueli and H. Lin, editors, Logic and Software Engineering, Proc. Int. Workshop in Honor of C.S. Tang,Beijing, 14-15 August 1995, pages 43–71. World Scientific, 1996. 8. R. Feenstra and R. Wieringa. LCM 3.0: A Language for Describing Conceptual Models - Syntax Definition. Technical report, Faculty of Mathematics and Computer Science, Vrije Universiteit Amsterdam, 1993. 9. A. Grau and M. Kowsari. A Validation System for Object-Oriented Specifications of Information Systems. In R. Manthey and V. Wolfengagen, editors, Proc. of the First East-European Symposium on Advances in Databases and Information Systems (ADBIS’97), St. Petersburg, 2-5 September. EWiC, Springer, 1997. 10. Paul Harmon and Mark. Watson. Understanding UML- The Developer’s Guide. Morgan Kaufmann, San Francisco, California, 1998. 11. P. Hartel, G. Denker, M. Kowsari, M. Krone, and H.-D. Ehrich. Information systems modelling with TROLL formal methods at work. Information Systems, 22(2-3):79–99, 1997. 12. T. Hartmann, G. Saake, R. Jungclaus, P. Hartel, and J. Kusch. Revised Version of the Modelling Language Troll (Version 2.0). Informatik-Bericht 94–03, Technische Universit¨ at Braunschweig, 1994. 13. R. Jungclaus, G. Saake, T. Hartmann, and C. Sernadas. Troll – A Language for Object-Oriented Specification of Information Systems. ACM Transactions on Information Systems, 14(2):175–211, April 1996. 14. J. K¨ uster Filipe. Modelling Parameterisation in Concurrent Object Systems. Logic Journal of the IGPL, 5(6):877–879, November 1997. In Conference Report: Workshop on Logic, Language, Information and Computation (WoLLIC)’97, Fortaleza, Cear´ a, Brazil, August 20-22. 15. J. K¨ uster Filipe. On a Distributed Temporal Logic for Modular Object Systems. Technical Report 98-06, Technical University Braunschweig, 1998. 16. K. Lano. Z++: an Object-Oriented Extension to Z. In J. Nicholls, editor, Z Users Workshop: Proc. 4th Annu. Z User Meeting, pages 151–172. Workshops in Computing. Springer-Verlag, Berlin, 1991. 17. K. Lodaya, R. Ramanujam, and P.S. Thiagarajan. Temporal Logics for Communicating Sequential Agents. Int. Journal of Foundations of Computer Science, 3(2):117–159, 1992. 18. M. Nielsen, G. Plotkin, and G. Winskel. Petri Nets, Event Structures and Domains, part 1. Theoretical Computer Science, 13:85–108, 1981. 19. O. Pastor, E. Insfran, V. Pelechano, J. Romero, and J. Meseguer. OO-Method: An OO Software Production Environment Combining Conventional and Formal Methods. In A. Olive and J. A. Pastor, editors, 9th International Conference on Advanced Information Systems Engineering (CAiSE’97), Barcelona, pages 145– 158, June 1997. 20. A. Ruiz-Delgado, D. Pitt, and C. Smythe. A Review of Object-Oriented Approaches in Formal Methods. The Computer Journal, 38(10):777–784, 1995. 21. J. Rumbaugh, M. Blaha, W. Premerlani, F. Eddy, and W. Lorensen. Object– Oriented Modeling and Design. Prentice Hall, New York, 1991. 22. A. Sernadas, C. Sernadas, and H.-D. Ehrich. Object-Oriented Specification of Databases: An Algebraic Approach. In P.M. Stoecker and W. Kent, editors, Proc. 13th Int. Conf. on Very Large Databases VLDB’87, pages 107–116. VLDB Endowment Press, Saratoga (CA), 1987.

Process Failure in a Rapidly Changing High-Tech Organisation: A System Dynamics View Bruce Campbell Joint Research Centre for Advanced Systems Engineering Macquarie University, Australia [email protected]

Abstract. IS is still suffering from high rates of failure and dissatisfaction by users with the delivered product. Respondents to a recent survey [19] said that the major reason for IS failure or poor performance is due to the lack of consideration of “softer” organisational issues - not technological reasons. This is hardly surprising given that most current modelling techniques are based on the structured analysis paradigm and do not easily lend themselves to modelling softer issues. Furthermore, there is a growing consensus that much remains to be done in order to determine exactly how softer factors influence the implementation of computer systems. This paper reports on research in progress at an Australian high-tech organisation where a number of modelling techniques were used to determine why processes within an implemented system were failing over time. The modelling techniques included data flow diagrams, flow charts and system dynamics modelling. The latter proved highly effective at demonstrating the impact of the softer organisational issues on the processes. The paper further emphasises the need to consider a system holistically during the requirements specification and design phases.

1.

Introduction

The action research described in this paper is part of an Australian Government sponsored collaborative project being undertaken with a large Australian high-tech organization. The initial request from this organization was for help to provide a much improved user documentation facility for its customer service centres as it had been observed that processes within these centres were deteriorating over time and were costing the organization a considerable amount of money. Initial analysis T.W. Ling, S. Ram, and M.L. Lee (Eds.): ER’98, LNCS 1507, pp. 291−300, 1998.  Springer-Verlag Berlin Heidelberg 1998

292

B. Campbell

indicated that the problem was much deeper than just poor documentation and it was decided to analyse and model the processes and how they were being affected by the organisation into which they had been installed. Preliminary findings indicate that pre-existing management policies are preventing the system operating as designed and are costing the organization a considerable amount of money. Although this is a single case study there is a very strong suspicion that these types of policies are generic and not restricted to the organization under study. This leads us to believe that many information systems cannot work at optimal efficiency resulting in a considerable loss of money.

2.

Background

There is growing evidence in the literature that many of the reasons for poor performance of information systems (IS) are due to a lack of consideration of the “softer” organizational issues. Jackson [16] refers to a report from the Organizational Aspects Special Interest Group (OASIG). The OASIG study was carried out in the U.K. and supported by the Economic and Social Research Council and the U.K. Department of Trade and Industry1. The report [19], representing the experiences of some 14,000 U.K. organizations, indicates in part that:• 80-90% of IT investments do not meet their performance objectives; the reasons for this are rarely purely technical in origin. • Most organizations are not good at evaluating the performance and impact of their investments in information technology (IT) [19] Other writers who have indicated the lack of consideration of the “softer” aspects include Curtis, Krasner & Iscoe [6], Ginzberg [13] and McGrath [21]. To reduce this problem a number of writers have recommended that business processes be modelled from a number of perspectives or views [5, 6, 14, 18, 19]. Despite this recommendation, research shows that although organisational issues receive substantial attention during some periods of the system development life cycle, they receive little consideration during the system modelling and development stages [10]. The question then is “why aren’t the softer factors modelled?” A possible reason may be that most current modelling techniques are based on a paradigm of structured analysis and do not lend themselves to modelling softer issues [19]. This could be due to differing philosophical perspectives of modelling which have been neatly summarised by Klein and Hirschheim [17]. The argument is that, in simple terms, modelling of hard and soft factors in systems analysis and business process modelling 1

A full copy of the report, the result of interviews with 45 researchers/consultants drawing on results from 14,000 UK organisations, can be obtained at the following URL:http://www.shef.ac.uk/~iwp/resprogs/itperf.html.

Process Failure in a Rapidly Changing High-Tech Organisation

293

(BPM) reside at opposite ends of an ontological/epistemological continuum [17] and that there is some difficulty in integrating the two views into one technique. The issue then is whether the available choice of IS techniques influence the value systems of designers, or whether the value systems of the designers influence the selection of development methodologies [12]. Gasson and Holland also argue that IS professionals tend to choose hard systems analysis techniques over soft analysis techniques because they emphasise the technical aspects of analysis [12]. They argue it is the technical expertise of IS professionals which gives them their basis of power within an organisation. IS professionals will tend to select hard analysis techniques to maintain their power base, but these will be in conflict with other methods which may take a more behavioural approach. The research presented in this paper indicates that soft factors can determine the performance of an implemented process and must be considered. The research is taking place within a rapidly changing Australian high-tech organisation for which the pseudonym Gigante will be used. Gigante, and the specific problem under research will be described in the next section. The fourth section describes the use of system dynamics modelling to reveal the effects of some of the softer issues on the business process under study. The fifth section provides some of the results and the final section outlines further research required in the area and identifies some of the issues which have arisen from this work.

3.

Gigante

Gigante is a large Australian information services provision company which has adopted a product differentiation strategy [25] in the past few years as a response to increased competition. This strategy has meant that its product mix is rapidly changing, making it difficult for systems and customer services representatives (CSR’s) to keep pace. Each time a new product is introduced, or a minor change made to an existing product, a new product code to identify the product is generated. There are currently in excess of 40,000 product codes. Gigante also has separate systems for service provisioning and customer billing with a number of front ends used at the user interface. Often, different products require slightly different procedures to enter data into the provisioning system. Each of these variations is documented and CSR’s are expected to know how to enter orders for each product type into the provisioning system. Incorrect entry of the product code into the provisioning system results in service records, generated as customers use a service, dropping to an “error bucket”. CSR’s must correct the initial product code entry before service records in the error bucket can be re-introduced to the billing system and the customer billed for the service. Gigante has a policy which excludes charging for services provided more than 12 months earlier. As a result of this policy, and the difficulties encountered in correcting errors, many service records are deleted from the error bucket before errors are corrected. This results in loss of revenue to Gigante.

294

B. Campbell

The problem observed by Gigante is that a new service provisioning process maintains its integrity for about 3 months and then starts to deteriorate with ever more data entry errors being made. The objective of this research is to find out why and to recommend corrective action. A system dynamics approach has been taken to model the interactions between the IS and the larger system within which it resides.

4.

Modelling with System Dynamics

Previous research has shown that power and politics can be extremely influential during the analysis, design and construction phases of an IS [20]. The current research, however, focuses on the effect of pre-existing policies and practices on an already implemented process. The interest is the dynamics between the IS process and the organizational system into which it has been placed. According to Forrester [9] “System dynamics focuses on policy and how policy determines behavior”. Put another way, a premise of system dynamics is that the behaviour of a system is entirely due to its structure and not to factors outside its boundary [29]. This is due to system dynamics’ concentration on feedback loops within a system. An earlier report made reference to items such as staff turnover, level of training and work load.2 It appeared to staff at Gigante that the computer system processes were deteriorating due to organisational factors such as these. In other words, the structure of the system appeared to be giving rise to its behaviour, though how this was occurring was not understood. Initially, a causal loop diagram was developed using a group modelling technique [30]. This involved a number of personnel from Gigante who were familiar with the computer system and process under investigation and the policies and working arrangements within the customer service centres (CSC’s). The causal loop diagram, shown at Fig. 1, indicates the variables identified within the system and their causal relationships. The addition or subtraction signs near the variable names indicate whether the predictor variable will cause the dependent variable to move in the same direction (a positive link) or in the opposite direction (a negative link). For example, from Fig. 1, if Knowledge Gap increases then Required Training also increases. Conversely, a decrease in Knowledge Gap will cause a decrease in Required Training. This is an example of a positive link. A negative link, in Fig. 1, exists between Lost Revenue and CSR Nos. If Lost Revenue increases there will be pressure to decrease CSR Nos. to reduce costs. If the sum of all the negative links within a feedback loop is a positive number, the loop will exhibit the behaviour of either a vicious or virtuous cycle, or, in system dynamic terms, a “positive” or reinforcing feedback loop. That is, a change in any of the variables within the loop will cause the overall effect of the loop to move away from equilibrium, often in an exponential manner. This is shown in Fig. 1 as a 2

The source cannot be cited as it would identify Gigante, but a report was prepared by a well known multi-national consulting firm.

Process Failure in a Rapidly Changing High-Tech Organisation

295

“snowball” icon within the loop. Conversely, if the sum of the negative links is negative - creating a “negative” or balancing feedback loop - the loop will attempt to return to equilibrium after a change in any of the variables. This often results in an oscillation of the variables within the loop around some mean value.3

Defections

+

-

Lost Revenue

Customer Satisfaction

+ Unbilled Revenue

CSR Numbers +

+ -

+ Enquiries Errors

Capability of CSC

+

+

+ Corrective Action

Knowledge Gap +

+

-

Knowledge +

Training Hours

-

Work Load + + Target Productivity

New Products

+ +

Required Training

Fig. 1. Simplified Causal Loop Diagram of the Product Provisioning Process within Gigante

5.

Results

The development of the causal loop diagram created many insights for the group modelling exercise participants [2, 7, 22, 27, 30]. It is apparent that although the computer process works as designed a number of other softer, organisational issues are affecting the behaviour of the entire process. For example, from the loops within Fig. 1, it can be seen that an increase in errors leads to an increase in workload, which leads to a decrease in available training time, leading to a decrease in knowledge, which leads to an increase in the knowledge gap, which leads to a decrease in the 3

For a discussion of causal loop diagrams, and some of their problems, see [26].

296

B. Campbell

capability of the CSC which in turn leads to an increase in errors... This is an example of a positive feedback loop, in this case a vicious circle. If the size of the error bucket, or any other factor, results in an unacceptably high backlog of work, it has been management policy to increase Target Productivity (see Fig. 1) in the short term. The causal loop diagram indicates that this will trigger the vicious cycle just described invalidating management’s decision. 4 The immediate effect is a reduction in the backlog, but due to the time delay involved in the loop, the long term effect is an increase in errors and the size of the error bucket. A side effect reported by Senge [28] is that quality of work will continually erode. Although no attempt has been made to measure quality of work to date, it could be argued that the increase in errors is an indication of erosion of quality. This investigation has revealed that a number of the organisational policies within Gigante are having an adverse effect on the product provisioning process. For example, the increase in workload, the poor pay structure and the limited opportunity for advancement (also noted in the report referred to earlier) are leading to poor motivation. This in turn is leading, in part, to a very high turnover of staff within the CSC’s - up to 200% per year being reported in the earlier report (see Footnote 2). This is affecting the ability of Gigante to provide necessary training, both formal and on the job, to CSR’s. CSR performance is measured on quantity of work done - not quality. Brown [4] discusses the unintended consequences of inappropriate measures of performance and the consequences of considering personnel as part of the machine - inert objects that perform given tasks without variance. Although not tested as yet, it would appear that CSR’s within Gigante learn, over time, that they can enter incorrect product codes into the system and have them accepted. This increases their throughput at the time and satisfies their performance measurements, but increases the number of errors in the long term. It is not known whether this action is a conscious attempt to “beat the system” or an innocent passing on of incorrect information from CSR to CSR. In summary, the group model building exercise to develop the causal loop diagram at Fig. 1 indicated that the deterioration of the product provisioning process, over time, within Gigante is due to a number of organisational issues. These include:• The product differentiation strategy adopted, leading to constant change of the process • Personnel policies which lead, at times, to an abnormally high turnover of staff • Inappropriate personnel performance measurements • An unrealistically high performance target for CSR’s leading to a high workload per CSR. This in turn makes it difficult to allow time to train staff. The causal loop diagram indicates that the poor performance of the system leads to either a loss of revenue to Gigante in a number of ways: (i) directly, due to customers not being billed for service records held in the error bucket, and (ii) loss of customers due to poor customer service. 4

A similar situation is described by Abdel-Hamid and Madnick [1] when investigating software project management.

Process Failure in a Rapidly Changing High-Tech Organisation

297

Causal loop diagrams are a qualitative analysis technique. How the system reacts over time cannot be determined from a causal loop diagram [29], and research has indicated that even most people experienced in system dynamics modelling are unable to determine the behaviour of a system that has more that one feedback loop without the use of computer simulation [8]. For these reasons a formal quantitative system dynamics (stock and flow) model is being developed from the previous causal loop diagram, a simplified version of which is shown at Fig. 2.

Fig. 2. Quantitative System Dynamics model of Gigante’s Product Provisioning Process [22].

System dynamic models are made up of stocks (represented by rectangles, eg Customer Satisfaction in Fig. 2), flows (represented as arrows with circular flow regulators, eg satisfn varn in Fig. 2) and converters (represented by circles, eg change programme in Fig. 2). Only flows can add to, or subtract from, a stock and the units of measure of a stock and its flows must be identical. Converters, as the name

298

B. Campbell

suggests, are used to convert measurements from one unit of measure to another. They also have a number of other functions [15]. The mathematics of a system dynamics model is a series of difference equations attached to the flows, and many of the converters In totality, system dynamics models may be used to both model the dynamics of a system over time, and closely represent an observed system [5]. Data obtained during the research is being used to quantify the variables and parameters of the stock and flow model. Intangible variables are quantified using available data and the clients’ knowledge of their behaviour. Although these cannot be accurately quantified due to their intangible nature it has been argued that omitting them assumes that their value is zero - about the only value that it is known they can’t have - and for this reason they should be included in a quantitative system dynamics model [10]. Once the model has been fully quantified and validated, using the techniques appropriate to system dynamics [3, 10, 11, 24], its simulation capabilities may be used to determine which of the policies and practices within the CSC’s are having the most effect on the provisioning process, and which offer the most effective leverage to gain an improvement in the overall process. This is done by manipulating the policy variables and then graphing the dependent variables over time. Because of the structure of the model, it is possible to link the effect of policies to lost revenue even though these may not be closely aligned in time. Note that because of the level of aggregation in many system dynamics models, and the measurement of some variables (for example, Knowledge is measured in “units of knowledge”, an artefact which cannot be accurately quantified), the graphs produced by system dynamics models of social systems should normally be taken as indicating trends only - not absolute amounts.

6.

Further Research

The current research has modelled the softer factors (policies and practices) which affect a process which has already been implemented. Later work in this project will attempt to quantify the effects of these factors on the business performance of Gigante to determine which factors offer the most leverage to improve the overall performance of the system. Those factors so far identified as being likely candidates are not within the IS system itself, but belong to the softer, organisational, issues. The policies and practices which have so far been identified as affecting the performance of the provisioning system within Gigante appear to be common to many organisations - especially within the service industry. They have been reported by other writers such as Senge [28] and Abdel-Hamid and Madnick [1]. If these are, in fact, common policies and practices it would indicate that many well designed processes cannot perform optimally due to the system/situation into which they have been placed.

Process Failure in a Rapidly Changing High-Tech Organisation

299

A question then is, when should these factors be modelled? Should they be considered during analysis and design, or immediately prior to, or after, implementation? It could also be argued that they are normal management problems and not directly related to the design and implementation of an information system. The latter view carries some risk to IS developers due to the demonstrated impact of these factors on an implemented system. More research is needed to: (i) determine whether there are common policies and practices which are affecting implemented systems, and (ii) quantify these effects.

References 1. 2.

3. 4.

5. 6.

7. 8.

9.

10. 11.

12.

13.

Abdel-Hamid T., and Madnick S.E., 1991, Software Project Dynamics: An Integrated Approach, Prentice Hall, New Jersey Anderson D.F., Richardson G.P., and Vennix J.A.M., “Group model building: adding more science to the craft”, in System Dynamics Review, Vol. 13, No. 2, (Summer 1997), John Wiley & Sons Ltd. Barlas Y., 1996, “Formal aspects of model validity and validation in system dynamics”, in System Dynamics Review, Vol. 12, No. 3, Fall 1996, John Wiley and Sons, pp 183-210. Brown W.B., 1969, “The Organization and Socio-Technical Controls”, in Scott W.G. (ed), Organization Concepts and Analysis, Dickenson Publishing Company, Belmont, California Curtis B., Kellner M.I., and Over J., 1992, “Process Modelling” in Communications of the ACM, Vol. 35, No. 9, September 1992, pp 75-90 Curtis B., Krasner H. and Iscoe N., 1988 “A Field Study of the Software Design Process for Large Systems” in Communications of the ACM, Vol. 31, No. 11, November 1988, pp 1268-1287 de Geus A.P., 1994, “Modelling to Predict or to Learn?” in Morecroft J.D.W., and Sterman J.D. (eds), Modeling for Learning Organizations, Productivity Press, Oregon Dangerfield B. and Roberts C., 1995, “Projecting dynamic behavior in the absence of a model: an experiment”, in System Dynamics Review, Vol. 11, No. 2, John Wiley & Sons, pp 157-172 Forrester J.W., 1980, “System Dynamics - Future Opportunities” in Legasto A..A., Forrester J.W. & Lyneis J.M. (eds), System Dynamics, North Holland Publishing Co., Amsterdam, pp 7-22 Forrester J.W., 1961, Industrial Dynamics, M.I.T. Press and John Wiley & Sons, New York. Forrester J.W. and Senge P.M., 1980, Tests for Building Confidence in System Dynamics Models, in Legasto A.A., Forrester J.W. and Lyneis J.M. (eds), System Dynamics, TIMS Studies in Management Sciences, Vol. 14, North-Holland Publishing, New York, pp 209228 Gasson S., and Holland N., 1996, “The Nature and Processes of IT-Related Change” in Orlikowski W.J., Walsham G., Jones M.R., and DeGross J.I., (eds), Information Technology and Changes in Organizational Work, Chapman and Hall, London, pp 213234 Ginzberg M.J., 1981, “Early Diagnosis of MIS Implementation Failure: Promising Results and Unanswered Questions”, Managment Science, Vol. 27, No. 4, April 1981, pp 459-478

300

B. Campbell

14. Gruhn V., and Wolf S., 1995, “Software Process Improvement by Business Process Orientation”, in Software Process - Improvement and Practice”, Pilot Issue, John Wiley & Sons, August 1995 15. HPS, 1994, “Introduction to Systems Thinking and iThink”, High Performance Systems Inc., Hanover, USA 16. Jackson, M.C. 1997, “Critical Systems Thinking and Information Systems Development” in Proceedings, 8th Australasian Conference on Information Systems, 1997, Flinders University, South Australia 17. Klein H.K., and Hirschheim R.A., 1987, “A Comparative Framework of Data Modelling Paradigms and Approaches”, in The Computer Journal, Vol. 30, No. 1 18. Kueng P. and Kawalek P., 1997, “Process Models: a help or a burden?” in online Proceedings, Association for Information Systems 1997 Americas Conference, Indianapolis, Indiana, August 1997, URL: http://hsb.baylor.edu/ramsower/ais.ac.97/papers/kueng.htm 19. OASIG, 1996, “The performance of Information Technology and the role of human and organizational factors” Report to the Economic and Social Research Council, UK. URL:- http://www.shef.ac.uk/~iwp/resprogs/itperf.html or in Word 2 format from:- ftp://ftp.shef.ac.uk/pub/uni/academic/I-M/iwp/itperf.doc 20. McGrath G.M., 1992, Migrating Information Systems Through the Analysis of Power, Its Determinants and Distribution, PhD thesis presented to the School of Mathematics, Physics, Computing and Electronics, Macquarie University, Sydney, Australia 21. McGrath G.M., 1997, “A Process Modelling Framework: Capturing Key Aspects of Organisational Behaviour”, in Proceedings of the Australian Software Engineering Conference - 1997, Sydney, 29 Sep. - 2 Oct., 1997, pp 118-126 22. McGrath G.M., Campbell B.R., More E. and Offen R.J., 1998 “Intra-Organisational Collaboration in a Complex, Rapidly-Changing Environment: A Field Study”, a paper prepared for the Seminar on Collaboration, held at the University of Technology, Sydney, on 8 May, 1998. 23. Morecroft J.D.W., (1994), “Executive Knowledge, Models, and Learning”, in Morecroft J.D.W. and Sterman J.D., (eds), Modeling for Learning Organizations, Productivity Press, Oregon 24. Peterson D.W. and Eberlein R.L., 1994, “Reality Check: a bridge between systems thinking and system dynamics”, in System Dynamics Review, Vol. 10, Nos. 2-3, SummerFall 1994, John Wiley & Sons Ltd. 25. Porter M.E., 1980, Competitive Strategy: Techniques for Analysing Industries and Competitors, Free Press, New York 26. Richardson G.P., 1986, “Problems with causal-loop diagrams” in System Dynamics Review”, No. 2, Summer 1986, John Wiley & Sons, London, pp 158-170 27. Richmond B., 1997, “The Strategic Forum: aligning objectives, strategy and process”, in System Dynamics Review, Vol. 13, No. 2, John Wiley & Sons Ltd. 28. Senge P.M., 1992, The Fifth Discipline: The Art & Practice of The Learning Organization, Random House, Australia 29. Sterman J.D., 1994, “Learning in and about complex systems” in System Dynamics Review, Vol. 10, Nos. 2-3 (Summer-Fall 1994), John Wiley & Sons, pp 291-330 30. Vennix J.A.M. 1996, Group Model Building: Facilitating Team Learning Using System Dynamics, John Wiley & Sons Ltd, Chichester, England

ROL2: A Real Deductive Object-Oriented Database Language Mengchi Liu and Min Guo Department of Computer Science, University of Regina Regina, Saskatchewan, Canada S4S 0A2 {mliu,guomin}@cs.uregina.ca

Abstract. ROL is a strongly typed deductive object-oriented database language. It integrates many important features of deductive databases and object-oriented databases. However, it is only a structurally objectoriented language. In this paper, we describe our extension of ROL called ROL2. ROL2 keeps all the important features of ROL. In addition, it incorporates important behaviorally object-oriented features such as rulebased methods and encapsulation so that it is a real deductive objectoriented database language. It supports object identity, complex objects, class hierarchy, methods, non-monotonic multiple structural and behavioral inheritance with overriding and blocking.

1

Introduction

In the past decade, a number of deductive object-oriented database languages have been proposed, such as O-logic [21], revised O-logic [15], IQL [1], LOGRES [7], Datalogmeth [2], CORAL++[26], Gulog [10], Rock & Roll [3] Flogic [14], and ROL [18,19]. However, most of them are only structurally objectoriented. Important behaviorally object-oriented features such as methods and encapsulation common in object-oriented database systems/languages such as GemStone [6], ONTOS [25], O2 [9], Orion [16], Iris [11], ObjectStore [17], and ODMG-93 [8] are not properly supported. In particular, O-logic, revised O-logic, IQL, LOGRES, DLT, Gulog, and ROL do not support methods, let alone encapsulation of methods. In Datalogmeth [2] and Datalog [12], methods can be defined using rules and can be encapsulated. However, the methods can only perform deduction rather than updates and their signatures cannot be defined. Besides, the semantics is given indirectly by translating method rules into Datalog rules. In F-logic, rule-based methods are supported and their signatures can be defined. However, its semantics is quite involved and the methods cannot perform updates and cannot be encapsulated and inherited. On the other hand, several deductive object-oriented languages that are both structurally and behaviorally object-oriented simply provide two kinds of incompatible languages: rule-based declarative language for deduction and queries, and imperative language for data definitions and manipulations, and complex tasks. For instance, CORAL++ provides rule-based query language CORAL [24] augmented with C++ expressions for accessing attributes and invoking rules and T.W. Ling, S. Ram, and M.L. Lee (Eds.): ER’98, LNCS 1507, pp. 302–315, 1998. c Springer-Verlag Berlin Heidelberg 1998

ROL2: A Real Deductive Object-Oriented Database Language

303

imperative programming language C++ augmented with new types and constructs. Similarly, Rock & Roll offers the imperative programming language Rock and rule-based language Roll. Unlike CORAL++, both Rock and Roll are based on a single underlying object-oriented data model. Offering two incompatible languages suffers various degrees of impedance mismatch and lacks logic-based semantics. In [4], general rule-based methods that can perform deduction as well as updates and encapsulation of these methods are discussed informally. The proposal is in the spirit of Prolog++ [22] in which Datalog facts and rules with action predicates of LDL [23] for specific objects are encapsulated together. However, proper syntax and semantics for the proposed framework are not defined. In Transaction F-logic [13], it is proposed to use F-logic [14] to model the structural aspect and Transaction Logic [5] to model the behavior aspects. Methods for updates can be supported but not the encapsulation. The reason is that there is no clear separation between the notions of schema and instance. In this paper, we describe our extension of ROL [18,19], called ROL2. ROL2 keeps all the important features of ROL. In addition, it incorporates important behaviorally object-oriented features such as rule-based methods and encapsulation so that it is a real deductive object-oriented database language. It supports object identity, complex objects, classes, class hierarchies, methods, non-monotonic multiple structural and behavioral inheritance with overriding and blocking. This paper is organized as follows. Section 2 focuses on schema and class definitions in ROL2. Section 3 discusses class hierarchy and multiple inheritance with overriding and blocking in ROL2. Section 4 describes object creation and databases in ROL2. Sections 5 and 6 discuss how to query and update a ROL2 database respectively. Section 7 concludes the paper.

2

Schema

In ROL2, the schema is a set of class definitions. A class describes the structure and behavior of a set of objects. The structural part of a class is a set of attribute definitions and default values. The behavioral part of a class is a set of methods defined using rules. In this section, we describe classes, attribute definitions and default values, methods, and class definitions. Classes ROL2 is a language that centers on objects. Objects that share common structural and behavioral properties are classified into classes. A class has two aspects: it denotes a collection of objects; it specifies the attributes and operations applicable to its objects. An object in the collection denoted by a class is an instance of the class. Four kinds of classes are distinguished: value classes, oid classes, functor classes and set classes. The following are examples of classes:

304

M. Liu and M. Guo

Value classes integer, real, string, integer(0..100), string({0 M 0 ,0 F 0 }) Oid classes person, student, dept, course, supplier, part Functor classes f amily({person}), supplies(supplier, part) Set classes {persons}, {f amily({person})} Note that ROL2 supports subranges of integers and reals such as integer (0..100) and enumerated strings such as string({0 M 0 ,0 F 0 }). The collections which value classes denote are fixed. The collections which oid or functor classes denote are user specified. The collections which set classes denote depend on the collections which the component classes denote. In addition, ROL2 supports two built-in classes object and none. The former denotes all objects while the latter has no instance at all. Attribute Definitions. The attribute applicable to all instances of a class must be defined on the class. Consider the following example: person[birthyear ⇒ integer(1900..2000); gender ⇒ string({0 M 0 ,0 F 0 }); spouse ⇒ person; parents ⇒ {person}] It declares attributes birthyear, gender, spouse and parents to be partial mappings from the oid class person to value classes integer(1900..2000) and string ({0 M 0 ,0 F 0 }), itself, and set class {person} respectively. If bob is an instance of person then bob[birthyear → 1960] specifies that bob has value 1960 for the attribute birthyear. It is well-typed wrt the attribute definition. But we cannot have both bob[birthyear → 1960] and bob[birthyear → 1950] in the database as birthyear here is no longer a mapping. Besides, the attribute values of some objects can be unknown as attributes are partial mappings. Default Values. It is more and more common to see that software systems provide various defaults to ease the use of the system. For example, Oracle allows the user to give default values. Besides, deductive object-oriented languages such as F-logic [14] and Datalog++ [12] also support default values (by different name though). In ROL2, we use a syntax similar to F-logic for default values. Consider the following example: person[birthyear •→ 1950; gender •→0 M 0 ] It says for every person, if the value for the attribute birthyear or gender are not given, then it takes the default values. Otherwise, the given attribute values override the default values. As ROL2 supports both partial and complete set terms of ROL, we can provide different attribute values for set-valued attributes. Consider the following example: f aculty[degrees •→ h0 P hD0 i]; f aculty[degrees •→ {0 P hD0 }];

ROL2: A Real Deductive Object-Oriented Database Language

305

The first one uses a partial set h0 P hD0 i saying that if the degrees of a faculty member are not given, or given partially with a partial set, then P h.D is one additional degree but if the degrees of a faculty member are give fully with a complete set, then they override the default value. The second one uses a complete set {0 P hD0 } saying that if the value for the degrees of a faculty is not given at all, then Ph.D is the only degree of the faculty. Otherwise, the given attribute value overrides the default value. Methods. Objects can be queried and manipulated using methods in ROL2. A method is one or more rules attached to a specific class which can be applied to objects of this class. A method has two components: signature and implementation. The signature of a method specifies the class to which it is attached to, the name of the method, the class of the arguments the method takes, and the class of the result it returns if any. The following is a signature definition: person[age() ⇒ integer(0..100); children() ⇒ {person}; ancestors() ⇒ {person}; divorce(); marry(person)] It declares 0-ary methods age, children, ancestors and divorce, and the class of the result they return and a unary method marry that does not return any values. Note that attribute definitions differ from method signatures in ROL2. In ROL2, methods are implemented using rules formed from object expressions. The following are examples of method implementations of age, children, ancestors and marry in the class person: person[P.age() → A :– P.birthyear → B, A = 1998 − B; P.children() → hCi :– C.parents → hP i; P.ancestors() → hAi :– P.parents → hAi; P.ancestors() → hAi :– P.parents → hBi, B.ancestors() → hAi; P.divorce() :– delete P.spouse → S, delete S.spouse → P ; P.marry(S) :– not P.spouse → S1, not S.spouse → S2, insert P.spouse → S, insert S.spouse → P ] The first one specifies how to calculate a person’s age from his/her birth year. The second specifies how to find all person’s children by using a partial set term hCi. The next two rules specifies how to find a person’s ancestors from his/her parents recursively. The fifth rule specifies how to let two married couple divorce. The last one specifies how to let two unmarried persons P and S be spouse of each other. Note that the last two rule uses the update command insert and delete. As rules must be encapsulated in their class definition in ROL2, it is not necessary to explicitly specify all variables in the rules. The above rules can be simplified into the following equivalent rules under variable renaming:

306

M. Liu and M. Guo

person[age() → A :– birthyear → B, A = 1998 − B; children() → hCi :– C.parents → hSelf i; ancestors() → hAi :– parents → hAi; ancestors() → hAi :– parents → hP i, P.ancestors() → hAi; divorce() :– delete spouse → S, delete S.spouse → Self; marry(S) :– not spouse → S1, not S.spouse → S2, insert spouse → S, insert S.spouse → Self ] where Self is a reserved variable. It can be seen that only the variable Self that appears in the beginning of an object expression is omitted from the rules. Method Overloading As with object-oriented programming languages such as C++ and Java, ROL2 supports method overloading. That is, a method name can be used for several methods with different signatures in the same class. Consider the following method signatures: person[children() ⇒ {person}; children(person) ⇒ {person}] Both methods have the same name children. The first one is a 0-ary method while the second is a unary one. The following are their implementations: children() → hCi :– C.parents → hSelf i; children(S) → hCi :– C.parents → {Self , S}, Self 6= S; Class Definitions Based on the above discussion, a class has attribute definitions, default values, and method with signature and implementation. In ROL2, we have to encapsulate all the above information in the class definition. For example, the class person discussed above can be declared in ROL2 using the insert command as follows: insert person[ birthyear ⇒ integer(1900..2000); gender ⇒ string({0 M 0 ,0 F 0 }); ... ] Note that instead of creating a class all at once, several insert commands can be used to create a complete class.

3

Class Hierarchy and Inheritance

In ROL2, we can organize classes into a class hierarchy with the two builtin classes object and none at the top and bottom respectively by using direct subclass/superclass definitions and take advantage of non-monotonic multiple structural and behavioral inheritance. A subclass in ROL2 inherits all attribute definitions, default values, and methods of its direct superclasses and can introduce additional attribute definitions, default values, and methods. Besides, every instance of a subclass is a non-direct instance of its superclasses.

ROL2: A Real Deductive Object-Oriented Database Language

307

Consider the following class definition: employee isa person [ salary ⇒ real; salary •→ 1000; raise(real); raise(R) :– delete salary → S1 , S2 = S1 + S1 ∗ R, insert salary → S2 ] It says that employee is a subclass of person. Therefore, employee inherits all attribute definitions, default values, and methods from person. In addition, it also has its own attribute definition salary, default value for salary, and method raise. Every instance of employee is also a non-direct instance of person. For direct subclasses of object, we can omit object in the class definition. For example, the class person can be defined in two different ways in ROL2: person isa object[...] person [...] Inheritance with Overriding and Blocking. In ROL2, a subclass can also override or block attribute definition, default value and method inheritance from its superclasses. Consider the following two class definitions: patient [treatedBy ⇒ physician; treatedBy •→ smith] alcoholic isa patient [treatedBy ⇒ psychologist] The first one says patients are treated by physician with smith as the default physician. In this case, smith must be a physician at the time the class definition is added to the database. Otherwise the class definition is not well-typed. The second says alcoholics are patients but are treated by psychologist. However, psychologists usually are not physicians. The subclass alcoholic introduces a new definition for attribute treatedBy which overrides the attribute definition treatedBy and the corresponding default value inherited from patient. The following example shows how to override default values without overriding attribute definitions. student isa person[ takes ⇒ courses; enroll(course); drop(course)] birthyear •→ 1975; gender •→0 F 0 ; enroll(C) :– insert takes → hCi; drop(C) :– delete takes → hCi] As a subclass of person, student inherits all attribute definitions, default values and methods of person. However, its own attributes takes, default values for birthyear and gender which override the inherited ones, and methods drop and takes.

308

M. Liu and M. Guo

The next example shows how to block attribute and method inheritance from a superclass. bachelor isa person[ spouse ⇒ none; marry(person) ⇒ none; divorce() ⇒ none] The built-in class none is used here to block the inheritance of the corresponding attribute and methods (both signature and implementation) from person to bachelor. As ROL2 supports method overloading, it is necessary to specify the method name and the class of its arguments for blocking. Besides, attribute definitions and their default values if any are bound together in terms of inheritance. That is, if a subclass inherits an attribute definition from one class, then it can only inherit the default value if any for the corresponding attribute from the same class. If the attribute definition is overridden or blocked, then the corresponding default value is also blocked. Non-Monotonic Multiple Inheritance. ROL2 supports multiple inheritance. That is, a subclass can be a direct subclass of more than one superclass. Therefore, inheritance conflicts may happen. Consider the following class definitions: employee [room ⇒ string] student [room ⇒ integer; room •→ 321] wstudent isa employee, student Here wstudent is a subclass of employee and student. It inherits attributes and methods from both employee and student and a conflict on the attribute room happens. ROL2 provides four different ways for conflict resolution with user-specified priority as the default mechanism: (1) User-Specified Priority. (2) Overriding. (3) Inheritance Path Selection. (4) Explicit Renaming. User-Specified Priority In ROL2, there is a built-in left-to-right ordering of superclasses in a class definition. If two or more superclasses have the attribute definitions and default values with the same name or methods with the same signature either directly defined or inherited, then the one found in the class appearing first in the list is inherited. Continue with the previous example, wstudent automatically inherits attribute room from employee rather than from student. It does not inherit the default value for room from student. However, if two or more superclasses have methods with the same name but different signatures, then all of them are inherited by the subclass. In this case, the method is just overloaded. Overriding As ROL2 supports overriding, the inheritance conflicts may be resolved by using overriding.

ROL2: A Real Deductive Object-Oriented Database Language

309

Continue with the previous example. If we want the class wstudent to inherit the attribute definition and default value for room from student instead of employee without changing the order of superclasses, we can repeat the definition of student in wstudent as follows so that it overrides the inherited one from employee: wstudent isa employee, student [ room ⇒ integer; room •→ 321] However, this approach breaks the connection between the attribute room of student and the attribute room of wstudent so that any changes to the attribute definition or default value of room in student will not be reflected in wstudent. If this is a problem, then the following alternative should be used. Inheritance Path Selection If a subclass needs to inherit one attribute definition, default value, or method from one superclass and another from a different superclass, then user-specified priority and overriding cannot meet these demands as discussed earlier. To solve this problem, ROL2 allows the user to specify the inheritance path to override user-specified priority using the from keyword. Continue with the previous example of wstudent. Suppose that employee and student also have a method income with same signature but different implementations. If we want wstudent to inherit the attribute room and method income from student instead of employee, then we can use the following class definition: class wstudent isa employee, student [room, income() f rom student] Note that with this class definition, wstudent inherits not only attribute definition but also default value of room from student. If the method income is overloaded in wstudent, only the 0-ary method is inherited with the above definition. Explicit Renaming This alternative is used to avoid conflicts by renaming one or more conflicting attributes or methods so that they can be inherited under different names. Continue with the example of wstudent. A working student may have two different rooms, one as a student and the other as an employee. The previous alternatives disallow this possibility. In this case, we can use explicit renaming as follows: wstudent isa employee, student [ employee.room as eroom; student.room as sroom] As the room attribute from student and employee have different names in wstudent, there is no inheritance conflict.

4

Object Creation and Databases

In ROL2, the user can creat new objects and assign their attribute values using the insert commands. Unlike other object-oriented systems, oid objects in ROL2

310

M. Liu and M. Guo

are created by the user rather than the system. They can only be a direct instance of one class and they have to be system-wide unique. The following are object creation examples: insert person : tom insert tom [birthyear → 1930] insert person : pam [birthyear → 1935] The first command creates a new oid object tom to be a direct instance of person. If tom is already an instance of class, then the operation fails. The second assigns 1930 as the value of the attribute birthyear of tom. The attribute value to be assigned is first checked with respect to the corresponding class definition. Only if it is well-typed with respect to the class definition, then it can be assigned to the object. The third command creates a new oid object pam and also assigns 1935 as the value of its birthyear attribute. A ROL2 database is just a set of objects together with their direct classes and attribute values. The following is an example of a ROL2 database. person : tom [birthyear → 1930 ; spouse → pam] person : pam [birthyear → 1935 ; spouse → tom] employee : ann [spouse → jim] employee : jim [spouse → ann; salary → 150] employee : bob [ birthyear → 1953 ; parents → {pam, tom}; salary → 210] student : sam [ parents → hjimi] studnet : pat [ parents → {bob}, salary → 100] wstudent : joe [ parents → {sam, pat}] wstudent : liz [ birthyear → 1970, parents → {bob}]

5

Queries

The ROL2 database can be queried using the query commands. There are two kinds of queries in ROL2: schema query and object query. In this section, we describe them using the examples given in the previous sections. Schema Queries. Schema queries are used to retrieve information about classes. Schema queries can be roughly classified into class queries, attribute definition queries, default value queries, and method queries. Class Queries This kind query is used to query classes and subclass relationships. Consider the following examples: (1) (2) (3) (4)

query query query query

C isa object C isa ∗ object wstudent isa C C isa ∗ person, not C isa person

The first query asks for each direct subclass of the built-in class object. The second asks for every subclass of object, that is, every user defined classes. The third asks for each direct superclass of wstudent. The answers are student and

ROL2: A Real Deductive Object-Oriented Database Language

311

employee. The last one asks for each non-direct subclasses of person. The answer is wstudent. Attribute Definition Queries This kind of query is used to retrieve directly defined or inherited attribute definitions of classes. The following are several examples: (1) (2) (3) (4)

query query query query

person [ spouse ⇒ ] person [ A ⇒ C ] bachelor [ spouse ⇒ ] C [ A ⇒ person ]

The first query asks whether the class person has an attribute spouse where ’ ’ is an anonymous variable. The answer is yes based on the class definition given before. The second asks for each attribute definition of person. The third asks whether the class bachelor has an attribute spouse. The answer is no because of inheritance blocking. The last one asks for each class and its attribute that is defined on person. Default Value Queries ROL2 supports default attribute values and queries on them. The following are several default value queries: (1) (2) (3) (4)

query query query query

person [ birthyear •→ 1975 ] C [ birthyear •→ 1975 ] person [ A •→ V ] C [ A •→ V ] isa ∗ person

The first query asks if the class person has default value 1975 for attribute birthyear. The answer is no based on the class definition given before. The second asks for classes that has default value 1975 for the attribute birthyear. The answers are student and wstudent based on the class definitions given before. The third asks for each default value and the corresponding attribute of the class person. The last one asks for each default value and the corresponding attribute of each direct and non-direct subclass of person. Method Queries A method query can be used to query either method signatures or method implementations (i.e., rules). If a method query contains variables, then it is used to query method signatures. Otherwise, it is used to query method implementations. The following are several method queries: (1) (2) (3) (4)

query query query query

person [ age() ⇒ C ] person [ M () ⇒ C ] person [ age() ] student [ enroll(course) ]

The first one is a method signature query asking for the class of the result that the method age returns. The second is also a method signature query asking for each 0-ary method of person and the class of the result. The next two are

312

M. Liu and M. Guo

method implementation queries asking for the implementation of the methods age of person and enroll of student. Object Queries. Object queries are used to retrieve information about objects and manipulate object through method invocation. Object queries can be roughly classified into object class queries, object attribute queries, and method invocation. Object Class Queries This kind of query is used to ask for class information of objects. The following are several such examples: (1) (2) (3) (4)

query query query query

person : bob C : bob C : ∗ bob C:O

The first query asks whether the class person is the direct class of the oid bob. The answer is no based on the database in the last section. The second asks for the direct class of bob. The answer is employee. The third asks for each direct or non-direct classes that bob is an instance of. The answers are person and employee. The last one asks for each class and each direct instance of the class. Object Attribute Queries This kind of query is used to ask for attribute values of objects. Consider the following examples: (1) (2) (3) (4)

query query query query

tom [ birthyear → B ] jim [ birthyear → B ] person : O [ birthyear → B ] O [ A → S]

The first query asks for the birth year of tom. The answer is 1930. The second asks for the birth year of jim. As the value is not given for jim in the database, the default value 1950 is returned as the result. The third asks for every direct instance of person and the person’s birth year. The last one is a very general query asking for each object and each of its attribute values. Method Invocation Rule-based methods in ROL2 can perform deduction as well as updates. They are invoked using object queries as well. Consider the following examples: (1) (2) (3) (4)

query query query query

liz [ age() → V ] liz [ ancestors() → hjimi ] liz [ M () → V ] ann [ marry(sam)]

The first query invokes the age method on liz. The second invokes the ancestors method on liz and check whether or not jim is an element in the result. The third invokes each 0-ary method that returns a value on liz. The last one invokes the marry method to let ann and sam be married.

ROL2: A Real Deductive Object-Oriented Database Language

6

313

Updates

ROL2 provides two elementary update commands: insert and delete, with which complex update commands can be formed. An update command in ROL2 is treated as a transaction which can either succeed or fail. If it fails, it has no effect on the database at all. Update commands are used to define and manipulate databases. They can be classified loosely into two kinds: schema updates and object updates. Schema Updates In ROL2, the user can insert, delete or modify class definitions using schema updates. Sections 2 and 3 have shown how to use the insert command for class definitions. The following examples show how to delete and modify classe definitions: (1) delete person[birthyear ⇒ ] (2) delete student[birthyear ⇒ ] (3) delete student[enroll()] (4) delete student The first command says delete the directly defined attribute birthyear from person. It will fail if any of its instance has a value for this attribute. The second is similar to the first one. However, as birthyear is inherited, it can only be blocked rather than deleted from the class student. The next one says delete both the method signature and implementation of enroll from student. The last one says delete the class student. It will fail if the class has any instance or subclass or is used in other classe definitions. Object Updates In ROL2, oid objects are system-wide unique that can be created or destroyed but cannot be modified. Their attribute values may change from time to time though method invocations or object updates. The insert and delete commands are used to to create or destroy objects and to assign or delete their attribute values. The combination of insert and delete are used to modify attribute values. Section 4 has shown how to use the insert command to create objects and assign their attribute values. The following are several further examples of object updates: (1) delete bob [birthyear → ] (2) delete bob [parents → hpami] (3) delete employee : bob (4) delete pat [salary → S1 ], S2 = S1 + 100, insert pat [salary → S2 ] The first command says delete the value of the attribute birthyear of bob. As this value is given by the user, it is deleted so that the value is now the default value. If the value is not given by the user, then the operation fails as the default value cannot be deleted with object updates. The second says delete pam from the Bob’s parents set. As the parents set is a complete set {pam,tom}, the attribute value after the operation is still a complete set {tom}. The next command says delete bob from its direct class employee as well as all of its attribute values if any. The last one says increase pat’s salary by 100.

314

M. Liu and M. Guo

In ROL2, we can combine query and update commands to specify complex updates. The following are several complex update examples: (1) query E[salary → S], S > 100, delete E (2) query P [age() → A], A > 40, delete P [salary → S1 ], S2 = S1 + 100, insert P [salary → Z] The first command says fire every employee who makes more than 100. The second one says increase the salary of every employee who are 40 years old by 100.

7

Conclusion

We have described a real deductive object-oriented database language ROL2 which is an extension of a structurally object-oriented deductive language ROL. Unlike other approaches, ROL2 supports almost all important features that one may expect an object-oriented language to have. We have partially implemented ROL2 on top of ROL. The prototype is available over the Internet from the Web page http://www.cs.uregina.ca/∼ mliu/ROL2. The ROL2 system runs on SUN, DEC, SGI workstations and IBM-PC compatible machines. The implementation of the complete ROL2 language as described in this paper, is in progress and will be released as a new version. The semantics for methods that update a database and their encapsulation in ROL2 has been investigated [20]. We are working on a more general semantics for ROL2.

References 1. S. Abiteboul and P.C. Kanellakis. Object Identity as a Query Language Primitive. In Proceedings of the ACM SIGMOD International Conference on Management of Data, pages 159–173, Portland, Oregon, 1989. 2. S. Abiteboul, G. Lausen, H. Uphoff, and E. Waller. Methods and Rules. In Proceedings of the ACM SIGMOD International Conference on Management of Data, pages 32–41, Washington, D.C., 1993. 3. M. L. Barja, A. A. A. Fernandes, N. W. Paton, M. H. Williams, A. Dinn, and A. I. Abdelmoty. Design and implementation of ROCK & ROLL: a deductive object-oriented database system. Information Systems, 20(3):185–211, 1995. 4. E. Bertino and D. Montesi. Towards a Logical Object-oriented Programming Language for Databases. In Proceedings of the International Conference on Extending Database Technology, pages 168–183, Vienna, Austria, 1992. Springer-Verlag. 5. A.J. Bonner and M. Kifer. Transaction Logic Programming. In Proceedings of the International Conference on Logic Programming, pages 257–279, Budapest, Hungary, 1993. MIT Press. 6. P. Butterworth, A. Otis, and J. Stein. The Gemstone Object Database Management System. Communications of the ACM, 34(10):64–77, 1991. 7. F. Cacace, S. Ceri, S. Crepi-Reghizzi, L. Tanca, and R. Zicari. Integrating ObjectOriented Data Modelling with a Rule-Based Programming Paradigm. In Proceedings of the ACM SIGMOD International Conference on Management of Data, pages 225–236, 1990.

ROL2: A Real Deductive Object-Oriented Database Language

315

8. R.G.G. Cattell, editor. The Object Database Standard: ODMG-93, Release 1.2. Morgan Kaufmann, 1996. 9. O. Deux et al. The O2 System. Communications of the ACM, 34(10):35–48, 1991. 10. G. Dobbie and R. Topor. On the Declarative and Procedural Semantics of Deductive Object-Oriented Systems. Journal of Intelligent Information System, 4(2):193– 219, 1995. 11. D. H. Fishman, B. B., H. P. Cate, E. C. Chow, T. Connors, J. W. Davis, N. Derrett, C. G. Hoch, W. Kent, P. Lyngbaek, B. Mahbod, M. A. Neimat, T. A. Ryan, and M. C. Shan. Iris: An object-Oriented Database Management System. ACM Trans. on Office Information Systems, 5(1):48–69, 1987. 12. H.M. Jamil. Implementing Abstract Objects with Inheritance in Datalogneg . In Proceedings of the International Conference on Very Large Data Bases, pages 46– 65, Athens, Greece, 1997. Morgan Kaufmann Publishers, Inc. 13. M. Kifer. Deductive and Object Data Language: A Quest for Integration. In T. W. Ling, A. O. Mendelzon, and L. Vieille, editors, Proceedings of the International Conference on Deductive and Object-Oriented Databases, pages 187–212, Singapore, 1995. Springer-Verlag LNCS 1013. 14. M. Kifer, G. Lausen, and J. Wu. Logical Foundations of Object-Oriented and Frame-Based Languages. Journal of ACM, 42(4):741–843, 1995. 15. M. Kifer and J. Wu. A Logic for Programming with Complex Objects. J. Computer and System Sciences, 47(1):77–120, 1993. 16. Won Kim. Introduction to Object-Oriented Databases. The MIT Press, 1990. 17. C. Lamb, G. Landis, J. Orenstein, and D. Weinreb. The ObjectStore System. Communications of the ACM, 34(10):50–63, 1991. 18. M. Liu. ROL: A Deductive Object Base Language. Information Systems, 21(5):431 – 457, 1996. 19. M. Liu. An Overview of Rule-based Object Language. Journal of Intelligent Information Systems, 10(1):5–29, 1998. 20. M. Liu. Incorporating Methods and Encapsulation into Deductive Object-Oriented Database Languages. In Proceedings of the 9th International Conference on Database and Expert System Applications (DEXA ’98), Vienna, Austria, August 24-28 1998. Springer-Verlag LNCS. 21. D. Maier. A logic for objects. Technical Report CS/E-86-012, Oregon Graduate Center, Beaverton, Oregon, 1986. 22. C. Moss. Prolog++. Addison-Wesley, 1994. 23. Shamim Naqvi and Shalom Tsur. A Logical Language for Data and Knowledge Bases. Computer Science Press, 1989. 24. R. Ramakrishnan, D. Srivastava, S. Sudarshan, and P. Seshadri. Implementation of the CORAL Deductive Database System. In Proceedings of the ACM SIGMOD International Conference on Management of Data, pages 167–176, Washington, D.C., 1993. 25. V. Soloviev. An Overview of Three Commercial Object-Oriented Database Management Systems: ONTOS, ObjectStore, O2. SIGMOD Record, 21(1):93–104, 1992. 26. D. Srivastava, R. Ramakrishnan, D. Srivastava, and S. Sudarshan. CORAL++: Adding Object-Orientation to a Logic Database Language. In Proceedings of the International Conference on Very Large Data Bases, pages 158–170, Dublin, Ireland, 1993. Morgan Kaufmann Publishers, Inc.

Multiobjects to Ease Schema Evolution in an OODBMS Lina Al-Jadir, Michel Léonard Centre Universitaire d’Informatique (C.U.I.), Université de Genève 24 rue Général-Dufour, 1211 Genève 4, Switzerland {aljadir, leonard}@cui.unige.ch

Abstract. The multiobject mechanism is a pertinent way to implement specialization in an object database and differs from the classical mechanism used in most object-oriented database systems. It supports multiple instantiation, automatic classification and object migration. Consequently it is well suited to take into account schema evolution. It makes schema changes more pertinent, easier to implement, and less expensive than with the classical implementation of specialization indeed. The multiobject mechanism is implemented in the F2 database system which supports schema evolution.

1 Introduction In the classical implementation of specialization in object-oriented database systems (OODBMS) an object is an instance of one most specific class. It is completely stored in this class, i.e. all attribute values on local and inherited attributes are present in the object. This has several shortcomings for object modelling and object evolution. Since an object is an instance of only one class, one must use multiple inheritance to model real-world entities that have many facets at once. This can lead to a combinatorial explosion of sparsely populated classes, as pointed out in [34] [33] [27] [28]. Once an object is created in a class, it stays in that class until it is deleted from it. This is a serious limitation, since one is forced to model real-world entities that evolve dynamically with objects that can not, as pointed out in [30] [33] [2] [28]. Several approaches (see section 4) have been proposed to overcome these shortcomings. In this paper we address other shortcomings of the classical implementation of specialization which are related to schema evolution. Schema evolution is an essential feature of a database system to allow database applications to run in a dynamic environment. Updating the schema of a populated database has repercussions on database objects in order to keep the database in a consistent state. Supporting single-instantiated and static objects restricts schema changes. Moreover, it requires to copy data when modifying the schema which is extremely time-consuming. Suppose for example that a class Swiss is added as a subclass of Person. With a classical implementation, it is not possible to make some of the existing Person objects belong now to the Swiss subclass. For all swiss citizens, the database administrator has to create copies of the Person objects in the new subclass Swiss and to delete the T.W. Ling, S. Ram, and M.L. Lee (Eds.): ER’98, LNCS 1507, pp. 316−333, 1998.  Springer-Verlag Berlin Heidelberg 1998

Multiobjects to Ease Schema Evolution in an OODBMS

317

corresponding objects in class Person. This may be a very expensive operation if there are many swiss persons. Supporting classical inheritance makes many schema changes difficult to understand and to implement. Orion [8] requires complex propagation rules to disambiguate the effect of schema changes. Gemstone [29] does not support some schema changes because their motivation is to only provide schema changes which are “well understood and have a reasonable implementation”. We propose the multiobject mechanism to implement specialization in an objectoriented database system. This mechanism supports multiple instantiation, automatic classification and object migration. It makes schema changes more pertinent, eases their implementation and understanding, and reduces their execution time. The multiobject mechanism is implemented in the F2 OODBMS. F2 is a general purpose database system developed at C.U.I. and used to experiment several features such as: updatable views [16], information system design methods [15], knowledge databases [17], database integration [13], schema evolution [7] [6] [3]. It is written in Ada and runs under SunOS, DEC/ALPHA, MacOS and Windows 95. The remainder of the paper is organized as follows. In section 2, we present the multiobject mechanism. In section 3, we show its advantages with respect to schema evolution. In section 4, we describe related approaches and compare the multiobject mechanism to them. In section 5 we conclude with a summary.

2 Multiobject Mechanism We describe in this section the multiobject mechanism. We introduce first the multiobjects and describe then the methods to manipulate them. 2.1

Multiobjects

Defining a Multiobject. In the F2 model [6] an object is an instance of a class. Objects structure is defined by class attributes. Objects behaviour is defined by primitive methods and triggered methods. A class, called subclass, can be declared as a specialization of another class called superclass. The class hierarchy is a forest, i.e. a set of specialization trees: a subclass has only one superclass (single inheritance), and there is not a root system-defined class. On a subclass may be defined specialization constraints. An object belongs to a subclass if and only if it satisfies the specialization constraints of the subclass. The ancestors of a subclass are its direct and indirect superclasses. The descendants of a class are its direct and indirect subclasses. We assume that the reality consists of entities. Entities have several facets. For example, a human being may be seen as a person, an employee, a tennis player, a student, etc. An entity is implemented in the multiobject mechanism by a set of objects in distinct classes of a specialization tree, Mo = {oC1, oC2, ..., oCn}, called multiobject. Each object oCi denotes a facet of the entity and carries data specific to its corresponding class Ci. This is referred to as multiple instantiation. A multiobject Mo satisfies the following constraint: if oCi, 1 ≤ i ≤ n, belongs to Mo and Ci is a subclass of Cj then there must be

318

L. Al-Jadir and M. Léonard

an object oCj, 1 ≤ j ≤ n and j ≠ i, which belongs to Mo. In other words, if an entity possesses an object in class C, then the entity must also possess objects for all the ancestors of C. For example, the class Student is a subclass of Person. A student is implemented by a multiobject containing two objects rStudent in Student and rPerson in Person. Subclasses can be inclusive, i.e. a multiobject may contain two objects oCi and oCj where Ci and Cj are sibling classes. For example, the class Person has another subclass Employee (see fig. 1.a). A person who is a student and an employee is implemented by a multiobject containing three objects: rPerson in Person, rStudent in Student and rEmployee in Employee (see fig. 1.b). There is therefore no need to add an artificial intersection class Student&Employee as with the classical implementation of specialization. C

: class C (italic items next to C are its attributes) Person

name nationality Person jobs

Student regist# hobby

Employee emp# salary

(a) classes

: IS-A link

rPerson

rStudent

rEmployee

Student

Employee

oid of rPerson: < idPerson, idr >, oid of rStudent: < idStudent, idr >, oid of rEmployee: < idEmployee, idr >, (rEmployee = rStudent) = FALSE, (rEmployee @= rStudent) = TRUE, relatedTo(rStudent, Employee) = rEmployee.

(b) instances

(c) operators

Fig. 1. Implementing a student-employee

An object oC in class C has the oid where idC is the class identifier and ido the instance identifier within C (as in Orion [8]). Two objects pE and qF are related, i.e. they belong to the same multiobject, if: − their classes E and F are in the same specialization tree, and − they have the same instance identifier (idp = idq). Two objects pE and qF are identical, if: − they have the same oid (i.e. same class identifier and same instance identifier). The operator “=” checks if two objects are identical, while the operator “@=” checks if two objects are related. The function relatedTo(oD, C) returns the object in class C which is related to oD (C and D being two classes in the same specialization tree). In other words, this function returns the object which has the oid . If no such object exists in C, it returns the unknown object. Note that in other approaches this function is called casting or coercion. Examples are given in figure 1.c. Since an entity may gain and lose facets during its life-time, objects can be added to and removed from its corresponding multiobject (see §2.2). Querying a Multiobject. In the multiobject mechanism, attributes are not inherited in the classical sense of inheritance; they are reached by navigating in a specialization tree. This contrasts with the classical implementation of specialization. For the objects of a subclass C, only the values on attributes locally defined at C are stored. The values on attributes defined at the superclass S of C are not stored with C objects but with their

Multiobjects to Ease Schema Evolution in an OODBMS

319

related S objects. For example, in figure 1, if the name of rPerson is “Dupont” and rEmployee in Employee is related to rPerson then rEmployee is named Dupont. The name values are stored with the objects of class Person. While traditional inheritance is upwards, reaching attributes in the multiobject mechanism can be upwards, downwards and sideways. The get(oC, att) primitive method reads the value that the multiobject containing the object oC takes on the attribute att. Its algorithm (see appendix) is the following: if att is a local attribute of C then it returns the value of oC on att, else if att is a local attribute of a class D belonging to the specialization tree of C then it returns the value of oD on att where oD is the object in class D which is related to oC. Figure 2 shows some examples related to figure 1. let rPerson be named “Dupont”, rStudent having the registration number 98755, and rEmployee earning a salary of 2000 francs. Let pStudent be another student (not employee) and qPerson a baby (neither student nor employee).

Person

qPerson pPerson rPerson

Student rStudent pStudent

name: ”Dupont”

Employee rEmployee

regist#:98755 salary:2000

name(rStudent) = “Dupont”, /* upwards */ salary(rPerson) = 2000, /* downwards */ regist#(rEmployee) = 98755, /* sideways */ emp#(pStudent) raises error2, /* pStudent is not related to an employee object */ regist#(qPerson) raises error2, /* qPerson is not related to a student object */ birthdate(rEmployee) raises error1. /* birthdate is not an attribute in the specialization tree of Employee */

Fig. 2. Reaching attributes upwards, downwards and sideways

An attribute of class C is always reached in the descendants of C (because each of their objects is related to an object in C) while it may be reached in the ancestors and sibling classes of C if some of their objects are related to an object in C. We forbid homonym attributes in classes belonging to the same specialization tree; nevertheless we have the same potential of information as O2 [1] and Goose [26]. 2.2

Manipulating Multiobjects

The algorithms for creating, deleting and updating a multiobject are provided in the appendix and are implemented in the F2 OODBMS. We briefly describe them hereafter. Creating a Multiobject. The create(C, [a1:v1, a2:v2, ..., ap:vp]) primitive method creates a multiobject including an object in class C. The automatic classification algorithm searches the classes of the multiobject in the specialization tree of C (beginning from the root), SC = {C1, C2, ..., Cn}, according to the attribute values [a1:v1, a2:v2, ..., ap:vp] and to the classes’ specialization constraints. If C does not belong to SC or if the origin class of one of the attributes aj does not belong to SC, an error is returned.

320

L. Al-Jadir and M. Léonard

Otherwise, an object oCi is added to each class Ci of SC and all these objects carry the same instance identifier. Each attribute value is stored with the object oCi which belongs to the origin class of the attribute. This contrasts with the classical implementation of specialization where an object is created in one most specific class. For example, in figure 3.a, the class Person has two subclasses: Employee which in turn has two subclasses ManEmp (for men employees) and WomEmp (for women employees), and Student which in turn has a subclass SwissSt (for swiss students). The following expression (in F2-DML) creates a multiobject containing four objects: oPerson in Person (root class of the specialization tree), oEmployee in Employee (the constraint jobs includes “employee” is satisfied), oManEmp in ManEmp (the constraint sex = “m” is satisfied) and oStudent in Student (the constraint jobs includes “student” is satisfied) (see fig. 3.b). oEmployee := create Employee’ [name: “Dupont”, sex: “m”, nationality: “french”, jobs: (“employee”, “student”), emp#: 125, salary: 2500]; name: “Dupont” oPerson sex: “m” nationality: “french” jobs: (“employee”, “student”)

name sex Person nationality jobs regist# hobby Student

SwissSt

emp# Employee salary

ManEmp

oStudent

WomEmp

emp#: 125 oEmployee salary: 2500

oManEmp

scholarship (a) classes

(b) instances Fig. 3. Creating a multiobject

Note that if we did not give a value on the jobs attribute, the creation would be rejected because the multiobject would not include an object in Employee (Employee constraint not satisfied). Deleting a Multiobject. The delete(oC) primitive method deletes the multiobject containing the object oC, i.e. it removes oC and all its related objects. The algorithm searches the classes of the multiobject in the specialization tree of C (beginning from the root), SC = {C1, C2, ..., Cn}, and removes its object oCi from each class Ci of SC. This contrasts with the classical implementation of specialization where an object is deleted from one class. For example, the following expression (in F2-DML) deletes a multiobject (representing a male student-employee) by removing all its objects oPerson, oEmployee, oManEmp and oStudent (see fig. 4). delete oEmployee;

Multiobjects to Ease Schema Evolution in an OODBMS

If instead one dismisses the employee, the entity remains as a student and a person. This can be done by updating the jobs attribute of the corresponding multiobject.

321

Person oPerson X Student oX Student

Employee X oEmployee

Updating a Multiobject. The update(oC, [att:val]) primitive method sets the value of the multiobject containing the object oC on X oManEmp the attribute att to val. Like the get method, SwissSt ManEmp WomEmp update searches the attribute att upwards, downwards and sideways. Since the Fig. 4. Deleting a multiobject attribute att could be used in specialization constraints on the descendants SD of its origin class Orig, the multiobject may gain new objects or/and lose existing objects in SD because it may now (with the new value val) validate or invalidate those specialization constraints. This is referred to as object migration. The automatic classification algorithm searches in the specialization tree of C, beginning from Orig: (i) the set of gained classes and adds an object (carrying the same instance identifier as oC) to each of them; (ii) the set of lost classes and removes the related object to oC from each of them. This contrasts with the classical implementation of specialization where an object stays in its class until it is deleted from it. For example, figure 5.a shows a swiss female student implemented by a multiobject containing three objects {pPerson, pStudent, pSwissSt}. The following expression (in F2-DML) expresses that this person ceases to be a student and becomes an employee. As a result (see fig. 5.b), (i) pEmployee is added to Employee and pWomEmp is added to WomEmp, (ii) pStudent and pSwissSt are removed from Student and SwissSt respectively. The multiobject contains now the objects {pPerson, pEmployee, pWomEmp}. update pPerson jobs:(“employee”); sex: “f” Person pPerson nationality: “swiss” jobs: (“student”) Student pStudent

pSwissSt SwissSt

Employee

sex: “f” Person pPerson nationality: “swiss” jobs: (“employee”) Student

Employee pEmployee

pWomEmp ManEmp WomEmp (a) before update

SwissSt

ManEmp

WomEmp

(b) after update

Fig. 5. Updating a multiobject (jobs attribute)

322

L. Al-Jadir and M. Léonard

3 Advantages of the Multiobject Mechanism for Schema Evolution In this section we first provide briefly the framework of schema evolution in F2. Then we discuss the advantages of the multiobject mechanism with respect to schema evolution. 3.1

Schema Evolution in F2

Set of Schema Changes in F2. An important feature of the F2 DBMS is the uniformity of its objects described in [6] [3]. We consider objects of three levels: database objects, schema objects and meta-schema objects. Uniformity of objects in F2 includes: − uniformity of representation. The same structures are used in F2 to represent database objects, schema objects and meta-schema objects; − uniformity of access and manipulation. The same primitive methods are used in F2 to access and manipulate database objects, schema objects and meta-schema objects. Thanks to the uniformity of the F2 DBMS, we built the set of schema changes in F2 as follows [6] [3]: for each class of the F2 meta-schema we apply the primitive methods create, delete and update on its objects (see fig. 6). Semantics of Schema Changes. We defined the semantics of each schema change in F2 with pre-conditions and post-actions [6] [3] such that the F2 model invariants are preserved. Pre-conditions must be satisfied to allow a schema change to occur; otherwise it is rejected. Post-actions are repercussions to be executed on schema objects and database objects in order to keep the database structurally consistent. We implemented pre-conditions and post-actions by triggered methods [6] [3]. Propagation of Schema Changes. In F2 schema changes are propagated immediately [3], i.e. the repercussions of a schema change are executed as soon as the schema change is performed. 3.2

Multiobject Mechanism and Schema Evolution

Since the multiobject mechanism implements specialization, we consider among the schema changes of F2 (fig. 6) those which are involved in specialization: create a subclass (1.3), delete a class (2), change the superclass of a subclass (3.4), update a class from non-subclass to subclass (3.5) and the reverse (3.6), create (4) and delete an attribute (5), change the domain class (6.4) and the origin class of an attribute (6.5), create (10) and delete a specialization constraint (11), change the list of subclasses on which is defined a specialization constraint (12.2). By examples we will show that the multiobject mechanism makes these schema changes more pertinent and easier to implement than with the classical implementation of specialization. We will compare F2 with the following OODBMS which support schema evolution: Orion [8], Gemstone [29], OTGen [23], Cocoon [37], Goose [26] and O2 [18]. All these systems support the classical

Multiobjects to Ease Schema Evolution in an OODBMS

(1) Create a new class (1.1) Create an atomic class (1.2) Create a tuple class (1.3) Create a tuple subclass (2) Delete an existing class (3) Update an existing class (3.1) Change its name (3.2) Change its interval if atomic class (3.3) Change its maximal length if atomic string class (3.4) Change its superclass (3.5) Make it a subclass, i.e. attach it to a specialization tree (3.6) Make it a non-subclass, i.e. detach it from a specialization tree (4) Create a new attribute of a class (5) Delete an existing attribute (6) Update an existing attribute (6.1) Change its name (6.2) Change its maximal cardinality (6.3) Change its minimal cardinality (6.4) Change its domain class (6.5) Change its origin class (7) Create a new key of a class (8) Delete an existing key (9) Update an existing key (9.1) Change the class on which it is defined

323

(9.2) Change its attributes (9.3) Enable / disable it (10) Create a new specialization constraint (11) Delete an existing spec. constraint (12) Update an existing spec. constraint (12.1) Change its name (12.2) Change the list of subclasses on which it is defined (13) Create a new trigger (14) Delete an existing trigger (15) Update an existing trigger (15.1) Change the event for which it is defined (15.2) Change the list of methods it triggers (16) Create a new event (17) Delete an existing event (18) Update an existing event (18.1) Change the class on which it is defined (18.2) Change its kind (18.3) Change its attribute (19) Create a new triggered method (20) Delete an existing triggered method (21) Update an existing triggered method (21.1) Change its name

Fig. 6. F2 schema changes

implementation, except Cocoon which supports multiple instantiation, class predicates and automatic classification. Create a Subclass. Example: The class Person has several attributes including nationality. It has four objects {a, b, c, d}, two of them {a, d} take the value “swiss” on the nationality attribute. The class Car has an attribute owner whose domain is Person. Now one creates the class Swiss as a subclass of Person. The wanted effect is that a and d become objects of the Swiss class. • Multiobject approach: The class Person has four objects {aP, bP, cP, dP} (see fig. 7.a). When the Swiss subclass and a specialization constraint on it are added, the objects aS and dS are automatically added to the Swiss subclass because they satisfy its specialization constraint (see fig. 7.b). Each of the multiobjects a and d contain now two objects. The attribute values of objects aP and dP are not copied because attribute values are locally stored and attributes of Person are reached in Swiss. Cocoon supports this schema change.

324

L. Al-Jadir and M. Léonard

: class C (underlined items next to C are its specialization constraints)

Person a bP cP P dP

Person a bP cP P dP Swiss aS dS

(a) before

nationality = ”swiss” (b) after

Fig. 7. Add the Swiss subclass with a specialization constraint

• Classical approach: Most of the OODBMS supporting schema evolution leave a subclass empty (without objects) when it is newly created. Thus in our example, a tool should be developed to: i) create two new objects in Swiss; ii) copy the value of objects a and d, on all the attributes of Person, to the newly created objects respectively; iii) delete the objects a and d in Person. If the objects a and d were referenced by Car objects through the owner attribute, the tool has also to update these references (to reference now the new objects in Swiss). Only O2 (thanks to migration functions) and OTGen (thanks to boolean expressions) allow subclass creation with object migration down. Delete a Class. Example: The class Swiss is a subclass of Person and has two objects {a, d}. The class Chalet has an attribute owner whose domain is Swiss. Now one is no more interested to classify the swiss persons and decides to delete the Swiss class. The wanted effect is that a and d become objects of the Person class (keep the swiss people as persons). • Multiobject approach: The Swiss class has two objects aS and dS which are related to aP and dP respectively in Person (see fig. 7.b). When the subclass Swiss is deleted, its objects are removed while their related objects {aP, dP} remain in Person (see fig. 7.a). The attributes of Swiss are deleted and the domain of the owner attribute is updated to Person, as with the classical approach. The values on the owner attribute remain unchanged (see update the domain of an attribute). Cocoon supports this schema change. • Classical approach: Most of the OODBMS supporting schema evolution delete the objects of a class when the class is deleted (Gemstone prevents the deletion of a class if it is not empty). This implies loss of information. Thus in our example, a tool should be developed to: i) create two new objects in Person; ii) copy the value of objects a and d, only on the inherited attributes from Person, to the newly created objects respectively; iii) delete the Swiss class (consequently its objects are deleted). If the objects a and d were referenced by Chalet objects through the owner attribute, the tool has also to update these references (to reference now the new objects in Person). Note that if the wanted effect in our example was not to keep the swiss persons, this could be achieved in the multiobject approach by first deleting the multiobjects containing Swiss objects and then deleting the Swiss class. Thus both semantics are possible in

Multiobjects to Ease Schema Evolution in an OODBMS

325

our approach and the database administrator can choose the most suitable for a given situation. Update the Superclass of a Subclass. Example: The class Person has two subclasses Student and Employee. Student has a subclass Young (see fig. 8.a). Now one updates the superclass of Young from Student to Employee. The wanted effect is that the Young class stores the young employee objects instead of the young student objects, and that it inherits the attributes of Employee instead of those of Student. • Multiobject approach: The class Young has the specialization constraint age < 30. Changing its superclass to Employee reclassifies automatically its objects: i) for each object in Young, if it is not related to an Employee object, it is removed from Young; ii) for each object in Employee whose age value is under 30, a related object to it is added to Young (if it does not already exist). The Young class reaches now another set of attributes; the physical storage of its objects remains unchanged. • Classical approach: Most of the OODBMS supporting schema evolution keep the same objects in a subclass when modifying its superclass. This may lead to an inconsistent semantics. They re-evaluate the inheritance of the subclass according to defined rules. In our example, the Young class inherits now the attributes of Employee instead of those of Student. This results in the modification of the physical storage of Young objects. To reclassify objects in the Young class, a tool should be developed to: i) migrate the Young objects up to Student (like when deleting a class); ii) migrate some objects of Employee down to Young (like when creating a subclass). Note that Gemstone does not support this schema change in order to only provide well understood schema changes. In F2, updating a subclass to non-subclass and the reverse are special cases of updating the superclass of a subclass. Due to lack of space we do not describe them. Create an Attribute. Example: One adds the attribute hobby to class Person which has several descendants. The wanted effect is that the descendants of Person inherit hobby. • Multiobject approach: Adding the new attribute hobby to class Person does not need to be propagated to the descendants of Person; instead hobby will be reached in them. • Classical approach: Most of the OODBMS supporting schema evolution, as Orion and Gemstone, propagate recursively this schema change to Person’s subclasses according to propagation rules. On storage, this requires to physically add the attribute to the objects of each descendant of Person inheriting it. Delete an Attribute. Example: One deletes the attribute hobby from class Person. The wanted effect is that the descendants of Person do no longer inherit hobby. • Multiobject approach: Removing the attribute hobby from class Person does not need to be propagated to the descendants of Person; instead hobby will not be reached in them. • Classical approach: There are different approaches, Orion removes recursively the attribute hobby from subclasses inheriting it while Gemstone does not. In the latter case, one has to delete the hobby attribute from each class inheriting it. On storage, both approaches require to physically delete the attribute from the objects of each descendant of Person inheriting it.

326

L. Al-Jadir and M. Léonard

Update the Domain Class of an Attribute. Example: The class Book has an attribute owner whose domain is Person. Student is a subclass of Person. The book book1 is owned by c (student object) and book2 is owned by d (person object). Now one updates the domain of owner to Student. • Multiobject approach: As with the classical approach, the owner value of book1 is unchanged: cS in Student is related to cP in Person and they have the same instance identifier. In F2, only instance identifiers are physically stored in attribute values (the class identifier is the same for all values on the same attribute; it can be obtained by getting the domain class of the attribute). The owner value of book2 is replaced by the unknown object. Updating the domain class of owner does not need to be propagated to the descendants of Book, because the owner values are stored with Book objects. Note that one can update the domain of owner to an ancestor, a descendant or a sibling class of Person. • Classical approach: All the OODBMS supporting schema evolution support this schema change but some with restrictions. For example, Orion allows only to generalize a domain while Gemstone can generalize and specialize it. Orion propagates this schema change to subclasses inheriting the attribute according to propagation rules while Gemstone does not. Update the Origin Class of an Attribute. Example: The class Young is a subclass of Student which is a subclass of Person (see fig. 8.a). Class Person has the objects {d}, class Student has {c} and class Young has {a, b}. Class Employee, another subclass of Person, has the objects {e}. Hobby is a local attribute of Student. Now one wants to store the hobbies of all persons and updates the origin class of hobby to Person. Updating the origin class of an attribute is very useful and is not equivalent to dropping the attribute and adding it to another class because in this case the values taken on the attribute are lost. • Multiobject approach: The classes and their objects are shown in figure 8.a. When the origin class of the hobby attribute is updated to Person (see fig. 8.b), the objects aP, bP, cP keep the same hobby value as aS, bS, cS respectively. The hobby value of dP and eP is set to unknown. Thanks to the transposed storage of objects [6] in F2, the hobby values are not copied. Note that one can update the origin class of hobby from Student to Person (ancestor), to Young (descendant) or to Employee (sibling class). Changing the origin class of an attribute does not need to be propagated to the descendants of its old and new origin classes; the attribute will be reached in another set of classes. Cocoon supports this schema change. • Classical approach: Most of the OODBMS supporting schema evolution do not support this schema change. In our example, a tool should be developed to: i) create a new attribute hobby2 in Person (it becomes inherited in Student, Young and Employee); ii) copy the hobby values on this new attribute for Student objects {c} and Young objects {a, b}; iii) delete the attribute hobby of Student; iv) rename the attribute hobby2 to hobby. Note that if instead one creates a new attribute hobby in Person, the semantics would be different because the attribute hobby of Student would not be considered as inherited from Person. Only Goose allows to update the origin class of an attribute with retaining values for objects.

Multiobjects to Ease Schema Evolution in an OODBMS

Person

Person

c aP bP d P eP P Employee

Student aS bS cS hobby aY bY Young

eE

(a) before

c aP bP d P eP hobby P Employee

Student aS bS cS

aY bY Young

327

eE

(b) after

Fig. 8. Update the origin class of the hobby attribute

Create/Delete a Specialization Constraint. Example: The class Person has a subclass European (citizen of a country of the european community) which has a subclass Young (under 30 years old) (see fig. 9.a). Class Young has one object a (young european), class European has one object b (old european), class Person has two objects c (young african) and d (young swiss). Suppose now that Switzerland joins the european community. One replaces (delete followed by create) then the specialization constraint of European by a new one taking into account Switzerland. The wanted effect is that the object d belongs now to Young instead of Person. • Multiobject approach: The classes and their objects are shown in figure 9.a. Specialization constraints are defined on the subclasses European and Young. When the specialization constraint of European is replaced, the objects dE and dY are automatically added to the classes European and Young respectively (see fig. 9.b). Cocoon supports class predicates and allows to change them. Person a bP cP P dP European aE bE Young

Person a bP cP P dP

nationality in {“french”, “german”,...} aY

(a) before

age < 30

nationality in European {“french”, aE bE dE “german”,..., “swiss”} age < 30 Young a d Y Y (b) after

Fig. 9. Replace the specialization constraint of European

• Classical approach: Most of the OODBMS do not support specialization constraints. In our example, a tool should be developed to migrate the object d down to Young. In F2, a specialization constraint (e.g. age < 30) can be defined on several subclasses (e.g. YoungStudent and YoungEmployee). Changing the list of subclasses of a specialization constraint is similar to create/delete a specialization constraint.

328

L. Al-Jadir and M. Léonard

4 Related Work Approaches for Multi-faceted and Dynamic Entities. We proposed and implemented a previous version of the multiobject mechanism in the extended entity-relationship DBMS Ecrins [20]. Several object approaches have been proposed to model the multifaceted and dynamic nature of entities. We summarize them in the following table. Then we compare our approach to them. Null entries (--) mean not known (we do not have the data). approach (how an entity is implemented) Object hierarchies [34]

Roles in ORM [30]

creation

extension

create an add/remove an entity: set of objects (object object hierar- object to/from an chy object hierarchy hierarchy) (parent attribute)

object internal representation

inheritance

local upwards in attributes object hierarchy (perobject)

entity: object of a class + role instances

create an object in a class

add/remove a role instance to/from an object

--

no inheritance

Aspects [33] entity: object of a class + aspect instances (same oid)

create an object in a class

add an aspect to an object

--

no inheritance (aspect exports selected parts of a class)

Roles in Fibonacci [2]

entity: object with a set of roles (same identity)

create an add/drop a role to/ local upwards in object with from an object attributes role hierarchy several roles

create an add/remove a role Category entity: object of classes [28] several classes object in sev- to/from an object eral classes (manual), or (roles) update an object (same oid) (automatic) Objectentity: concep- create a conslicing [22] tual object + set ceptual object of implementa- with implementation tion objects objects (bi-directional link)

create/delete an implementation object

--

upwards in class hierarchy

local attributes

upwards in class hierarchy

local upwards, Our entity: set of create a multi- update a multiobapproach: objects (multi- object (auto- ject (objects are attributes downwards and sideways added/removed to/ matic Multiobjects object) in specializa(same instance classification) from a multiobject tion tree automatically) identifier)

Multiobjects to Ease Schema Evolution in an OODBMS

329

The multiobject mechanism differs from ORM roles and aspects because they do not integrate the class hierarchy. As in object hierarchies, Fibonacci roles, category classes and object-slicing, an entity in our approach is implemented by a set of objects which can be enlarged and reduced (automatically as in category classes). However, our approach has several differences: − an object can access attribute values of related objects not only upwards but also downwards and sideways in the class hierarchy. − two related objects are neither linked by a parent attribute as in object hierarchies, nor by a bi-directional link via a conceptual object as in object-slicing. Both objects have the same instance identifier (same identity in Fibonacci roles and category classes). Thus there is no overhead when accessing related objects in our approach. − we provide the algorithms to manipulate multiobjects. − we use the multiobject mechanism not only to ease object modelling and object evolution but also to ease schema evolution. OODBMS Supporting Schema Evolution. Several OODBMS support schema evolution. We can classify them in three categories: schema evolution without versioning (Orion [8], Gemstone [29], OTGen [23], Cocoon [37], Goose [26], O2 [18]), schema evolution with versioning (Encore [35], Orion, Goose, Closql [25]), schema evolution with views (Contexts [7], Goose, Views [11], TSE [32]). The two last categories are out of the scope of this paper. Approaches of the first category differ by the set of supported schema changes, the semantics of schema changes, and the propagation of schema changes (immediate, deferred, mixed).

5 Conclusion We presented the multiobject mechanism to implement specialization in object-oriented databases. We described how this mechanism models multi-faceted and dynamic realworld entities. We showed its advantages with respect to schema evolution. The multiobject mechanism makes schema changes more pertinent than with the classical implementation of specialization. It makes them easier to implement and less timeconsuming since it needs neither to copy objects nor to propagate schema changes. It makes schema changes easier to understand since it avoids complex propagation rules. We implemented the multiobject mechanism in F2. We submitted the F2 DBMS to the OO7 benchmark [14]. We obtained in [3] good results for the queries and traversals, because objects in the multiobject approach have smaller size than objects in the classical object model; this consequently reduces the number of input/output operations. For the insert and delete operations of the benchmark, the multiobject approach is more expensive than the classical object model; this is due to the automatic classification (which is not supported in the classical object model) and to the fact that several objects are created/deleted instead of one. The execution times could be improved by using a parallel algorithm; this issue deserves to be investigated. Extensions of the multiobject mechanism in F2 include: i) allow several objects of the same class in a multiobject; ii) allow several attributes with the same name in a spe-

330

L. Al-Jadir and M. Léonard

cialization tree; iii) handle methods and virtual binding (see interpretation of messages in [2]); iv) model the life-cycle of an entity [4] and restrict the way objects may be added to and removed from a multiobject. Extensions of schema evolution in F2 include: i) take into account schema changes on methods; ii) study the behavioural consistency of the database; iii) test schema changes in a real application.

References 1. Adiba M., Collet C., Objets et bases de données: le SGBD O2, Hermès, 1993. 2. Albano A., Bergamini R., Ghelli G., Orsini R., An object Data Model with Roles, Proc. Int. Conf. on Very Large Data Bases, VLDB, Dublin 1993. 3. Al-Jadir L., Evolution-Oriented Database Systems, Ph.D. thesis, Faculty of Sciences, University of Geneva, 1997. 4. Al-Jadir L., Falquet G. , Léonard M., Context Versions in an Object-Oriented Model, Proc. Int. Conf. on Database and Expert Systems Applications, DEXA, Prague 1993. 5. Al-Jadir L., Le Grand A., Léonard M., Parchet O., Contribution to the Evolution of Information Systems, in: Methods and Associated Tools for the Information Systems Lifecycle, A.A. Verrijn-Stuart & T.W. Olle (eds), IFIP, Elsevier, 1994. 6. Al-Jadir L., Estier T., Falquet G., Léonard M., Evolution Features of the F2 OODBMS, Proc. Int. Conf. on Database Systems for Advanced Applications, DASFAA, Singapore 1995. 7. Andany J., Léonard M., Palisser C., Management of Evolution in Databases, Proc. Int. Conf. on Very Large Data Bases, VLDB, Barcelona 1991. 8. Banerjee J., Kim W., Kim H-J., Korth H.F., Semantics and Implementation of Schema Evolution in Object-Oriented Databases, Proc. Int. Conf. on Management Of Data, ACM SIGMOD, San Francisco 1987. 9. Barbedette G., Schema Modifications in the LISPO2 Persistent Object-Oriented Language, Proc. European Conf. on Object-Oriented Programming, ECOOP, Geneva 1991. 10. Bellahsene Z., An Active Meta-model for Knowledge Evolution in an Object-oriented Database, Proc. Int. Conf. on Advanced Information Systems Engineering, CAISE, Paris 1993. 11. Bertino E., A View Mechanism for Object-Oriented Databases, Proc. Int. Conf. on Extending Database Technology, EDBT, Vienna 1992. 12. Bertino E., Jajodia S., Modeling Multilevel Entities Using Single Level Objects, Proc. Int. Conf. on Deductive and Object-Oriented Databases, DOOD, Phoenix 1993. 13. Bonjour M., Falquet G., Concept Bases: A Support to Information Systems Integration, Proc. Int. Conf. on Advanced Information Systems Engineering, CAISE, Utrecht 1994. 14. Carey M.J., DeWitt D.J., Naughton J.F., The OO7 Benchmark, Proc. Int. Conf. on Management Of Data, ACM SIGMOD, Washington 1993. 15. Estier T., Falquet G., Guyot J., Léonard M., Six Spaces for Global Information Systems Design, in: The Object Oriented Approach in Information Systems, F. van Assche & B. Moulin & C. Rolland (eds), IFIP, North-Holland, 1991. 16. Falquet G., Interrogation de bases de données à l’aide d’un modèle sémantique, Ph.D. thesis, Faculty of Sciences, University of Geneva, 1989. 17. Falquet G., Léonard M., Sindayamaze J., F2Concept: a Database System for Managing Classes’ Extensions and Intensions, in: Information modelling and knowledge bases V, H. Jaakola et al. (eds), IOS Press, 1994.

Multiobjects to Ease Schema Evolution in an OODBMS

331

18. Ferrandina F., Meyer T., Zicari R., Ferran G., Madec J., Schema and Database Evolution in the O2 Object Database System, Proc. Int. Conf. on Very Large Data Bases, VLDB, Zürich 1995. 19. Hauck F.J., Inheritance Modeled with Explicit Bindings: An Approach to Typed Inheritance, Proc. Conf. on Object-Oriented Programming Systems, Languages and Applications, OOPSLA, Washington 1993. 20. Junet M., Falquet G., Léonard M., ECRINS/86: An Extended Entity-Relationship Data Base Management System and its Semantic Query Language, Proc. Int. Conf. on Very Large Data Bases, VLDB, Kyoto 1986. 21. Kambayashi Y., Peng Z., Object Deputy Model and Its Applications, Proc. Int. Conf. on Database Systems for Advanced Applications, DASFAA, Singapore 1995. 22. Kuno H.A., Ra Y-G., Rundensteiner E.A., The Object-Slicing Technique: A Flexible Object Representation and Its Evaluation, Technical Report, CSE-TR-241-95, University of Michigan, 1995. 23. Lerner B.S., Habermann A.N., Beyond Schema Evolution to Database Reorganization, Proc. Conf. on Object-Oriented Programming Systems, Languages and Applications, OOPSLA, Ottawa 1990. 24. Ling T.W., Teo P.K., Object Migration in ISA Hierarchies, Proc. Int. Conf. on Database Systems for Advanced Applications, DASFAA, Singapore 1995. 25. Monk S.R., Sommerville I., A Model for Versioning of Classes in Object-Oriented Databases, Proc. British National Conf. on Databases, BNCOD, Aberdeen 1992. 26. Morsi M.M.A., Navathe S.B., Kim H-J., A Schema Management and Prototyping Interface for an Object-Oriented Database Environment, in: Object Oriented Approach in I.S., F. Van Assche & B. Moulin & C. Rolland (eds), IFIP, North-Holland, 1991. 27. Nguyen G.T., Rieu D., Escamilla J., An Object Model for Engineering Design, Proc. European Conf. on Object-Oriented Programming, ECOOP, Utrecht 1992. 28. Odberg E., Category Classes: Flexible Classification and Evolution in Object-Oriented Databases, Proc. Int. Conf. on Advanced Information Systems Engineering, CAISE, Utrecht 1994. 29. Penney D.J., Stein J., Class Modification in the GemStone Object-Oriented DBMS, Proc. Conf. on Object-Oriented Programming Systems, Languages and Applications, OOPSLA, Orlando 1987. 30. Pernici B., Objects with Roles, Proc. IEEE Conf. on Office Information Systems, 1990. 31. Peters R.J., Özsu M.T., An Axiomatic Model of Dynamic Schema Evolution in Objectbase Systems, ACM Transactions on Database Systems, vol. 22, no 1, march 1997. 32. Ra Y.G., Kuno H.A., Rundensteiner E.A., A Flexible Object-Oriented Database Model and Implementation for Capacity-Augmenting Views, Technical Report, CSE-TR-215-94, University of Michigan, april 1994. 33. Richardson J., Schwarz P., Aspects: Extending Objects to Support Multiple, Independent Roles, Proc. Int. Conf. on Management Of Data, ACM SIGMOD, Denver 1991. 34. Sciore E., Object Specialization, ACM Transactions on Information Systems, vol. 7, no 2, april 1989. 35. Skarra A.H., Zdonik S.B., Type Evolution in an Object-Oriented Database, in: Research Directions in OO Programming, B. Shriver & P. Wegner (eds), MIT Press, 1987. 36. Smith J.M., Smith D.C.P., Database Abstractions: Aggregation and Generalization, ACM Transactions on Database Systems, vol. 2, no 2, june 1977. 37. Tresch M., A Framework for Schema Evolution by Meta Object Manipulation, Proc. Int. Workshop on Foundations of Models and Languages for Data and Objects, Aigen 1991.

332

L. Al-Jadir and M. Léonard

Appendix 1. Get Algorithm Let att be an attribute and oC be an object of class C (an object carries the class it is instance of, thus C can be known from oC); origin_class(att) be a function which returns the origin class of attribute att; root(C) be a function which returns the root class of the specialization tree of class C. function get(oC, att) return val is begin /* call the check_valid_attribute procedure which checks that att is a local attribute of C (sets obj to oC) or a reached attribute by oC (sets obj to the object in the origin class of att which is related to oC) */ check_valid_attribute(att, oC, obj); return the value val of obj on att; end get; procedure check_valid_attribute(att, oC, obj) is begin Orig := origin_class(att); /* if att is a local attribute of class C */ if (C = Orig) then obj := oC; /* if classes Orig and C are in same spec. tree */ elsif (root(C) = root(Orig)) then oOrig := relatedTo(oC, Orig); /* if oC is related to an object in Orig */ if (oOrig ≠ unknown_object) then obj := oOrig; else error2; end if; else error1; end if; end check_valid_attribute;

2. Create Algorithm Let C be a class and [a1:v1, a2:v2, ..., ap:vp] be an array of pairs; subclasses(C) be a function which returns the direct subclasses of class C; satisfy_constraints([a1:v1, a2:v2, ..., ap:vp], C) be a function which indicates whether the given attribute values satisfy all the spec. constraints of class C; root() and origin_class() have been defined before. function create(C, [a1:v1, a2:v2, ..., ap:vp]) return oC is begin TheRoot := root(C); /* initialize SC to the empty set */ SC := {}; /* call the classify procedure which puts in SC the classes of the new multiobject */ classify(TheRoot, [a1:v1, a2:v2, ..., ap:vp], SC); /* check that C belongs to SC */ if (C not in SC) then error1; end if; /* check that all the given attributes will be

reached by the new multiobject */ for each attribute ai in [a1:v1, a2:v2, ..., ap:vp] loop if (origin_class(ai) not in SC) then error2; end if; end loop; /* add objects, store attribute values */ ID := the instance identifier for the objects of the new multiobject; for each class Ci in SC loop add an object oCi (having the instance identifier ID) to Ci; store the value of oCi on the attributes whose origin class is Ci; end loop; return oC; end create; procedure classify(C, [a1:v1, a2:v2, ..., ap:vp], TheSet) is begin /* add class C to TheSet */ TheSet := TheSet ∪ C; Sub := subclasses(C); for each Si in Sub loop /* if the attribute values satisfy all the spec. constraints of class Si */ if satisfy_constraints([a1:v1, a2:v2, ..., ap:vp], Si) then classify(Si, [a1:v1, a2:v2, ..., ap:vp], TheSet); -- recursive call end if; end loop; end classify;

3. Delete Algorithm Let oC be an object of class C; root() and subclasses() have been defined before. procedure delete(oC) is begin TheRoot := root(C); oRoot := relatedTo(oC, TheRoot); /* initialize SC to the empty set */ SC := {}; /* call the facets procedure which puts in SC the classes of the multiobject containing oRoot */ facets(oRoot, SC); /* remove objects */ for each class Ci in SC loop oCi := relatedTo(oC, Ci); remove the object oCi from Ci; end loop; end delete; procedure facets(oC, TheSet) is begin /* add class C to TheSet */ TheSet := TheSet ∪ C; Sub := subclasses(C);

Multiobjects to Ease Schema Evolution in an OODBMS for each Si in Sub loop oSi := relatedTo(oC, Si); /* if an object in Si is related to oC */ if (oSi ≠ unknown_object) then facets(oSi, TheSet); -- recursive call end if; end loop; end facets;

4. Update Algorithm Let oC be an object of class C and [a:v] be an pair; constraint_on_att(C, att) be a function which indicates whether class C has a specialization constraint involving the attribute att; satisfy_constraints_att([a:v], C) be a function which indicates whether the value v satisfies the specialization constraints of class C involving the attribute a; check_valid_attribute(), origin_class(), subclasses(), satisfy_constraints(), facets() and classify() have been defined before. procedure update(oC, [a:v]) is begin check_valid_attribute(a, oC, obj); Orig := origin_class(a); /* initialize ToAdd and ToRemove to empty set */ ToAdd := {}; ToRemove := {}; /* call the reclassify procedure which puts in ToAdd the classes to which a related object will be added and in ToRemove the classes from which a related object will be removed */ reclassify(oC, [a:v], Orig, ToAdd, ToRemove); /* add objects */ for each class Ci in ToAdd loop add an object oCi (having the same instance identifier as oC) to Ci; end loop; /* remove objects */ for each class Ci in ToRemove loop oCi := relatedTo(oC, Ci); remove the object oCi from Ci; end loop; /* store the new attribute value */ store the new value v of the object obj on the attribute a; end update; procedure reclassify(oC, [a:v], D, TheSetAdd, TheSetRemove) is begin Sub := subclasses(D); for each Si in Sub loop /* if class Si has a spec. constraint involving the attribute a */ if constraint_on_att(Si, a) then /* call the migrate procedure which (i) adds Si (and possibly its descendants) to TheSetAdd if a related object oSi should be added, (ii) adds Si (and possibly its descendants) to TheSetRemove if the related object oSi should be removed, (iii) sets

333

continue to true if the related object oSi exists and stays in Si */ migrate(oC, [a:v], Si, TheSetAdd, TheSetRemove, continue); if continue then reclassify(oC, [a:v], Si, TheSetAdd, TheSetRemove); -- recursive call end if; /* else if an object in Si is related to oC */ elsif (relatedTo(oC, Si) ≠ unknown_object) then reclassify(oC, [a:v], Si, TheSetAdd, TheSetRemove); -- recursive call end if; end loop; end reclassify; procedure migrate(oC, [a:v], D, TheSetAdd, TheSetRemove, continue) is begin continue := false; /* is_related indicates if oC is related to an object in D (before update); will_be_related indicates if oC will be related to an object in D (after update) */ oD := relatedTo(oC, D); is_related := (oD ≠ unknown_object); if is_related then /* does the new attribute value satisfy the spec. constraints of D involving the attribute a */ will_be_related := satisfy_constraints_att([a:v], D); else /* does the object oC (take into account the new attribute value) satisfy all the spec. constraints of D */ will_be_related := satisfy_constraints( attribute values of oC, D); end if; /* case 1: there is a related object in D and it will stay in it */ if (is_related and will_be_related) then continue := true; /* case 2: there is a related object in D but it will be removed from it */ elsif (is_related and not will_be_related) then /* add D and each of its descendants where an object is related to oC to TheSetRemove */ tempRemove := {}; facets(oD, tempRemove); TheSetRemove := TheSetRemove ∪ tempRemove; /* case 3: no related object in D but a new one will be added to it */ elsif (not is_related and will_be_related) then /* add D and possibly its descendants to TheSetAdd */ tempAdd := {}; classify(D, attribute values of oC, tempAdd); TheSetAdd := TheSetAdd ∪ tempAdd; /* case 4: no related object in D and no one will be added to it */ else null; -- nothing to do end if; end migrate;

Implementation of Automatic Lock Determination in C++-Based OODBMSs Yong S. Jun1 , Eunji Hong2 , and Suk I. Yoo3 1

Computer System Division, Hyundai Information Technologies, Jeokseon-Dong, Jongro-Ku, Seoul, Korea [email protected] 2 Department of Computer Science, SungKongHoe University, Hang-Dong, Kuro-Ku, Seoul, Korea [email protected] 3 Department of Computer Science, Seoul National University, Shilim-Dong, Kwanak-Ku, Seoul, Korea [email protected]

Abstract. In most C++-based OODBMSs, the concurrency control has been implemented by using the locking mechanism. However, application programmers for those databases should be involved in locking the objects that would be accessed from their applications. So they had to spend a lot of time on issuing locks, and thus the productivity of application development has become poor. This paper presents a mechanism of automatic lock determination in C++-based OODBMSs in order to solve the above problem. In this mechanism, the locks for the objects accessed in applications can be automatically determined at compile-time and the locks are requested automatically to the lock manager just before the objects are accessed at run-time. This mechanism is entirely transparent to application programmers, and thus it can improve the productivity of application development in C++-based OODBMSs.

1

Introduction

In recent years, a rapid growth in computer technology has opened the market to new database applications. These applications include CAD/CAM, CASE, multimedia, and office information systems. However, the modeling power and the computational power of the relational data model are very limited to satisfy the requirements of these applications [1]. In order to overcome these limitations, some OODBMSs(Object-Oriented DataBase Management Systems) have been developed. In these OODBMSs, applications are usually written in their own APIs(Application Programming Interfaces). Most of these APIs have been implemented by extending C++ language [2], probably because of the wide popularity of C++. In this paper, a C++-based OODBMS is defined as an OODBMS with a C++-based API. In applications written in these C++-based APIs, objects can be updated directly as well as through query languages. For example, in the statement T.W. Ling, S. Ram, and M.L. Lee (Eds.): ER’98, LNCS 1507, pp. 334–347, 1998. c Springer-Verlag Berlin Heidelberg 1998

Implementation of Automatic Lock Determination

335

peter→age = 20;, we can see that the object peter is directly updated. Unfortunately, however, it is impossible for C++-based OODBMSs to detect these direct updates. Thus, C++-based OODBMSs are not capable of knowing whether or not an object will be updated before it is updated. This means that it is impossible for these OODBMSs to automatically request the lock manager to acquire appropriate locks on the objects. To overcome this limitation, conventional C++based OODBMSs have provided their own mechanisms for lock requests in their own C++-based APIs. However, these previous mechanisms have imposed a lot of burdens on application programmers. This paper presents a mechanism for automatic lock determination in a C++based API, named spareC++ [12], that is designed for writing applications on C++-based OODBMSs. In this mechanism, locks on objects, which will be accessed from applications, can be automatically determined at compile-time and the locks can be automatically requested to the lock manager before the objects are accessed at run-time. Thus, this mechanism is entirely transparent to application programmers. Since they can concentrate on writing applications without paying attention to the locking, the productivity of application development in C++-based OODBMSs can improve. Also, this mechanism can be easily ported on any C++-based OODBMSs since it is independent of operating systems and lock managers. The organization of the rest of the paper is as follows. In Sect. 2, we introduce the spareC++ OODBPL(Object-Oriented DataBase Programming Language). The lock determination at compile-time and the lock request at run-time are described in Sects. 3 and 4, respectively. In Sect. 5, we compare our mechanism with some previous works. The conclusion is presented in Sect. 6.

2

spareC++ OODBPL

This work has been implemented on the Obase C++-based OODBMS developed at Seoul National University and Hyundai Electronics Industries in Korea. In Obase, objects can be defined and manipulated through spareC++ OODBPL, which is designed for writing applications on OODBMSs, particularly on Obase. This language is based on one integrated data model for both database manipulation and general computation. In this section, with emphasis on subjects relating to automatic lock determination, we will outline the object definition and object manipulation in spareC++. The details of Obase and spareC++ can be found in [7] and [12], respectively. 2.1

Object Model

In Obase, data types are divided into atomic types(such as int, float, and char) and class types. A class type corresponds to a class defined in a database. On the basis of multiple inheritance, a class can inherit from multiple superclasses and it may have any number of subclasses. Classes in a database are again divided into system-defined classes(such as OBJECT, String, Set, and ClassDesc), which

336

Y.S. Jun, E. Hong, and S.I. Yoo

are initially defined at the time when the database is created, and user-defined classes which are defined as a subclass of existing classes in a database. The class inheritance hierarchy of a database is rooted at the system-defined class OBJECT. Thus, all the classes inherit from the root class OBJECT. An object is defined as an instance of a class, and an object is either temporary or persistent. When an object is created within a transaction, it is initially temporary. A temporary object exists only within the transaction and it is automatically deleted right after the transaction ends. Conversely, a persistent object, which exists in the database after the transaction where it has been created and committed, can be accessed in other transactions. A temporary object may be changed to a persistent one either by users’ explicit requests or by being transitively referenced by any other persistent objects. Persistent objects can be deleted only by users’ explicit requests. Every object is associated with a system-generated OID(Object IDentifier) consisting of a database identifier, a class identifier, and an instance identifier. A database identifier indicates the database where its corresponding object is stored. A class identifier indicates the class to which its corresponding object belongs. When an object is created as an instance of a class, the class gives the object a serial number that is used as its instance identifier. Thus, no two distinct objects can have the same OID. Since an OID is unique over a number of databases, objects can be referenced through their OIDs. Therefore, the actual value of a class-type attribute is the OID corresponding to the object which is referenced via the attribute. A database consists of a schema and an extension. The extension is a union of class extensions of classes defined in the database. A class extension of a class is a set of all its instances. The schema is a set of type descriptions for the classes defined in the database. An instance of the system-defined class ClassDesc corresponds to a certain class and it contains a type description of its corresponding class. This means that a schema is a class extension of the class ClassDesc. Thus, the instance identifier for ClassDesc object is equivalent to each instance’s class identifier of its corresponding class. Therefore, it can be said that the type description of an object can be always accessed at run-time. 2.2

Class Definition and Object Manipulation

In spareC++, a class definition consists of two parts: structure definition and behavior definition. Direct superclasses, attributes, and method signatures are specified in the structure definition while bodies of methods are contained in the behavior definition. For example, Fig. 1 shows the structure definition of the class Emp that is defined as a subclass of the class OBJECT, and Fig. 2 shows the behavior definition of the class Emp. As shown in this example, the class definition of spareC++ is similar to that of C++, except that only atomic types and pointers to the class types can be allowed. Also, application programmers can manipulate objects uniformly irrespective of their persistent states. That is, operators defined on objects are used in spareC++ for the same operations as in C++. Thus, objects can be created by

Implementation of Automatic Lock Determination

1 class Emp: virtual public OBJECT { 2 public: 3 String *name; 4 int salary; 5 Emp *manager; 6 Set<Skill*> *skills; 7 Emp *getTopManager(); 8 void addSkill(Skill *newSkill); 9 void changeSalary(int newSalary); 10 }; Fig. 1. Structure Definition 1 2 3 4 5 6 7 8 9 10 11 12

Emp *Emp::getTopManager() { if (manager == NULL) return this; return manager->getTopManager(); } void Emp::addSkill(Skill *newSkill) { if (skills == NULL) skills = new Set("Skill"); skills->add(newSkill); } void Emp::changeSalary(int newSalary) { salary = newSalary; } Fig. 2. Behavior Definition

1 main(int argc, char **argv) 2 { 3 Transaction::begin(); 4 Set<Emp*> *result = new Set("select * from e in Emp;"); 5 while (result->more()) { 6 Emp *emp = result->next(); 7 emp->salary += emp->salary * 0.1; 8 } 9 Transaction::commit(); 10 } Fig. 3. Application Program

337

338

Y.S. Jun, E. Hong, and S.I. Yoo

using new operators, deleted by using delete operators, and referenced by using → operators. In addition, spareC++ supports two facilities for fetching objects. One is the naming facility that uses the mapping between objects and alias names. This facility enables application programmers to associate an object with an alias name, fetch the object through the name, and deprive the object of the name. This functionality is implemented as methods(or friend functions) defined in the root class OBJECT such as attachName, getObject, and detachName. The other is the query facility that provides the set-at-a-time processing similar to those found in relational databases [11]. Application programmers should specify queries written in OQL++ [10] as the argument of the constructor of the class Set to use this facility. Therefore, query results are always temporary Set objects and they can be uniformly manipulated just like ordinary objects. For example, Fig. 3 shows the application program that increases salaries of all Emp objects by 10%. In this example, a new Set object, whose elements are the selected Emp objects, is created through the query facility.

3 3.1

Lock Determination at Compile-Time spareC++ Preprocessing

In spareC++, objects can be specified only through memory pointers pointing to them. However, in most OODBMSs objects can be specified only through their OIDs. Therefore, OIDs are entirely hidden from spareC++ programmers. In order to seamlessly couple spareC++ with OODBMSs, spareC++ preprocessors translate class definitions and application programs written in spareC++ into C++ files, where pointer variables are transformed into OID variables. Above all, structure definitions of classes are preprocessed into C++ header files by the spareC++ class preprocessor, named falcon. After checking the definitions against the current schema, falcon registers the classes into the schema. This means that a type description of a class is created as a ClassDesc object. Then, falcon translates the definitions into C++ by generating an OID class per each class, generating a creation function per each class, and replacing the pointer type to a class with its corresponding OID class type. For example, Fig. 4 shows the output of preprocessing the structure definition in Fig. 1. For the class Emp, as shown in this example, falcon generates the OID class EmpOID and the creation function newEmpOID. Also, we can see that the type of the attribute skills and the return type of the method getTopManager are replaced by SetOID and EmpOID, respectively. Thus, as described above, the value of a class-type attribute is the OID of the object that is referenced through the attribute. OID classes, their methods, and creation functions will be detailed in the next section. Both behavior definitions and application programs are translated into C++ files by the spareC++ program preprocessor, named stealth. By using the type descriptions managed in the schema, stealth (i) replaces the pointer type to a

Implementation of Automatic Lock Determination

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15

struct EmpOID: virtual public OBJECTOID { EmpOID(); EmpOID(EmpOID& oid); Emp *Z(Lock lock); // Access Method }; EmpOID newEmpOID(Emp*); // Creation Function class Emp: virtual public OBJECT { public: StringOID name; int salary; EmpOID manager; SetOID skills; EmpOID getTopManager(); void addSkill(SkillOID& newSkill); void changeSalary(int newSalary); }; Fig. 4. Output of Preprocessing Fig. 1

1 2 3 4 5 6 7 8 9 10 11 12 13

EmpOID Emp::getTopManager() { EmpOID THIS = getID(); if (THIS.Z(R)->manager == NULL) return THIS; return THIS.Z(R)->manager.Z(R)->getTopManager(); } void Emp::addSkill(SkillOID& newSkill) { EmpOID THIS = getID(); if (THIS.Z(R)->skills == NULL) THIS.Z(W)->skills = newSetOID(new Set("Skill")); THIS.Z(R)->skills.Z(R)->add(newSkill); } void Emp::changeSalary(int newSalary) { EmpOID THIS = getID(); THIS.Z(W)->salary = newSalary; } Fig. 5. Output of Preprocessing Fig. 2

1 main(int argc, char **argv) 2 { 3 Transaction::begin(); 4 SetOID result = newSetOID(new Set("select * from e in Emp;")); 5 while (result.Z(R)->more()) { 6 EmpOID emp = result.Z(R)->next(); 7 emp.Z(W)->salary += emp.Z(R)->salary * 0.1; 8 } 9 Transaction::commit(); 10 } Fig. 6. Output of Preprocessing Fig. 3

339

340

Y.S. Jun, E. Hong, and S.I. Yoo

class with its corresponding OID class type, (ii) inserts the code .Z(W) or .Z(R) in front of each operator → which is used to access an attribute or method of an object, and (iii) transforms an operator new into a creation function for the class. In the case of method definitions except constructors, prior to the above translations, stealth (iv) initializes the variable THIS as the OID of the object for which the method is invoked, (v) replaces the keyword this with the variable THIS, and (vi) inserts the code THIS→ in front of an attribute or method of the class where the method is defined. For example, Fig. 5 shows the output file after the translation of the behavior definition in Fig. 2. The method getID, which is defined in the class OBJECT, returns the OID of the object that invokes itself. As shown in line 2 of Fig. 5, this method is used to initialize the variable THIS, which is generated in each method. This output file is compiled into a method object file by the C++ compiler. Also Fig. 6 shows the output file of translating the application program in Fig. 3. Then, this output file is compiled into an object file by the C++ compiler, and then the object file is dynamically linked with the related libraries and the method object files of classes that are used in the application program. By using the dynamic linking, application programmers do not need to recompile application programs, although they may add new methods or change codes of existing methods. 3.2

Update Detection

The method Z generated in each OID class is invoked for a particular OID. This method returns the memory pointer to the object that is associated with the invoking OID. Thus, a call to this method corresponds to an object access and this method is called as the access method. As shown in Fig. 4, the access method has one argument that specifies the lock mode for the object access. According to the translation (ii), stealth should determine the lock mode of each access method at compile-time. In this subsection, we will discuss how stealth determines the argument for the access method. The details of the access method will be given in the next section. First of all, a class-type attribute is defined as an attribute whose type is a class, and a class-type method is defined as a method whose return type is a class. In spareC++, an object expression is defined as f0 →f1 →...→fn−2 →fn−1 →fn (n≥1) where (1) f0 is a variable for an object, (2) fi (1≤i≤n-1) is a class-type attribute or method of fi−1 , and (3) fn is an attribute or method of fn−1 . In order to observe this definition, stealth does the translations (iv), (v), and (vi). In this definition, fk (0≤k≤n-1) actually corresponds to an object because it is a class-type attribute or method. Attribute expression is an object expression whose fn is an attribute of fn−1 ; otherwise, it is defined as method expression. In spareC++ like C++, lvalue is the expression referring to a region of storage or a function [2]. An obvious example of lvalue expression is the name of the region for storage. This implies that an object expression is an lvalue expression. An lvalue expression is modifiable if it is not a function(or method) name, an array name, or a variable specified with the

Implementation of Automatic Lock Determination

341

keyword const [2]. Thus, attribute expressions are modifiable lvalue expressions and method expressions are not. The type of object expression is the type of its fn , and the type of attribute expression is atomic(such as int, float, and char) or class-type. The fn of an int or char attribute expression can be updated iff it is used as an operand of the increment operator ++ or decrement operator --, or a left-hand operand of the assignment operators such as =, +=, and -=. The fn of float attribute expression can be updated iff it is used as a left-hand operand of assignment operators. The fn of class-type attribute expression can be updated iff it is used as a left-hand operand of the operator =. As a result, in spareC++, the database can be updated only at the execution of the statement with an attribute expression used in the location where its fn can be updated. Thus, object expressions are divided into updatable expressions and non-updatable expressions. Any attribute expression, which is used in the location where its fn can be updated, is defined as an updatable object expression; otherwise, it is defined as non-updatable object expression. The value of its fn is updated only when the statement that includes an updatable object expression executes at run-time. Also, it can be said that only the object fn−1 among the objects in the updatable object expression can be updated. In spareC++, it is possible to update objects indirectly through pointers [2] like C++. According to the above definition, however, these updates cannot be detected by stealth. While these indirect updates are necessary in programming language areas, they are not in database areas. Therefore, we decided that spareC++ programmers are prohibited from using indirect object updates through pointers. To achieve this, stealth makes a compilation error whenever it finds a statement where an object expression is used as an operand of address-of operator &. For example, consider the statement e1→manager→age = e2→age; where both e1 and e2 are variables for Emp objects. The object expression on the right-hand side is non-updatable because its type is int and it is not used in any updatable locations. The left-hand side expression is updatable because its type is int and it is used as a left-hand operand of the operator =. Whenever this statement executes at run-time, the value of attribute age of the object e1→manager will be set as the value of object e2 and the object e1→manager will be actually updated too. For another example, the method expression invokes the method fn for the object fn−1 . According to the above definition, this expression is non-updatable although any attribute of the calling object may be updated within the method. However, the updates within the method can be done only in updatable object expressions and they can be detected by stealth. After determining whether an object expression is updatable, stealth converts non-updatable one into f0 .Z(R)→f1 .Z(R)→...→fn−2 .Z(R)→fn−1 .Z(R)→fn and updatable one into f0 .Z(R)→f1 .Z(R)→...→fn−2 .Z(R)→fn−1 .Z(W)→fn . The only difference between the two is the argument of the access method for object fn−1 . As described above, it is because object fn−1 is the only object that will be

342

Y.S. Jun, E. Hong, and S.I. Yoo

actually updated among the objects involved in an updatable object expression. There are some examples for this conversion in Fig. 5 and Fig. 6. Consequently, stealth can detect all the updates of the objects while compiling behavior definition files and application program files written in spareC++. This update detection is regarded as static because it is done at compile-time. Based on this static update detection, stealth can determine an appropriate lock mode for each object access operations in application programs and behavior definitions. Since the static update detection and the lock determination are entirely transparent to application programmers, this mechanism enables programmers to spend less time on locking objects and to concentrate on writing application programs. So, the productivity of application development can improve.

4

Lock Request at Run-Time

As described in the last section, falcon generates an OID class per each class at compile-time and stealth determines a lock mode per each object access. Creation functions and access methods request the lock manager to acquire determined locks for the objects at run-time, just before objects are accessed through the object buffer manager. In addition, they play an important role in manipulating objects uniformly irrespective of their persistent states and also irrespective of whether or not they are resident in virtual memory. 4.1

Run-Time Object Management

As mentioned above, an object is associated with a unique OID and referenced through its OID by other objects. Thus, the request to access an object from an application is always specified through its OID and sent to the object buffer manager that manages objects in virtual memory. For mapping OIDs to the objects, as briefly illustrated in Fig. 7, the object buffer manager maintains an object buffer table which is implemented as a hash table in order to speed up associative searches based on OIDs. The key of this table is the OID, and the value is the pointer to the descriptor of the object associated with the OID, called IMOD(In-Memory Object Descriptor). An IMOD has one-to-one correspondence to an object resident in virtual memory, and it consists of four fields which indicate the current status of the object. The oid field in the IMOD contains OID of the object. The body field contains a pointer through which the contents of the object can be accessed. The lock and persist fields indicate the current lock mode and persistence state of the object, respectively. On receiving a request to access an object through its OID, the object buffer manager detects if an entry for the object is already in the object buffer table. If the entry exists in the table, the manager returns the pointer in the body field of its corresponding IMOD. Otherwise, the buffer manager fetches the object through the storage manager, registers an IMOD for the object in the table with the OID as the key, and returns a pointer in the body field of the IMOD.

Implementation of Automatic Lock Determination

OID

OID

343

OID

Application Buffer Manager OID

ObjectBufferTable OID OID OID ......

OID

OID

oid

char *body IMOD

IMOD

IMOD

Object

Object

Object

Body

Body

Body

Lock lock Persist persist

Fig. 7. Object Buffer Manager

4.2

Creation Function and Access Method

Figure 8 shows the codes of creation function for the class Emp. In the case of other classes, falcon generates their creation functions only by substituting their class names for ”Emp”. Hence, we will continue the discussion about the creation function with the class Emp. This creation function is used to create an Emp object. First, the creation function calls the storage manager to load the ClassDesc object that maintains the type description for the class Emp and then it generates a new OID. The database identifier of the new OID is the identifier of the open database. The class identifier of the new OID is the instance identifier of the OID for the ClassDesc object. The value of the attribute nextId of the ClassDesc object is increased by one and assigned to the instance identifier of the new OID. Secondly, the generated OID is assigned to the attribute id of the passed Emp object. The class Emp inherits the attribute id from the class OBJECT. Thirdly, the function creates an IMOD for the passed Emp object. The oid field of the IMOD is the generated OID and the body field is the pointer to the passed Emp object. Since every created objects are temporary and considered to be updated, the lock field is set as W and the persist field is set as TEMPORARY. Lastly, the function calls the object buffer manager to locate the IMOD in the object buffer table. An object created in a transaction can be used within that transaction. That object can be accessed from other transactions only after it is made persistent and the transaction commits. Thus, it is unnecessary to manage temporary objects in the object buffer table and to consider locks for them. However, temporary objects are managed by the object buffer manager and locked with W lock mode to uniformly manipulate objects irrespective of their persistent states. Figure 9 shows the codes of access method defined in the OID class EmpOID. In the case of other classes, falcon generates their access methods only by substituting their class names for ”Emp”. Hence, we will continue the discussion only with the access method defined in EmpOID. As described above, the lock argument of the access method is statically specified by stealth. First, the access method finds the IMOD for its invoking OID. To do this, the method calls the

344

Y.S. Jun, E. Hong, and S.I. Yoo

1 EmpOID newEmpOID(Emp *body) 2 { 3 ClassDesc *desc = StorageManager->load("Emp", R); 4 EmpOID oid; 5 oid.databaseId = Database::db->id; 6 oid.classId = desc->id.instanceId; 7 oid.instanceId = ++desc->nextId; 8 body->id = oid; 9 IMOD *imod = new IMOD(oid, body, W, TEMPORARY); 10 BufferManager->locateIMOD(oid, imod); 11 return oid; 12 } Fig. 8. Creation Function

object buffer manager to search the entry of the OID of the object buffer table. If the entry already exists in the table, the method gets IMOD to invoke OID since the entry has the pointer to it. Otherwise, the method calls the storage manager to fetch the object that is associated with the OID, creates an IMOD for the object, and calls the object buffer manager to locate the IMOD in the object buffer table. At this time, the fetched object is persistent and intact. Thus, the lock field of the created IMOD is set as R and persist field is set as PERSISTENT. Secondly, if the lock field of IMOD is R and the passed lock argument is W, the lock manager is called to acquire W lock on the object and the lock field of IMOD is set as W. Thus, it can be easily shown that the lock mode for the object accessed in a transaction can be changed from R to W only once in the transaction. Then, the method returns the memory pointer to the object. The access method initially requests the lock manager through the storage manager to acquire R lock for an object just before a persistent object is fetched for the first time in a transaction. R lock for the object can be changed to W lock only if the above condition to change the lock is satisfied. Thus, the locks, which stealth statically determines at compile-time, can be dynamically requested at run-time. For example, if the condition in line 8 of Fig. 5 is not satisfied, the next statement cannot be executed. This implies that it is avoidable to request the lock manager to acquire an unnecessary W lock. Therefore, this dynamic locking is able to optimize the concurrency between transactions. After requesting the lock manager, the access method waits until the lock manager acquires a lock. If a deadlock occurs, the deadlock manager detects the deadlock, and then breaks it by choosing one of the deadlocked transactions as the victim, aborting it, and thereby releasing its locks. The locks, which are acquired in a transaction, are immediately released just after the transaction ends. It is noted that the access methods just send locking requests to the lock manager and the physical locking is entirely done by the lock manager. Depending on the

Implementation of Automatic Lock Determination

345

1 Emp *EmpOID::Z(Lock lock) 2 { 3 IMOD *imod = BufferManager->findIMOD(*this); 4 if (imod == NULL) { 5 char *object = StorageManager->load(*this, R); 6 imod = new IMOD(*this, object, R, PERSISTENT); 7 BufferManager->locateIMOD(*this, imod); 8 } 9 if (imod->lock == R && lock == W) { 10 LockManager->changeLock(*this, W); 11 imod->lock = W; 12 } 13 return (Emp*) imod->body; 14 } Fig. 9. Access Method

implementation of the lock manager, the actual unit, which is physically locked, may be an object, a group of objects, or a page. This implies that this mechanism is independent of physical locking units and it can be easily integrated with any lock managers. Since this mechanism is handled with spareC++ preprocessors, it is independent of operating systems. Thus, this locking mechanism can be ported on other C++-based OODBMSs with minor efforts.

5

Related Work

In this section, we compare previous mechanisms of determining locks in relational DBMSs and previous C++-based OODBMSs. In relational databases, applications are written in embedding SQL statements in a host programming language [11]. In these applications, databases can be accessed only through embedded SQL statements. However, SQL has no declarative locking facilities. Instead, implementations of this language require the lock manager to acquire appropriate locks dynamically as necessary [5]. For example, an UPDATE statement automatically causes an appropriate lock to be acquired on the table containing the data that will be updated. Thus, the locking is entirely transparent to application programmers. In Ontos [3], the lock determination is entirely the programmer’s responsibility. For example, the following is a statement in an Ontos application program; Person *peter = (Person *) OC getObject(”Peter”, WriteLock);. In this statement, Ontos application programmers must recognize whether an object will be updated or not, and he should access it with an appropriate lock. So, application programmers should handle object locking as well as application programming. Moreover, if an application programmer requests incorrect locks in his application program, he must modify and recompile the program. This makes the productivity of application development poor.

346

Y.S. Jun, E. Hong, and S.I. Yoo

There are some C++-based OODBMSs such as Versant [13] and ODMG [8] which require application programmers to notify any direct object updates to the system. In these systems, an application programmer should always explicitly invoke the special notification method for an object in order to notify the direct update to the system before he directly updates the object. The special notification method of ODMG is mark modified and one of Versant’s is dirty. In this mechanism, the lock determination is also placed under the control of application programmers. Thus, this mechanism also makes the productivity of application development poor. In some OODBMSs such as UniSQL [9] and Orion [1], application programmers can update objects only by means of special modification functions. These functions update objects after acquiring W locks for the objects that will be updated. For example, in UniSQL, objects can be only updated by the function db put. Since application programmers do not need to explicitly request locks for each object access, this mechanism imposes less work than the above mechanisms. However, it is very restrictive that the application programmers who are familiar with C++ cannot access and update objects directly. Namely, the UniSQL programmers cannot write the statement peter→age = 20;. This restriction causes the impedance mismatch problem [1] and it also makes the productivity of application development poor. Ode [6], which is the C++-based OODBMS built in AT&T, can only detect the updates that are caused through methods. To support this mechanism, the system automatically modifies the methods by inserting codes for update detection and lock requests. However, direct updates, such as those caused by directly assigning a new value to an attribute, cannot be detected. For example, upon executing the statement peter→age = 20;, Since Ode cannot detect the update to object peter it cannot send a request to acquire a W lock on the object. Thus, this mechanism may cause databases inconsistent. The C++-based OODBMS built by Object Design, ObjectStore [4], can automatically set appropriate locks on objects by using page protection violation and keep track of what has been modified. While this mechanism makes the lock determination transparent to programmers, it has some restrictions in portability. This mechanism cannot be easily ported on OODBMSs with the object-based lock manager because it is based on the page-based lock manager. In addition, [4] said that this mechanism depends on the operating systems, which provide the page protection violation. Therefore, this mechanism is difficult to port on the operating systems, which have no page protection violation.

6

Conclusions

This paper presents automatic object locking for C++-based OODBMSs. At compile-time, spareC++ program preprocessor detects the object expressions that need to access objects in applications and determines the lock mode for each object access. Then, the preprocessor transforms each object access by inserting a call to an access method whose argument is a determined lock mode. At run-

Implementation of Automatic Lock Determination

347

time, the access method, which the spareC++ class preprocessor generates in each OID class, dynamically requests the lock manager to acquire a specified lock for the accessed object. The physical locking is the responsibility of the lock manager. The contribution of this paper requires considerations from the viewpoint of the application development. This locking mechanism is entirely transparent to application programmers. When spareC++ application programmers write applications, there is no need to spend time on locking objects and instead they can concentrate on writing applications. Thus, this mechanism improves the productivity of application development. We have implemented the spareC++ preprocessors [12] in C++ on UNIX. Currently spareC++ and this locking mechanism have been integrated with the Obase C++-based OODBMS [7]. However, they can be easily ported on other C++-based OODBMSs since they are based on the fundamental objectoriented data model and they are also independent of operating systems and lock managers.

References 1. Kim, W.: Introduction to Object-Oriented Databases. MIT Press (1990). 2. Ellis, M., Stroustrup, B.: The Annotated C++ Reference Manual. Addison-Wesley (1990) 3. Andrews, T., Harris, C., Sinkel, K.: Ontos: A Persistent Database for C++. Object-Oriented Database with Applications to CASE, Networks, and VLSI CAD, Prentice-Hall (1991) 4. Lamb, C., Landis, G., Orenstein, J., Weinreb, D.: The ObjectStore Database System. Communication of ACM 34 (1991) 5. Date, C., Darwan, H.: A Guide to SQL Standard. Addison-Wesley (1993) 6. Agrawal, R., Dar, S., Gehani, N.: The O++ Database Programming Language: Implementation and Experience. Proceedings of the 9th International Conference on Data Engineering (1993) 7. Yoo, S., Hong, E., Lee, Y., Park, H., Lee, J., Jun, Y., Kim, T., Chang, H.: Obase: An Object- Oriented Database Management System. Journal of the Korea Information Science Society 21 (1994) 8. Cattell, R.: The Object Database Standard: ODMG-93. Morgan Kaufmann (1994) 9. UniSQL: UniSQL/4GE Application Development Environment. UniSQL Inc. (1994) 10. Lee, Y., Yoo, S.: Semantic Query Optimization in OODB Systems. Proceedings of the 5th International Conference on Database and Expert Systems Applications (1995) 11. Korth, H., Silberschatz, A.: Database System Concepts. McGraw-Hill (1995). 12. Jun, Y., Yoo, S.: Design and Implementation of the spareC++ OODBPL. Proceedings of the 6th International Conference on Database and Expert Systems Applications (1995) 13. Versant: Versant C++ Language Interface Manual. Versant Object Technology Inc. (1995)

Do We Need Information Modeling for the Information Highway? Bernhard Thalheim Brandenburg Technical University at Cottbus, Computer Science Institute POBox 101344, D - 03013 Cottbus, Germany, [email protected]

Information in the Hoystack? Web sites, intranet and the internet are getting larger and more complex. Users have a frustating time finding the information they need. Web site developers and web administrators are more challenged than ever before to effectively develop and maintain the growing and evolving sites. While the technical problems will find their appropriate solution the information variety in the web is so large, so heterogeneous and sometimes to doubtful that casual users find the information in a chaotic manner similar to Brown’s movement of atoms. Indeed, companies selling information products have now web sites which are illegible. In general, this situation reminds the state we faced before the advent of database management systems. Information Modeling as the Silver Bullet? During the panel discussion problem such as the following are to be discussed: – Are there differences between information, data, news and knowledge? What is information? How the user can measure information quality? Does everybody require information or just only data? – What are the differences between the web and databases? Should everybody treat the web as a database? Should be the web understood as a data warehouse? – Are communication and information technologies merging now? – Is information modeling an appropriate approach? Should everybody use information modeling while designing his/her pages? Is information modeling powerful enough for the web? Is it too powerful? Are there peoble who model their web application? Octavian Circle The Panel is going to be organized according the rules of the Octavian cirle: On a certain topic a round table-table discussion is to be organized with 8 (Octavian) people in the round and much more people as listeners. Only those people can talk who are in the round. The talk time is restricted to 3 minutes. If somebody in the round feels that he cannot or should not make any further contribution then he leaves the round and will be a listener. One listeners can now replace this panelist. Questions from outside can be forwarded through the moderator. The moderator cannot leave the round. His contribution is limited to 1 minute. The first two rounds of the panelists are pre-selected what does not mean that this will be the order of making contributions. T.W. Ling, S. Ram, and M.L. Lee (Eds.): ER’98, LNCS 1507, p. 348, 1998. c Springer-Verlag Berlin Heidelberg 1998

Design and Analysis of Quality Information for Data Warehouses *

1

2

Manfred A. Jeusfeld , Christoph Quix , and Matthias Jarke 1

2

Tilburg University, INFOLAB, Tilburg, The Netherlands [email protected] 2 RWTH Aachen, Informatik V, Aachen, Germany {quix,jarke}@informatik.rwth-aachen.de

Abstract. Data warehouses are complex systems that have to deliver highlyaggregated, high quality data from heterogeneous sources to decision makers. Due to the dynamic change in the requirements and the environment, data warehouse system rely on meta databases to control their operation and to aid their evolution. In this paper, we present an approach to assess the quality of the data warehouse via a semantically rich model of quality management in a data warehouse. The model allows stakeholders to design abstract quality goals that are translated to executable analysis queries on quality measurements in the data warehouse’s meta database. The approach is being implemented using the ConceptBase meta database system.

1

Introduction

Data warehouse systems are ideally suited to investigate the design and analysis of quality information because they are consisting of many components, they involve a large number of stakeholders with different goals, and they are constantly being monitored via administration tools. Figure 1 shows the traditional understanding of data warehouse. They scan by so-called wrappers huge data sets and materialize them in a central (or distributed) database system. Clients for data analysis and decision making access the materialized data sets to generate and validate hypotheses about the enterprise. Before data warehouses draw the attention of researchers in the early nineties [CGMH+94,KLSS+95,HGMW+95] the integration of heterogeneous data sources was investigated using semantic data models that tried - with limited success - to resolve inhomogeneities to the largest possible extent [Wie92,HZ96]. They concentrated mainly on improving the consistency of the global schema. Data warehouses

*

This work was supported in part by the European Commission in ESPRIT Long Term Research Project 22469 DWQ (Foundations of Data Warehouse Quality).

T.W. Ling, S. Ram, and M.L. Lee (Eds.): ER’98, LNCS 1507, pp. 349−362, 1998.  Springer-Verlag Berlin Heidelberg 1998

350

M.A. Jeusfeld, C. Quix, and M. Jarke

tackle the problem in a broader way that interestingly makes things easier. Besides consistency, quality goals like timeliness, accessibility and others [WSF95] come into play which reduce the previously absolute consistency goals to a relative one. Moreover, data warehouses are read-only for the clients and materialize only data of interest to decision making. Via aggregation, errors in individual data sources tend to be minimized by the statistical law of the great numbers.

Clients

GIS

OLAP

DSS

Data

Data

Mart

Mart

Administration Agent Meta Database

Data Warehouse

Wrappers/ Loaders

Sources

Text File

DB

External data

Fig. 1. Current Understanding of a Data Warehouse [JJQ*98]

We claim however that the design and analysis of the quality of a data warehouse is not well-understood and a great problem from the perspective of the users [Jans88]. To tackle the problem, a rich semantic data model was proposed [JJQ*98] for the components of a data warehouse linked to a quality model. The architecture model (see Figure 2) provides for modeling objects at source, data warehouse and client level with perspectives for the conceptual view (a variant of the ER semantic data model), the logical view (relational data model enhanced by aggregation data types), and the physical view (an extension of data flow diagrams). Essentially, we provided in [JJQ*98] a method for representing data warehouse objects in the meta database. Data warehouse objects can come from all levels (source, data warehouse, client), from all perspectives (conceptual, logical, physical), and from any abstraction layer. For example, an instance like a specific object of a table can be represented in the meta database just like its relation definition and even the data model. The same holds for transport agents both at the class level (the transport agent as a program defini-

Design and Analysis of Quality Information for Data Warehouses

351

tion) and the instance level (a transport agent process running at a certain time and performing a specific data transport in the data warehouse system). As a consequence, the meta database of the data warehouse has a rather comprehensive view of the structure of the data warehouse (the class layer) and its current state (the instance layer). The idea of [JJQ*98] was to make all data warehouse objects subject to quality assessment. The problem left open in that paper was how to represent the quality model and how to use it both for quality goal formulation and for quality measurement. Here is where this paper continues.

Logical Perspective

Conceptual Perspective OLAP

Client Model

?

Client Schema

DW Schema

Observation

Operational Department Model

Client Data Store

Aggregation/ Customization

Enterprise Model

Source Schema

Transportation Agent

DW Data Store

Wrapper

OLTP

Physical Perspective

Transportation Agent

Source Data Store

Fig. 2. Levels and perspectives for data warehouse models [JJQ*98]

We argue that the meta database of a data warehouse is the right place to explicitly represent quality goals of stakeholders and to transform them into executable queries on results of quality measurement. The results of quality measurements are also stored in the meta database. Thus, analysis of the quality of a data warehouse shall be done by querying the meta database. Our proposal follows the Goal-Question-Metric approach [OB92] originally developed for software quality management. The main difference in our proposal is representation of ‘quality questions’ as executable queries on the semantically rich meta database. The meta modeling language Telos [MBJK90] together with the query capabilities of ConceptBase [JGJ*95] form the basis of the presentation throughout the paper. All concepts and queries can be directly inserted into ConceptBase to form a prototype for the data warehouse quality management. In the next section, a collection of important concepts for quality management is discussed along with informal examples. Then, a formal meta model is

352

M.A. Jeusfeld, C. Quix, and M. Jarke

derived from it and applied to the two tasks of quality goal design and quality analysis. Finally, we discuss limitations of the approach and further work.

2

Basic Concepts for Quality in Data Warehouses

Quality is an issue depending on objects at all levels and perspectives of a data warehouse (Fig. 2), and depending on quality goals of different stakeholders. A measurable object is an object in the conceptual, logical or physical perspective of a data warehouse. This is the common base class for all classes in the architectural model. Examples are the conceptual schema of a data source, the logical schema of the data warehouse, a wrapper agent, the physical location of a multidimensional data cube and so on. Labeling them measurable object means that some measurable quality goals may be attached to them. A quality goal is an abstract requirement as defined in the Goal-Question-Metric approach of Basili et al. [OB92], it is a related to an object and has a stakeholder, dimension and purpose (see below). As such, a quality goal is a natural-language statement by a stakeholder about her/his quality requirement. It is typically not operational in the sense that there is a unique method to check whether the goal is achieved or not. Examples are ”improve the availability of the source s1 until the end of the month (in the viewpoint of the stakeholder DW administrator)” and ”increase the efficiency of the data loading process (in the viewpoint of the stakeholder source administrator)”. By nature, a quality goal is describing some future state of the data warehouse system that is typically not fulfilled in present time. One may also compare them to user requirements in a software development process. However, in the data warehouse setting, quality goals can be formulated at any time in the life cycle of the data warehouse. A quality query operates on quality measurements to check if a quality goal is currently fulfilled or how the measured quality has changed in a certain period of time. The quality query does not only compare expected range with the achieved value of the quality measurement, it can have a more complicated method to check if the quality goal is fulfilled or not (e.g. by the combination of several quality measurements). An example query would be whether the availability value of data source S1 is in the expected range of 97%. Quality queries are the formal (and executable) specifications on how to check whether the quality goal for which it provides evidence is achieved or not. In our model, we assume that any information necessary to decide that is in the DW meta database. Thus, a query to the meta database is sufficient to evaluate a quality query. The algorithms for quality measurements are hidden in the metric agents (see below): they store a quality value as result of a measurement in the meta database.

Design and Analysis of Quality Information for Data Warehouses

353

A quality dimension is a term used to formulate quality goals. The terms are arranged in a specialization hierarchy. Each stakeholder may have his/her own hierarchy of preferred quality dimensions. Examples are data usage quality, accessibility, consistency, timeliness etc. The difference of quality dimension to quality goal is that the latter has a) a direction like ‘improve’ or ‘achieve’, and b) a measurable object it is applied to. Quality dimensions provide the vocabulary to formulate quality goals. Each stakeholder may have a different vocabulary, i.e. hierarchy of quality dimensions. A quality measurement is the documented activity to measure the quality of some measurable objects. The documentation establishes a static relationship between a measurable object and the measured quality value. It uses a quality metric to get an actual quality value of the object, it has an interval of expected values for the quality value, and it is related to a quality dimension. Quality measurements may be specialized for measuring a specific kind of DW objects, e.g. measurement of Non-NullTuples applies only to relations being part of the logical schema. An example would be like ”the current value for availability of source s1 is 23.5 hours per day”. A metric unit specifies the dimension of a quality value. It is analog to physical units like ”meter/second” for measuring speed. We allow that for a given kind of quality measurement, multiple units are possible. For example, one could have ”nullvalues / tuples” or ”nullvalues / attributes” as metric units for measuring the completeness of a relation. Like for physical units, there can be an algebra for transforming a quality value from one metric unit to another. A quality domain specifies permissible quality values which themselves are the results of measurements. Quality values are instances of a quality domain. Domains should be totally ordered, i.e. for two values x,y of a domain either x < y, or x=y or y<x holds. Quality values are different from quality measurements: a measurement is the activity to attach a quality value to a measurable object. Examples are 23 hours and 95%. The pure value has no meaning by itself. Only the link to a measurable object and (via a quality query) the link to a quality goal establishes the context necessary to interpret such a value! A quality range is a set of quality values, in particular of expected quality values. The most common form of ranges are intervals. We generalize this to subsets of quality domains. Finally, a stakeholder is a person who formulates quality goals for the data warehouse. Typically, a stakeholder receives as feedback answers to the quality queries which provide evidence for his/her quality goals. The DW administrator, the decision maker and the source administrator are all stakeholders with different quality goals.

354

3

M.A. Jeusfeld, C. Quix, and M. Jarke

The Quality Meta Model

The above glossary could used be used to understand the issues of quality management. By using a meta modeling approach, it can be made part of the meta database schema of the data warehouse. By doing so, the stakeholders can represent their quality goals explicitly and the meta database maintains the relationship to quality values of measurable objects. Thus, the quality meta model can be used for both design and analysis purposes. To do so, one has to take into account that a quality measurement maps an arbitrary object of a data warehouse (e.g., the number of null values of source relation ‘Sales’) to some value. Thus, a quality measurements relates concepts with different abstraction levels, here a schema concept to a number. Because of this, we propose a quality meta model (see Fig. 3) whose concepts are all at the meta class level (indicated by light gray boxes) with the notable exception of ‘MeasurableObject’ being a meta meta class (white box). The quality meta model provides the notation for formulating quality goals, queries, and measurements. The upper part around ‘QualityGoal’ allows stakeholders to formulate quality requirements for ‘MeasurableObject’. Typical instances of ‘MeasurableObject’ are ‘Relation’, ‘Agent’ etc., i.e. all kinds of objects present in a data warehouse and represented in its meta database. The purpose of a quality goal is interpreted as a direction, e.g. ‘increasing’ or ‘achieving’ some quality goal. It can be used for mapping the quality goal into a query. The description is just a string like the examples for quality goals in the previous section. Finally, a quality goal is linked to the quality dimensions, e.g. availability.

Purpose

direction

description

! QualityGoal

String

forPerson

Stakeholder concreteBy dimension

imposedOn

prefers

! QualityQuery

Quality Dimension

evaluates

isSubDimOf dimension

dependsOn

Measureable Object

onObject

Quality Measurement

expected achieved

of

unit

when Measuring Agent

PowerSet

QualityDomain

Timestamp MetricUnit hasResult

Fig. 3. The quality meta model

Design and Analysis of Quality Information for Data Warehouses

355

While the upper third around quality goal is abstract and design-oriented, the lower third is concrete and analysis-oriented. A measurable object is linked to a quality value by a quality measurement, not incidentally displayed like an ER relationship type. The quality measurement also links a measurable object to expected an unexpected ranges for the quality values. It can be compared to a temperature meter showing a certain value on a scale with green and red ranges. By attaching such a quality measurement explicitly to a measurable object, we express that we want to monitor the quality of that object. It is essential that MeasurableObject is at a different abstract level. Just consider a concept like the relation ‘SalesEurope’. It is at the abstraction level of a simple class whose instances are the tuples of the relation. However, a quality value on the availability of this relation is just a number, i.e. an instance object. In functional terms, a quality measurement maps a concept (like SalesEurope) to a concrete number. In our current model, quality dimensions are related to stakeholders and they form a subsumption hierarchy, e.g. traceability is sub-dimension of Design & administration quality (DW Designer/Administrator viewpoint). We want to stress that the hierarchy of quality dimensions depends on the stakeholder, too. Every stakeholder prefers its own quality dimensions and arranges them in her own hierarchy. In ConceptBase, different viewpoints (roughly hierarchies of quality dimensions) can implemented in different modules, so that the viewpoints are independent from each other. The analysis of these goals are supported by queries working on quality measurements. The relationship between quality queries and quality measurements will be mostly defined implicit through the query classes of ConceptBase.

4

Specialization and Instantiation of the Model

When we instantiate quality goals, we encode actual quality goals of stakeholders like shown above. Instances of quality measurements encode the plan to measure an instance of DW_Object, e.g. the relation ‘SalesEurope’ for its quality value on availabiity. The figure below defines a quality goal ‘AvailGoalforRel’ that states that stakeholders ‘DW Administrator’ may in principle be interested in achieving a certain level of availability for (source) relations. As mentioned above, the natural language definition of quality goal is taken from the GQM model. Goals are represented there as a template of purpose, issue (called quality dimension here), object and viewpoint (called stakeholder here). Consider the following two examples of quality goals: Purpose: QualityDimension: Measurable Object:

Achieve Availability source relation SalesEurope

356

M.A. Jeusfeld, C. Quix, and M. Jarke

Stakeholder: Description:

Data Warehouse Administrator ”... at least once per week”

Purpose: QualityDimension: Measurable Object: Stakeholder: Description:

Increase Efficiency process data loading Source administrator ”... increase speed of data loading by 25%”

Measureable Object

Purpose String

description

imposedOn direction

byPerson

! QualityGoal

Stakeholder prefers

in dimension

Quality Dimension

Relation in

isSubDimOf

objectType

“This goal expresses that ...”

Achieve purpose

in

longtext who

SalesEurope

! AvailGoalforRel

DW-Administrator

whatDim

prefAvail

Availability ot1 in

! Goal_1

in g1

in

Mokrane

Fig. 4. Use of the quality model for goal formulation

Figure 4 shows how quality goals are encoded using the quality meta model. The dark gray boxes are simple classes (schema level). They are pattern to express instance quality goals like ‘Goal_1’ of stakeholder ‘Mokrane’. The figure is not a oneto-one translation of the above textual examples. Instead, one part of the example goes to the simple class level (gray boxes) , and the other to the instance level (black boxes). At the simple class level, a goal ‘AvailGoalforRel’ is formulated that refers to object type ‘Relation’. The purpose is set to ‘Achieve’ and a description is given. Moreover, it is stated that the DW Administrator is in principle interested in such a goal. At the instance level, ‘Goal 1’ represents the fact that ‘Mokrane’ (being a real DW administrator) has instantiated this goal for the (source) relation ‘SalesEurope’.

Design and Analysis of Quality Information for Data Warehouses

357

Note the different abstraction levels of the ‘Goal 1’ and ‘SalesEurope’! By simple instantiation, the same quality goal can be attached/detached to multiple data warehouse objects, here relations. The middle layer of figure 4 represents patterns of quality goals rather than actual quality goals which are found in the lower layer. Thus, the middle layer constitutes re-usable quality goals that only have to be parameterized by the object type (here: an instance ‘Relation’) and the stakeholder (here an instance of ‘DW-Administrator’). A company using our approach may populate this middle layer by their specific collection of quality goal patterns. This collection encodes the domain knowledge of that company about formulating quality goals. Figure 5 instantiates the quality measurement part of the meta model. Again, it defines a pattern for measurements at the simple class level. Here, the class ‘measureNullValues’ is defined to measure relations into percentages of null values per tuple (100% means that all values of all tuples are NULL). The exact interpretation is hidden in the description of the metric unit ‘#nullvalues/#tuples’. A range ‘[0;2]’ is defined as the expected interval The metric agent is ‘nv_counter’ which possibly only accesses a small portion of the relation to estimate the achieved quality value. Timestamp

Measureable Object

dependsOn onObject

Quality Dimension

dimension

when Quality Measurement

expected

PowerSet

achieved

in Measuring Agent

of Quality Domain

MetricUnit in

DateTime

Relation

qualifies measure NullValues NV-counter

domain

P[0;100] [0;100]

#nullvalues/#tuples

in

in

SalesEurope

expectedvalue

o1

23-Apr-1998/14:05

[0;2]

qm1 1

o1 29-Apr-1998/07:39

qm2

Fig. 5. Use of the quality model for measurements

[0;3]

358

M.A. Jeusfeld, C. Quix, and M. Jarke

The definition of the expected range as a query class in ConceptBase is done by queries defining subsets out of the possible quality domain. QueryClass [0;2] isA [0;100] with constraint c: $ (this >= 0.0) and (this =< 2.0) $ end

Class [0;100] in QualityDomain isA Real with constraint c: $ (this >= 0.0) and (this =< 100.0) $ end A quality domain is defined by an ordinary class with membership constraint. Note that query classes automatically classify the instances of their super class which fulfill the constraint of the query class. At the instance level, two measurements are shown. The first ‘qm1’ results in an expected quality value, the second is an incomplete instance that represents a future measurement. Because the ranges are formally defined as queries, the measured quality values are automatically classified to be in the expected or unexpected range. The expected range is attached at the instance level. Thereby, each quality measurement can have its own expected range. Like measured quality values, the expected ranges may change over time. A partial instance like ‘qm2’ is interpreted as the plan to do the measurement of the specified class on the specified object at the specified time expecting the specified range. What remains open is the role of the quality query in the meta model. It is placed between quality goal and quality measurement. The purpose of quality query is to mediate between the quality goal (an abstract requirement that cannot be assessed directly) and a measurement (yielding a concrete quality value). QualityQuery TooManyNullValues isA Source,Relation with constraint c: $ exists m/MeasureNullValues (this hasMeasure m) and not (m in MeasureNullValues^exprange) $ end That query is evidence for a quality goal like ‘AchieveCompl’ (achieve completeness of source relations). Note that the query restricts the relations to be measured. A more complicated query has to be defined for the purpose of increasing a certain quality goal, e.g. increase the number of source relations which have an expected quality

Design and Analysis of Quality Information for Data Warehouses

359

value for the percentage of null values. The instances of the query ‘BetterOnNullValues’ shown below are all source relations of the data warehouse which perform better on the null values (based on two measurements). The attribute ‘after’ was not shown in the meta model to ensure the readability. The objects ‘Source’ and ‘Relation’ locate the kind of data warehouse objects that are of interest to this query: it is the source schema of the logical perspective in Fig. 2. QualityQuery BetterOnNullvalues isA Source,Relation with constraint c: $ exists m1,m2/MeasureNullValues (this hasMeasure m1) and (this hasMeasure m2) and (m2 after m1) and exists v1,v2/[0;100] (m1 qualityvalue v1) and (m2 qualityvalue v2) ==> (v1 > v2) $ end In the implementation of the quality model, a metric agent would store the precedence of measurements in the meta database. More detailed information about measurements is also possible. For example, the attribute ”when: Timestamp” specifies for a quality measurement when it was (or will be taken). Then, an instance like ‘qm2’ could lack a quality value but could have a ‘when’ attribute sometime in the future. Such incomplete measurement objects denote a schedule of measurements very much like a crontab file in UNIX. The quality meta model of Fig. 3, a quality goal is made concrete by multiple qualIty queries. Quality queries themselves accesses quality measurements attached to data warehouse objects. Data warehouse objects are classified in the DW meta database according to level, perspective, and abstraction layer as introduced in Sect. 1. Hence, quality queries aggregate information of rather heterogeneous nature in order to provide evidence for the fulfillment or non-fulfillment of a quality goal. The usefulness of the quality queries depends on the completeness, timeliness, accuracy, and consistency of the meta database, particularly of the quality measurements materialized there. The meta database itself can be regarded as a view on the data warehouse system which has to be maintained according to secondary quality goals. We will however not elaborate on this problem here.

360

M.A. Jeusfeld, C. Quix, and M. Jarke

5

Conclusions

We presented a quality meta model for data warehouses that can be used for both design of quality and analysis of quality measurements. The model can be directly incorporated into the meta database system ConceptBase. The main advantages are: • •

•

•

Any kind of measurable object is allowed as long it is represented in the meta database of the data warehouse. Specifically, static objects like schema concepts and dynamic objects like wrappers are supported. Quality goals can be formulated from the perspectives of an extensible set of stakeholders. Each stakeholder can assess the data warehouse quality from his/her perspective by evaluating the quality queries attached to his/her quality goals. Quality queries are executable queries on the meta database. Their answers are the evidence for a stakeholder to decide whether the quality is appropriate or not. At any time quality queries can be inserted, extended, modified, and removed. This is due to the fact that the meta database is not just a CASE repository but an integral part of the runtime data warehouse system. Quality measurements are explicitly stored in the meta database. By materializing sequences of quality measurements of the same type in the meta database, one can realize more advanced quality goals about trends by appropriate quality queries.

Besides the integrated quality meta model, a major achievement is the separation of quality information at the instance and schema level. The latter provides for a collection of reusable quality goals and measurements that are activated by simple instantiation in the meta database. There are a couple of research questions to be addressed. First, a suitable collection of quality metrics for data warehouses has to be investigated. Starting point is the research and practice on metrics in the software development area. Second, a suitable strategy for materializing the quality measurements is missing. For the moment, we assume that there are external metric agents that compute some quality value for a given measurable object. But when should the agent be activated? Supposedly, measuring the quality of a component like a source relation is computationally expensive. One simply cannot afford to measure the quality for all components continuously. Interestingly, the quality of the materialized quality measurements can be assessed like the quality of any other component. They have a certain accuracy, a certain timeliness etc. The interdependencies between quality measurements has not been addressed in this paper. One can assume that the quality of interacting data warehouse components depend on each other. For example, the completeness of a materialized data cube depends on the completeness of the data sources. Would it be possible to estimate the

Design and Analysis of Quality Information for Data Warehouses

361

quality value of the data cube based on the quality values of the data sources, or vice versa? The approach is currently implemented in the Esprit Project Foundations of Data Warehouse Quality (DWQ). Test beds are being provided by associated partners from the telecommunications industry and the insurance business. A prototype showing the design and analysis capabilities is finished and being demonstrated at data warehouse conferences and workshops [GJJ*98]. Research on quality management for data warehouses is still at its genesis. Data warehouses are an excellent research object in this respect: there has been little investigated so far and the need for a solution is imminent. The quality meta model elaborated on in this paper provides the structure for representing knowledge on how to do the quality management for a given application. The actual selection of quality goal patterns and quality measurement plans is subject to the application domain. Thus, an important next step is to populate the meta database with domain knowledge on how to organize quality management for a given application. A further research goal is to extend to method to the design of a data warehouse which includes view refreshment strategies, selection of the right source databases, filters, transport agents etc. based on their quality properties. Our approach currently concentrates on the assessment of the quality of a given data warehouse solution. By linking it to a quality-oriented design/evolution method, we hope to establish a feedback cycle for continuous improvement of a data warehouse solution according to changing quality goals. Acknowledgments. Many thanks go to our DWQ partners, especially to Mokrane Bouzeghoub, Maurizio Lenzerini, Panos Vassiliadis, and Enrico Franconi for commenting on earlier versions on the quality meta model.

6

References

[CGMH+94]

[GJJ*98]

[HGMW+95]

[HZ96]

[Jans88]

S. Chawathe, H. Garcia-Molina, J. Hammer, K. Ireland, Y. Papakonstantinou, J. Ullman, J. Widom. The TSIMMIS project: integration of heterogeneous information sources. In Proc. of IPSI Conference, Tokyo (Japan), 1994. M. Gebhardt, M. Jarke, M.A. Jeusfeld, C. Quix, S. Sklorz. Tools for data warehouse quality. In Proc. 10th Intl. Conf. on Scientific and Statistical Database Management (SSDBM'98), Capri, Italy, July 1-3, 1998. J. Hammer, H. Garcia-Molina, J. Widom, W. Labio, Y. Zhuge. The Stanford Data Warehousing Project. Data Eng., Special Issue Materialized Views on Data Warehousing, 18(2), pp. 41-48, 1995. R. Hull, G. Zhou. A Framework for supporting data integration using the materialized and virtual approaches. Proc. ACM SIGMOD Intl. Conf. Management of Data, pp. 481–492, Montreal, 1996. M. Janson. Data quality: the Achilles heel of end-user computing,

362

M.A. Jeusfeld, C. Quix, and M. Jarke

[JGJ+95]

[JJQ*98]

[JV97]

[KLSS95]

Omega J. Management Science, 16, 5, 1988. M. Jarke, R. Gallersdörfer, M.A. Jeusfeld, M. Staudt, S. Eherer. ConceptBase - a deductive object base for meta data management. In Journal of Intelligent Information Systems, 4, 2, pp. 167-192, 1995. M. Jarke, M.A. Jeusfeld, C. Quix, P. Vassiliadis: Architecture and quality in data warehouses. In Proc. 10th Intl. Conf. CAiSE*98, Pisa, Italy, June 8-12, 1998, pp. 93-113, Springer-Verlag, Berlin. M. Jarke, Y. Vassiliou. Foundations of data warehouse quality -- a review of the DWQ project. Proc. 2nd Intl. Conf. Information Quality (IQ-97), Cambridge, Mass., 1997. T. Kirk, A.Y. Levy, Y. Sagiv, and D. Srivastava. The Information Manifold.

Proc. AAAI 1995 Spring Symp. on Information Gathering from Heterogeneous, Distributed Environments, pp. 85-91, 1995. [MBJK90]

[OB92] [Wie92] [WRK95] [WSF95]

J. Mylopoulos, A. Borgida, M. Jarke, M. Koubarakis. Telos – a language for representing knowledge about information systems.. In ACM Trans. Information Systems, 8, 4, pp. 325-362, 1990. M. Oivo, V. Basili. Representing software engineering models: the TAME goal-oriented approach. IEEE Trans. Software Eng. 18, 10, 1992. G. Wiederhold. Mediators in the architecture of future information systems. IEEE Computer, pp. 38-49, March 1992. R.Y. Wang, M.P. Reddy, H.B. Kon. Towards quality data: an attribute-based approach, Decision Support Systems, 13, 1995. R.Y. Wang, V.C. Storey, C.P. Firth. A framework for analysis of data quality research, IEEE Trans. Knowledge and Data Eng. 7, 4, 1995.

Data Warehouse Schema and Instance Design Dimitri Theodoratos? and Timos Sellis Department of Electrical and Computer Engineering, Computer Science Division, National Technical University of Athens, Zographou 157 73, Athens, Greece {dth,timos}@dblab.ece.ntua.gr

Abstract. A Data Warehouse (DW) is a database that collects and stores data from multiple remote and heterogeneous information sources. When a query is posed, it is evaluated locally, without accessing the original information sources. In this paper we deal with the issue of designing a DW, in the context of the relational model, by selecting a set of views to materialize in it. Views allow to compute both the schema and the instance of the DW from the schemas and the instances of the source relations. We briefly present a theoretical framework for the DW design problem, which concerns the selection of a set of views that (a) fits in the space allocated to the DW, (b) answers all the queries of interest, and (c) minimizes the total query evaluation and view maintenance cost. We then formalize it as a state space search problem by taking into account multiquery optimization over the maintenance queries (i.e. queries that compute changes to the materialized views) and the use of auxiliary views for reducing the view maintenance cost. Finally, incremental algorithms and heuristics for pruning the search space are presented.

1

Introduction

Data warehousing is an in-advance approach to the integration of data from multiple, possibly very large, distributed, heterogeneous databases and other information sources. In this approach, selected information from each source is extracted in advance, filtered and transformed as needed, merged with relevant information and loaded in a repository (Data Warehouse - DW). The Data Warehousing approach allows for decoupling as much as possible On-Line Analytical Processing (OLAP) from On-Line Transaction Processing (OLTP). Data warehouse architecture: Figure 1 shows a typical DW architecture [2]. The data at each layer is derived from the data of lower layers. At the lowest layer there are the distributed operational data sources. The central layer is the global or principal Data Warehouse. The upper layer contains the local DWs or Data Marts. Data Marts contain highly aggregated data for extensive analytical processing [7]. They are also probably less frequently updated than global DWs. We see a DW as a set of materialized views [17] in the framework of the relational ?

Research supported by the European Commission under the ESPRIT Program LTR project “DWQ: Foundations of Data Warehouse Quality”

T.W. Ling, S. Ram, and M.L. Lee (Eds.): ER’98, LNCS 1507, pp. 363–376, 1998. c Springer-Verlag Berlin Heidelberg 1998

364

D. Theodoratos and T. Sellis Data Marts

... Maintenance queries

Queries

Global DW

Queries Maintenance queries

Data sources

...

Fig. 1. DW architecture

data model. Views allow to compute both the schema and the instance of the DW from the schemas and the instances of the source relations. Users address their queries to the global DW or to the local DWs. These queries must be evaluated locally, at each DW, without accessing the (remote) data of the lower layer. In this work we are going to address global DW design issues (Local DW design encounters similar problems but queries and views are mainly grouping/aggregation queries). Thus, in the following, DW refers to the global DW. We call the queries that are issued by the users against the DW simply queries. DW view maintenance: When changes to the data in the lowest layer occur, they must be propagated to the data of the higher level. Different maintenance policies can be applied. Usually, a Data Warehouse is updated periodically. Though, there are applications issuing queries to the Data Warehouse that need current data. In this case an immediate [1] or a “on demand” deferred update policy is adopted. In order to update the materialized views of a DW, after a change to the source relations, an incremental or a rematerialization strategy can be employed. In both cases, queries are issued against the source relations. The DW evaluates theses queries by appropriately sending queries to the source relations and receiving the answers. It then performs the updating of the materialized views. We call the queries that are issued by the DW against the source relations, for maintenance reasons, maintenance queries. Multiquery optimization on maintenance queries: The changes taken into account for maintaining the materialized views at the DW may affect more than one view. Then multiple maintenance queries defined over the source relations are issued for evaluation. These maintenance queries may contain subexpressions that are identical, equivalent, or more generally subexpressions such that one can be computed from the other. We describe these subexpressions by the generic term common subexpressions [3]. The techniques of multiple query optimization [13,14] allow possibly non-optimal local query evaluation plans to be combined into an optimal global plan, by detecting common subexpressions between queries.

Data Warehouse Schema and Instance Design

365

Using materialized views to reduce the maintenance cost of other views: A global evaluation plan for some maintenance queries can be executed more efficiently if some intermediate subqueries are kept materialized in the DW [11], or can be computed from views that are kept materialized in the DW. It is worth noting that an optimal global evaluation plan without materialized subqueries can be completely different than the optimal global evaluation plan when materialized views are used. The existence of these materialized views can greatly reduce the cost of evaluating maintenance queries. Indeed, the computation of the corresponding subqueries is avoided or simplified. Further, since DWs are typically distributed systems, access of the data sources and expensive data transmissions are reduced. 1.1

The Data Warehouse Design Problem

Here we deal with the problem of selecting a set of views to materialize in a DW. DWs are usually used for OLAP and Decision Support. These applications require high query performance. Selections of views though that guarantee high query performance, may require also an important view maintenance cost. In fact low query evaluation cost and low view maintenance cost are conflicting requirements. Low maintenance cost is desired because otherwise frequent updating cannot be achieved and current data is a key requirement for many DW applications. Thus, we are seeking here for sets of views that minimize a combination of the query evaluation and the materialized view maintenance cost (operational cost). Another issue somehow orthogonal to minimizing the operational cost is the space occupied by the materialized views. Clearly, the materialized views should fit in the DW. Thus, their total size should not exceed the space available at the DW. The problem: The problem we address (DW design problem) consists on selecting a set of views to materialize in the DW such that: 1. The materialized views fit in the space available at the DW. 2. All the queries can be answered using this set of materialized views (without accessing the remote source relations). 3. The combination of the query evaluation cost and the view maintenance cost (operational cost) of this set of views is the minimal among all the sets of views that satisfy the previous two constraints. 1.2

Contribution and Outline

In this paper we state formally the DW design problem and we provide a method for solving it for a certain class of relational queries and views. The approach was first introduced in [15] and is extended here in order to take into account space constraints, multiquery optimization over the maintenance queries and the use of views in the maintenance process of other views. The former views may be

366

D. Theodoratos and T. Sellis

materialized ad-hoc without being used for answering queries in which case they are called auxiliary views. The main contributions of the paper are the following: – We set up a theoretical basis for the DW design problem. – We provide a method for solving this problem by taking into account multiquery optimization over the maintenance queries and the use of views in the maintenance process of other views. – The solution is constructive. Thus, we provide both a set of views to materialize in a DW and a complete rewriting of all the queries over it that minimizes the operational cost. – We design incremental algorithms and we suggest heuristics for pruning the search space. The rest of the paper is organized as follows. The next section contains related work. In Sect. 3, we formally state the DW design problem and we provide some intuition on how to deal with it. Section 4 determines the search space and shows that it contains a solution to the problem. Incremental algorithms and heuristics are presented in Sect. 5. Finally, Sect. 6 contains concluding remarks and possible extension directions. Because of lack of space, the proofs are omitted. They can be found in [16].

2

Related Work

Design problems using views usually follow the following pattern: select a set of views to materialize in order to optimize the query evaluation cost, or the view maintenance cost or both, eventually in the presence of some constraints. The papers [6,5,4] aim at optimizing the query evaluation cost: In [6], the problem is addressed in the context of aggregations and multidimensional analysis under a space constraint. This work is extended in [5], where greedy algorithms are provided, in the same context, for selecting both views and indexes. In [4], greedy algorithms are provided for queries represented as AND/OR graphs. The works [11,8] aim at optimizing the view maintenance cost: In [11], given a materialized SQL view, an exhaustive approach is presented as well as heuristics for selecting additional views that optimize the total view maintenance cost. [8] considers the same problem for select-join views and indexes. Given a selectproject-join view, [10] derives, using key and referential integrity constraints, a set of auxiliary views other than the base relations that eliminate the need to access the base relations when maintaining both the initial and the auxiliary views (i.e. that makes the views altogether self-maintainable). The works [12,18] aim at optimizing the combined query evaluation and view maintenance cost: [12] provides an A* algorithm in the case where views are seen as sets of pointer arrays under a space constraint. [18] considers the same problem for materialized views but without space constraints. Nevertheless, the maintenance cost model does not take into account multiquery optimization over the maintenance queries or the use of materialized views when maintaining other views. [4] provides a formalization of the problem of selecting a set of views that

Data Warehouse Schema and Instance Design

367

minimizes the combined cost under a space constraint but it does not provide any algorithm for solving the problem in the general case. His approach considers a centralized DW environment where all the source relations are available locally for computation. Note that none of these approaches require the queries to be answerable exclusively from the materialized views as is the case in the present work. In [12,18,4] the materialized views are chosen in a (preprocessed) global evaluation plan for the queries resulting by a bottom-up merging of local plans. There might be though views in the set of views that minimizes the combined cost that do not appear in this plan. In the present paper we follow a method that decomposes and merges views in a top-down way and we show that the optimal view set appears in our search space. [15] follows an approach similar to the one we present here but it does not take into consideration space constraints or multiquery optimization over the maintenance queries.

3

Definitions and Formal Statement of the Problem

We consider that a non-empty set of queries Q = {Q1 , . . . , Ql } is given, defined over a set of source relations R. The DW contains a set of materialized views V over R such that every query in Q can be rewritten completely over V. Thus, all the queries in Q can be answered locally at the DW, without accessing the source relations in R. Let Q be a query over R. By QV , we denote a complete rewriting of Q over V. This notation is extended to sets of queries. Thus, we write QV , for a set containing the queries in Q, rewritten over V. Given Q, a DW configuration C is a pair < V, QV >. Consider a DW configuration < V, QV >. We call simple views those views in V that appear in QV and auxiliary views the rest of the views in V. The intuition behind this definition is the following: the simple views in V are those that are used for answering the queries in Q. The auxiliary views may be used in reducing the maintenance cost of other simple or auxiliary views. Simple views too may be used in the same manner, but they have to appear in QV . The cost of evaluating the queries in QV is denoted E(QV ), while the maintenance cost of the views in V is denoted M (V). The operational cost T (C) of a DW configuration C =< V, QV > is T (C) = E(QV )+cM (V). The parameter c indicates the relative importance of the query evaluation and view maintenance cost. The storage space needed for materializing the views is denoted V. Our approach for dealing with the DW design problem is independent of the way materialized view storage space, query evaluation and view maintenance cost is assessed. The solutions suggested, though, do depend on the specific cost model used. We state now the DW design problem as follows. Input: A set of source relations R and a set of queries Q = {Q1 , . . . , Ql } over R The functions E for the query evaluation cost and M for the view maintenance cost.

368

D. Theodoratos and T. Sellis

The function S for the materialized views space. The space available in the DW for materialization t. A parameter c. Output: A DW configuration C =< V, QV > such that S(V) ≤ t and T (C) is minimal.

4

The Search Space

In this section, we model the DW design problem as a state space search problem based on a multiquery graph representation of the views. We then prove that under the assumption of a monotone cost model, our search space is guaranteed to contain a solution to the problem (if such a solution exists). We consider the class of relational views (queries) of the form σF (R1 × . . . × Rk ). The formula F is a conjunction of comparisons of the form x op y + c or x op c where op is one of the comparison operators =, <, ≤, > and ≥, c is an integer valued constant, and x, y are attribute names. Conjuncts involving attributes from only one relation are called selection predicates, while conjuncts involving attributes from two relations are called join predicates. Attributes of every Ri are involved with those of at least one other Rj in a predicate join in F . All the Ri s are distinct (no self- joins). Without loss of generality, we consider that attribute names in different relations are distinct 4.1

States

In order to define states, we use the notion of the multiquery graph to represent a set of views V. Given a view V = σF (R1 × . . . × Rk ), its query graph GV is a multigraph defined as follows: 1. The set of nodes of GV is the set of relations appearing in V . 2. For every join predicate p in V involving attributes of the relations Ri and Rj there is an edge between Ri and Rj labeled as V : p. Such an edge is called join edge. 3. For every selection predicate p in V involving attributes of the relation Ri , there is a loop edge on Ri labeled as V : p. If V = Ri , there is a loop edge on Ri labeled as V : T . The symbol T denotes here a valid formula. Both these edges are called selection edges. Views in the multiquery graph can be marked. We represent marked views in GV by preceding their names by a ∗. Clearly, a multiquery graph GV contains all the information about the views in V. A state s is a pair < GV , QV >. Thus, a state s is essentially the DW configuration C =< GV , QV >. In a state s, we use the letter W to refer to simple views, the letter Z to refer to auxiliary views while the letter V is used to refer to both of them indiscreetly.

Data Warehouse Schema and Instance Design

369

Example 1. Consider the source relation schemas R(A, B), S(C, D), T (E, F ), P (G, H) and U (K, L). Let V = {W1 , W2 , W3 } be a set of views over these relations, where the views W1 , W2 and W3 (in a form using joins) are defined as follows: W1 = R 1A≤C σD>5 (S) 1C<E+3 T 1F ≥G σH=7 (P ) W2 = R 1A=C S 1C<E σE=F (T ) and W3 = σL≤D (S 1C=E T 1E>K U ) The corresponding multiquery graph GV is depicted on Fig. 2. 2

W2: C<E

W2: E=F W1: H=7

W2: A=C S W1: A<=C R

W1: C<E+3

T P

W3: C=E

W1: D>5 W3: L<=D

U

W1: F>=G

W3: E>K

Fig. 2. The multiquery graph GV

With every state s, a cost is associated through the function cost(s). This is the operational cost T (C) of the DW configuration C. Also, a size is associate through the function size(s). This is the space S(V) needed for materializing the views in V. 4.2

Transitions

Transitions between states are defined through the following six state transformation rules that can be applied to a state s =< GV , QV >. Each state transformation rule transforms the multiquery graph GV and in most cases introduces new view names in it. If this transformation of GV affects a simple view, the rule rewrites the queries in QV over the new view set. – Selection edge cut: If e is a selection edge on a node R of an unmarked view in GV labeled as V : p, where p 6= T , construct a new multiquery graph as follows: (a) if e is the unique edge of R in GV labeled by V , then replace its label by V1 : T , where V1 is a new view name (in this case, V1 represents the source relation R). (b) otherwise, remove e from GV and replace every occurrence of V in GV by a new view name V1 . New view names should not already appear in GV . They have to respect the convention on the simple and auxiliary view names. That is, if V is the simple view W the new view name is W1 and similarly for auxiliary views.

370

–

–

–

–

–

D. Theodoratos and T. Sellis

If V is a simple view, replace any occurrence of V in QV , by the expression σp (V1 ). Join edge cut: If e is a join edge of an unmarked view in GV labeled as V : p and the removal of e from the query graph GV of V does not divide GV into two disconnected components, then: Remove e from GV , and replace every occurrence of V in GV by a new view name V1 . Note that the removal of e does not divide GV into two disconnected components if there are also other edges in GV between the same nodes, or if GV is cyclic, and e is part of a cycle. If V is a simple view, replace any occurrence of V in QV , by the expression σp (V1 ). View split: If e is a join edge of an unmarked simple view in GV labeled as W : p and the removal of e from the query graph GW of W divide GW into two disconnected components then: Remove e from GV and replace every occurrence of W in GV in the one component of GW in GV by a new view name W1 and in the other component by a new view name W2 . Replace any occurrence of W in QV by the expression σp (W1 × W2 ). View augmentation: If the predicate p of an unmarked view V in GV implies a predicate p0 of a different simple view W in GV then: Replace V : p in GV by V : p0 and then replace any occurence of V in GV by a new view name V1 . If V is a simple view, replace any occurrence of V in QV , by the expression σp0 (V1 ). A predicate p implies a predicate p0 if p is more restrictive than p0 . For instance, A ≤ B +1 is more restrictive than A ≤ B +5. Clearly the implication entails that both predicates involve the same attributes. View elimination: If the simple view W1 and the unmarked simple view W in GV have the same set of nodes and each predicate of W1 is implied by a predicate of W then: Remove all the edges labeled by W in GV . Replace any occurrence of W in QV by σp1 ∧...∧pn (W1 ), where p1 , . . . , pn are the predicates of W1 that are not implied by a predicate of W . If there is no such predicate, simply replace any occurrence of W in QV by W1 . Auxiliary view generation: If R1 , . . . , Rk are some (but not all the) nodes of a view V in GV and the subgraph of the query graph of V defined by these nodes is a connected graph then: Mark the view V in GV . Let Z be a new auxiliary view name. For every edge in GV on Ri or between Ri and Rj , i, j = 1, . . . k, labeled as V : p (or ∗V : p if the view V is marked) add an edge in GV between the same nodes, labeled as Z : p. We say that the auxiliary view Z is based on the view V .

Data Warehouse Schema and Instance Design

371

The set QV is not modified, i.e. the queries in QV are not rewritten over a new view set. Example 2. Consider the query set Q = {Q1 , Q2 , Q3 } and the view set V = {W1 , W2 , W3 } of example 4.1 where each query Qi is defined as the view Wi . Let QV1 = W1 , QV2 = W2 and Q3 = W3 . We apply in sequence state transformation rules to the state < GV , QV > and we depict the resulting state. In Fig. 3a we show GV after the application of the selection edge cut rule to the edge labeled as W1 : D > 5. The query Q1 is rewritten as follows: QV1 = σD>5 (W4 ). The queries Q2 and Q3 are not affected by this transformation. W4 is a new view name. New view names may be introduced in the multiquery graph and in the rewritings of the query definitions during the application of the transformation rules. W2: C<E

W2: C<E

W2: E=F W4: H=7

S W4: A<=C R

W2: E=F W4: H=7

W2: A=C

W2: A=C W4: C<E+3

S

T

W4: A<=C

P W3: C=E

W3: L<=D

U

(a)

W4: F>=G

W3: E>K

R

W4: C<E+3

T P

W5: C=E

U

W4: F>=G

W5: E>K

(b)

Fig. 3. GV after an application of (a) the selection edge cut rule and (b) the join edge cut rule

Figure 3b shows GV after the application of the join edge cut rule to the join edge labeled as W3 : L ≤ D. This transformation rule can be applied because the join edge is part of a cycle in the view W3 . The query Q3 is now rewritten as follows: QV3 = σL≤D (W5 ). The queries Q1 and Q2 are not affected. In Fig. 4a, the view split rule has been applied to the join edge labeled as W4 : F ≥ G of GV . Only the query Q1 is affected. Its rewriting is: QV1 = σD≥5∧F ≥G (W6 × W7 ) or (in a form using joins) QV1 = σD≥5 (W6 1F ≥G W7 ). The views W2 and W6 are defined over the same set of nodes and the predicates A ≤ C and C < E + 3 of W6 (these are the only predicates of W6 ) are implied by the predicates A = C and C < E of W2 respectively. Thus, we can apply the view elimination rule to the view W2 . The resulting multiquery graph is depicted on Fig. 4b. The only affected view is rewritten as follows: QV2 = σA=C ∧ C<E (W6 ). The view W6 appears now in the rewriting of both queries Q1 and Q2 . Notice that no new view name is introduced by this transformation. Up to now, there are no auxiliary views in GV . The application of the auxiliary view generation rule to the nodes S and T of the view W5 generates the auxiliary view Z1 depicted on Fig. 5a. Auxiliary views are represented by dashed

372

D. Theodoratos and T. Sellis W2: C<E

W2: E=F W7: H=7

W7: H=7

W2: A=C S W6: A<=C

W6: C<E+3

S

T

W6: A<=C

P

R

U

T P

R

W5: C=E

W6: C<E+3 W5: C=E

W5: E>K

U

(a)

W5: E>K

(b)

Fig. 4. GV after an application of (a) the view split rule and (b) the view elimination rule

lines on the figures. Note that the view W5 is now marked. No query rewriting is needed for this transformation. W7: H=7

Z1: C=E

S W6: A<=C R

W6: C<E+3

S

T

W6: A<=C

P R

*W5: C=E

U

(a)

W5: E>K

W7: H=7

Z2: C<E+3

W6: C<E+3

T P

*W5: C=E

U

W5: E>K

(b)

Fig. 5. GV after an application of (a) the auxiliary view generation rule and (b) the view augmentation rule

The predicate C = E of the view Z1 implies the predicate C < E + 3 of the view W6 . By applying the view augmentation rule to these views we obtain the multiquery graph of Fig. 5b. Note that the auxiliary view Z2 can be used for maintaing both: the view W5 and the view W6 . Since no simple view is modified by this transformation, the queries need not be rewritten. 2 By applying any of the six state transformation rules to a state s we obtain the 0 multiquery graph GV of a set of views V0 over R and a complete rewriting of Q 0 0 over V0 , i.e. a new state < GV , QV >. There is a transition T (s, s0 ) from state s to state s0 iff s0 can be obtained by applying any of the six state transformation rules to s. 4.3

Completeness of the State Transformation Rules

We start by providing some definitions. A Boolean expression of predicates implies another such expression if every assignment of values to the attribute names

Data Warehouse Schema and Instance Design

373

that satisfies the first expression, satisfies also the second. Two Boolean expressions are equivalent if they imply each other. We say that a query Q contains another query Q0 if the materialization of Q is a superset of the materialization of Q0 for any instance of the base relations. Two queries are equivalent if they mutually contain each other. Definition 1. A query Q = σF (R1 × . . . × Rk ) is in full form when (a) if a predicate p is implied by F , then there is a predicate in F that implies p, and (b) F is not redundant in the sense that there is no predicate in F that is implied by another predicate in F . 2 Proposition 1. Any query Q = σF (R1 × . . . × Rk ) can be put in an equivalent full form in polynomial time 2 A view V 0 is a subview of a view V if V can be rewritten using V 0 . A view V 0 can be used in maintaining a view V if V can be partially rewritten [9] using V 0 . The following theorem is a completeness statement for the state transformation rules. Theorem 1. Let Q be a set of queries in full form, C =< V, QV > be a DW configuration, sub(V) be a set of subviews of the views in V that contains all the views in V, and X be a set of complete rewritings of some views in 0 0 sub(V) over views in sub(V). Then, there is a state < GV , QV > obtained by applying in sequence a finite number of state transformation rules to the state s0 =< GQ , QQ > such that: (a) There is a set sub(V0 ) and a mapping f from sub(V) to sub(V0 ) such that ∀V ∈ sub(V), V contains f (V ). 0 (b) For every query QV ∈ QV , the query QV involves exactly the images of the views in QV with respect to f . (c) For every complete rewriting in X of a view V over the views V1 , . . . , Vn , where V, V1 , . . . , Vn ∈ sub(V), there is a complete rewriting of f (V ) over the views f (V1 ), . . . , f (Vn ). 2 This theorem generalizes results in [15] where multiquery optimization on the maintenance queries and the use of auxiliary views are not taken into account. 4.4

Search Space Definition

Our search space is the graph that can be obtained by considering exhaustively all the possible transitions from the initial state s0 =< GQ , QQ > and the subsequent states thus considered. Clearly, this a not merely a tree but a graph without cycles rooted at the state s0 . A cost model is monotone if the cost of computing a query Q defined over the views V1 , . . . , Vn is not greater than the cost of computing a query Q0 , contained by Q, and defined over the views V10 , . . . , Vn0 , when Vi contains Vi0 , i ∈ [1, n]. Under the assumption of a monotone cost model for evaluating queries and maintenance queries, and as a consequence of theorem 4.1, there is a path in the search space from the state s0 to a state corresponding to a DW configuration having a minimal operational cost.

374

5

D. Theodoratos and T. Sellis

Incremental Algorithms and Heuristics

We present in this section an exhaustive incremental algorithm and a greedy one and we suggest heuristics for pruning the search space. In [16]we provide a generic cost model and we show how the cost and the size of a new state can be computed incrementally when transiting from one state to another. An exhaustive algorithm. For obtaining the lowest maintenance cost when maintaining a view using another view defined over a fixed set of base relations, only one such view is needed. All the other views defined over the same set of base relations are useless. Thus, in the following, we consider that the state definition and the auxiliary view generation rule are modified in a way that the application of the rule to the nodes R1 , . . . , Rk of a view V is prohibited, if an auxiliary view based on V and defined over R1 , . . . , Rk has already been generated [16]. The exhaustive algorithm is a graph search algorithm that generates all the states and chooses one that has minimal cost among those that satisfy the space constraint. An r-greedy algorithm. An exhaustive algorithm can be very expensive for a big number of complex queries. Even though the design of a DW is a procedure that is not meant to be done very frequently we present here a greedy algorithm and we suggest some heuristics. The r-greedy incremental algorithm proceeds in two phases. In the first phase it endeavors to find a state that satisfies the space constraint. Starting with the state s0 it iteratively expands the states in the search space to a depth r, and chooses one having minimal size. If no state satisfying the space constraint is found, the algorithm returns a fail. Otherwise, it proceeds to the second phase. In the second phase, the algorithm endeavors to find a state that has minimal cost. Starting with a state that satisfies the space constraint, it iteratively expands the states in the search space to a depth r, by considering only states satisfying the space constraint, and chooses one having minimal cost. The algorithm stops when the expansion of a state does not produce any state satisfying the space constraint. Heuristics. We explore various heuristics that can be used in order to improve the performance of the algorithms. A two phase application of the rules. The distinction of the views in simple and auxiliary, entails a respective distinction of the state transformations: simple view transformations are those that modify or eliminate a simple view. These transformations modify also the query rewritings in QV . Transformations that do not modify or eliminate a simple view are called auxiliary view transformations. These transformations modify or generate an auxiliary view and do not modify the query rewritings. This distinction suggests for the following two phase heuristic application of the rules: during the first phase use one of the algorithms to find a state s, starting from the state s0 , by performing only simple view transformations. During the second phase use one of the algorithms to

Data Warehouse Schema and Instance Design

375

compute the final state starting from state s, by performing only auxiliary view transformations. Guiding auxiliary view generation by frequent transaction types. Not all the auxiliary views based on a view V are useful in maintaining V after the propagation of the changes specified by a transaction type. Thus, we can generate only, for every view V , and for every frequent transaction type Ti that affects V , the maximal auxiliary views based on V that do not contain a relation changed by Ti . Maximality is meant with respect to set inclusion on the set of the relations appearing in the definition of the views. Frequent transaction types are considered because they are expected to contribute more than the others to the view maintenance cost. Prohibiting transformations of the auxiliary views. The auxiliary view join edge cut, selection edge cut and augmentation transformations can be deactivated. In fact an auxiliary view, in the form it has when it is generated, can be used more efficiently in maintaining the view on which it is based: the corresponding maintenance query can be rewritten by joining the auxiliary view with other views or base relations, without applying any selection condition on it. The usefulness of transforming an auxiliary view relies on the possibility of using the transformed auxiliary view in maintaining other materialized views besides the view on which it is based. Thus, this pruning of the search space is done at the expense of using an auxiliary view this way.

6

Conclusion and Possible Extensions

We have addressed the problem of selecting a set of views to materialize in a DW that fits in the space allocated for materialization, allows all the queries of interest to be answered using exclusively these views and minimizes the combined query evaluation and view maintenance cost (DW design problem). The problem is formalized as a state space search problem for a class of relational queries that takes into account both: multiquery optimization over the maintenance queries and the use of (auxiliary) views in the computation of the maintenance queries. A solution provides the optimal view set and a complete rewriting of the queries over it. We have designed incremental algorithms and have presented heuristics to reduce their execution time. The approach can be easily extended to deal with self-joins and projections. Self-joins can be handled by assigning different new names to the multiple occurrences of the same relation in a query definition. Projections can be treated by keeping with each node of the multiquery graph the attributes that are projected out in each view, labeled by the corresponding view name. The design problem of Data Marts which contain highly aggregated data also constitutes an important extension direction. In this paper we have studied the static case of the DW design problem. DWs though are entities that need to evolve in time. Dynamic interpretations of the DW design problem involve the incremental design of a DW when different

376

D. Theodoratos and T. Sellis

input parameters to the problem are modified; this issue is also an interesting extension to the presented work and is at the focus of our current research work. Finally, we are seeking experimental validation of our results by assessing the efficiency of the greedy algorithm and the effectiveness of the heuristics.

References 1. S. Ceri and J. Widom. Deriving production rules for incremental view maintenance. In Proc. of the 20th Intl. Conf. on Very Large Data Bases, pages 577–589, 1991. 2. S. Chaudhuri and U. Dayal. An Overview of Data Warehousing and OLAP Technology. SIGMOD Record, 26(1):65–74, 1997. 3. S. Finkelstein. Common Expression Analysis in Database Applications. In Proc. of the ACM SIGMOD Intl. Conf. on Management of Data, pages 235–245, 1982. 4. H. Gupta. Selection of Views to Materialize in a Data Warehouse. In Intl. Conf. on Database Theory, pages 98–112, 1997. 5. H. Gupta, V. Harinarayan, A. Rajaraman, and J. D. Ullman. Index Selection for OLAP. In Proc. of the 13th Intl. Conf. on Data Engineering, pages 208–219, 1997. 6. V. Harinarayan, A. Rajaraman, and J. D. Ullman. Implementing Data Cubes Efficiently. In Proc. of the ACM SIGMOD Intl. Conf. on Management of Data, 1996. 7. W. Immon and C. Kelley. Rdb/VMS: Developing the Data warehouse. QED Publishing Group, Boston, Massachussets, 1993. 8. W. Labio, D. Quass, and B. Adelberg. Physical Database Design for Data Warehousing. In Proc. of the 13th Intl. Conf. on Data Engineering, 1997. 9. A. Levy, A. O. Mendelson, Y. Sagiv, and D. Srivastava. Answering Queries using Views. In Proc. of the ACM Symp. on Principles of Database Systems, pages 95–104, 1995. 10. D. Quass, A. Gupta, I. S. Mumick, and J. Widom. Making Views Self Maintainable for Data Warehousing. In PDIS, 1996. 11. K. A. Ross, D. Srivastava, and S. Sudarshan. Materialized View Maintenance and Integrity Constraint Checking: Trading Space for Time. In Proc. of the ACM SIGMOD Intl. Conf. on Management of Data, pages 447–458, 1996. 12. N. Roussopoulos. View Indexing in Relational Databases. ACM Transactions on Database Systems, 7(2):258–290, 1982. 13. T. K. Sellis. Multiple Query Optimization. ACM Transactions on Database Systems, 13(1):23–52, 1988. 14. K. Shim, T. K. Sellis, and D. Nau. Improvements on a heuristic algorithm for multiple-query optimization. Data & Knowledge Engineering, 12:197–222, 1994. 15. D. Theodoratos and T. Sellis. Data Warehouse Configuration. In Proc. of the 23nd Intl. Conf. on Very Large Data Bases, pages 126–135, 1997. 16. D. Theodoratos and T. Sellis. Designing Data Warehouses. Technical Report, Knowledge and data Base Systems Laboratory, Electrical and Computer Engineering Dept., National Technical University of Athens, pages 1–29, 1997. 17. J. Widom, editor. Data Engineering, Special Issue on Materialized Views and Data Warehousing, volume 18(2). IEEE, 1995. 18. J. Yang, K. Karlapalem, and Q. Li. Algorithms for Materilaized View Design in Data Warehousing Environment. In Proc. of the 23nd Intl. Conf. on Very Large Data Bases, pages 136–145, 1997.

Reducing Algorithms for Materialized View Updates Tetsuya Furukawa and Fei Sha Dept. Economic Engineering, Kyushu University, JAPAN Abstract. Materialized views, which are used for such as data warehousing, have to be consistent with their original data. They are updated according to updates of the original data. Materialized view updates, however, become complicated when the view definition includes projection because one view data corresponds to several original data. This paper shows incremental view update algorithms when a materialized view defined with projection is updated. The algorithms proposed in this paper reduce the intermediate results by checking as early as possible whether the view data corresponding the updated original data indeed have to be updated. The paper also gives algorithms for multiple updates, which can realize deferred updates.

1

Introduction

In database systems, query processing performance is one of the most important aspects of database efficiency. To increase query processing efficiency, query optimization methods reduce the amount of data to be processed by reducing intermediate results. It is particularly important to reduce intermediate results in warehousing systems [15] [1], where data warehouses store materialized views from remote data sources by networks, because communication costs depend on the size of the intermediate results. In this paper we discuss how to reduce intermediate results to update materialized views incrementally. Materialized views are copies of data derived from original data. When original data is updated, the materialized views may also be required to change to keep database consistency. Since it costs a great deal to rebuild the materialized views, incremental re-computation methods are proposed for relational databases [13] [4] [12] [8] [7]. One of the performance problems occurs when views are updated. Deferred updates of materialized views, such as periodically or on-demand updates, are also proposed to increase the efficiency, which refresh materialized views for several updates of original data [9] [14] [5] [2]. On the other hand, there are cases that immediate updates are required [6]. These methods are based on relational expressions showing which tuples are inserted to or deleted from the materialized views. Materialized view updates become complicated when the view expressions contain projection [4] [3] [8]. Suppose that a tuple t is inserted to or deleted from a base relation, original data in a relational database. In general, the corresponding tuples of view relations are inserted or deleted according to the update of the T.W. Ling, S. Ram, and M.L. Lee (Eds.): ER’98, LNCS 1507, pp. 377–392, 1998. c Springer-Verlag Berlin Heidelberg 1998

378

T. Furukawa and F. Sha

base relation. There can be, however, view relations which do not change because the view tuples may be generated by other tuple connections rather than the connections including t. If all the view tuples are derived from other tuples of the base relation than t, the materialized view relation does not change when t is updated. Methods to maintain materialized views with projection in their definitions are proposed as counting algorithms [4] [8] and super-key approach [10] [11]. Let base relations be R1 , R2 , and R3 as shown in Fig. 1 (a). If view relation V is defined as projection on ACE of these relations’ join R1 ∗ R2 ∗ R3 , V is the relation shown in Fig. 1 (b). Some tuples of V are produced by several tuples of R1 ∗ R2 ∗ R3 , for example (a2 , c2 , e2 ) by (a2 , b2 , c2 , d2 , e2 ), (a2 , b3 , c2 , d2 , e2 ), and (a2 , b4 , c2 , d2 , e2 ). This fact causes a problem when a base relation is updated. When tuple (a2 , b2 ) of R1 is deleted, only (a2 , b2 , c2 , d2 , e2 ) disappears in these tuples. Since the rest two tuples still exist, (a2 , c2 , e2 ) is not deleted from V . R1

A a1 a2 a2 a2

B b1 b2 b3 b4

R2

B b1 b2 b2 b3 b4 b4

C c1 c2 c2 c2 c2 c3

D d1 d1 d2 d2 d2 d3

R3

D d1 d1 d2 d2 d3

E e1 e4 e2 e4 e3

(a)

V

A a1 a1 a2 a2 a2 a2

C c1 c1 c2 c2 c2 c3

E e1 e4 e1 e2 e4 e3

(b)

Fig. 1. Base relations and a view

We need the following three processes to update materialized views incrementally when a base relation is updated. 1. Get new generated or disappeared tuple connections in base relations. 2. Check whether there are other tuple connections producing the same view tuples as the view tuples of the tuple connections in 1. 3. Update the view if the check in 2 is false. This paper gives algorithms to get the tuples actually inserted into or deleted from the materialized views whose view expressions include projection. We discuss such problem with relational databases, where the original data is base relations and views are defined by relational expressions on the base relations. The algorithms are based on combining Processes 1 and 2. For example, when the deleted tuple (a2 , b2 ) of R1 is joined with tuple (b2 , c2 , d2 ) of R2 in Process 1, we can find that the tuples in V derived from (a2 , b2 , c2 , d2 ) are also derived from other tuples such as (a2 , b3 , c2 , d2 ), which still exists after the deletion. There are cases in which Process 2 finishes in an early stage of Process 1, that is, the problem caused by projection disappears when the algorithms are used.

Reducing Algorithms for Materialized View Updates

379

The counting algorithms store the multiplicity of tuple duplicates [4] [8]. In the algorithms, Process 1 counts how many times view tuples derived from the deleted tuple, and subtracts the numbers from the corresponding multiplicity. In Process 2, it is checked whether the stored numbers are zero or not. If the multiplicity of a view tuple is zero, the tuple is deleted. While the counting algorithms always count the multiplicity of the view tuples derived from a modified tuple, the reducing algorithms proposed in this paper can finish the view update at the time when it is found that the multiplicity of view tuples does not become zero. The Delete and Rederive algorithm has also been proposed [8], which uses the similar notation of our algorithms. The algorithm deletes all the view tuples of Process 1 and recovers the view tuples of Process 2. That is, the algorithm deletes view tuples derived from a deleted tuple of a base relation, finds out such deleted view tuples that have alternative derivations, and inserts those tuples to the view relation. The reducing algorithms have the following advantages. – Each algorithm is independent of the others. We can adopt an arbitrary combination of the algorithms at each join independently. Also the algorithms are independent of classical query optimization methods such as selection and projection as early as possible, and we can hence combine both of the reducing algorithms and the query optimization methods together. – Each base relation is accessed only once independently. The algorithms consequently do not require any extra costs even if they are used in autonomous distributed databases such as multidatabases. This paper is outlined as follows. Section 2 gives basic concepts for this paper which is the base of the reducing algorithms proposed in this paper. In Sect. 3, basic algorithms to reduce intermediate results are shown for deletion of a tuple, which are put together into one algorithm REDUCE. In Sect. 4, classical query optimization methods are applied to the algorithms. Insertion of a tuple and multiple updates are discussed in Sect. 5. It is shown that every algorithm for deletion of a tuple given in Sect. 3 also works for insertion of a tuple. The algorithms for multiple updates reflect several insertion and deletion of tuples to a materialized view in a time. Section 6 is the conclusions.

2

The Idea Behind the Algorithms

A relation R(X) is a set of tuples on attributes X. σC R, πY R, and R1 ∗R2 denotes selections of R by condition C, projection of R on attributes Y (Y ⊆ X), and join of R1 and R2 , respectively. If C is a set of conditions, the selection condition is conjunction of the elements of C, c1 ∧ c2 ∧ · · · ∧ cm for C = {c1 , c2 , · · · , cm }. Although join is assumed to be natural join for simplicity in this paper, it can be easily extended to θ-join with a slight modification. t[Y ] is the projection of tuple t on attributes Y .

380

T. Furukawa and F. Sha

A materialized view relation V (XV ) is defined as f (R), where f is a relational expression on a set of relations R={R1 , R2 , · · · , Rn }. View V is consistent if V = f (R). V is defined as selection by condition C and projection on XV of R1 ∗ R2 ∗ · · · ∗ Rn , that is f (R) = πXV σC R1 ∗ R2 ∗ · · · ∗ Rn . There are three types of updates, deletion of a tuple t from a relation Ri , insertion of a tuple t to a relation Ri , and modification of values of a tuple t in a relation Ri . First, we discuss deletion of a tuple. Insertion of a tuple is handled as the same way as shown in Sect. 5, where modification of values of a tuple from t1 to t2 is also treated as deletion of t1 and insertion of t2 . If a relation Ri is updated, the update have to be reflected to V to keep the database consistency. Since join operations are commutative, we assume R1 is the relation to be updated without a loss of generality. For a tuple t (or a set of tuples S) of R1 , we say a tuple tV of V is derived from t (S) if tV is in f ({{t}, R2 , · · · , Rn }) (f ({S, R2 , · · · , Rn }), respectively). Suppose a tuple t is deleted from R1 . Some of the tuples derived from t may not be deleted from V because they may be also derived from R1 − {t}. The tuples derived from t are the candidates to be deleted from the view relation and the tuples derived from R1 − {t} still exist after the deletion of t. Definition 1. Let a tuple t be deleted from R1 . The candidate relation and the existing relation of the update for view V = f (R) is VC = f ({{t}, R2 , · · · , Rn }) and VE = f ({R1 − {t}, R2 , · · · , Rn }), respectively. VC is the set of tuples of V which are derived from t and VE is the set of tuples of V which are derived from R1 − {t}. VE is the same view relation as the resulting view relation VR after the deletion of t from R1 because R1 becomes R1 − {t}. Obviously, the resulting view relation VR after the deletion is V − (VC − VE ). While VC is the relation of candidate tuples to be deleted from V , VE may include some tuples of VC . Only the tuples in VC − VE are deleted from V . If VC ∩ VE 6=∅, there is at least one tuple of R1 other than t which derives tuples in VC . If a view expression does not include projection, VC ∩ VE is the empty set. We can get VR by deleting VC from V in this case. Definition 2. Let a tuple t be deleted from R1 . The target relation of the update is VT = VC − VE and the failure relation of the update is VF = VC ∩ VE . The target relation is the difference between V and VR , which is the relation of the tuples deleted from the view V . The failure relation is the relation of the tuples which are derived from t but not deleted from the view V because they are also derived from other tuples of R1 than t. Figure 2 shows the relationship among these relations. Example 1. Suppose tuple (a2 , b2 ) is deleted from R1 in the database shown in Fig. 1. The tuples of V derived from (a2 , b2 ) may have to be deleted. VC in Fig. 3 is the relation of such candidate tuples, the candidate relation. There are, however, tuples of R1 other than (a2 , b2 ) which derive some of the tuples of VC .

Reducing Algorithms for Materialized View Updates

R1

381

V

R1 - { t }

derive VE = VR

t

VF VT

VC

Fig. 2. Candidate, existing, target, and failure relations

VE in Fig. 3 is the existing relation whose tuples are derived from R1 −{(a2 , b2 )}, the resulting relation VR of the deletion in this example. Since (a2 , c2 , e2 ) and (a2 , c2 , e4 ) of VC are also tuples of VE , only (a2 , c2 , e1 ), the tuple of the target relation VT , is deleted from V .

Fig. 3. Delete tuple (a2 , b2 ) from V

We could get VT by difference between VC and VE according to the definition of VT . It is not, however, a good method because we have to rebuild the resulting view relation VR as VE . We do not have to get the exact VC and VE to get VT . Proposition 3. Let VC0 and VE0 be such relations that VT ⊆ VC0 ⊆ VC and VF ⊆ VE0 ⊆ VE , respectively. Then VC0 − VE0 is the target relation VT . Proof. VC0 is a superset of VT and VC0 − VT is a subset of VE0 . Every tuple in VT but not in VE0 is in VC0 . Thus VC0 − VE0 is the target relation VT . The reducing algorithms given in this paper are based on Proposition 3. The algorithms show how to reduce VC0 and VE0 .

382

3

T. Furukawa and F. Sha

Reducing the Intermediate Relations

Intermediate results of creating VE0 and VC0 can be reduced. In this section, basic ideas of the reduction are shown as algorithms RIE and RIC, and they are put together to algorithm REDUCE. Although we can apply some selection and projection of the view definition during an early stage of the joins, the selection and the projection are applied after the joins in this section for the simplicity. Such optimization is discussed in Sect. 4. Suppose VCi and VEi are {t} ∗ R2 ∗ · · · ∗ Ri and (R1 − {t}) ∗ R2 ∗ · · · ∗ Ri , respectively. We can reduce VEi because there are cases in which some tuples in VEi that do not agree with tuples in VCi on the attributes of XV do not derive any tuples in VF . For example (a1 , b1 ) of VE1 and (a2 , b4 , c3 , d3 ) of VE2 do not derive the tuples in VE = πXV σC VE3 that intersect with tuples in VC = πXV σC VC3 . RIE is the algorithm to reduce such tuples. Algorithm 1. RIE (Reducing Intermediate results of the Existing relation) Si Let a view V (XV ) be defined by f (R), t be a tuple of R1 , and Yi be j=1 Xj ∩ XV . 1. VC1 = {t}, VE1 = σY1 =t[Y1 ] (R1 − {t}) 2. For i = 2 to n, VCi = VCi−1 ∗ Ri , VEi = σYi ∈πYi VCi (VEi−1 ∗ Ri ) 3. VT = πXV σC VCn − πXV σC VEn Lemma 4. The resulting relation VT of algorithm RIE is the target relation of the update which deletes t from R1 . Proof. πXV σC VCn is the candidate relation VC because VCn is {t} ∗ R2 ∗ · · · ∗ Rn . i−1 Let ti be a tuple in σYi ∈π ∗ Ri ). πXV σC ({ti } ∗ Ri+1 ∗ · · · ∗ Rn ) has no / Yi VCi (VE / VF [Yi ]. Thus VF ⊆ πXV σC VEn ⊆ VE . Proposition 3 tuple in VF because ti [Yi ] ∈ shows such relation VT = πXV σC VCn − πXV σC VEn is the target relation. Example 2. Figure 4 shows the process to generate VC0 = πACE VC3 and VE0 = πACE VE3 by algorithm RIE. (a1 , b1 ) is not a tuple of VE1 because the tuple of VC1 does not have the same value of Y1 . (a2 , b4 , c3 , d3 ) is not a tuple of VE2 either because there is no tuple in VC2 which has the same value of Y2 , (a2 , c3 ), and so on. Let N1 be the number of tuples of R1 and n1 be the number of such tuples of R1 that agree with t on Y1 . Intermediate results of RIE are about n1 /N1 as comparing with rebuilding the view even if VEi (2 ≤ i ≤ n) is not reduced, because the size of VC1 ∪ VE1 is n1 /N1 of R1 − {t}. Incrementally updating a materialized view is much more efficient than rebuilding the view even if the view definition includes projection. There exist tuples with the same Y2 value (a2 , c2 ) in VC2 and VE2 in Fig. 4. Since the tuples derived from (a2 , b2 , c2 , d2 ) of VC2 and (a2 , b3 , c2 , d2 ) of VE2 are the same tuples (a2 , c2 , e2 ) and (a2 , c2 , e4 ) of VC and VE , we can get VT = πXV σC VC3 − πXV σC VE3 even if (a2 , b2 , c2 , d2 ) is deleted from VC2 .

Reducing Algorithms for Materialized View Updates VC1

VE1

A B a2 b 2

A B a2 b 3 a2 b 4

VC2

VE2

A B C D a 2 b 2 c2 d 1 a 2 b 2 c2 d 2

A B C D a 2 b 3 c2 d 2 a 2 b 4 c2 d 2

VC3

VE3

A a2 a2 a2 a2

B b2 b2 b2 b2

C c2 c2 c2 c2

D d1 d1 d2 d2

E e1 e4 e2 e4

A a2 a2 a2 a2

B b3 b3 b4 b4

C c2 c2 c2 c2

D d2 d2 d2 d2

E e2 e4 e2 e4

πACE VC3

πACE VE3

A a2 a2 a2

C c2 c2 c2

383

E e1 e2 e4

A C E a2 c2 e2 a2 c2 e4

Fig. 4. Delete tuple (a2 , b2 ) from R1 with algorithm RIE

Although the AC value of (a2 , b2 , c2 , d1 ) of VC2 is also (a2 , c2 ), it cannot be deleted. It has the different value d1 from the D value of the tuples of VE2 which have (a2 , c2 ) on AC. Since (a2 , b2 , c2 , d1 ) is joined with different tuples of R3 , the tuples derived from (a2 , b2 , c2 , d1 ) may not derived from (a2 , b3 , c2 , d2 ) or (a2 , b4 , c2 , d2 ). Note that this reduction cannot be applied to VEi ; we cannot delete (a2 , b3 , c2 , d2 ) from VE2 because it may produce the same tuples as tuples produced by other tuples of VC2 than (a2 , b2 , c2 , d2 ), (a2 , c2 , e4 ) in this example. When the view definition has selection, we have to consider another problem. Suppose a tuple tiC of VCi agrees with a tuple tiE of VEi on the attributes in XV and on the join attributes of the later joins. tiC is still a candidate if tiE does not derive any tuple of VEn satisfying the selection. For example, if the view definition of the running example had selection B = b2 , (a2 , b2 , c2 , d2 ) of VC2 would be still a candidate because (a2 , b3 , c2 , d2 ) of VE2 would not derive any tuple of the view relation. Thus tiC must agree with tiE on the attributes which appear in selection condition, too, if we delete tiC from VCi . Algorithm RIC shows this reduction of the intermediate relations. In the ith step, we must keep join attributes which will be used in the kth step (k > i). Si Let Ji be the join attributes used in the (i+1)st step, j=1 Xj ∩ Xi+1 . Let Ji+ be the attributes Snin the all join attributes used in the rest steps after Si included the ith step, j=1 Xj ∩ k=i+1 Xk . Let attr(c) S be the set of attributes which appear in selection condition c, and attr(C) be c∈C attr(c) for a set of selection conditions C. Algorithm 2. RIC (Reducing Intermediate results of the Candidate relation) Si Let a view V (XV ) be defined by f (R), t be a tuple of R1 , Yi be j=1 Xj ∩XV , Si Sn Si Ji+ be j=1 Xj ∩ k=i+1 Xk , and Zi be Yi ∪ Ji+ ∪ (attr(C) ∩ j=1 Xj ). 1. VE1 = σY1 =t[Y1 ] (R1 − {t}), VC1 = σZ1 6∈πZ1 VE1 {t}

384

T. Furukawa and F. Sha

2. For i = 2 to n, VEi = VEi−1 ∗ Ri , VCi = σZi 6∈πZi VEi (VCi−1 ∗ Ri ) 3. VT = πXV σC VCn Lemma 5. The resulting relation VT of algorithm RIC is the target relation of the update which delete t from R1 . Proof. Let tiC ∈ VCi , tiE ∈ VEi such that tiC [Zi ] = tiE [Zi ]. If tnC = πXV (tiC ∗ ti+1 ∗ · · · ∗ tn ) (tj ∈ Rj , i < j ≤ n), tiE is joinable with ti+1 ∗ · · · ∗ tn because tiC [Zi ] = tiE [Zi ] leads tiC [Ji ] = tiE [Ji ]. Thus for every tnC ∈ VCn (tnC [Yi ] = tiC [Yi ]), there exists a tuple tnE ∈ VEn such that tnE [XV ] = tnC [XV ]. If tnC satisfies C, tnE also satisfies C because of tnC [attr(C)] = tnE [attr(C)]. tnC [XV ] is a tuple of VF because tnE [XV ] is a tuple of VE . If tnC does not satisfy C, tnC [XV ] is not a tuple of the view relation. In both of the cases, tiC does not derive any tuple of the target relation, that is, VT of RIC is a superset of the target relation. On the other hand, πXV σC VCn is πXV σC σXV 6∈πZn VEn (VCn−1 ∗ Rn ) = πXV σC (VCn−1 ∗ Rn ) − πXV σC VEn , which is a set of tuples in a subset of VC but not in πXV σC VEn = VE . Therefore πXV σC VCn is a superset of the target relation and does not include any tuple of VE , that is, the target relation. Example 3. If we use algorithm RIC to get the target relation for the running example, the intermediate results of the algorithm are as shown in Fig. 5. (a2 , b2 , c2 , d2 ) is deleted from VC2 because it agrees with (a2 , b3 , c2 , d2 ) and (a2 , b4 , c2 , d2 ) of VE2 on Z2 = ACD.

VC1

VE1

A B a2 b 2 A B a2 b 3 a2 b 4

VC2

VE2

A B C D a 2 b 2 c2 d 1 A a2 a2 a2

B b3 b4 b4

C c2 c2 c3

D d2 d2 d3

VC3

VE3

A B C D E a2 b2 c2 d1 e1 A a2 a2 a2 a2 a2

B b3 b3 b4 b4 b4

C c2 c2 c2 c2 c3

D d2 d2 d2 d2 d3

E e2 e4 e2 e4 e3

πACE VC3

πACE VE3

A C E a2 c2 e1 A a2 a2 a2

C c2 c2 c3

E e2 e4 e3

Fig. 5. Delete tuple (a2 , b2 ) from R1 with algorithm RIC

The reduction of RIC is one of the advantages of the reducing algorithms. The counting algorithms [4] [8] have to produce VC in order to get the multiplicity of the tuples. RIC finds some of the tuples of VCi which will derive only tuples in VF but not tuples in VT , and stops the derivation from such tuples. It often occurs that VCi becomes the empty set, that is VT is empty, which allows to quit the materialized view update without accessing Rk (i < k ≤ n).

Reducing Algorithms for Materialized View Updates

385

The algorithms RIE and RIC shows the basic ideas of how to reduce the intermediate results. These ideas can be used for each ith join in Step 2 individually, and they can be also combined in the ith join. Algorithm REDUCE is such algorithm that adopts both of the reducing algorithms. VEi can be further reduced in REDUCE because tuples of VEi for reduced tuples of VCi need not be kept. Algorithm 3. REDUCE Si Let a view V (XV ) be defined by f (R), t be a tuple of R1 , Yi be j=1 Xj ∩XV , Si Sn Si Ji+ be j=1 Xj ∩ k=i+1 Xk , and Zi be Yi ∪ Ji+ ∪ (attr(C) ∩ j=1 Xj ). 1. VE1 = σY1 =t[Y1 ] (R1 − {t}), VC1 = σZ1 6∈πZ1 VE1 {t} 2. For i = 2 to n, 0 0 VEi = VEi−1 ∗ Ri , VCi = σZi 6∈πZ V i 0 (VCi−1 ∗ Ri ), VEi = σYi ∈πYi VCi VEi i E 3. VT = πXV σC VCn Theorem 6. The resulting relation VT of algorithm REDUCE is the target relation of the update which deletes t from R1 . Proof. The reduced tuples of VEi by Yi ∈ πYi VCi in the ith step will not effect the reduction of VCk (i < k ≤ n) because no tuple derived by the reduced tuples agree with the tuples of VCk on Yi which is a subset of Zk . Therefore VC in REDUCE is the same relation as VC in RIC. Since Lemma 5 shows that πXV σC VCn in RIC is the target relation, πXV σC VCn in REDUCE is also the target relation. Example 4. Figure 6 is the intermediate results to get the target relation by algorithm REDUCE.

Fig. 6. Delete tuple (a2 , b2 ) from R1 with algorithm REDUCE

4

Optimization of the Reducing Algorithms

The reducing algorithms allow to adopt classical query optimization methods, selection and projection as early as possible. We discuss how the query optimization methods are applied to the reducing algorithms.

386

T. Furukawa and F. Sha

There are several query optimization methods to increase performance by applying selection as early as possible. There is also a materialized view update algorithm based on this method [12], which does not treat the projection problem. The reducing algorithms can adopt this optimization with slight modification. Si−1 Let Ci be the set of the selection conditions {c | c ∈ C − j=1 Cj , attr(c) ⊆ Si j=1 Xj }, which become newly applicable in the ith step. The reducing algorithms are modified as follows. Modifying the reducing algorithms for selection 1. the first step: VC1 = σC1 {t}, VE1 = σC1 σY =t[Y1 ] (R1 − {t}) 2. the ith step (2 ≤ i ≤ n): VEi−1 ∗Ri and VCi−1 ∗Ri are replaced with σCi (VEi−1 ∗ R1 ) and σCi (VCi−1 ∗ Ri ), respectively. 3. the last step: VT = πXV VCn − πXV VEn in RIE and VT = πXV VCn in RIC and REDUCE. Tuples (a2 , b3 , c2 , d2 ) and (a2 , b4 , c2 , d2 ) of VE2 in Fig. 6 derive the same tuples of πACE VE3 . Also there are cases that some tuples of VCi derive the same tuple of VC . It is enough to keep only one tuple of the tuples of VCi and VEi deriving the same set of tuples of πXV σC VCn and πXV σC VEn , respectively. The optimization by projection as early as possible can reduce such redundancy as well as reduces the number of attributes. Si Note that even if two tuples have the same values of attributes j=1 Xj ∩ XV , they can have different values of other than the attributes. If the values of the join attributes which are used for the joins in the rest steps after the ith step are different from each other, we have to keep them. For example, the two tuples of VC2 in Fig. 4 derive the different sets of tuples of πACE VC3 , which have S2 the same value (a2 , c2 ) on AC = j=1 Xi ∩ ACE, {(a2 , c2 , e1 ), (a2 , c2 , e4 )} and {(a2 , c2 , e2 ), (a2 , c2 , e4 )}, respectively. Furthermore attributes used by selection conditions must be kept, too. The attributes which have to be kept in the ith step are Si – Yi = j=1 Xj ∩ XV : attributes in XV , Si Sn – Ji+ = j=i Xj ∩ k=i+1 Xk : join attributes used in the rest steps, and Si – attr(C) ∩ j=1 Xj : attributes which appear in selection conditions. Si Let Zi be Yi ∪ Ji+ ∪ (attr(C) ∩ j=1 Xj ). The reducing algorithms are modified as follows. Modifying the reducing algorithms for projection 1. the first step: VC1 = πZ1 {t}, VE1 = πZ1 σY =t[Y1 ] (R1 − {t}) 2. the ith step (2 ≤ i ≤ n): VEi−1 ∗Ri and VCi−1 ∗Ri are replaced with πZi (VEi−1 ∗ R1 ) and πZi (VCi−1 ∗ Ri ), respectively. 3. the last step: VT = σC VCn − σC VEn in RIE and VT = σC VCn in RIC and REDUCE.

Reducing Algorithms for Materialized View Updates

387

When we use both of the optimization methods, selection and projection as early as possible, Zi should be changed to less attributes because attributes which appear in attr(Cj ) (1 ≤ j ≤ i) but not in attr(Ck ) (i < k ≤ n) are no Sn Si longer needed. Zi is Yi ∪ Ji+ ∪ ( k=i+1 attr(Ck ) ∩ j=1 Xj ) in this case, and the algorithms are modified as follows. Modifying the reducing algorithms for selection and projection 1. the first step: VC1 = πZ1 σC1 {t}, VE1 = πZ1 σC1 σY =t[Y1 ] (R1 − {t}) 2. the ith step (2 ≤ i ≤ n): VEi−1 ∗ Ri and VCi−1 ∗ Ri are replaced with πZi σCi (VEi−1 ∗ R1 ) and πZi σCi (VCi−1 ∗ Ri ), respectively. 3. the last step: VT = VCn − VEn in RIE and VT = VCn in RIC and REDUCE. Algorithm REDUCE+ is such reducing algorithm that is the result of applying both of the optimization methods to REDUCE. Algorithm 4. REDUCE+ (REDUCE with query optimization methods) Si Let a view V (XV ) be defined by f (R), t be a tuple of R1 , Yi be j=1 Xj ∩XV , Si Sn Sn Si Ji+ be j=1 Xj ∩ k=i+1 Xk , and Zi be Yi ∪ Ji+ ∪ ( k=i+1 attr(Ck ) ∩ j=1 Xj ). 1. VE1 = πZ1 σC1 σY1 =t[Y1 ] (R1 − {t}), VC1 = πZ1 σC1 σZ1 ∈V / E1 {t} 0

i−1 0 2. For i = 2 to n, VEi = πZi σCi (VEi−1 ∗ Ri ), VCi = πZi σCi σZi ∈V ∗ Ri ), / i (VC 0

VEi = σYi ∈πYi VCi VEi 3. VT = VCn

E

Theorem 7. The resulting relations V T of the resulting algorithms of modifying RIE, RIC, and REDUCE for selection, projection, or both of them are the target relation of the update which deletes t from R1 . Proof. Even if the selection σCi is applied in the ith step, VE and VC are the same relation as Ci is applied in the last step, because the tuples of VEi−1 ∗ Ri and VCi−1 ∗ Ri which do not satisfy Ci do not derive any tuple of VE and VC , respectively. Projection πZi in the ith step does not effect the later selection because VEi and VCi keep the attributes which appear in Ck or Yk (i < k ≤ n). Example 5. Fig. 7 shows the process to get the target relation of the running example by algorithm REDUCE+ . Although the view expression of the example does not contain selection we can observe the effect of query optimization by projection.

5

Generalizing the Algorithms

In Sects. 3 and 4 the algorithms for deletion of a tuple from a base relation are shown. In this section, it is shown that the algorithms also work for insertion of

388

T. Furukawa and F. Sha

Fig. 7. Delete tuple (a2 , b2 ) from R1 with algorithm REDUCE+

a tuple to a base relation without any change at first. Then algorithms to handle multiple updates are also proposed. / R1 ). Although the tuples of relation Suppose a tuple t is inserted to R1 (t ∈ VC = πXV σC ({t} ∗ R2 ∗ · · · ∗ Rn ) are candidates to be inserted to V , the same tuples may already exist in V . Thus, we define the candidate relation and the existing relation of a case of tuple insertion. Definition 8. Let a tuple t be inserted to R 1 . The candidate relation and the existing relation of the update for a view V = f (R) are VC = f ({{t}, R2 , · · · , Rn }) and VE = f ({R1 , R2 , · · · , Rn }), respectively. Note that VE is the current view relation V . Since R1 − {t} = R1 , the definitions of VC and VE for tuple deletion in Definition 1 can be regarded as the definition for tuple insertion. As the same way as deletion of a tuple, we define the target relation and the failure relation for insertion of a tuple. Definition 9. Let a tuple t be inserted to R 1 . The target relation of the update is VT = VC − VE and the failure relation of the update is VF = VC ∩ VE . Figure 8 shows the relationship among these relations. The candidate, existing, target, and failure relations of tuple insertion have the same relationship as tuple deletion. Let VC0 and VE0 be such relations that VT ⊆ VC0 ⊆ VC and VF ⊆ VE0 ⊆ VE , respectively. Then the target relation of insertion is VC0 − VE0 . In the same way of tuple deletion, algorithms RIE, RIC, and REDUCE work to get the target relation in the case of tuple insertion. Theorem 10. Algorithms RIE, RIC, and REDUCE output the target relation of the update which insert a tuple t to R1 . Proof. The candidate, existing, target, and failure relations for tuple insertion have the same properties as those of tuple deletion. The proof is the same as the proofs of Lemmas 4 and 5 and Theorem 6. As shown in Theorem 10, the algorithms for tuple deletion can be used for tuple insertion. Since the query optimization methods shown in Sect. 4 are also

Reducing Algorithms for Materialized View Updates

R1 U { t }

389

VR

R1

derive VE = V

t

VF VT

VC

Fig. 8. Candidate, existing, target, and failure relations for insertion

valid for insertion of a tuple, REDUCE+ works for tuple insertion. The proofs of these are the same as the proofs for tuple deletion. Tuple insertion, however, has a different property from tuple deletion, which is formally described as Lemma 11. Lemma 11. Let VC0 be such relation that VT ⊆ VC0 ⊆ VC . V ∪VC0 is the resulting relation VR of the insertion of a tuple. Proof. The resulting relation VR is V ∪ VT . Since VT = VC − VE by Definition 9 and VE = V for tuple insertion, VR = V ∪ (VC − VE ) = V ∪ (VC − V ) = V ∪ VC . For VC0 such that VT ⊆ VC0 ⊆ VC , VR = V ∪ VC0 because VR = V ∪ VT = V ∪ VC . We can get VR without VE0 as V ∪ VC0 . This property is quite different from tuple deletion. Lemma 11 allows to cut the computation of VE0 at some step in each algorithm. That is, the algorithms can stop at any ith step and then VC0 = πXV σC VCi ∗ Ri+1 ∗ · · · ∗ Rn is added to V . Since algorithms RIE, RIC, and REDUCE contain operations to reduce the intermediate results, there are cases that it is more efficient to cut the reducing process after some ith step. It is sometimes more efficient to update the view for several updates of a base relation than to update the view for each update of a base relation. This strategy is proposed as deferred updates in [9] [14] [5] [2]. Since modification of values can be transformed to multiple updates, deletion of a tuple and insertion of a tuple, the algorithms work for tuple modification. If the all updates are either insertions or deletions, we can get the reducing algorithms by replacing inserted or deleted tuple t with a set of tuples S. The algorithms consequently work for insertion or deletion of multiple tuples with changes in the selection condition Y1 = t[Y1 ] in Step 1 to Y1 ∈ πY1 S in each algorithms. Then we treat insertion of tuples and deletion of tuples together. Let SI and SD (SI ∩ SD = ∅) be sets of tuples which are inserted to and deleted from R1 , respectively.

390

T. Furukawa and F. Sha

Definition 12. VCI = f ({SI , R2 , · · · , Rn }) and VCD = f ({SD , R2 , · · · , Rn }) are the candidate relations of the insertion and the deletion, respectively. The existing relations of the insertion and the deletion are VEI = f ({R1 , R2 , · · · , Rn }) and VED = f ({R1 − SD , R2 , · · · , Rn }), respectively. The target and the failure relations of the insertion are VTI = VCI − VEI and VFI = VCI ∩ VEI , and the target and the failure relations of the deletion are VTD = VCD − VED and VFD = VCD ∩ VED , respectively. (V − VTD ) ∪ VTI is the resulting view relation VR of the updates which insert SI to R1 and delete SD from R1 . That is (V − VTD ) ∪ VTI = πXV σC ((R1 − SD ) ∪ SI ) ∗ R2 ∗ · · · ∗ Rn . Thus we can update the view relation using VTI and VTD , which we can get by RIE, RIC, or REDUCE. The advantage of the multiple updates is not only to reduce the times of materialized view updates but also to reduce the intermediate results. While VR is (V − VTD ) ∪ VTI , (V ∪ VTI ) − VTD cannot be VR because there can be a tuple t in VR such that t ∈ VTI and t ∈ VTD . Such t is derived from both SI and SD and is not in (V ∪ VTI ) − VTD . This fact indicates that we can consider the newly generated tuples by insertion of SI as tuples of the existing relation after the deletion. The intermediate results are reduced further by treating VTI as a part of the existing relation of the deletion. In the selection condition to reduce VCi i0 i0 i 0 in the ith step σZi ∈π / Z V i of RIC and REDUCE, VE becomes VE ∪ VC where i

0

E 0

VEi and VCi should be VEi D and VCi I , respectively, by the definition for multiple updates. MULTI is such the algorithm that applied this reduction to the multiple update version of REDUCE. Algorithm 5. MULTI Let a view V (XV ) be defined by f (R), SI and SD be sets of tuples inserted Si to and deleted from R1 (SI ∩ SD = ∅), respectively, Yi be j=1 Xj ∩ XV , Ji+ be Sn Si Si + j=1 Xj ∩ k=i+1 Xk , and Zi be Yi ∪ Ji ∪ (attr(C) ∩ j=1 Xj ). 1. VC1I = SI , VE1I = σY1 ∈πY1 SI R1 , VC1D = SD , VE1D = σY1 ∈πY1 SD (R1 − SD ) 2. For i=2 to n, 0 i−1 i i0 0 VEi I = VEi−1 ∗ Ri , VCi I = σZi ∈π / Z V i (VCI ∗ Ri ), VEI = σYi ∈πYi VCi VEI , I 0

i

∗Ri , VCi D = σZi ∈π VEi D = VEi−1 / Z D

3. VTI = πXV σC VCnI , VT0D =

EI

i−1 0 V i ∪VCi (VCD ∗Ri ), i ED I πXV σC VCnD

I

0

VEi D = σYi ∈πYi VCi VEi D D

Theorem 13. For VTI and VT0D of algorithm MULTI, (V − VT0D ) ∪ VTI is the resulting relation of the update, insertion of SI to R1 and deletion of SD from R1 . Proof. VTI is the target relation of the insertion because the process to produce VTI is the same as REDUCE. Obviously VT0D ⊆ VTD . Let TD be VTD − VT0D . The i tuples of TD are caused by the selection σZi ∈π / Zi VCi to reduce VCD in Step 2. I In the same way as the proof of Lemma 5, it can be shown that the tuples of

Reducing Algorithms for Materialized View Updates

391

TD are in VCI . Thus (V − VT0D ) ∪ VTI is equal to (V − VTD ) ∪ VTI , which is the resulting relation VR . The reducing algorithm for multiple updates MULTI can be also combined with the query optimization methods by the same way as the algorithms for single update shown in Sect. 4.

6

Conclusions

We have shown the incremental re-computation algorithms to update materialized views. The algorithms that handle the cases that view expressions include projection, where it is required to check if the candidate tuples are actually inserted to or deleted from the view relations, are there. One of the advantages of the reducing algorithms is to quit producing the tuples other than the tuples in the target relation. It is often found in an early stage of the update process that candidate tuples at the ith step do not derive any tuples of the target relation which allows to quit producing view tuples from the candidate tuples, while the counting algorithms always produce all view tuples derived from the candidate tuples. The algorithms can be combined with query optimization methods such as selection and projection as early as possible, and work for multiple updates of deferred updates as well. The algorithms are effective for views in warehousing systems as well because the communication costs depend on the size of the intermediate relations. They are also applicable to multidatabases because each step of them can be closed in an autonomous component. The ideas to reduce intermediate results were given as RIE and RIC. We need not apply these reducing methods in every step like REDUCE. If the efficiency of the reduction in the ith step is not expected to be well, it may be better to skip the reduction by RIE or RIC, because there is trade off between the size of data to join and the cost of selection operations for the reduction.

References 1. Agrawal, D., Abbadi, A.E., Singh, A., Yurek, T.: Efficient View Maintenance at Data Warehouses. Proc. ACM SIGMOD Int’l Conf. on Management of Data. (1997) 417–427 2. Adelberg, B., Garcia-Molina, H., Widom, J.: The STRIP Rule System for Efficiently Maintaining Derived Data. Proc. ACM SIGMOD Int’l Conf. on Management of Data. (1997) 147–158 3. Blakeley, J.A., Coburn, N., Larson, P.: Updating Derived Relations: Detecting Irrelevant and Autonomously Computable Updates. ACM Trans. on Database Syst., Vol. 14, No. 3. (1989) 369–400 4. Blakeley, J.A., Larson, P., Tompa, F.W.: Efficiently Updating Materialized Views. Proc. ACM SIGMOD Int’l Conf. on Management of Data. (1986) 61–71 5. Colby, L.S., Griffin, T., Libkin, L., Mumick, I.S., Trickey, H.: Algorithms for Deferred View Maintenance. Proc. ACM SIGMOD Int’l Conf. on Management of Data. (1996) 469–480

392

T. Furukawa and F. Sha

6. Colby, L.S., Kawaguchi, A., Lieuwen, D.F., Mumick, I.S., Ross, K.A.: Supporting Multiple View Maintenance Policies. Proc. ACM SIGMOD Int’l Conf. on Management of Data. (1997) 405–416 7. Gupta, A., Mumick, I.S.: Maintenance of Materialized Views: Problems, Techniques and Applications. IEEE Data Engineering Bulletin, Vol. 18, No. 2. (1995) 3–19. 8. Gupta, A., Mumick, I.S., Subrahmanian, V.S.: Maintaining Views Incrementally. Proc. ACM SIGMOD Int’l Contf. on Management of Data. (1993) 157–166 9. Hanson, E.N.: A Performance Analysis of View Materialization Strategies. Proc. ACM SIGMOD Int’l Conf. on Management of Data. (1987) 440–453 10. Konomi, S., Furukawa, T.: Updating Duplicate Values in Distributed Multidatabase Systems. Proc. IEEE Int’l Workshop on Interoperability in Multidatabase Syst.. (1991) 243–246 11. Konomi, S., Furukawa, T., Kambayashi, Y.: Super-key Classes for Updating Materialized Derived Classes in Object Bases. Proc. Int’l Conf. on Deductive and Object-Oriented Databases, Lecture Notes in Computer Sciences, Vol. 760. Springer-Verlag. (1993) 310–326 12. Qian, X., Wiederhold, G.: Incremental Recomputation of Active Relational Expressions. IEEE Trans. on Knowledge and Data Eng., Vol. 3, No. 3. (1991) 337–341, 13. Stonebreaker, M.: Implementation of Integrity Constraints and Views by Query Modification. Proc. ACM SIGMOD Int’l Conf. on Management of Data. (1975) 65–78 14. Segev, A., Park, J.: Updating Distributed Materialized Views. IEEE Trans. Knowledge and Data Eng., Vol. 1, No. 2. (1989) 173–184 15. Zhuge, Y., Garcia-Morina, H., Hammer, J., Widom, J.: View Maintenance in a Warehousing Environment. Proc. ACM SIGMOD Int’l Conf. on Management of Data. (1995) 316-327

Reengineering Conventional Data and Process Models with Business Object Models: A Case Study Based on SAP R/3 and UML Eckhart v. Hahn1, Barbara Paech1, and Conrad Bock2 1

Institut für Informatik, Technische Universität,München, D-80333 München, [email protected], [email protected] 2 Intellicorp Inc. 1975 W. El Camino Real, Suite 201, Mountain View, CA 94040-2216, USA [email protected]

Abstract.Today, the business logic of complex business applications is typically documented by separate data and process models. This documentation is adequate for users to understand the functionality of the system. However, these models do not reflect the business objects which constitute a flexible architecture for continuous adaption to changes in the business processes. Business objects structure the business domain model into independent units encapsulating business data and behaviour. In this paper we describe a modeling technique for business objects and a procedure to derive the business object model from conventional data and process models. In particular, we show how to translate structured entity relationship models and eventdriven process chains into UML class and activity diagrams. We then show how to enrich these diagrams with business object information.

1 Introduction Complex business applications today typically consist of a database and a separate layer to implement the functionality of the system. This layer may or may not be object-oriented. In both cases the business processes are built into the layer such that it is difficult to adapt this layer to changes. Therefore, business objects have been suggested as a separate layer mediating between the database on the one hand and the business logic on the other hand [Ren96]. Business objects structure the business domain model into independent units encapsulating business data and behaviour [BOM97]. However, there is no agreement yet on the contents and properties of business objects. In this paper, a business object contains a set of logically related classes (e.g. for order management) and all their instances. The documentation (and implementation, of course) of existing systems does not reflect this notion of business objects. First, the business logic is most often documented from the viewpoint of a users. Therefore these documents do not show in detail which data is affected by which operations. Second, data and behaviour are T.W. Ling, S. Ram, and M.L. Lee (Eds.): ER’98, LNCS 1507, pp. 393−406, 1998.  Springer-Verlag Berlin Heidelberg 1998

394

E.v. Hahn, B. Paech, and C. Bock

typically separated into different models. Thus, in order to derive a business object model, behaviour documented in the process models has to be integrated in the data model. Third, business processes typically affect several business objects. Thus, the behaviour described in the process models has to be divided between the business objects. In this paper we describe a modeling technique for business objects and a procedure to derive the business object model from conventional data and process models. The results have been gained within a case study based on SAP R/3 and UML [Hah98]. Thus, we started with structured entity relationship (SER) models and event-driven proces chains (EPC). Since we aimed at an object-oriented model, we first translated the SER models and the EPC into UML notation. The resulting diagrams are enriched to reflect the distribution of the business logic over different business objects. Finally, we integrated data and behaviour into business object models which are extended UML class diagrams. While the translation of SER models and EPC into UML is specific for SAP R/3 (or at least, these modeling techniques), the business object model and its derivation from data and process models is applicable to all kinds of business applications. The paper is structured as follows: first we introduce the SER and EPC notation with short examples. In section 3 we sketch the translation from SER models to UML class diagrams. The translation from EPC to UML activity diagrams is discussed in section 4. Section 5 introduces business object models as an extension of UML class diagrams and shows how to derive the business object model from the class and activity diagrams. We conclude with a summary and outlook.

2 SAP R/3 Data and Process Models SAP R/3 is divided into a hierarchy of high-level software components, each of which represents a part of the overall business logic in terms of data structures and business processes. These main components are Accounting, Logistics, and Human Resource Management. Each of them is further divided into subcomponents. In the case of the R/3 Logistics component, the subcomponents are for example Sales and Distribution, Materials Management, Quality Management, Production Planning and Control, Plant Maintenance. The components on the level below these subcomponents are called business objects. However, they only capture data structure, not behaviour (see [SAP96b] for an overview of the SAP R/3 structure). A similar, but different hierarchy exists for business processes. There, the units below the subcomponents are called business scenarios. They comprise several processes. Within the R/3 system, structured entity relationship models [Sin88] are used to describe concrete data structures of subcomponents. Processes are represented by event-driven process chains [KNS92]. In addition to the above models, R/3 utility functions allow the retrieval of the relationships between different business objects on entity type level.

Reengineering Conventional Data and Process Models with Business Object Models

395

Altogether, three different views on the software system are possible: • the Internal Static View: SER models show the entity types and their relationships within one Business object; • the External Static View: R/3 utility functions yield the relationships between business objects, derived from the relationships between the entity types of the different business objects; • the Behavioural View: EPC display the behaviour of the system in terms of business processes. Within the present R/3 implementation, the static and the behavioural views are separated, i.e. there are no model-inherent connections between SER models, business object relationships or EPC. The reengineering efforts described in the following sections therefore concentrate on an integration of the three views. In the following we introduce the model elements of the three views with examples of the SAP R/3 order mangement. A SER model consists of • Entity types; • Attributes: Entity attributes, Entity type attributes; • Relationships: - Hierarchical Relationships(H), - Aggregational Relationships (A), - Referential Relationships (R), - Cond.-agg. Relationships (CA), - Cond.-ref. Relationships (CR), and - Temporary-ref. Relationships (TR); • Specializations/ Generalizations; • Cardinalities of Relationships and Entity types

An EPC consists of • Functions, • Operators (logical AND, OR, XOR), • Events, and • Process Paths as connectors between different EPC.

Table 1. SER Model and EPC Elements

SER Model Table 1 lists the elements of the SER model, which are used in the R/3 system. Additional explanations of their definitions and use can be found in [SAP96a,Sin88]. The SER model of Fig. 1 describes the structure of an Order with a header (entity type "Order") and one or more Order Items. The Order itself may be arranged in an Order Hierarchy. This additional entity type represents the child/parent relationship between two different Orders. Thus, an instance of Order Hierarchy has two relationships to two different orders, namely the child and its parent. In addition, an Order may have Global Conditions (prices, discounts etc.) and may contain information about its execution ("Order Operation"). An Order Item is either a Material, a Service or an Outline Level and has a Schedule Line and optionally Billing Plan Dates as well as particular Item Conditions.

396

E.v. Hahn, B. Paech, and C. Bock

O rd er G lob al C on d ition

O rd er O p eration

H

H

O rd erItem C on d ition H

O rd er H ierarch y

A A

O rd er

CR

H

O rd erItem

O rd erItem B illin g P lan D ate

H

H

O rd erItem S ch ed u le Lin e

Material

S ervice

O u tlin e Level

Fig. 1. A SER Model with Excerpts from the R/3 Order Management

EPC Table 1 lists the elements of the EPC, which are used in the R/3 system. Additional explanations of their definitions and use can be found in [KNS92]. A function (denoted as a box with rounded angles) is a significant step in the advance of a business process performed by the system either with or without user interaction. It is triggered by one or more events and may result in one or more events. Events are denoted as hexagonals. They may be the result of a preceding function or may be an external stimulus. An operator (denoted by a small circle) connects events and functions. A process path (denoted as box with added triangular) is a connector between two EPC. In both EPC it serves as a placeholder for the respective other EPC.

Reengineering Conventional Data and Process Models with Business Object Models Order Notification arrived

Contract Date due

Legend

Event

XO R

< name > Create Order

Function Order created < name > AND

Process Path Determine Sold-to party

Determine Vendor

Sold-to party determined

Order created

< name >

Operator Op

AND

Determine Order Items

Order Items determined

Calculate Order

Order calculated

Release Order

Order released

AND

Order Confirmation

Goods Movement

Fig. 2. An EPC with Excerpts from the R/3 Order Management

397

398

E.v. Hahn, B. Paech, and C. Bock

The EPC of Fig. 2 describes an excerpt from the process of the R/3 Order Creation. The process presented here starts with two external events, Contract Date Due and Order Notification Arrived. Afterwards a new empty order is created with the Sold-to Party and the Vendor to be determined. Additionally, the Order Items are determined, the order is calculated and released by an employee. The process continues with two process paths to Goods Movement and Order Confirmation, which are separate EPC. Business Object Relationships

Business Partner

Material

Order

Warehouse

Conditions

Fig. 3. Relationships between Business Objects with Excerpts from the R/3 Order Management

Fig. 3 shows the external relationships of the business object Order to the business objects Business Partner, Material, Conditions and Warehouse. Each line represents one or more actual foreign key relations between a table (entity type) of the business object Order and one from the associated other business objects.

3 Translation of SER Models into UML Notation In this section we describe the translation of a SER model into an UML class diagram. This translation serves as the basis for the business object models, since each business object model will contain the classes covered by the business object. For definition of UML class diagram elements we refer the reader to [Rat97]. The translation of the SER model elements into UML class diagram elements is straightforward. We present the details in Table 2. Fig. 4 shows the result of the translation of Fig. 1.

Reengineering Conventional Data and Process Models with Business Object Models

399

As mentioned in line 6 of Table 2, the associations corresponding to the aggregational relationships do not capture all information. In particular, it is not possible to express the existence dependency of the aggregational entity type (in this case Order Hierachy) from the source entity types (in this case Order). The property {frozen} only captures the fact that the relationships cannot be changed after being established. The existence dependency would usually be expressed in UML by aggregation. This is not possible for aggregational relationships in general, since the aggregational entity type cannot be included in both source entity types.

1 2 3 4 5 6

7 8

9 10 11 12

SER model element

UML Class Diagram element

Comment

Entity type Entity Entity type Attribute Entity Attribute Hierarchical Relationship Aggregational Relationship

Class Instance (Object) Class-scope Attribute Instance (Object) Attribute Composition

-----------

Normal Association with multiplicity 0..1 on target side and property {frozen} Normal Association Normal Association with multiplicity 0..1 on source side

Slight loss of information: see above

Referential Relationship Conditionalaggregational Relationship Conditional-referential Relationship Temporary-referential Relationship Specialization Relationship Cardinality

Normal Association with multiplicity 0..1 on source side Normal Association with explicit changeability Specialization Association multiplicity

--see 6

---------

Table 2. Translating SER Model Elements into UML Class Diagram Elements

400

E.v. Hahn, B. Paech, and C. Bock 0..N

O rd er G lob al C on d ition

0..1

0..N 0..N

O rd er O p eration

0..N

O rd er H ierarch y

0..N

0..N 0..1

1 1

O rd er

1..N

O rd erItem

1

1..N

1

O rd erItem C on d ition

O rd erItem B illin g P lan D ate

O rd erItem S ch ed u le Lin e

Material

S ervice

O u tlin e Level

Fig. 4. An UML Class Diagram Corresponding to the SER-Model of Fig. 1

4 Translation of EPC into UML Notation In this section we present a translation of the EPC into the notation of UML activity diagrams. The later are advocated in [Rat97] as the adequate notation for business process modeling. Swimlanes are not considered in this section. However, they play an important role in Sect. 5. Table 3 lists those elements of the UML activity diagram which are used below for the translation. Further definitions and explanations of activity diagram elements can be found in [Rat97].

Reengineering Conventional Data and Process Models with Business Object Models

401

An UML Activity Diagram consists of • Action states (Activities) - excactly one Start State: • Transitions; • Transition bars; • Decision operators. Table 3. UML Activity Diagram Elements

The translation poses three problems: first, activity diagrams do not support operators. We therefore extend UML by adornment of transition bars with operators. Second, events are not allowed on transitions leaving activities. By looking at the EPC in detail, we found that the events which follow the activities typically only capture the completion of the activity. We therefore, decided to omit these events altogether in the UML activity diagrams. Thus, only external stimuli are allowed as events in the activity diagrams. Third, activity diagrams - as a special case of statechart diagrams do not allow events on transitions which join or fork, but only on the transition bar. Since this situation occurs for external stimuli triggering the EPC and triggered by the EPC (as exemplified in Fig. 2), we decided to extend UML in this respect. Given the preliminary status of the activity diagrams at the time of the case study, these extension seemed to be reasonable. Exclusive OR E1

...

EN

AND E1

XOR

...

UML Translation

OR EN

E1 . . .

AND

EN

E1 EN Op (AND | OR | XOR)

OR

F 1

F

2

F

3

F

4

F

5

F

6

F

XOR

E1 7

. . . E2 E

AND

E1 8

. . . E2 E

1-3 5,6 F

OR

E1 9

...

E2

E

Op (AND | OR)

EN

E1 8

E AND

XOR

AND

OR

F1

... F N

F1

... F N

F1

... F N

F1

. . . FN

F1

. . . FN

F1

. . . FN

XOR

10

E

AND

11

E

F1

...

FN

F1

...

FN

E

Op (AND | OR | XOR)

OR

12

E

10-12

Fig. 5- Combinations of EPC Functions, External Stimuli and Operators and Their Translation into UML

402

E.v. Hahn, B. Paech, and C. Bock

Figure 5 shows all possible combinations of functions, external stimuli and operators in EPC and their corresponding UML translation. Cases 7 and 9 are not allowed, since they are nondeterministic with regard to the selection of the following functions. Case 4 is translated into an UML decision operator. This is not shown in the table for reasons of space. C ontrac t D ate due

O rder N otific atio n arrived

XO R C re ate O rder

AND

D e term ine S old -to party

D e term ine V endo r

AND D e term ine O rder Item s

C a lculate O rder

R e lease O rder

AND

O rder C o nfirm atio n*

G oods M ovem en t*

Fig. 6. UML Activity Diagram Corresponding to Fig. 2

Reengineering Conventional Data and Process Models with Business Object Models

403

Altogether, we use the following extensions of UML activitiy diagrams: • Transition bars are adorned with XOR, AND or OR labels to show the logic of the branching. • Decision operators represent Exclusive OR branches (XOR). • External stimuli are allowed as adornments to transitions which join or fork.. • An asterisk on an activity denotes a former EPC Process Path and connects an activity-like placeholder of one activity diagram with the same placeholder contained in a different activity diagram. Figure 6 shows the UML activity diagram resulting of the translation of Fig. 2.

5 Integrating Structure and Behaviour within Business Object Models In this section we present the business object model as well as its derivation from class and activity diagrams. The most important step in this derivation is the introduction of swimlanes into the activity diagrams which make explicit the business objects the activities are assigned to. Since this assignment is not included in the EPC, it has to be derived a posteriori. The business object relationships diagram introduced in Sect. 2 shows all the partner business objects which are candidates for the assignment.The choice of the right candidate for each activity could only be done by domain experts, since this assignment is not documented in any existing model of SAP R/3. As a rule of thumb an activitiy is assigned to the business object which either provides most of the needed input data or which can be made responsible for the correct execution of the particular functionality. Activities which clearly denote manual user input without system intereference such as "Enter Order schedule lines" or "Release Order" are assigned to an artificial business object "User". Fig. 7 shows an UML activity diagram with swimlanes consistent with Fig. 6.

404

E.v. Hahn, B. Paech, and C. Bock O rder

C ontract D ate due

B usiness P artner

M aterial

C ondition

U ser

W arehouse

O rde r N otificatio n arrived

C re a te O rd e r

AND

D e te rm in e S o ld -to p a rty

D e te rm in e V endor AND

D e te rm in e O rd e r Ite m s

C a lc u la te O rd e r

R e le a s e O rd e r

AND

O rd e r C o n firm a tio n *

G o o d s M o ve m e n t*

Fig. 7. UML Activity Diagram with swimlanes, transformed from Fig. 6

The assignment information of these activity diagrams is used to integrate class and activity diagrams into business object models. Business object models are used to represent the classes covered by the business object and the behavioural dependencies on other business objects. A business object model therefore is a class diagram representing the classes (data and behaviour) encapsulated by the business objects and the relationships to other business objects. Therefore, the class diagram is extended with specialized objects representing partner business objects (denoted by boxes with a small box on top) and with labels on the associations to the partner business objects denoting operation calls to the classes of the business object. These calls may be realized with the open Business Application Programming Interfaces (BAPI) of SAP where the partner business objects collects all the relevant BAPI. The operations correspond to the activities of the activity diagrams. Activities which are assigned to the main business object are associated as operations to the corresponding internal class. Again, the assignment of operations and operation calls to classes within the business object has to be done by domain experts.

Reengineering Conventional Data and Process Models with Business Object Models

405

Fig. 8 shows the business object model for Order integrating Fig. 3, Fig. 4 and Fig. 7. B u sin ess O b ject "O rd er" C a lc u la te O rd e r()

Con d ition s

0..N

O rd er G lob al Con d ition

0..N

O rd er O p eration

0..N

O rd er Hierarch y

0..N

0..1

0..N

0..N 0..1

1 1

O rd er

1..N

O rd erItem

1

1

1..N

O rd erItem Con d ition

O rd erItem B illin g P lan Date

O rd erItem S ch ed u le Lin e

G o o d s M o v e m e n t*()

C re ate O rde r() O rd erC o nfirm a tio n*()

W areh ou se

Material D e te rm in e O rd e rI te m s () D e te rm in e S o ld -to P a rty ()

S ervice

D e te rm in e V e n d o r() R e le a s e O rd e r()

O u tlin e Level

User

B u sin ess P artn er

Material

Fig. 8. Business Object Model integrating Fig. 3, Fig. 4 and Fig. 7

6 Conclusion The introduction of process integrating business objects requires the reengineering, respectively the evolution, of existing systems and their documentation. In particular, the classes covered by the business object have to be identified, and the business logic has to be distributed over the business objects. In the case study, the classes covered had already been determined. The emphasis therefore was on the distribution of the business logic. To make the distribution explicit, we have introduced the notion of business object model. Based on two preliminary steps which translate data models into class diagrams and process models into extended activity diagrams, we have introduced an assignment of activities to business objects. This enabled us to identify the operations and dependencies of the business objects.

406

E.v. Hahn, B. Paech, and C. Bock

Reengineering of the documentation is clearly only a first step. It is necessary in order to explore different business object architectures. For implementation, one can eventually use the standard issued by the OMG [BOF96].

Acknowledgements. We would like to thank Dietmar Nowotny for his continuous feedback on earlier versions of this work.

References [BOF96]

Object Management Group, Common Facilities RFP4, Common Business Objects and Business Object Facility, OMG TC Document CF/96-01-04, 1996 [BOM97] Object Management Group, Business Object DTF, Common Business Objects, bom/97-12-04, Version 1.5, 1997 [Hah98] E. von Hahn: Development and Utilization of an Application Pattern Demonstrated on the SAP R/3 Order Management, Master Thesis, Computer Science,Technical University Munich, Germany, February 1998 [KNS92] G. Keller, M. Nuettgens, A.-W. Scheer: Semantische Prozeßmodellierung auf der Grundlage Ereignisgesteuerter Prozeßketten (EPK), Veröffentlichungen des Instituts für Wirtschaftsinformatik, Heft 89, Saarbrücken 1992 [Rat97] UML 1.1 documentation, Rational INC, http://www.rational.com/uml/ [Ren96] D.S. Renshaw, Application Model Trends in Distributed Object Systems, OOIS’96, pp.506-514, Springer, 1996 [SAP96a] Data Modeling with the Data Modeler, SAP Documentation R/3 System Release 3.0, SAP AG, 1996 [SAP96b] The SAP Information Model. Model supported Information Management in the System R/3, SAP Method Manual, SAP AG, 1996 [Sin88] E.J. Sinz: Das Strukturierte Entity-Relationship-Modell (SER Modell), Angewandte Informatik 5/88, pp.191-202, Vieweg Verlagsgesellschaft, 1988

An Active Conceptual Model for Fixed Income Securities Analysis for Multiple Financial Institutions Allen Moulton, Stéphane Bressan, Stuart E. Madnick, Michael D. Siegel Massachusetts Institute of Technology, Sloan School of Management, Cambridge, Massachusetts 02139 {amoulton, smadnick, msiegel}@mit.edu [email protected]

Abstract. The practical implementation and use of a mediator for fixed income securities analysis demonstrated the potential for extending the application of conceptual modeling from the system design stage to providing query access to both data and computational resources. The mediator product was designed as a general interpretive engine specialized and controlled by the declarative knowledge from the conceptual model. The fixed income conceptual model included securities ranging in complexity from Treasury bills to collateralized mortgage obligations, as well as standard and proprietary analytic methodologies and calculations. All information, whether computed or retrieved from databases, was offered to clients in the form of attributes of conceptual entities. Client preference entities and attributes were used to control selection among conceptually interchangeable source data and procedural components. Experience from this implementation provides insight into the requirements for successful application of a conceptual model in mediating among heterogeneous, autonomous users and information resources.

1 Introduction Effective fixed income investment decision-making involves integrating information from internal firm sources with industry standard information and specialized proprietary knowledge, all within a common framework of abstract analytic methodologies. Representing the information structure of abstract methodologies in high level conceptual models enabled the Data and Calculation Services (DCS) mediator to combine heterogeneous, autonomously controlled information resources needed by decision-makers. Decision support application developers could write queries directly against the conceptual model, with the mediator responsible for finding and combining component resources. The conceptual model provided a database-like framework for integrating databases, analytic calculations, and predictive models. End-users could independently control implicit assumptions and selection of resource components used by their applications. The mediator enabled the interchange and combination of information resources across organizational boundaries, while responding to the rapid T.W. Ling, S. Ram, and M.L. Lee (Eds.): ER’98, LNCS 1507, pp. 407−420, 1998.  Springer-Verlag Berlin Heidelberg 1998

408

A. Moulton et al.

evolution of financial products, transactions, and markets characteristic of the fixed income securities industry. The challenge of the DCS mediator was to bring together six highly competitive firms to cooperate in providing decision support information to their customers while retaining autonomous control of proprietary data and analytic resources. Wiederhold[1,2] described mediators as an extra software layer to provide client applications with access to the integrated knowledge available from heterogeneous, autonomous data servers. The DCS mediator extended the definition to include computational resources as well as data sources. Context refers to the implicit assumptions that add meaning to symbols. Autonomous organizations often develop different contexts, resulting in semantically heterogeneous representations of the same knowledge[3]. In making investment decisions, it is vital the information conveyed among parties be understood by both. Government regulators and industry associations, such as the Securities Industry Association[4] and the Public Securities Association[5], develop standards for terminology and generic analytic measures. Publications, such as Fabozzi’s series of books (viz. [6]) disseminate theories and practices. Beyond the basics, however, firms add value by developing richer, more complex predictive models for security valuation and portfolio management. Context mediation can help integrate industry and regulatory standards with proprietary data and analytic methods to serve the needs of the investing public. In COIN[7,8,9], sources and receivers independently describe their context to the mediator, which uses metadata to detect conflicts in data representation and interject conversion operations in the query plan. The DCS mediator supported context definition and automatic conversions for parameters and values of computational functions as well as source and receiver data. In addition, the DCS mediator experimented with context conversions among different representations of security valuation related through abstract models, often implemented in proprietary analytic software. The traditional use of conceptual modeling is in the planning and design of information systems and databases[10,11]. The DCS mediator was a practical experiment in making a conceptual model actively available for query as envisioned by Markowitz and Shoshani [12] and Parent et al.13]. The DCS conceptual model served as a domain model for fixed income decision-making information. SIMS[14], Information Manifold[15], Infomaster[16], and Garlic[17] employ a domain-specific ontology. As in these projects, DCS integrates data sources through a shared model of knowledge specific to the problem domain. DCS differs from the approach of TSIMMIS[18], which uses an ontology-free approach where data is integrated and delivered in its own terms, from Carnot[19] which draws on an ontology of general common-sense knowledge, and from Benaroch’s[20] use of macro-level ontological knowledge to integrate disparate knowledge-based systems for financial risk management.

2 The Fixed Income Securities Industry Information is the critical resource in the fixed income securities industry - information about securities and their issuers, information about markets, information about

An Active Conceptual Model for Fixed Income Securities Analysis

409

economic conditions and events, and information about methodologies and models (see Fig. 1). Billions of dollars of debt instruments trade every week. Firms on the “buy side” (institutional investors and investment managers) manage capital on behalf of investors and benefit plan sponsors. Firms on the “sell side” (investment banks, brokers, and dealers, often known as “Wall Street”) bring new securities to market and interact to create capital markets. Fixed income securities, such as bonds, are obligations to pay sums of money at points in time over the life of the security. Unlike equity securities, an investor has no stake in the financial entity that issued the security. To an investor, a fixed income security represents a stream of future cash flows. A purchase decision trades present money for future payments. Optional events may affect the cash flow stream and there may be risk of default. In essence, however, all fixed income securities are interchangeable investment vehicles. Cash flows from one or more obligations may even be repackaged by selling off rights to payments or by combining rights to payments into new composite securities. Repackaging, or “financial engineering” produces “derivative” securities. Faced with a vast array of combinations of cash flows, risks, and optional events, every industry participant needs timely information and effective methods for determining investment value from raw data. In 1990, six major Wall Street investment banking firms formed a partnership establishing an “Electronic Joint Venture” aimed at using advanced technology to improve communication with their buy-side clients. During the 1970s and 1980s, fixed income investment management had become substantially more complex. Interest rates had fluctuated over a wide range. Huge quantities of new debt instruments had been issued. New securities products had been regularly introduced. Analytic methodologies had advanced to cope with the new complexities. Markets had become more globally integrated and moved at a faster pace. To cope with the new complexity, sell-side firms hired bright young “rocket scientists” with mathematical and engineering backgrounds into “fixed income research”

Information

Funds invested

BUY SIDE

Security sales

SELL SIDE

PLAN Institutional Investment SPONSORS Investors Banks & Funds Security OTHER withdrawn purchases Brokers Investment INVESTORS Managers Dealers Benefit payments Large investment capital with low transaction volume

S

S

Underwriting ISSUERS

Trades

MARKETS

Relatively small capitalization with high transaction volume

Fig 1. The fixed income securities industry.

410

A. Moulton et al.

departments. They developed databases with detailed descriptive information about securities as well as historical prices, interest rates, and economic statistics. Proprietary real-time broadcast networks brought current market information and news to trader and investment analyst desktops. Application software was built to display securities data, perform securities and portfolio analytics, and apply the models developed by financial engineers. Information technology had also been advancing rapidly. Business mainframes stored databases, supercomputers performed the most complex calculations on an overnight basis, while a combination of dumb terminals, PCs, and workstations were piled high on trading and sales desks. In the midst of this jumble of technology, buy-side firms began to demand direct access to data, analytics, and models. Previously, the sell-side would offer ideas and supporting analysis on the phone and by FAX. The buy-side wanted more control over the analysis used to make investment decisions. Investors wanted to be able to integrate proprietary sell-side data, analytics, and models with their own internal data, analytics, and models so that they could make better decisions. The sell-side firms were willing to provide information resources to the buy-side, but wanted to protect proprietary information from being passed on to competitors. The sell-side also hoped to receive information from the buy-side about positions held in their portfolios in order to be able to make better trading recommendations. The mission of the Venture was to provide the infrastructure to facilitate the interchange of information between buy-side and sell-side firms. The technology plan involved six major areas: physical connectivity via a proprietary network, a desktop workstation platform, a suite of standard applications, standard databases, standard analytic calculations, and the data and calculation services (DCS) mediator. By offering “generic” or industry standard applications, databases, and analytic calculation software the venture would be able to replace duplicative efforts on the part of clients and its partner firms. All Venture-supplied applications, databases, and calculations were required to be designed on an open architecture that would allow partner firms and clients to develop their own proprietary extensions and substitute parts. A Venture partner might provide a proprietary application that would use and extend the Venture’s standard databases and calculation software. A partner firm might alternatively supply a proprietary algorithm for use within a standard application. The DCS context mediator was similar in some ways to an object broker approach. The difference was a design based on declarative knowledge used to dynamically decide on methods to materialize elements of the conceptual model.

3 Architecture Of The DCS Mediator The DCS mediator (see Fig. 2) was designed as a demand-driven general interpretive engine controlled by a static declarative knowledge. The core of the declarative knowledge was a conceptual model of fixed income data and analytic methodologies. Action specifications provided the means for realizing entities and attributes of the conceptual model from heterogeneous resources supplied by autonomous organizations. Declarative knowledge was prepared in a definition language script compiled into a

An Active Conceptual Model for Fixed Income Securities Analysis Control panel

Spreadsheet

Standard apps

Proprietary apps

411

Client apps

C API Client Context Control Objects

Static Declarative Knowledge Structure

Financial assumptions Source selection preferences

Sybase wrapper

Standard data

C call wrapper

Standard analytic libraries

Conceptual model Action specs

DCS Mediator Engine

Proprietary data

Object wrapper Proprietary analytic libraries

RPC wrapper

Client data

Client analytic libraries

Fig. 2. Architecture of the DCS Mediator.

binary structure for run-time interpretation. Application software accessed the mediator through a C-language application-programming interface (API). Ad-hoc user access was provided through a spreadsheet interface. Generic wrapper interfaces were developed for the Sybase relational DBMS, C function libraries, an object technology contributed by one of the partner firms, and remote procedure calls (RPC). Client preferences and financial assumptions were represented in context control objects that could be viewed and manipulated though a Control Panel application. By changing values in context objects, the user could control the financial assumptions, data sources, and analytic models used by the mediator in responding to other application queries. In essence, the Control Panel allowed the user, and the client firm’s management, to define the context – the implicit assumptions – of information to be received. 3.1 Conceptual Model Structure The DCS conceptual model defined the entities, attributes and relationships available through the mediator and served as a shared ontology for integrating heterogeneous, autonomous data and calculation resources. A high level conceptual model for fixed income securities analysis was published as the principal documentation of the functionality available through the mediator. Fig. 3 shows an example of conceptual model elements offered in DCS. Entity class A is a subclass of a parent class P, depicted by a hollow arrow. DCS supported single inheritance and polymorphism with subclass determined dynamically from an attribute

412

A. Moulton et al. P

subclass

real number D

A a1 : semantic type a2 : enum D a3 : ref B a4 : set C

B

C

G

a5 = a6 = $ a7 : struct a8 : type

Fig. 3. DCS conceptual model elements.

percentage quantity

percent fraction basis pts 32nds 64ths 275 2.24 2.75 .0275 2.48

Fig. 4. Semantic types for percentage quantities.

in the parent class. Class A has six attributes (a1…a6) and a substructure a7 containing an additional attribute a8. Attribute a1 illustrates a data attribute. Semantic type captures properties of data at the conceptual level, while a data type stands for its physical representation. Both types are associated with data values during mediation. Fig. 4 shows a semantic type hierarchy for several representations of percentage quantities. The five different numbers at the bottom all represent 2.75%. The mediator provided automatic conversion among compatible semantic and data types. Attribute a2 in Fig. 3 refers to an enumerated domain D. Arbitrary codes are often used to represent properties of entities. An enumerated domain could list the codes used in the conceptual model or codes used in a particular resource context. Mapping relations associated context-specific codes with conceptual model codes, allowing the mediator to convert codes across contexts. An attribute declared on an enumerated domain may also be used as a role name for a many-to-one relationship. Attribute a3, a reference to an object of class B, is a role name for the many-to-one (or one-to-one) relationship with class B. Attribute a4, similarly, is a role name for the one-to-many relationship with class C. Many-to-many relationships require an intervening entity. Attributes a5 and a6 are equated to other attributes reachable by a relative(a5) or global(a6) path. A path is a sequence of attribute names separated by dots, representing the traversal of a relationship. Each step, except the last, in a path must be the name of an attribute which refers to another object or to an enumerated domain. A relative path begins with the current object. For example, attribute a8 can be reached from an object of class A with the relative path “a7.a8.” A global path (indicated by a leading “$”) begins with the user’s global object G. Every object has an implicit many-to-one relationship with the global object. 3.2 Action Specifications Action specifications provide procedures for deriving output attribute values from input parameters (see Fig. 5). An action may be a database retrieval, a computational function, or an arithmetic expression. Each action is associated with a class in the conceptual model (the base class for the action). Attribute reference paths may be either relative to the action base class or global. Each action may have enabling conditions which serve

An Active Conceptual Model for Fixed Income Securities Analysis

413

as constraints that must hold for an action to be applicable. For the initial implementation, enabling conditions were limited to equality constraints specified as pairs of attribute paths and constant values. Database retrieval actions are defined by an SQL statement with slots for input parameter values. Output attribute paths in the same order as the SELECT clause are specified in an INTO clause. For single row retrievals, the action sets values into attributes of a single object of the base class and related objects. For set valued retrievals, an additional output attribute defined as a set of base class objects must be declared. Semantic types of attributes obtained from a database retrieval action are defined by the conceptual model. Data types are determined from the database query output. Computational functions may be legacy or new code, available in a local library or through a remote procedure call on a server. Functions can return a single value to multiple values through output arguments. . A function must return the same output values for each call with a given combination of inputs. Internal state may not affect outputs. Functions are declared to the mediator with both data and semantic types for each value and parameter. Action specifications match input and output attribute paths to function arguments and value. Expression evaluation actions use a built-in interpreter for simple arithmetic expressions. Independently supplied functions need not be changed when integrated into DCS, since the mediator automatically converts of input data to the semantic and data types required by the function. Values returned retain the functiondefined semantic and data types until conversion is required where used. The network of action inputs and outputs constitutes a dependency graph among attributes of the conceptual model. At a high level of abstraction, there may well be several different ways of representing functionally equivalent information. To allow heterogeneous sources, receivers, and computations to each use their preferred representation, the conceptual model may need to include functional redundancies resulting in potential cycles in the action dependency graph. Fig. 6 illustrates a cycle resulting from mutual dependency of the fullPrice and price attributes. To cope with some frequently occurring cycles, the DCS mediator provides for designation of groups of mutually determined attributes. This information is used to detect overspecified criteria in a query and to avoid trying actions known to result in cycles. 3.3 Mediation Algorithm The mediator algorithm provided the appearance to the user of a database capable of materializing all elements of the conceptual model. Queries are based on a single object with attributes of related objects reached using relative paths. The mediator processes queries of the form: SELECT <list of attribute paths> FROM WHERE

A query proceeds by creating a working object of the given class, “putting” values specified in the criteria into attributes of the object and related objects reached through it, then “getting” values for each selected attribute path from the object. Related objects

414

A. Moulton et al. action2

enabling conditions input parameter attribute paths

Action

Fig. 5. Action specifications.

output value attribute paths

fullPrice

accInt

price

action1

Fig. 6. Dependency graph cycle.

are automatically created when referenced. Each intermediary and final result value is held in a “semantic value package” structure together with its semantic and data types. The query logic follows a depth-first approach checking for cycles to avoid infinite recursion. The logic of the mediator can be understood by describing the “get” algorithm. Get is a recursive procedure that takes a reference object and a path and follows a depth first search strategy to find a value. The attribute path is processed in sequence, getting each object reference and applying get recursively to the remainder of the path until only one element remains. When the path has only a single attribute name, get first checks the conceptual model for an equivalence declaration. If the attribute is as equated, a recursive get is performed using the equated path. If the value of the attribute has previously been determined, it is returned. If not, the mediator attempts to find an action that can supply the requested value. Since the dependency graph for actions may well have cycles, the mediator marks the attribute as “in use” before proceeding. If another get operation is attempted on an attribute that is so marked, the operation fails. To derive a value for the attribute, the mediator uses the dependency graph to attempt each action capable of providing the required attribute as an output. Actions are tried in the order they are declared. When an action succeeds, the value is returned and alternative actions ignored. Each action may have enabling conditions, which require an attribute to match a specified constant value. For each enabling condition, a recursive get is performed on its attribute path based on current object. If the get fails or the returned value does not match, the action is cancelled and the mediator moves to the next action. If all enabling conditions hold, the mediator attempts to obtain values of all input parameters by calling get recursively for each. If the get on any input parameter fails, the action is discarded. If all input gets are successful, the mediator compares the semantic data types of the values returned to the types required for the action. Incompatible types cause the action to be discarded. If either semantic or data types are compatible but different, the mediator calls an appropriate semantic or data type conversion. Once all the inputs are collected, the mediator calls the appropriate wrapper to execute the action. A failure in the wrapper causes the action to be cancelled. The database wrapper prepares the SQL statement by substituting input values where marked. The statement is then sent to the database instance specified in the action. Each value returned is put using the path from the INTO list in the action specification. Data type is determined by the type returned from the database; semantic type, from the conceptual model. The function wrapper sets up and executes a local or remote procedure call. On

An Active Conceptual Model for Fixed Income Securities Analysis

415

return the function value and all output argument values are put to the path specified in the action specification. For functions, semantic and data types are given by the function declaration. When any action succeeds, get returns the value after storing the attribute value in the containing object. If all actions fail, get returns failure. In either case, the attribute is marked “not in use.” After the recursive call stack is unwound, the mediator converts the result value to the user-specified semantic and data types.

4 Applying the DCS Mediator to Fixed Income Securities Analysis Fixed income investment decision-making involves buying, selling, and swapping securities in order to improve the performance of an investment portfolio while meeting the sponsor’s objectives. Decision-makers use both general methodologies applied across all kinds of securities and specialized analytic techniques applicable to particular market segments. Analytic support tools must be extensible for new kinds of securities invented by financial engineers and for new analytic models within the structure of common methodologies. Decision-makers also need access to and control over assumptions, models and analytic components used. The DCS mediator offered a new paradigm for analytic application development. Traditionally, the application programmer is responsible for meshing database retrievals and calculation libraries. With DCS the application programmer works at a higher level of abstraction focusing on the results of applying analytic methodologies. The mediator takes responsibility for finding and integrating the right data and programs to fulfill the application’s needs. 4.1 Developing the Fixed Income Conceptual Model After about a year of design and development, a beta version of the DCS mediator was made available to application developers in 1991. Product release was in mid-1992. After the initial development of the mediator engine, the efforts of the DCS development team focused primarily on extending the domain model, verifying the correctness of calculations and data integrated through the mediator, and documentation. Documentation included “placemat” diagrams of object classes similar to the small samples in Fig. 8, along with explanations of the semantics of each attribute. A glossary of fixed income terminology (still available on the web at [21]) clarified definitions and semantics as used in the domain model. Fig. 7 illustrates part of the abstract conceptual model for fixed income securities and analytic methodologies. The Security entity represents a generic concept of a fixed income security. The PyCalc (price-yield calculator) entity represents an actual or hypothetical trade in a security together with internal rate of return (IRR) and option adjusted spread (OSA) analytic methodologies applied to the trade. At the abstract level, the Security has an issuing financial entity represented by Issuer and end-of-day prices from various sources in a set of Valuation entities. The global Assumptions class

416

A. Moulton et al. Assumptions

PyCalc PyBond

sec

Valuation

Security

Issuer

SecBond SecPool Collateral

PyMort SecMgen PyTranche

SecTranche

CMODeal

Fig. 7. Fixed income conceptual model fragment showing polymorphic subclass interrelationships.

provides context information on financial assumptions and client selection of data sources and analytic models. Polymorphic subclasses (shown with hollow arrows pointing to the parent generic class) represent different information structures for Security and different analytic implementations for PyCalc. In each case, the subclass is determined dynamically from the value of the meta-attribute type in the parent class. Security subclasses correspond to different kinds of descriptive data relevant to each type of security: SecBond for bonds, SecPool for mortgage pools, SecMGen for a generic mortgage model, and SecTranche for a tranche of a collateralized mortgage obligation (CMO). For the SecTranche subclass, the security is related to a CMODeal entity and in turn to a set of Collateral, each of which may be a position in a pool or a generic mortgage. Subclasses of PyCalc correspond to alternative action specifications for performing the functionality of the abstract class. The subclass of PyCalc is derived from the type of Security entity referenced: PyBond for bonds, PyMort for mortgage generics and pools, and PyTranche for CMO tranches. In some cases, a subclass may extend the generic PyCalc functionality with additional measures specific to one type of security. Fig. 8 shows some of the attributes of the generic PyCalc and Security classes in “placemat” form used in the documentation. The sec attribute in PyCalc provides a reference to a Security. The type attributes in each class control subclassing, with PyCalc type derived from Security type by an enumerated domain relation. Security type may PyCalc Security type : enum SecType coupon type : enum PyType optionAdjusted source : string sec : ref Security rate : percent assetID : dbid settleDate : date spread : basis points frequency : enum cusip : string fullPrice : percent CouponFreq ticker : string price : percent assumed = $Assumptions maturity description : string yield : string date : date issuer : ref Issuer yieldType : enum YieldType price : percent valuation : set Valuation nominalSpread : basis points standardYieldType : enum YieldType accruedInterest : percent

Fig. 8. Fragments of the “placemat” documentation for the generic PyCalc and Security classes.

An Active Conceptual Model for Fixed Income Securities Analysis

417

be implied (because a source is limited to a single type) or derived from stored information. The optionAdjusted part of PyCalc is also polymorphic based on security type and selection of OAS model in Assumptions. A Security object may be used to represent a hypothetical security, with attribute values filled in by the application program. More often, a Security object would be associated with stored data using key attributes such as cusip, ticker, or assetID. CUSIP[22] is an industry standard identifier used in the United States and Canada. An assetID was an artificially constructed globally unique key for all security data objects accessible through DCS. Venture-supplied security databases were designed with assetID attribute for each security. Autonomous databases are made accessible by extracting the primary keys of each security record into a lookup table and assigning a unique assetID to each. The assetID also contained codes indicating the source for the data and the security type. 4.2 The DCS Mediator at Work The following simple query illustrates an application request for a description string, yield and price for a trade in a security identified by CUSIP, settling on November 19, 1998, with valuation specified as a nominal spread of 275 basis points from an assumed benchmark yield curve. SELECT sec.description:string, yield:percent, price:fraction FROM PyCalc WHERE sec.cusip=”887315AT” AND settleDate=”11/19/98" AND nominalSpread=275

The DCS mediator begins by creating a generic PyCalc object and putting the values given for the three attributes in the WHERE clause. A generic Security object is automatically created in traversing the path sec.cusip. Attempting to obtain sec.description, the mediator must determine the security type and source. The mediator uses a database action requiring the known cusip attribute to obtain assetID. From assetID, source and type are derived. Since CUSIP 887315AT is a bond, the Security object is subclassed to SecBond. A second package based on assetID retrieves bond data from the identified source and puts it in attributes of the SecBond object. If the source does not provide a value for the description attribute, a function action constructs a description string from other attributes. The value returned would be: TIME WARNER 7.48% DEB 01/15/2008

As illustrated in Fig. 9, yield, price, and nominalSpread are alternative representations of security valuation, each mutually computable from one another. The example query specified nominalSpread. The action for computing yield from nominalSpread involves adding the value given for the spread to the yield value found at a point on a benchmark yield curve, which is derived from a scenario object in the global Assumptions object. By setting a scenario using the Control Panel, the user determines the contextual meaning of nominal spread in all applications. After calculating yield, the mediator converts the value, if necessary, to the percent form requested in the query.

418

A. Moulton et al. security valuation

price

fullPrice

yield

nominalSpread

OAS

Fig. 9. Alternative representations of security valuation.

Computing price from yield requires an algorithm specific to the type of security. Based on the security type of bond, PyCalc was subclassed to PyBond. The PyBond subclass has action specifications for computing accruedInterest and for fullPrice given yield. Each action requires settleDate and security characteristics attributes from the SecBond object. The mediator sets up each call, after converting the representation to meet the requirements of the called function, then computes price as fullPrice less accruedInterest. As a final step, the value for price is converted, if necessary, to the fractional form required by the user query. By decoupling applications from the methods and data used in computing analytic information, DCS allows plug’n play dynamic substitution of functionally equivalent components. For example, mortgage prepayment models are critical to projecting performance on portfolios holding derivative securities. When a Wall Street firm refines its prepayment model, it wants to make that new knowledge available to its customers quickly. Buy-side portfolio managers may have preferred portfolio analysis application tools that can gain immediate benefit from the new model delivered through the mediator. The mediator can also facilitate the sell-side analyst in developing proposed trades by bringing buy-side portfolio holdings data into the analyst’s applications. Improving information flow between buy-side and sell-side has the added benefit of improving capital market efficiency.

5 Conclusions DCS proved the concept of context mediation in the pragmatic world of an informationintensive and highly competitive industry. When the original design was presented to clients, response was enthusiastic. Developers liked the idea of integrating data and calculations, allowing them to avoid the work of connecting components together in software. They found the notion of treating computational results as data to be appealing. Producing a mediator for production business use requires quality assurance and training, yet the major quality assurance effort was not directed at the mediator itself. Clients saw the results delivered through the mediator as the responsibility of the mediator. In consequence, the mediator development team spent most of a year verifying data and testing and fixing calculation software as it was integrated. Considerable effort was also devoted to documentation and training, including a manual on domain model design using a tennis shoe sales example. For some users the structure imposed by the standard conceptual model was helpful. More advanced developers and client shops, however, wanted to be able to specify their needs as user views or in their own conceptual models. The mediator would need

An Active Conceptual Model for Fixed Income Securities Analysis

419

to go beyond context conversion for individual pieces of data, to be able to map the central domain model into the structure of client views and models. Clients also wanted to be able to treat the components of the domain model as building blocks that could be arranged to suit their purposes. For example, clients would have liked to pose queries that combined entities as needed rather than using pre-planned relationships. Use of UML[23] modeling capabilities and OQL[24] as a query language would have been helpful. Experience with DCS suggests the potential value of further research on query processing reasoning algorithms for mediating among application developers’ conceptual models and component supplier models using industry-specific abstract knowledge. Since the DCS mediator was built, Goh[7] has examined abductive logic programming for context conversion of semantic types. Our present research involves extending the abductive reasoning paradigm to multiple levels of abstraction in conceptual models.

References 1. 2. 3.

4. 5. 6. 7. 8.

9. 10. 11. 12. 13. 14. 15. 16.

G. Wiederhold, “Mediators in the architecture of future information systems.” IEEE Computer, March 1992, pp. 38-49. G. Wiederhold, “Mediation in information systems,” ACM Comp. Surveys vol. 27, no.2, June 1995, pp. 265-267. S. Madnick. “Are we moving toward an information superhighway or a Tower of Babel? The challenge of large-scale semantic heterogeneity.” Proc. IEEE International Conference on Data Engineering, 1996, pp. 2-8. Securities Industry Association. New York. http://www.sia.com Public Securities Association. New York. http://www.psa.com F. Fabozzi, ed., The Handbook of Fixed Income Securities, 3rd ed., Irwin, Homewood, Illinois, 1991. C. Goh. “Representing and reasoning about semantic conflicts in heterogeneous information systems.” Ph.D. Thesis, MIT Sloan School of Management, 1996. C. Goh, S. Bressan, S. Madnick, and M. Siegel. “Context interchange: new features and formalism for the intelligent integration of information.” MIT Sloan School of Management Working Paper #3941, Feb. 1997. M. Siegel and S. Madnick. “A metadata approach to resolving semantic conflicts.” Proc. 17th International Conference on Very Large Data Bases, 1991, pp. 133-145. C. Batini, S. Ceri, and S. Navathe. Conceptual Database Design. Redwood City, Calif.:Benjamin/Cummings, 1992. T. Teorey. Database Modeling and Design. San Mateo, Calif.: Morgan Kaufmann, 1990. V. Markowitz and A. Shoshani. “Abbreviate query interpretation in entity-relationship oriented databases.” Proc. 8th Int’l. Conf.. on Entity-Relationship Approach, 1989. C. Parent, H. Rolin, K.Yétongnon, S. Spaccapietra. “An ER calculus for the entityrelationship complex model.” Proc. 8th Int’l. Conf.. on Entity-Relationship Approach, 1989. Y. Arens and C. Knobloch. “Planning and reformulating queries for semantically-modeled multidatabase.” Proc. of the Intl. Conf. on Information and Knowledge Management, 1992. A. Levy, A. Rajaraman, and J. Ordille. “Query answering algorithms for information agents.” Proc. 13th National Conf. on Artificial Intelligence, Aug. 1996, pp. 40-47. M. Genesereth, A. Keller, O. Duschka. “Infomaster: an information integration system.” Proc. 1997 ACM SIGMOD Conference, May 1997, pp. 539-542.

420

A. Moulton et al.

17. Y. Papakonstantinou, A. Gupta, and L. Haas. “Capabilities-based query rewriting in mediator systems.” Proc. 4th Intl. Conf. on Paralled and Distributed Information Systems,1996. 18. H. Garcia-Molina. “The TSIMMIS approach to mediation: data models and languages.” Proc. Conf. on Next Generation Information Technologies and Systems, 1995. 19. D. Woelk, P. Cannata, M. Huhns, W. Shen, and C. Tomlinson. “Using Carnot for enterprise information integration.” Second International Conf. on Parallel and Distributed Information Systems, Jan. 1993, pp. 133-136. 20. M. Benaroch. “Toward the notion of a knowledge repository for financial risk management.” IEEE Trans. on Knowledge and Data Engineering, vol. 9, no. 1, Jan. 1997, pp. 161-167. 21. Bridge Info. Systems, “EJV Partners Financial Glossary” http://bondchannel.bridge.com/html/EJVGlossary.html 22. Standard & Poors CUSIP Service Bureau http://www.cusip.com 23. Rational Software Corp. http://www.rational.com/uml/index.shtml 24. R. Cattell (ed.). The object database standard: ODMG-93. San Francisco: Morgan Kaufmann, 1996.

An Entomological Collections Database Model for INPA 1

1

1

J. Sonderegger , P. Petry , J.L. Campos dos Santos , and N.F. Alves

2

1

National Institute for Amazon Research (INPA), Alameda Cosme Ferreira, 1756 - Aleixo, Caixa Postal 478, CEP 69.083-000 Manaus-Brazil {jurgson, petryp, lcampos}@inpa.gov.br http://www.inpa.gov.br 2 University of Amazonas, Av. General Rodrigo Octávio Jordão Ramos 3000 - Aleixo, CEP 69.083-000 Manaus-Brazil [email protected]

Abstract. The National Institute for Amazon Research, INPA, has more than 40 years tradition in studying the Amazon, the biggest region of biodiversity on this planet. Millions of organisms, collected from the rainforest and its rivers were deposited in the collections of the institute. They are divided into six botanical, four microorganismal and nineteen zoological collections, consisting of vertebrates and invertebrates. The majority of the invertebrates are insects, which comprise the entomological collection. The aim of this work is to present a database model for this entomological collection. Entomologists at INPA were interviewed and the results have been evaluated to determine the information scope. The models of the Association of Systematic Collections, the Ohio State University and the Museum for Vertebrate Zoology at the University of California, Berkeley, were used during the development of our model. The model presented here was constructed using concepts from the method Object Protocol Model (OPM).

1

Introduction

Biodiversity information systems are becoming a very important tool in the decision making process for worldwide policy development of environmental preservation and biodiversity conservation. The gradual shift in recent years from structurally based, species-oriented conservation to an approach rooted in preserving ecological processes and complexity will require increased information integration to be successful in achieving conservation and sustainable use of biological diversity. The Amazonian tropical rain forest is one of the most complex biomes on earth and concentrates the largest biodiversity on the planet. Since 1952 the National Institute for Amazon Research (INPA) in Manaus-Brazil, has been studying this region and has collected a large volume of data about it's biodiversity. These data are distributed among the different departments and collections of the Institute. One of the most challenging tasks for the future is to organize these data in the form of a Biodiversity Information System (BIS) with an efficient database design that allows for an T.W. Ling, S. Ram, and M.L. Lee (Eds.): ER’98, LNCS 1507, pp. 421−434, 1998.  Springer-Verlag Berlin Heidelberg 1998

422

J. Sonderegger et al.

intelligent and fast access to the stored information for the scientific community as well as for the community in general. The planned BIS should serve for the following purposes described in Biodiversidade na Amazônia [5] (institutional project proposal): A) Support of the scientific research in Amazon. The information generated by scientific research should be stored in databases that can be accessed and used for further studies. B) Improvement of the knowledge that is relevant for humans, as the influence of anthropological activities on the environment and its diversity, the economical advantage of biodiversity and the problems to humans that are caused by biodiversity, e.g. tropical pests and illnesses. C) Support the development allowing for the formulation of environmental politics as laws and decisions about the management and the preservation of the existing biodiversity. The information to be collected, captured and included in the system is very heterogeneous and diverse. The final version of the system should consist of data about scientific collections, taxonomy, systematic, evolution, ecology, epidemiology, bio-geography, agriculture and forestry among other areas of knowledge. To capture all the different types of information it will be necessary to create different databases, which will have to be related to one another in order to minimize redundancy. The first set of data to be stored in databases is the information of the scientific collections. INPA has 6 botanical, 4 microorganismal and 19 zoological collections. The aim of this work was to develop a database model for one biological collection. Because of the volume of information and the lack of a standard procedure to catalogue specimens, we concluded that the entomological collection would be an ideal case study. Entomological research is widespread among several departments at INPA, leading to a situation in which the entire insect collection is comprised of 16 different parts and every part has its own set of information and procedures. In order to achieve common ground, the data requirements were analyzed together with the scientists that are working with insects. Beneath the diversity of the data, it had to be considered that the model must be reusable for the development of other collection’s models, which should have a similar pattern. One important characteristic of a data base approach is providing some level of data abstraction by hiding details of data storage that are not needed by most database users [4]. A data model is the main tool that represents this level of abstraction. By definition a data model is a set of concepts that can be used to express the structure of a database, its data types, relationships, and constraints. The majority of data models include a set of basic operations for specifying retrievals and updates in the data base. Advanced data models include concepts which specify behavior by defining a set of valid user defined operations in addition to the basic operations provided by the data model [4], [7].

2

Requirements and Existing Models for Biological Collections Databases

The requirements of a database system arises from the usage demand of the stored information. The most obvious reason for Museums and Scientific Institutions to

An Entomological Collections Database Model for INPA

423

develop databases for biological collections is to improve the collections data management and replace the traditional register-card-archives by digital data. Tools, processes and interfaces help curators to control and manage the material of the collections safely, and thus allows users a fast and modern way to access collection information. Data of digital databases can be accessed over computer networks by the members of the network. Considering internet capabilities the data will be accessible worldwide, given the proper restrictions to sensitive information. An exemplary approach to developing a collections management system was made by the Museum of Vertebrate Zoology (MVZ) in Berkeley, defining at first the functional and system requirements [8], [9] and then developing the model [10], [11] based on the results of the requirements analysis. In the first part of the requirements analysis [8] the four main functions called High-level Museum Functions and their specific functionalities were defined, Fig. 1.

Fig. 1. Interactions between Functions and Collections of the MVZ [8]

In the center lies the Collections Management that builds the link between the physical collection, the databases and the other functions. The Education and Conservation functions are furnished with information by the Collections Management. The relationship between Collections Management and Research is bidirectional. Research produces the input to the databases and collections and uses the data for analysis. This fundamental philosophy is also a good example for a whole information system and the integration of collections with data of different fields of research as it is planned for the BIS of INPA. The requirements analysis, as well as the models of the MVZ, have been used as references during the development of our model. Another important model that was considered during this work is the ASC biological collections model [1]. It is a guideline that provides the background for the future connection of biological collection’s databases worldwide. The integration of data from different institutions implies that standards or norms for data exchange

424

J. Sonderegger et al.

must exist. The ASC has been working on this task since 1992. The ASC model was used as the base for this work, because the possibility to exchange data using it's structures represents a very important requirement. We considered further the OSU database model of the Ohio State University entomological collection [6]. It is a model for an insect collection and contains general attributes for biological collections as well as specific attributes of entomological data.

3

Data Requirements for INPA's Entomological Collection Database

The construction of a database model for a collection requires the understanding of the physical construction of the collection, as well as the knowledge of the data of it's objects that will be stored in the database. It is important that the users of the collection information accept the database structure and data. For these reasons the users were directly included in the data requirements analysis. During the definition phase we interviewed 17 scientists at INPA that work with the entomological collection material. Each one is studying a taxonomic group or a certain biological aspect of some taxonomic groups, e.g. beetle and other insects that attack wood. Interviews had an open format, researchers were asked the same general questions, and allowed to elaborate on how they conducted field sampling and recorded information. The information captured during a collecting event consists of a general part, which holds the information that is normally collected in all studies (e.g. date, time, locality description), and a specific part which corresponds to the scientific interest of a study (e.g. the altitude of a locality or the moon-phase may be of interest in one study but not in another). Interviewing scientists that work on different studies helps to differ between information that is important for all and that which is used by just a few scientists. The results of the interviews are represented in Table 1. The column Data contains the attributes of the objects in a collection. The information is divided into the groups Collections Information, Taxonomic and Identification Information, Collecting Event Information and Social and Ecological Information. The group Collections Information contains collections management information and attributes that describe the place and state of an object in the collection. The o Access N represents a unique identifier for all specimens that have been collected during one sampling expedition and all the documentation derived from its collecting o events. The same Access N will be assigned to objects of different collections if they o were collected together. The Collection N is the identifier of a specimen or a lot in a collection. Fixation, Conservation and Preparation are descriptions which were required by nearly all scientists. Rack and Drawer describe the physical place of an object in the collection, which of them is used depends on the kind of insect collection. The Fitter is the person which added an object into the collection. This object attribute and the attributes Cataloger, Loan and Availability are important data for collection management.

An Entomological Collections Database Model for INPA

425

Table 1. Collections information, M indicates information recorded by the interviewed researchers (Aca=Acarina, Col = Coleoptera, Dip = Diptera, Eph = Ephemoptera, Hem = Hemiptera, Hym = Hymenoptera, Lep = Lepidoptera, Odo = Odonata, Ort = Ortoptera) Data

Type Aca Col

Access No Collection No Fixation

String String Table Table Table Collections Preparation Number Information Rack Drawer Number Fitter Table Cataloger (Digitizer) Table Loan Binary Availability Binary Material Condition String Order Table Super-Family Table Family Table Sub-Family Table Table Taxonomic Genus Sub-Genus Table And Table Identification Specie Table Information Sub-Specie Status/Type Table Taxonomic History Table Identifier String Identification Date Date o Field N String Country Table State Table Community Table Locality Table Micro-Locality Table Collecting Coordinates Table Event String Information Altitude Date Date Hour Hour Clima/Conditions String Collecting Method Table Habitat String Moon String Collector Table Sex Binary Parasite Binary Social and Solitary Binary Ecological Table Information Social Mature/Immature String

M M M

M M M

M

Dip

M M M

Entomological field Eph Hem Hym Lep Odo Ort

M M M

M

M

M M M M M

M M M M M

M M M M

M M M M

M M M M M M

M

M

M

M

M M M

M

M

M

M M M M M M M M M

M M

M M

M M M M M M

M M M M M M

M

M M M M M

M M

M M M

M

M

M

M M M M M

M

M

M

M M

M M

M M

M M

M M

M M

M M M M M

M M M M M M M M

M M M M M M M M M M M M M M M M

M M M M M M

M M M M M M

M M M M M M

M M M M M

M

M M

M

M M M M M M

M M M M M M M

M

M

M

M

M M M

M M

M M M M M

M M

M M

M M

M M M

M M M M

M

M

426

J. Sonderegger et al.

The attribute Material Condition may help a curator to decide whether he should loan an object or not. The attributes not marked in this table are not of scientific interest but they are important for the management of the collection. The group Taxonomic and Identification Information contains the identifiers of the biological classification as Order, Family, Genus and Species and their Super- and Sub-variations and nearly all of them were used. Status/Type describes the meaning of an object for the taxonomy, e.g. a holotype specimen is the one that has been used as the standard for the original description of a species. The Identifier and identification date of an object are important for all scientists. The Taxonomic History contains the information about the taxonomic names, synonyms, their authors and the date of the description. The information recorded during a collecting event is included in the group o Collecting Event Information. Field N is an identification number for the event and is normally correlated with the field notes that a collector makes for the event. The remaining attributes define the collectors, the place and the time of an event. Generally all entomologists use the same attributes. Exception are the altitude and moon phase that are just used in special studies. Specific attributes of collection objects have been put into the group Social and Ecological Information. The attributes Sex and Mature/Immature are important in all research areas. It is even desirable that the latter attribute gives a more precise impression than just the 'mature' or 'immature'. It should contain the description of the life stage that the collection object is in. The attributes Parasite, Solitary and Social describe a social or ecological aspect of an animal and are not used by all scientist, nevertheless they must be considered in the model of the database.

4

The Entomological Database Model of INPA (EDMI)

The method used for the development of the database model is called ObjectProtocol-Model and is described in [2] and [3]. The diagrams are illustrated in a form that is slightly different from that used in the OPM model of ASC. In both types of diagrams the boxes represent classes. In the ASC diagrams [1] one figure has been made for each class, which illustrates the relations of this class to its attributes. Relations are illustrated by lines and belong to the class on the left end of the line. In our diagrams relations are represented by arrows and they belong to the class in which the arrows begin as shown in Fig. 2. This form of illustration has been chosen because the understanding of the relations seems to be easier, and more than one class can be fully illustrated in one diagram. A special type of relation is the derived relation. The value of a derived relation depends on other objects and must be determined using the attributes of other objects. For example an object in the class Reference_Work, shown in Fig. 8, contains as Subjects information about an object of the class Taxon_Name which is mentioned in the reference. This class is described in Fig. 6 and its relation back to the class Reference_Work is derived. This means that before a Taxon_Nameobject knows its references, it must be investigated in which objects of class Reference_Work it is contained as a subject.

An Entomological Collections Database Model for INPA

427

The classes of EDMI are sorted in 6 clusters which represent functional groups, e.g. the fundamental collections classes. The cluster Collection Classes includes Collection, Collection_Object, Lot, Specimen and Object_Situation. The other clusters are Collecting Event Classes, Locality Classes, Taxonomy Classes, Agent Classes and Reference Classes. All classes of the model contain the three attributes Created, Updated and UpdatedBy which contain information about the date and time of creation, the last update of an object, and about the person that made the update. These attributes serve as tools to track possible data corruption events.

Fig. 2. The OPM-derived notation

4.1

Cluster Collection Classes:

The cluster Collection Classes is shown in Fig. 3 and represents the core of the model. It contains the classes Collection, Collection_Object, Lot, Specimen, Object_Situation and Loan which hold the information about the collection, its objects, the kind of objects and the physical location of an object in the collection or whether it is on loan. Systematic information, such as the taxonomic name of an object and its place in the system is part of the Determination which may be found in the cluster Taxonomy Classes. 4.2

Cluster Collecting Event Classes:

The cluster Collecting Event Classes is shown in Fig. 4 and contains just one class, the Collecting_Event. This class allows access to information about the date, locality and participating collectors of an event. Furthermore there are derived relations to the Collection_Objects which have been collected at the event and to the Reference_Works where the event is mentioned.

428

J. Sonderegger et al.

Fig. 3. Classes of the Cluster Collection Classes and their relations and attributes.

Fig. 4. Collecting_Event class and it's relations and attributes.

An Entomological Collections Database Model for INPA

4.3

429

Cluster Locality Classes:

The classes Locality, Named_Place, Habitat, Cartographic_Reference, Geo_Reference_Object, Chain, Line and Point belong to the Locality Classes cluster as shown in Fig. 5. They define a geographic position, name and the habitat type of a locality where a collecting event took place. A locality shape of a collecting event may be described as a couple of points - class chain, a line between two points - class line or an area around a point - class point. A Cartographic_Reference defines a map on which the locality may be found.

Fig. 5. Classes of the Cluster Locality Classes and their relations and attributes.

4.4

Cluster Taxonomy Classes:

The classes Determination, Taxon_Name and Taxon_Relation belong to the taxonomy classes cluster described in Fig. 6. A Determination is valid for exactly one

430

J. Sonderegger et al.

Collection_Object. It represents the link between an object of the collection and its taxonomic information as the name of the taxon and its place in the taxonomic system. The class Taxon_Name may be used for the definition on all taxonomic levels as phyla, class, family, genus or species. The attribute Rank contains the information of the taxonomic level and the relation Parent Taxon is a link to the next taxon on a higher taxonomic level to which the actual taxon belongs. Therefore the whole systematic structure may be constructed with one class, the Taxon_Name. The determination of a Collection_Object may be made on the level that is desired, e.g. family, genus or species, without different taxon classes or different kinds of relations being required. Furthermore the class Taxon_Name contains the information about the original reference, the author that described the taxon, a list of Taxon_Relations, the name of the classification that contains the name and whether it is considered as valid or not. If there is a number of synonyms for one taxon, each of them is one object of the class Taxon_Name but just one of these objects may be considered as valid and the others contain a link called ValidName to it. The Taxon_Relation describes relations between taxa, for example, symbiosis or parasitism. The names of the taxa must be within the accepted nomenclature and will be based on authority files which ensure integrity. The entomological collection represents the class insects, so the class will be the highest taxonomic level that is required in this database.

Fig. 6. Classes of the Cluster TaxonName Classes and their relations and attributes.

An Entomological Collections Database Model for INPA

4.5

431

Cluster Agent Classes:

Agent classes cluster shown in Fig. 7 contains the classes Agent, Person and Organization which were used to define curators, authors, collectors or determiner of objects. In the general class Agent the contact information contains address and phone or fax numbers. The person’s or organization's specific information is situated in the classes Person and Organization. For each Person- or Organization-object there must be one Agent-object which contains the general information. The attribute TypeOfAgent in the class Agent defines whether an agent is a person or an organization.

Fig. 7. Classes of the Cluster Agent Classes and their relations and attributes.

4.6

Cluster Reference Classes:

The classes Reference_Work, Book, Book_Section, Journal, Journal_Article and Scientific_Study belong to the Reference classes cluster, shown in Fig. 8. Reference_Work is the general class which contains lists of references to the subjects that are named in the work. Referenced subjects may belong to the classes Collection_Object, Collecting_Event, Taxon_Name, Classification and Locality. A reference work may be a book, a book section, a journal article or a description of a

432

J. Sonderegger et al.

scientific work, e.g. a dissertation or field notes. The information in these classes must guarantee that the reference may be found by an interested person. For an article in a journal there must be known information about the journal. For this reason the class Journal_Article contains the class Journal which itself is not specific enough to be a Reference_Work.

Fig. 8. Classes of the Cluster Reference Classes and their relations and attributes.

4.7

The Logical Model Description of EDMI

The logical description of all objects that define the attributes, values and relations, as well as the description of the controlled value classes that define specific value types can be found at http://www.inpa.gov.br/colecoes/entomol/model.html.

5

Considerations for the Implementation of a Prototype

For a first prototype the Data Base Management System (DBMS) PostgreSQL will be used because its features allow to fulfill our basic requirements and a later migration to an other DBMS based on SQL-standard is simple and fast. The decision, which DBMS will be used for the final implementation including the whole Biodiversity

An Entomological Collections Database Model for INPA

433

Information System, will be made in the near future. As a first step in the prototype development, the model will be translated into a relational model consisting of the table definitions in the database. These will then be implemented in a PostgreSQL database. The first prototype must include the graphical user interfaces for the data input and the possibilities to access and browse the database over the internet. That will be also a first contribution to the society of taxonomists for which it is of fundamental importance to have access to collections informations and materials. Depending on the users echo about the prototype the functionality will be expanded and if necessary also the model will be accomodated.

6

Concluding Remarks

The use of an object oriented model leads to a more natural representation of the database entities and their relations. Modelers can use the object oriented notation as the means to translate users expectations into a structured and robust design, which contemplates the users needs making the model understandable to them. The opinion of the users is one of the most important criterias because a system will just be sensefull if the people for whom it is made also accept and use it. The inclusion of them in the modeling phase is as well important for the data definition as for the analysis of the functional requirements of the product. Researchers will be benefited by having their requirements tested in the model and translated into the implementation. At the institutional level the model will allow data integration across different databases. With the development of other collections databases, the demand for integrating all of the information arises, so that the relations between objects of different collections are part of the information system. This is an important issue, that will guide the extensions of the model. The implementation of the model and it's use as template for other collections will result into the normalization of the data, which in turn will facilitate information exchange within the scientific community also at an interinstitutional level. Acknowledgments. We would like to thank the Brazilian National Council for Scientific and Technological Development, CNPq, for funding this project, the Brazilian Research Institute INPA, where the project took place, Dr. Célio Magalhães of INPA for supporting the project, Dr. Stanley Blum, who helped us with information about the ASC project and hints about the modeling method, Dr. Norman F. Johnson for the information about the OSU insect database, all scientists of INPA, who participated the interviews and Dr. Victor Py-Daniel and JoLynne Wightman for the revision of the manuscript.

434

J. Sonderegger et al.

References 1.

Association of Systematic Collections: The ASC Reference Model, 1997, http://gizmo.lbl.gov/DM_TOOLS/OPM/BCSL/LIB/ASC.html 2. Chen, I.A., and Markowitz, V.M.: An Overview of the Object-Protocol Model (OPM) and OPM Data Management Tools, Information Systems, Vol. 20, No. 5, 1995 Or http://gizmo.lbl.gov/opm.html 3. Chen, I.A., and Markowitz, V.M.: The Object-Protocol Model (Version 4.1). Documentation: Technical Report LBNL-32738 (revised), 1996. http://gizmo.lbl.gov/DM_TOOLS/OPM/OPM_4.1/OPM/OPM.html 4. R. Elmasri e S. Navathe: Fundamentals of Database Systems. 2ª ed., Benjamin Cummings, 1994. 5. Instituto Nacional de Pesquisas da Amazônia: Biodiversidade na Amazônia, Manaus - AM, Setembro de 1996 (institutional project proposal, not published). 6 Norman F. Johnson: The Ohio State University Insect Collection - OSUC Database Structure, November 1997, http://iris.biosci.ohio-state.edu/projects/tpp/cluster.html. 7. H. F. Köth e A. Silberschatz. Sistemas de Banco de Dados. São Paulo, Makron Books, 2ª ed. Revised, 1995. 8. Museum of Vertebrate Zoology: Requirements Analysis - Functional Requirements. University of California, Berkeley - California, August de 1995, http://www.mip.berkeley.edu/mvz/cis.html. 9. Museum of Vertebrate Zoology: Requirements Analysis - System Requirements, University of California, Berkeley - California, October de 1995, http://www.mip.berkeley.edu/mvz/cis.html. 10. Museum of Vertebrate Zoology: The MVZ Information Model, University of California, Berkeley - California, October de 1996, http://www.mip.berkeley.edu/mvz/cis.html. 11. Museum of Vertebrate Zoology: The MVZ Logical Model, University of California, Berkeley - California, November de 1996, http://www.mip.berkeley.edu/mvz/cis.html.

A Global Object Model for Accommodating Instance Heterogeneities Ee-Peng Lim1 and Roger H.L. Chiang2 1 Centre for Advanced Information Systems School of Applied Science, Nanyang Technological University Singapore 639798, SINGAPORE 2 Information Management Research Centre Nanyang Business School, Nanyang Technological University Singapore 639798, SINGAPORE

Abstract. To completely address database integration problems in the context of multidatabase[10] and data warehousing systems, one has to examine various integration and query requirements. Due to various reasons such as poor data quality in local databases, ongoing local database updates, and instance heterogeneities, some instance differences have to be accommodated by the integrated databases. We have therefore proposed a new object-oriented global data model, called OORA , that can accommodate attribute and relationship instance heterogeneities in the integrated database. In addition, the OORA model has been designed to allow database integrators and end users to query both the local and resolved instance values using the same query language.

1

Introduction

To fully address the schema and instance integration issues in both multidatabase and data warehousing systems, one has to examine the database integration process at the macro level. Throughout the entire database integration process, inter-database heterogeneities should be handled appropriately. While there has not been a well accepted database integration methodology, we proposed to divide the entire integration process into three phases, namely Analysis, Derivation, and Evolution. – Analysis: Analysis is essentially a knowledge acquisition phase. In this phase, database integrators are expected to understand pre-existing databases at both the conceptual and implementation levels. Database integrators are also required to find out from the integrated database users their global application requirements in order to derive the global schema and instances. – Derivation: The actual derivation of global schema and integrated instances is done in this phase. Once the derivation is done, queries on the integrated database can be evaluated. It is in this phase a complete mapping from local schemas to the global schema, as well as a mapping from local instances to global instances are specified. – Evolution: Due to the autonomy of local database systems, updates to the local databases may violate the mapping from local instances to global T.W. Ling, S. Ram, and M.L. Lee (Eds.): ER’98, LNCS 1507, pp. 435–448, 1998. c Springer-Verlag Berlin Heidelberg 1998

436

E.-P. Lim and R.H.L. Chiang

instances. Evolution therefore refers to the ongoing refinement of integrated databases as the local database schemas and instances evolve. It becomes the most important phase to maintain a multidatabase or a data warehouse. Among the above three phases, evolution has been largely ignored in the database integration research primarily due to two reasons. Firstly, most researchers focus on schema integration issues[1,3,4,11,14]. While a lot of schema integration issues have to be investigated for different databases during the derivation phase, it is uncommon to investigate schema integration issues during the evolution phase due to rare modification to pre-existing local schemas. Secondly, research on the integration of instances has been pre-occupied by query processing issues instead of local database updates during the evolution phase. In this paper, we argue that instance integration may not be complete in the derivation phase. During the evolution phase, one also has to consider local database updates which lead to new instance conflicts that cannot be handled by pre-defined integration methods. Hence, new global data models that can accommodates instance heterogeneities become necessary. Literature Review Most previous database integration research focused on resolving schema conflicts. Lately, as researchers begin to address instance integration problems, several solutions of instance conflict resolution have been proposed [7,8,13]. We review some data modeling research in handling instance conflicts as follows. – Polygen model [13] was proposed to capture source information of attribute values that come from different local relations. A source value is associated with every attribute value of the tuples of polygen relations. The source information captured include the sites the attributes originated from and the intermediate sites at which they are processed. The model, however, does not provide the mechanism to accommodate or resolve instance heterogeneities. – TS Relational model [5] was proposed to accommodate entity and attribute conflicts in a relational integrated database. A special source attribute is assigned to every relations. An extended relational algebra has been proposed to manipulate the TS relations. Like the Polygen model, TS Relational model is not designed to represent resolved instance values. – Role-based Multidatabase model [9] considered the different roles (or relations) assumed by real-world objects. Queries on a role-based multidatabase are decomposed into queries on different combinations of roles. Apart from not handling resolved instance values, the role-based multidatabase model does not classify between tolerable and intolerable relationship and attribute value conflicts (see Sect. 2). Research Objectives Our research addresses the problem of accommodating instance heterogeneities (conflicts) in the global data model adopted for integrated databases. There are a number of reasons for accommodating instance heterogeneities:

A Global Object Model for Accommodating Instance Heterogeneities

437

– Resolving all instance differences may not be desirable because some global applications may want to retain and view these differences. For example, the different prices for the same product sold in different stores may be required to be retained and queried in the integrated database. – Preserving the instance heterogeneities allows database integrators to apply different resolution techniques on the same integrated database for different global application requirement. – Resolving all instance conflicts may not be possible because the information and knowledge required for the complete conflict resolution is not available during the moment of instance integration. – Resolving all instance conflicts may not be feasible because the amount of processing time to resolve conflicts for large number of instances may be so much that integrated information may not be available on time. In this paper, we present an object-oriented global data model that can accommodate instance heterogeneities for attributes and relationships in the integrated database. The new global data model supports different integration and query requirements from the database integrators and database users during the derivation and evolution phases of database integration. To our best knowledge, this is the first attempt in developing a global data model to support queries on integrated databases containing resolved and unresolved instances.

2

Instance Heterogeneities

Instance heterogeneities can be classified into entity conflicts, attribute conflicts, and relationship conflicts [6,8]. Entity conflicts arise when it is not known which entity instances from matching entity types1 correspond to the same real-world entities. Relationship conflicts occur when it is not known which relationship instances from matching relationship types correspond to the same real-world relationships. Attribute conflicts arise when the matching entity (or relationship) instances (determined by resolving entity or relationship conflicts) do not have the same attribute values. As pointed out by a number of researchers [2,5], instance integration may be difficult due to imperfect data quality in the legacy databases. The methods used for resolving discrepancies may also be different for different attributes. In this paper, we will further point out that the tolerance of instance conflicts varies among different attributes. In fact, it is common that not all instances from legacy databases can be properly integrated during the derivation phase. Although instances from different databases can be properly integrated during the derivation phase, one still has to handle integration issues arising from the updates to local database(s) during the evolution phase. For example, new data instances could be added to a database making it necessary to perform instance integration on the new instances. Similarly, instance integration is required for changes to attributes of some pre-existing data instances. There are 1

Matching entity types are determined by schema integration.

438

E.-P. Lim and R.H.L. Chiang

essentially two approaches to handle instance integration problems during the derivation and evolution phases. The first approach requires the database integrator to anticipate all possible integration scenarios during the derivation phase and define the instance integration methods accordingly, hoping that all integration problems in the evolution phase can be predicted in advance. When the integration scenarios cannot be predicted in advance (which is often the case), one has to resort to accommodating instance conflicts in the integrated database before these conflicts can be finally resolved sometime in the future or may not be resolved at all. 2.1

Entity Conflicts

To resolve entity conflicts, the knowledge for identifying instances representing the same real-world entities is required. For simple cases, common keys among entity instances could be used to match instances. For example, the employee name attribute can be used to match data from DBA and DBB . Complicated entity conflicts arise when there is no common attribute that can be used to match instances from different databases. Although it may not be possible to resolve all entity conflicts, instances that could not be determined to represent the same real-world entities can still be retained as separated instances in the global database. 2.2

Attribute Conflicts

Given two instances that represent the same real-world entity, the differences in their equivalent attributes are known as attribute conflicts. We distinguish two main types of attribute conflicts, namely tolerable and intolerable attribute conflicts, that should be handled in database integration. Tolerable attribute conflicts are those expected by a database integrator at the time an integrated database is derived. Intolerable attribute conflicts, on the other hand, refer to attribute conflicts that should not be resolved automatically by any predefined resolution methods. To distinguish between the above two types of attribute conflicts, we introduce the concept of threshold predicate. When the difference between two or more conflicting attribute values is smaller than a threshold value or when the conflicting attribute values differs in expected patterns, there is an straightforward pre-defined approach to handle the conflicts. The exact conflict handling approach can be readily specified during the derivation phase of database integration. The primary purpose of a threshold predicate is therefore to explicitly capture the criteria to be satisfied by tolerable attribute conflicts. In other words, we define tolerable attribute conflicts to be those satisfying the threshold predicates defined for the attributes involved. It is necessary to resolve tolerable attribute conflicts derived from different local databases. To do so, resolution functions should be defined to reconcile the corresponding tolerable attribute values. However, it is not always possible to apply resolution functions to resolve all possible tolerable attribute conflicts.

A Global Object Model for Accommodating Instance Heterogeneities

439

Sometime, one may not know the correct resolution function to be specified or used. In other occasions, the attribute conflicts are considered to be valid and acceptable by the integrated database users. Thus, no resolution function is required. Attribute conflicts which fail the specified thresholds are defined to be intolerable. Database integrators should be alerted for intolerable attribute conflicts by having the intolerable attribute conflicts recorded in a log file. 2.3

Relationship Conflicts

Relationship conflicts, first discussed in [6], arise when the relationship between two real-world entities may not be represented consistently in different databases. In [6], different types of relationship conflicts have been derived and they can be caused by incorrect schema integration, incorrect entity conflict resolution and inaccurate database content. Like other instance-level conflicts, a complete resolution of relationship conflicts may not always be possible. When relationship conflicts cannot be resolved by the multidatabase system or data warehousing system, they should be retained and accommodated.

3

A Database Integration Example

As both multidatabase and data warehousing systems preserve the autonomy of local database systems, updates to local databases can often introduce new instance conflicts to integrated databases. Some of these new instance conflicts could be handled automatically by the resolution methods predefined by database integrators. For new instance conflicts that cannot be resolved automatically, database integrators have to be called upon to handle them. Nevertheless, before any actions can be taken by the database integrators, these new conflicts have to be accommodated by the global data model and the end users should be allowed to continue using the integrated database. We employ an integration scenario to demonstrate the attribute and relationship conflicts. Figure 1 depicts the object-oriented schemas of the local databases (DBA and DBB ) containing company information. Here, we assume that schema integration has been performed and the schemas of existing databases have been made compatible to facilitate instance comparisons. The instances of DBA and DBB are shown in Figs. 2 and 3 respectively. Suppose all entity conflicts are resolved by matching ename and dname of employee and department instances respectively. We notice that John in DBA works in the Marketing department but John in DBB works in the Research department. This is a relationship conflict. On the other hand, the difference in salary for Chen in DBA and DBB is an attribute conflict. Figure 4 depicts the integrated schema derived from DBA and DBB .

440

E.-P. Lim and R.H.L. Chiang work_in Employee position

ename

work_in

managed_by qual

Department floor

Employee

dname

budget

salary

ename position

(a) Schema of DB A

managed_by

salary

Department floor

dname

budget

(b) Schema of DB B

Fig. 1. Schemas of Local Databases Employee instances ename = john position = trainee salary = 1000 qual = diploma ename = kim position = secretary salary = 1500 qual = NULL ename = chen position = engineer salary = 2500 qual=MEng ename = mark position = manager salary = 3000 qual = BBus ename = daniel position = engineer salary = 2400 qual = BEng

Department instances work_in

work_in

dname = marketing floor = 3 budget = 2M

work_in managed_by work_in managed_by work_in managed_by

dname = planning floor = 2 budget = 4M

dname = library floor = 1 budget = 1M

Fig. 2. Instances of DBA

4

The OORA Object-Oriented Data Model

In this section, we propose the OORA 2 , the extended object-oriented data model, to accommodate instance heterogeneities in the integrated databases. Specifically, the OORA model is able to accommodate attribute and relationship conflicts. The OORA data model is also designed to support queries on the integrated databases. Furthermore, the OORA data model ensures that the source of instance heterogeneities can be identified in order to support subsequent integra2

R represents the relationship conflicts. A represents the attribute conflicts. The name OORA indicates that both relationship and attribute conflicts can be accommodated.

A Global Object Model for Accommodating Instance Heterogeneities Employee instances

441

Department instances

ename = kim position = assistant salary = 1500

work_in

work_in

ename = chen position = leader salary = 2600

dname = marketing floor = 3 budget = 2.1M

managed_by

ename = stacy position = sales rep salary= 3500

work_in

ename = john position = trainee salary = 1200

work_in

work_in

ename = sugimoto position = fellow salary = 10000

dname = research floor = 6 budget = 1M

managed_by

ename = kain position = engineer salary = 5000

work_in

Fig. 3. Instances of DBB work_in Employee

managed_by qual

ename position

floor

Department

budget

dname

salary

Fig. 4. Integrated Schema

tion work on the partially integrated database. OORA differs from the traditional OO data model in a number of ways: – – – –

Identification of matching criteria for deriving global objects; Specification of threshold predicates and resolution functions; Representation of original and resolved attribute values; and Uniform treatment of attribute and relationship conflicts.

In the following, we describe the unique features of OORA model in detail.

442

4.1

E.-P. Lim and R.H.L. Chiang

Global Objects

A global object in the integrated database is derived from one or more local objects that represent the same real-world entity. Like in the traditional OO data model, each global object is assigned a unique global object id (oid). In OORA , we assume that local objects corresponding to the same global objects (or real-world entities) can be matched by examining some common attribute values. These common attribute(s) can be specified as matching criteria by the database integrator. For example, a database integrator may use ename to match Employee objects, and dname to match Department objects from DBA and DBB . The following two data definition statements have been used to identify matching local objects: DERIVE EMP from Employee@DB A, Employee@DB B USING Employee@DB A(ename), Employee@DB B(ename);

DERIVE DEPT from Department@DB A, Department@DB B USING Department@DB A(dname), Department@DB B(dname);

4.2

Threshold Predicates and Resolution Functions

A threshold predicate and a resolution function can be defined for each attribute in the global schema. Given an attribute in a class of global objects, the threshold predicate determines for each global object if a difference between local values of the attribute is tolerable. The resolution function is then specified to resolve tolerable attribute conflicts automatically. Depending on the characteristics of attributes, different threshold predicates and resolution functions should be defined and be implemented using system-defined functions/operators or general programs. Given an attribute in the global schema, three combinations of threshold predicates and resolution functions can be constructed3 : – Both the threshold predicate and resolution function are undefined: This implies that any difference between the corresponding attribute values is considered an intolerable attribute conflict. Unless all corresponding attribute values given are identical, the resolved attribute value is always NULL. – The threshold predicate is defined, but not the resolution function: This implies that tolerable attribute conflicts can exist among distinct instances. These conflicts are also acceptable. However, unless the acceptable attribute conflict involves identical values, the resolved attribute value is always NULL. – Both the threshold predicate and resolution function are defined: This implies that tolerable attribute conflict can exist among distinct instance and the resolution function will return the resolved attribute values.

4.3

Elements of Attribute Values

In the OORA model, every non-oid attribute has a domain consisting of three elements, namely the original values (denoted by ovalue), resolved values (denoted by 3

Note that when the threshold predicate is not defined for an attribute, it is meaningless to define the resolution function for the attribute since any difference between corresponding attribute values is considered intolerable, and such conflict shouldn’t be resolved by a resolution function automatically.

A Global Object Model for Accommodating Instance Heterogeneities

443

rvalue) and conflict type (denoted by conflictType). The resolved value, original value, and conflict type of an attribute A are represented by A.rvalue, A.ovalue and A.conflictType respectively. The A.ovalue of a global object is defined to be a set of (value, database id) pairs where value denotes the attribute value contributed by the corresponding object from the existing database identified by database id. The A.rvalue of a global object is defined to be any A value contributed by local objects if there is no attribute conflict. If a difference is found among the local A values, the tolerance of the conflict is first determined using A.threshold(). If the conflict is tolerable, A.rvalue is obtained by applying A.resolution() on the local attribute values. In the event where the conflict is intolerable or A.resolution() is undefined, NULL is assigned to A.rvalue. Depending on the original attribute values and the threshold predicate defined for the attribute, different conflict types can be derived and be captured in A.conflictType. A.conflictType is assigned NULL if there is no conflict, Resolvable if there is a tolerable conflict that can be resolved by the pre-defined resolution function, Acceptable if there is a tolerable conflict and there is no pre-defined resolution function for resolving the conflict, and Intolerable if there is a intolerable conflict. In our integrated database example, we can define the threshold predicates and resolution function for the salary and position attributes as follows: DEFINE salary.threshold@EMP(s1,s2) = DEFINE salary.resolution@EMP(s1,s2) = DEFINE position.threshold@EMP(p1,p2) =

(abs(s1-s2) ≤ 100) max(s1,s2) (p1=p2) or (p1=secretary and p2=assistant)

With the above definition, the salary values of 2500 and 2600 for the employee Chen constitute a resolvable attribute conflict. The global object for the employee Chen will have a resolved salary value of 2600 computed by the resolution function. On the other hand, the salary values of 1000 and 1200 for the employee John constitute an intolerable attribute conflict. In this situation, database integrators should be alerted and the conflict should be resolved manually. Since only threshold predicate is defined for the position attribute, the position values of secretary and assistant constitute an acceptable conflict.

4.4

Relationship Conflicts

In the OORA model, global relationships are derived from relationships between objects of existing databases. The global relationships, represented as reference attributes in the global schemas, relate global objects from different classes in the integrated database. Similar to the attribute conflict, we represent the original and resolved values of a reference attribute R in the global schema by R.ovalue and R.rvalue respectively. Threshold predicates and resolution functions can also be defined on reference attributes. For illustration, let say the research department is in fact part of the marketing department. The following threshold predicate and resolution function can be defined. DEFINE work in.threshold@EMP(o1,o2) = (o1 = o2) ∨ (∀ o, o.oid ∈ {o1,o2}, o.dname ∈ {research,marketing}) DEFINE work in.resolution@EMP(o1,o2) = o1 if (o1=o2), marketing otherwise In the above statements, o1 and o2 denotes global object ids of DEPT objects.

444

4.5

E.-P. Lim and R.H.L. Chiang

Integrated Database Instances

The OORA objects of the integrated database are shown in Tables 1 and 2. Note that the Attribute-element columns in the above tables are included simply to illustrate the three elements of attribute values. As shown in Tables 1 and 2, the OORA data model retains both attribute and relationship conflicts while holding the matching objects from different databases together by assigning global object ids to them. Respective resolution functions are defined to perform various resolutions of instance conflicts when they are tolerable. For example, the following threshold predicate and resolution function are defined for budget attribute. abs(b1−b2) DEFINE budget.threshold@DEPT(b1,b2) = ( max(b1,b2) ≤ 5%) DEFINE budget.resolution@DEPT(b1,b2) = min(b1,b2)

Table 1. EMP’s Global Objects Attrib-element oid ename position ovalue e1 (john,A)(john,B) (trainee,A) (trainee,B) rvalue john trainee conflictType NULL NULL ovalue e2 (kim,A)(kim,B) (secretary,A) (assistant,B) rvalue kim NULL conflictType NULL Acceptable ovalue e3 (chen,A)(chen,B) (engineer,A) (leader,B) rvalue chen NULL conflictType NULL Intolerable ovalue e4 (mark,A) (manager,A) rvalue mark manager conflictType NULL NULL ovalue e5 (daniel,A) (engineer,A) rvalue daniel engineer conflictType NULL NULL ovalue e6 (stacy,B) (sales rep,B) rvalue stacy sales rep conflictType NULL NULL ovalue e7 (sugimoto,B) (fellow,B) rvalue sugimoto fellow conflictType NULL NULL ovalue e8 (kain,B) (engineer,B) rvalue kain engineer conflictType NULL NULL

5

salary (1000,A) (1200,B) NULL Intolerable (1500,A) (1500,B) 1500 NULL (2500,A) (2600,B) 2600 Resolvable (3000,A) 3000 NULL (2400,A) 2400 NULL (3500,B) 3500 NULL (10000,B) 10000 NULL (5000,B) 5000 NULL

qual work in (diploma,A) (d1,A)(d4,B) diploma NULL (NULL,A)

d1 Resolvable (d1,A)(d1,B)

NULL NULL (MEng,A)

d1 NULL (d1,A)(d1,B)

MEng NULL (BBus,A) BBus NULL (BEng,A) BEng NULL (NULL,B) NULL NULL (NULL,B) NULL NULL (NULL,B) NULL NULL

d1 NULL (d2,A) d2 NULL (d3,A) d3 NULL (d1,B) d1 NULL (d4,B) d4 NULL (d4,B) d4 NULL

OORA Query Language and Examples

To query the global objects represented in the OORA data model, one has to formulate queries in a language we refer to as OOQLRA . OOQLRA adapts the existing SQL syntax for object-oriented queries. In addition, it is specially designed to support the query requirement for an integrated database containing attribute and relationship conflicts in the derivation and evolution phases of database integration. A OOQLRA SELECT query statement can be expressed as:

A Global Object Model for Accommodating Instance Heterogeneities

445

Table 2. DEPT’s Global Objects Attribute-element ovalue rvalue conflictType ovalue rvalue conflictType ovalue rvalue conflictType ovalue rvalue conflictType

oid dname d1 (marketing,A)(marketing,B) marketing NULL d2 (planning,A) planning NULL d3 (library,A) library NULL d4 (research,B) research NULL

floor (3,A)(3,B) 3 NULL (2,A) 2 NULL (1,A) 1 NULL (6,B) 6 NULL

budget (2M,A)(2.1M,B) 2M Resolvable (4M,A) 4M NULL (1M,A) 1M NULL (1M,B) 1M NULL

managed by (e3,A)(e3,B) e3 NULL (e4,A) e4 NULL (e5,A) e5 NULL (e7,B) e7 NULL

SELECT , ..., FROM , ..., WHERE <predicate expression>; Unlike the usual SQL statements, every non-oid attribute (say A) found in a OOQLRA query statement must be in one of the forms, A, A.ovalue, A.ovalue(D), A.rvalue and A.conflictType where D is some local database id. Only attributes of the forms A.ovalue, A.ovalue(D), A.rvalue and A.conflictType can be used in the WHERE clause. In other words, the attribute in the form of attribute name can only appear in the SELECT clause. In the following subsections, we will use several query examples to illustrate other essential features of OOQLRA .

Queries on Original Attribute/Relationship Values The original attribute and relationship values in the existing databases have to be examined by the database integrators during the process of deriving objects in the integrated databases in both the derivation and evolution phase. For example, the following OOQLRA statement (Q1) could be used to identify unresolved intolerable attribute conflict in the EMP class. Example (Q1): SELECT E.oid,E.ename.ovalue,E.position.ovalue, E.salary.ovalue,E.qual.ovalue FROM EMP E WHERE E.ename.conflictType=Intolerable OR E.position.conflictType=Intolerable OR E.salary.conflictType=Intolerable OR E.qual.conflictType=Intolerable; Since the OORA model accommodates all the original attribute and relationship values in the integrated database, users can query local databases via the global schema using OOQLRA statement. An example of such queries is illustrated in Q2. Example (Q2): SELECT E.oid,E.ename.ovalue(A),E.position.ovalue(A), E.salary.ovalue(A),E.qual.ovalue(A),E.work in(A).dname.ovalue(A) FROM EMP E;

446

E.-P. Lim and R.H.L. Chiang Table 3. Query Result of Q1 oid ename.ovalue position.ovalue salary.ovalue qual.ovalue e1 (john,A)(john,B) (trainee,A)(trainee,B) (1000,A)(1200,B) (diploma,A) e3 (chen,A)(chen,B) (engineer,A)(leader,B) (2500,A)(2600,B) (MEng,A)

Table 4. Query Result of Q2 oid e1 e2 e3 e4 e5

ename.ovalue(A) john kim chen mark daniel

position.ovalue(A) trainee secretary engineer manager engineer

salary.ovalue(A) 1000 1500 2500 3000 2400

qual.ovalue(A) diploma NULL MEng BBus BEng

work in.ovalue(A).dname.ovalue(A) marketing marketing marketing planning library

Queries on Resolved Attribute/Relationship Values Once an integrated database is derived, OOQLRA allows end users to query only the resolved attribute and relationship values in the integrated database while hiding the conflicts from the users. Example (Q3): SELECT E.ename.rvalue,E.work in.rvalue.dname.rvalue, E.work in.rvalue.budget.rvalue FROM EMP E; Table 5. Query Result of Q3 ename.rvalue john kim chen mark . . .

E.work in.rvalue.dname.rvalue marketing marketing marketing planning . . .

E.work in.rvalue.budget.rvalue 2M 2M 2M 4M . . .

Evolution of Local Databases When an integrated database is first derived during the derivation phase, all conflicts between local instances may be fully resolved. As the local database evolves, new records are added to the databases, some old ones are removed, and other old ones get updated. These local changes may lead to un-anticipated conflict(s) in the integrated database. In this case, queries similar to Q1 can be used to identify unresolved attribute and relationship conflicts. Example (Q4): SELECT E.oid,E.ename,E.position,E.salary,E.qual,E.work in FROM EMP E WHERE E.ename.conflictType=Intolerable OR E.position.conflictType=Intolerable OR

A Global Object Model for Accommodating Instance Heterogeneities

447

E.salary.conflictType=Intolerable OR E.qual.conflictType=Intolerable OR E.work in.conflictType=Intolerable; Table 6. Query Result of Q4 oid ename e1 (john,A)(john,B) john NULL e3 (chen,A)(chen,B) chen NULL

position (trainee,A)(trainee,B) trainee NULL (engineer,A)(leader,B) NULL Intolerable

salary (1000,A)(1200,B) NULL Intolerable (2500,A)(2600,B) 2600 Resolvable

qual (diploma,A) diploma NULL (MEng,A) MEng NULL

work in (d1,A)(d4,B) d1 Resolvable (d1,A)(d1,B) d1 NULL

To resolve the identified conflicts, one has to examine the cause of conflicts. If the conflicts are due to flaws in the derivation of integration database, we can define attribute threshold predicates and resolution functions using the DEFINE statements. Otherwise, the conflicts may be caused by erroneous information introduced to some local database, e.g. typographical errors made during data entry.

6

Conclusions

This research introduces a novel approach to examine the database integration process. To support the query and integration activities in all phases of database integration, we believe that some amount of instance-level conflicts have to be accommodated by the integrated databases. Furthermore, not all instance conflicts can always be resolved during database integration. This paper examines the impact of instance conflicts on global data model. The concept of threshold predicate and resolution function have been adopted to handle both attribute and relationship conflicts. An extended objectoriented data model called OORA has been proposed to accommodate attribute and relationship conflicts. Its query language OOQLRA and some query examples were given. This research can be seen as an initial effort to systematically devise different solutions to resolve as well as to accommodate instance heterogeneity in the integrated databases. This is in contrast to past database integration research which often emphasized on conflict resolution only.

References 1. C. Batini, M. Lenzerini and S.B. Navathe. A Comparative Analysis of Methodologies for Database Schema Integration. ACM Computing Survey, 18(4), December 1986. 2. M. Garcia-Solaco, F. Saltor and M. Castellanos. Semantic Heterogeneity in Multidatabase Systems, chapter 5, pages 129–202. Prentice Hall, 1996. 3. W. Kim and J. Seo. Classifying Schematic and Data Heterogeneity in Multidatabase Systems. IEEE Computer, December, 1991. 4. J.A. Larson, S.B. Navathe and R. Elmasri. A Theory of Attribute Equivalence in Databases with Application to Schema Integration. IEEE Transactions in Software Engineering, 15(4), April 1989.

448

E.-P. Lim and R.H.L. Chiang

5. E.-P. Lim, R.H.L. Chiang and Y.Y. Cao. Tuple Source Relational Model: A SourceAware Data Model for Multidatabases. in Data & Knowledge Engineering, Forthcoming, 1999. 6. E.-P. Lim and R.H.L. Chiang. Resolving instance-level relationship conflicts in database integration. In Workshop on Information Technologies and Systems (WITS’97), Atlanta, Georgia, December 1997. 7. E.-P. Lim, J. Srivastava, S. Prabhakar, and J. Richardson. Entity Identification Problem in Database Integration. In Proceedings of IEEE Data Engineering Conference, Vienna, Austria, 1993. 8. E.-P. Lim, J. Srivastava, and S. Shekhar. Resolving Attribute Incompatibility in Database Integration: An Evidential Reasoning Approach. In IEEE International Conference on Data Engineering, Houston, TX, February 1994. 9. P. Scheuermann, and E.I. Chong. Role-based Query Processing in Multidatabase Systems. In International Conference on Extending Database Technology, Cambrige, U.K., March 1994. 10. A.P. Sheth and J.A. Larson. Federated database systems for managing distributed heterogeneous, and autonomous databases. ACM Computing Surveys, 22(3), September 1990. 11. S. Spaccapietra, C. Parent and Y. Dupont. Model Independent Assertions for Integration of Heterogeneous Schemas. Very Large Database Journal, 1(1), 1992. 12. Y.R. Wang and S.E. Madnick. The Inter-database Instance Identification Problem in Integrating Autonomous Systems. In IEEE International Conference on Data Engineering, 1989. 13. R. Wang and S. Madnick. A Polygen Model for Heterogeneous Database Systems: The Source Tagging Perspective. In International Conference on Very Large Data Bases, pages 519–538, Brisbane,Australia, 1990. 14. M.W.W. Vermeer and P.M.G. Apers. On the applicability of schema integration techniques to database interoperation. In Entity-Relationship Conference, Cottbus,Germany, 1996.

On Formalizing the UML Object Constraint Language OCL Mark Richters and Martin Gogolla University of Bremen, FB 3, Computer Science Department Postfach 330440, D-28334 Bremen, Germany {mr|gogolla}@informatik.uni-bremen.de, WWW home page: http://www.db.informatik.uni-bremen.de

Abstract. We present a formal semantics for the Object Constraint Language (OCL) which is part of the Unified Modeling Language (UML) – an emerging standard language and notation for object-oriented analysis and design. In context of information systems modeling, UML class diagrams can be utilized for describing the overall structure, whereas additional integrity constraints and queries are specified with OCL expressions. By using OCL, constraints and queries can be specified in a formal yet comprehensible way. However, the OCL itself is currently defined only in a semi-formal way. Thus the semantics of constraints is in general not precisely defined. Our approach gives precise meaning to OCL concepts and to some central aspects of UML class models. A formal semantics facilitates verification, validation and simulation of models and helps to improve the quality of models and software designs.

1

Introduction

Methods for object-oriented analysis and design have brought a great number of languages and notations supporting many aspects of the software specification process. The Unified Modeling Language (UML) is the result of an effort in developing a single standardized language for object-oriented modeling [22]. It integrates and enhances concepts from many analysis and design methods, including Booch [4], OMT [25], and OOSE [16]. Part of the latest UML proposal is the Object Constraint Language (OCL) [24]. This language is intended to facilitate specification of model properties in a formal yet comprehensible way [28]. Previously, UML support for describing constraints was limited to annotations in form of uninterpreted textual comments. The introduction of a formal constraint language therefore is an important step towards proper formalization of complex models. However, the OCL itself is mainly defined in a semi-formal way by using English text descriptions, a context-free grammar specifying the concrete syntax of OCL and many examples illustrating the intended meaning of expressions. While this style of presentation is perfectly well-suited to introduce and demonstrate the concepts of OCL, we argue that a thorough understanding of OCL semantics requires a formal definition. The authors themselves emphasize that T.W. Ling, S. Ram, and M.L. Lee (Eds.): ER’98, LNCS 1507, pp. 449–464, 1998. c Springer-Verlag Berlin Heidelberg 1998

450

M. Richters and M. Gogolla

currently no complete formal semantics exists [18]. Due to this semi-formal nature of the OCL definition some issues have already been identified that may lead to problems or open questions [14,15]. As a simple example, we present an OCL expression where the result cannot be unambiguously determined. Consider a query which builds a list containing the names of persons working at a car rental station. We could use the following OCL expression for this purpose. (1)

rs :RentalStation rs.employee->iterate(p :Person; names :String = "" | names.concat(p.lastname))

The expression iterates over the elements in a set of persons (rs.employee) and adds the last name of each person in this set to an initially empty string. The result will be a string containing all last names of a rental station’s employees. One problem here is that there is no statement in OCL about the order of evaluation, hence evaluations may yield different results caused by different iteration sequences. This shows that important aspects of the iterate expression are underspecified, even worse, since most operations on collections are defined in terms of iterate expressions their behavior is also not precisely defined. In our paper we will present a solution to this problem by defining a deterministic evaluation semantics and by identifying necessary preconditions for application of iterate expressions. We present a formalization of OCL and certain aspects of UML class models. To this end, we define syntax and semantics of OCL expressions and give examples for their evaluation. It is our goal to keep as close as possible with the intended semantics of OCL. However, where the intended semantics is not obvious or appears problematic we do not refrain from suggesting an alternative approach. In our view a formalization of the OCL is beneficial for achieving the following goals: (1) Improvement of the OCL itself by pointing out some potential problems with its current definition, (2) a more precise understanding of UML class models and their interpretation, (3) a solid foundation for implementing CASE tools supporting simulation and validation of UML models, and (4) integration of UML with classical approaches for conceptual modeling of information systems. Our work is related to recent approaches handling formal aspects of UML and other object-oriented methods. Work has been done on the basis of wellestablished traditional approaches to specification like Z and VDM: [26,11,12, 10] focuses on the UML type system and class models, a general integration of Z with object-orientation is discussed in [9], and in [19,3] an object calculus enhancing the expressibility of object-oriented notations has been proposed. Other approaches treat in detail the UML predecessor OMT [2,27], and in particular class diagrams [5] in connection with Larch are discussed. [6] sketches a general scenario for most UML diagrams without going into technical details, and [21] presents a general framework for relationships and Use Cases in UML. In the information systems field we are particularly interested in data models and their query languages. We see a strong resemblance between the UML

On Formalizing the UML Object Constraint Language OCL

451

language and notation for class diagrams and traditional Entity-Relationship approaches, e.g., [8,13]. An example comparing a UML class diagram with an EER diagram and integrity constraints in OCL with constraints specified in the EER calculus can be found in [14]. The UML might also be appropriate for modeling object-oriented database systems according to the ODMG standard [7]. In fact, OCL seems to be not so far away from the ODMG query language OQL. The rest of the paper is organized as follows. In Sect. 2 we briefly introduce the OCL by examples and illustrate their application to UML class models for specifying constraints and queries. We start our formal framework by first developing a formal description of basic and complex types in Sect. 3. In Sect. 4 we propose a model to formalize structural properties of class models which may be referenced in OCL expressions. Syntax and semantics of expressions is then defined in Sect. 5. We close with a summary and point out problems that still need some further consideration.

2

UML and the Object Constraint Language

The primary purpose of OCL is to augment a model with additional information that often (if at all) cannot be expressed appropriately in UML. This information is given by constraints which in general are easier to specify in a textual notation than in a graphic-oriented language. The OCL provides a flexible and concise notation for specifying a broad range of constraints. For example, a simple constraint might just restrict attribute values to some subset of an otherwise fixed domain (e.g.,, stating that the age of a person must be a positive integer), or it might describe some complex relationship between several objects. The intended effect of a constraint specification is a restriction on possible system states and transitions with respect to a given model. An OCL expression is declarative in the sense that an expression says what constraint has to be maintained, not how this is accomplished. Therefore, specification of constraints is done on a conceptual level, where implementation aspects are mainly irrelevant. We only give some basic examples of constraints to illustrate the general structure of OCL expressions. Refer to [24] for a complete overview of OCL. All examples in the following sections refer to the class diagram shown in Fig. 1. The diagram is a simplified model of a car rental company having several rental stations each employing a number of persons. A company has a fleet of cars and motorcycles which are generalized into the class vehicle. We do not claim this model to be adequate for representing the real world, rather our intention here is to provide a simple example using the UML concepts of class, attribute, operation, association, and generalization. Evaluation of an expression always requires a contextual instance of a specific type. In the example (2) given below, the expression self refers to an object of type Person. Class properties like attributes (e.g.,, age) may be referenced by using a dot notation. Thus expressions generally expand from left to right. Person (2) self.age > 0

452

M. Richters and M. Gogolla Company

Vehicle registration :String numWheels :Integer

Owns 0..*

name :String 0..1 /numEmployees :Integer

RentalStation 1

1..*

location :String

stockPrice() :Real

Car category :enum{compact, midsize, luxury}

Motorcycle numSaddles :Integer cc :Integer

managed 0..1 Station

0..1 employer

manager 1

0..* employee

Person firstname :String lastname :String age :Integer isMarried :Boolean income(d :Date) :Integer

Fig. 1. UML class diagram

The term “property” is also used for referring to association ends (i.e.,, the role an object plays in an association, e.g.,, a person is an employee in a rental station) and side effect-free operations (e.g.,, income). The next expression (3) ensures that all employees of a station have a minimum income at a given date. forAll is a predefined OCL operation testing whether a predicate holds for each member of a collection (a set of employees in this case). (3)

RentalStation self.employee->forAll(p | p.income("98/03/01") > 2000)

The OCL is also suitable for specifying queries. In fact, constraints can be considered a special kind of queries which are required to have a boolean result type whereas a query may have any result type. The expression (4) selects for a given company all vehicles which have a registration number starting with the string "R24". Due to subtyping rules, the result type Set(Vehicle) is capable of holding both car and motorcycle instances. The subexpression vehicle refers to the left end of the association Owns and provides the set of all vehicles owned by a company which is then further restricted by the select expression. (4)

3

Company self.vehicle->select(v | v.registration.substring(1,3) = "R24")

Types

We start developing our formal definition of the Object Constraint Language by first looking at concepts for representing types and associated operations. In a way quite similar to algebraic specification, we use a signature to describe

On Formalizing the UML Object Constraint Language OCL

453

sorts and operation symbols. After defining basic sorts we proceed with sort constructors and subsort relations. Figure 2 gives an overview of types available in OCL. Arrows indicate a subtype relationship between types. Further types introduced by UML model elements will be considered in Sect. 4. OclAny

Real Boolean String Enumeration Collection OclType OclExpression <Model Element>

Integer

Set Sequence Bag

Fig. 2. OCL Type Hierarchy

3.1

Basic Types

Basic data types defined in OCL are Boolean, Integer , Real , and String. These types are independent of any particular UML model. In our approach, the syntax of basic types and their properties are described by a data signature ΣD = (SD , ΩD ) with SD being a set of data sorts, and ΩD a set of operations. The signature includes at least the following simple sorts as required by the OCL definition of basic types: {Boolean, Integer , Real , String} ⊆ SD . Interpretation of basic sorts is as usual but we extend each set with a special value ⊥ denoting the undefined value. A sort s ∈ SD is mapped to a semantic domain Ds by a function I : s 7→ Ds as follows (A∗ is the set of all sequences over a finite alphabet A): I(Integer ) = Z ∪ {⊥}, I(Real ) = R ∪ {⊥}, I(Boolean) = {true, false} ∪ {⊥}, I(String) = A∗ ∪ {⊥}. Enumeration types used in a UML model extend the set of basic types. Their domain is given by an enumeration of literal values. For example, the enumeration enum{compact,midsize,luxury} in Fig. 1 can be assigned a type CategoryKind which is interpreted as I(CategoryKind ) = {compact, midsize, luxury} ∪ {⊥}. A set of operations is defined on basic sorts. Note that some operations have the same (overloaded) name symbol and can only be distinguished by also looking at their argument sorts. {∨, ∧ : Boolean × Boolean → Boolean, ¬ : Boolean → Boolean, . . . } ∪ {+, −, ∗, /, div, mod, max : Integer × Integer → Integer , . . . } ∪ {<, ≤, ≥, >: Integer × Integer → Boolean, . . . } ∪ {+, −, ∗, /, max, min : Real × Real → Real , floor : Real → Integer , . . . } ∪ {<, ≤, ≥, >: Real × Real → Boolean, . . . } ∪ {concat : String × String → String, size : String → Integer , . . . } ⊆ ΩD .

454

M. Richters and M. Gogolla

We only give two examples for interpretations of operations. In OCL, care has to be taken when passing undefined values as arguments. In general, all operations have a strict evaluation semantics. If at least one argument is undefined, then the whole operation yields the undefined value as a result. Exceptions of this rule are the boolean operations ∧ and ∨. False and-ed with anything yields false, and true or-ed with anything yields true. ( i1 + i2 , if i1 6= ⊥ and i2 6= ⊥ I(+)(i1 , i2 ) = ⊥, otherwise   true, if b1 = true or b2 = true I(∨)(b1 , b2 ) = false, if b1 = false and b2 = false   ⊥, otherwise The OCL rules for undefined values lead to a rather unexpected behavior of exists and forAll expressions. If one element of a collection is undefined then the result of the whole expression will be undefined. However in case of forAll we would expect a result of false, and in case of exists, there well might be some other element making the whole expression true. Thus a more natural behavior could be achieved by requiring all operations with boolean result to yield false in case of undefined arguments. 3.2

Complex Types

In OCL multi-valued expressions are described by collection types. Sets, bags, and lists can be constructed by providing a type parameter for their element type. For example, the type Sequence(Integer) denotes list values with integer elements. However, collection types may not be nested, therefore the explicit construction of more complex types is not possible in OCL. When an evaluation yields a value with deeper nesting, a flattening process “automatically” transforms this value to a set, bag or sequence containing only basic data values. We lift this restriction for the sake of orthogonality by allowing an arbitrary number of constructor applications. Complex structures can not be avoided in non-trivial models. For example, the expression c.rentalStation->collect(employee) with c being an object of class Company selects all rental stations of a company together with their employees. The result type should be Bag(Set(Person)). However, according to the OCL definition the result gets automatically flattened into a value of type Bag(Person). How this flattening works is described only by means of the following example [24]: Set{ Set{1,2}, Set{3,4}, Set{5,6} } ⇒ Set{ 1, 2, 3, 4, 5, 6 } The naive rule of removing all inner collections and just repeating the elements in order of their appearance in the source expression seems to suffice in this example, but we can easily construct a case where it does not produce a well-defined result. Consider the expression Sequence{Set{p1,p2}} with

On Formalizing the UML Object Constraint Language OCL

455

p1,p2 being variables holding objects of type Person. Is the result of flattening Sequence{p1,p2} or Sequence{p2,p1}? The problem here is that mapping a set into a sequence requires an order on the set elements. For achieving a deterministic flattening semantics, and therefore equal results in the previous example, one would have to induce an order on the set elements, which is not a priori defined on all types. The requirement of generally having an order on all types seems rather strong. We therefore suggest to keep the structure of evaluation results. Where flattening of nested structures is desirable, explicit transformations have to be defined and used. Thus, the following definition generalizes the idea of nested data structures by the notion of sort expressions that can be recursively applied to construct complex structures. Definition 1 (Syntax of Sort Expressions) Let S be a set of sort symbols. The set of sort expressions SExpr (S) is defined as follows: i. ii. iii. iv.

If s ∈ S then s ∈ SExpr (S). If s ∈ SExpr (S) then Set(s), Sequence(s), Bag(s) ∈ SExpr (S). If s ∈ SExpr (S) then Collection(s) ∈ SExpr (S). Any ∈ SExpr (S).

The initial set of sort expressions (i) is given by the set of basic sorts. For now, we consider only the data sorts in SD as defined in the previous section. Later, in Sect. 4, we will extend this set by classifier types introduced by UML class models. Complex structures can be modeled by using sort constructors Set, Sequence, and Bag (ii). A Collection type (iii) is not strictly necessary since it is defined in OCL as an abstract super type of Set, Sequence, and Bag. Thus from a semantic point of view, there will be no values having the most specific type Collection. However, it is convenient to define operations common to type Set, Sequence, and Bag only once by means of an abstract super type. OclAny (or just Any) is the top of the type hierarchy in OCL (iv). All other types are either direct or indirect subtypes of Any. Sort expressions are interpreted as sets of possible values. In the following definition, F(I(s)) denotes the set of all finite subsets of I(s), and B(I(s)) is the set of all finite multisets over I(s). Definition 2 (Semantics of Sort Expressions) The semantics of sort expressions s ∈ SExpr (S) is defined as follows: i. I(s) is defined in Sect. 3.1. ii. I(Set(s)) = F(I(s)) ∪ {⊥}, I(Sequence(s)) = (I(s))∗ ∪ {⊥}, I(Bag(s)) = B(I(s)) ∪ {⊥}. iii. I(Collection(s)) = I(Set(s)) ∪ I(Sequence(s)) ∪ I(Bag(s)). S iv. I(Any) = s∈SExpr (S)\{Any} I(s). A set of operations ΩSExpr(S) is defined on sort expressions. These operations can be used to express functions (also called features in OCL) over collection types. Some examples are listed in the following table.

456

M. Richters and M. Gogolla

OCL Feature Operation set->union(set2 :Set(T)) :Set(T) union : Set(s) × Set(s) → Set(s) bag->including(object :T) :Bag(T) including : Bag(s) × s → Bag(s) collection->size :Integer size : Set(s) → Integer size : Sequence(s) → Integer size : Bag(s) → Integer The left column displays signatures of features as they are defined in [24]. The left-hand side of an arrow1 denotes an instance of the indicated type. A corresponding operation in our language is given in the right column for each case. Inherited features on type Collection (e.g.,, size) require specialized operations for each subtype. The appropriate operation will be selected by determining whether the collection expression results either in a set, bag, or sequence value. A further group of operations serves as constructor functions for representing literals in OCL. For example, a literal Set{2,3} will be expressed as an operation MkSet(2, 3). Features like exists, forAll, etc. are operations with an argument of type OclExpression. We do not support this type since it mixes different levels of abstraction [14]. Instead we define these operations as part of the expression syntax in Sect. 5. In this section, we introduced sort expressions with associated operations. By combining them with our initial data signature we obtain an extended signature Σ = (SExpr (SD ), ΩSExpr (SD ) ∪ ΩD ).

4

UML Class Models

The relationship between the UML metamodel and OCL is informally explained in [24]. Since components of a UML model form the basic building blocks for constraint expressions, we emphasize the need for a precise interface. The following features of a UML class model are accessible in OCL expressions: – Classifiers: Each Classifier2 introduces a distinct type (e.g.,, Person, Company, etc.). Class types are probably the most frequently used types in OCL expressions. Further kinds of Classifiers in the core model are Datatypes and Interfaces. Since basic data types are already included in our data signature we do not consider them here again. Furthermore the subtype hierarchy induced on classifiers by generalization/specialization is utilized to employ a kind of inclusion polymorphism in OCL. – Attributes: Structural properties of classifiers are described by attributes (e.g.,, name, location, isMarried, etc.). – Associations: An association connects two or more classifiers (not necessarily being distinct). Complex expressions referring to several class types can be constructed by navigation along these associations (e.g.,, “retrieve 1 2

An arrow indicates a collection expression whereas a dot generally is used for singlevalued expressions. Terms set in a serif-less font refer to elements of the UML metamodel (see [23]).

On Formalizing the UML Object Constraint Language OCL

457

all persons working as employees for a company”). For navigation purposes, the association ends are uniquely identified by role names. Aggregational and compositional relationships can be treated uniformly like associations in OCL. – Operations: Operations and methods that are marked as being side effect-free (i.e.,, they have the property isQuery) may be used in an OCL expression (e.g.,, p.income(date) returns the income of a person p at a specific date). – Enumerations: Enumeration types extend the set of basic types (see Sect. 3.1). With regard to the formalization of OCL it is sufficient to consider only the previously listed items. These properties all describe static aspects of a UML model that do not change over time. For the specification of temporal aspects OCL provides pre- and postconditions. Since these are primarily used to specify effects of method definitions, we do not consider them here. Let Aattr be a set of attribute names and Arole a set of role names. We define a UML class model as a structure M = (C, ≺, Attc , Opc , Assocc ) with the following meaning: – C is a set of classifiers. – ≺ is a partial order on C reflecting the generalization hierarchy of classifiers. – Attc is a set of operation signatures ac : c 7→s for functions mapping an object of type c to an associated attribute value of sort s. The attribute symbol ac is an element of Aattr . – Opc is a set of signatures ωc : c × s1 . . . sn 7→s for side effect-free operations defined for a classifier c having result sort s. The first argument denotes the instance to which a function is applied (sometimes called the receiver of a message). – Assocc is a set of operation signatures r : c 7→s for functions returning instances associated with an object of type c. The operation symbol r ∈ Arole is chosen according to the unique role name assigned to an association end. Depending on the multiplicity of an association, the result sort s is either c0 for single-valued associations, Set(c’) for multi-valued associations (e.g.,, having a multiplicity of 0..*), or Sequence(c’) for ordered associations. Example: A model MCompany for the class diagram in Fig. 1 can be defined as follows. C = { Vehicle, Car , Motorcycle, Company, RentalStation, Person} ≺ = { (Car , Vehicle), (Motorcycle, Vehicle)} Attc = { nameCompany : Company → String, isMarriedPerson : Person → Boolean, . . . } Opc = { stockPriceCompany : Company → Real , incomePerson : Person × Date → Integer } Assocc = { employee : RentalStation → Set(Person), manager : RentalStation → Person, . . . } With each classifier c ∈ C we associate a sort having the same name. A possible interpretation of a classifier sort is a set of object identifiers denoting

458

M. Richters and M. Gogolla

the currently existing object instances in a system state. In order to guarantee safe expressions, we postulate the set of objects in any system state being finite. We will consider the interpretation of a model in more detail in the next section. 4.1

Subsort Hierarchy

The OCL supports polymorphism by a subtype hierarchy defined on basic and complex types. This hierarchy is established by a subtype relation on the basic types Integer and Real , and a transitive subtype relation on collection types. Furthermore, it is extended by generalization hierarchies in UML models. We formalize this subtype hierarchy in our language by a subsort relation defined on sorts. Definition 3 (Subsort Relation) Let s, s0 ∈ S and c, c0 ∈ C with C ⊂ S. A subsort relation ≤ is defined on S by the following rules: i. ii. iii. iv.

s ≤ Any. Integer ≤ Real . If c0 ≺ c then c0 ≤ c. If s0 ≤ s then Set(s0 ) ≤ Set(s), Sequence(s0 ) ≤ Sequence(s), and Bag(s0 ) ≤ Bag(s). v. If s0 ∈ {Set(s), Sequence(s), Bag(s)} then s0 ≤ Collection(s).

The semantics of the subsort relation is given by a subset condition. We require that for each two sorts s0 , s ∈ S with s0 ≤ s the domain of the specialized sort s0 is a subset of the more general sort s, i.e., I(s0 ) ⊆ I(s). 4.2

System State

A UML model describes the static structure and dynamic behavior of a system. With respect to our discussion of OCL, we do not need to consider evolution of a system state over time. In general, we can concentrate on a system state given at a discrete point in time. This system state provides a context for the evaluation of OCL expressions. The set of existing object instances, their attribute values, and associations among instances constitute the state of a model. In this section, we present a simple model for objects, attributes, and navigational expressions by giving an interpretation for a model M. For each classifier c ∈ C there is an infinitely large set of unique object identifiers I(c) = {c1 , c2 , . . . } ∪ {⊥}. Subsorts imply a subset condition on object identifiers: c ≤ c0 ⇒ I(c) ⊆ I(c0 ). The interpretation of classifiers is pairwise disjoint if two classifiers are not related by subtyping: ∀c, c0 ∈ C : ¬(c ≤ c0 ∨ c0 ≤ c) ⇒ I(c) ∩ I(c0 ) = {⊥}. We describe a model state σM at a given point t in time by a hypergraph G, i.e., σM (t) = G. Nodes of the graph represent objects whereas edges represent associations among objects. An edge represents a link in UML terminology, i.e., an instantiated association. Since associations may be n-ary, edges may connect

On Formalizing the UML Object Constraint Language OCL

459

any number of nodes in the resulting graph, thus requiring the concept of hyperedges. Nodes and edges are labeled with additional information. Each object node carries information about attribute values, and each edge is labeled with information about role names. Definition 4 (System State) Let M be a class model. A system state σM (t) is a labeled hypergraph G = (V, E, nodes, roles, ostate) defined as follows: S i. V ⊂ c∈C I(c) is a finite set of nodes (object identifiers). ii. E is a set ( of edges (association links) connecting nodes. E →V∗ iii. nodes : is a function connecting edges with nodes. e 7→(v1 , . . . , vn ) ( E → A∗role iv. roles : is a function labeling each edge with a tuple of e 7→(r1 , . . . , rn ) role names. ( S V → {f | f : Aattr → s∈S I(s)} is a function labeling each v. ostate : v 7→ {(a1 , s1 ), . . . , (an , sn )} node with a set of attribute/value mappings. Figure 3 shows a UML object diagram representing part of a system state for the car rental model. The graph in the diagram has nodes for a rental station object rs1 and two persons p1 and p2 . A set of attribute names and values is attached to each node, respectively. Edges indicate links between the station and persons. Note that edge labeling by role names enables us to distinguish the manager and employee association between rs1 and p1 . In cases like this, where the same set of objects are linked by multiple edges, it is necessary that each edge can be unambiguously assigned to a distinct association. We identify associations with unique combinations of role names and require: ∀e1 , e2 : nodes(e1 ) = nodes(e2 ) =⇒ roles(e1 ) 6= roles(e2 ). manager managedStation rs1 :RentalStation location = "NY/Airport"

p1 :Person

lastname = "Clark" employee age = 47

employer p2 :Person employer

lastname = "Barnes" employee age = 23

Fig. 3. Object Diagram representing a System State for MCompany

The definition of system state is a prerequisite for fixing the meaning of expressions referring to attributes and associations. These features are operations

460

M. Richters and M. Gogolla

defined as part of a model M. Attribute operations can be interpreted as follows: An operation ac : c → s in Attc with ac ∈ Aattr , c ∈ C, s ∈ S is a function mapping objects to attribute values. For a system state with graph G the semantics of ac is given as ( ostate(c)(ac ), if c ∈ V and ac ∈ dom(ostate(c)) I(ac )(c) = ⊥, otherwise. The attribute age of a person object, for example, is given by I(agePerson )(p1 ) = ostate(p1 )(agePerson ) = 47. Finally, association operations allow the construction of navigation expressions. An n-ary association induces n · (n − 1) operations allowing navigation among participating classes in each direction. Association ends have to be labeled by role names for this purpose. An operation r : c → s in Assocc with r ∈ Arole , c, c0 ∈ C, s ∈ {c0 , Set(c’), Sequence(c’)} maps an object of sort c either to a single object, or to a set or sequence of associated objects of sort c0 .  0  I(c) → F(I(c )) I(r ) : c 7→ {c0 | e ∈ E ∧ ∃i, j : πi (nodes(e)) = c ∧ πj (nodes(e)) = c0   ∧πj (roles(e)) = r} For the result, all edges being incident on the source object and target objects are considered (πi is a function projecting a tuple to its ith element). Since there may be multiple edges (representing different associations) between the same set of objects, the edge leading to a target object must be labeled with the role name given as operation symbol. Thus for the example in Fig. 3, the interpretation of association operations yields the following results: I(employee )(rs1 ) = {p1 , p2 } I(manager )(rs1 ) = {p1 } I(employer )(p1 ) = {rs1 } I(employer )(p2 ) = {rs1 } I(managedStation )(p1 ) = {rs1 }

5

Expressions

We now define the expression syntax and its semantics for constraints and queries. First we give a complete syntax of expressions before we explain semantics and application to OCL in detail. The definition of expressions is based upon the signature we developed in the previous sections. A signature ΣM = (SM , ≤, ΩM ) provides a set of sorts SM = SExpr (SD ∪ C) containing basic sorts, class sorts, and complex sorts constructed by sort expressions. A partial order ≤ on sorts reflects subtyping. On these sorts, a set of operations ΩM = ΩD ∪ Attc ∪ Opc ∪ Assocc is defined. The signature contains the initial set of syntactic elements and is basically the same as used in order-sorted algebraic specification. Based upon the signature, we define expressions.

On Formalizing the UML Object Constraint Language OCL

461

An OCL expression is always evaluated in context of a classifier instance. Note that we distinguish between basic sorts and classifier sorts. Hence, in our framework expressions cannot be built in context of simple data values like integers. Definition 5 (Syntax of Expressions) Let Var = {Vars }s∈S be a set of variables indexed by sort s. The syntax of expressions over a classifier c ∈ C is a set Expr(c) = {Expr(c)s }s∈S indexed by sort which is defined as follows: i. self ∈ Expr(c)c . ii. v ∈ Expr(c)s , if v ∈ Vars . iii. ω(τ1 , . . . , τn ) ∈ Expr(c)s , if ω : s1 × · · · × sn → s ∈ Ω and τi ∈ Expr(c)s0i , where s0i ≤ si for i = 1, . . . ,(n. Expr(c)s1 , if s2 ≤ s1 iv. if ϕ then τ1 else τ2 fi ∈ Expr(c)s2 , if s1 ≤ s2 if ϕ ∈ Expr(c)Boolean and τ1 ∈ Expr(c)s1 , τ2 ∈ Expr(c)s2 . v. (τ as s0 ) ∈ Expr(c)s0 , if τ ∈ Expr(c)s and s0 ≤ s. vi. iterate(v1 : τ1 , v2 : τ2 , τ3 ) ∈ Expr(c)s2 , if v1 ∈ Vars1 , v2 ∈ Vars2 , τ2 , τ3 ∈ Expr(c)s2 , and τ1 ∈ Expr(c)Sequence(s1 ) ∪ Expr(c)Set(s1 ) ∪ Expr(c)Bag(s1 ) . A self expression (i) provides a reference to a classifier the instance of which giving a context for evaluation. Variable bindings (ii) are introduced in features which are based on the iterate abstraction (select, exists, etc.) [24]. Operation expressions (iii) include predefined operations for e.g., integer arithmetic as well as for accessing object attributes and associations. Alternatively, an operation expression may also be written in OCL path syntax as τ1 .ω(τ2 , . . . τn ). If τ1 denotes a collection value the dot may be replaced by an arrow symbol: τ1 → ω(τ2 , . . . τn ). An if expression (iv) provides an alternative selection of two expressions. An as expression (v) can be used in cases where static type information is insufficient. It corresponds to the oclAsType feature in OCL. The iterate feature of OCL is captured by iterate expressions (vi). A large group of query constructs can be reduced to iterate expressions [24]. The semantics of each kind of expression is fixed in the following definition. A context for evaluation is given by a classifier instance c ∈ I(c) and a variable assignment β : Vars → I(s). Definition 6 (Semantics of Expressions) Let the set of all variable assignments be B. The semantics of an expression e ∈ Expr(c)s is a function I[[ e ]] : B × C → I(s) defined by induction on the structure of e: i. I[[ self ]](β, c) = c. ii. I[[ v ]](β, c) = β(v). iii. I[[ ω(τ1 , . . . , τn ) ]](β, c) = I(ω)(I[[ τ 1 ]](β, c), . . . , I[[ τn ]](β, c)).  I[[ τ1 ]](β, c), if I[[ ϕ ]](β, c) = true iv. I[[ if ϕ then τ1 else τ2 fi ]](β, c) = I[[ τ2 ]](β, c), if I[[ ϕ ]](β, c) = false   ⊥, otherwise.

462

M. Richters and M. Gogolla

( I[[ τ ]](β, c), if I[[ τ ]](β, c) ∈ I(s0 ) v. I[[ (τ as s ) ]](β, c) = ⊥, otherwise. vi. I[[ iterate(v1 : τ1 , v2 : τ2 , τ3 ) ]](β, c) = I[[ iterate0 (v1 : τ1 , τ3 ) ]](β 0 , c) where β 0 is a modified assignment β 0 = β{v2 /I[[ τ2 ]](β, c)}. 0

a) ( if τ1 ∈ Expr(c)Sequence(s1 ) then I[[ iterate0 (v1 : τ1 , τ3 ) ]](β, c) = I[[ v2 ]](β, c), if I[[ τ1 ]](β, c) =<> I[[ iterate0 (v1 : tail(τ1 ), τ3 ) ]](β 00 , c), if I[[ τ1 ]](β, c) =< e1 , . . . , en > b) if τ1 ∈ Expr(c)Set(s1 ) ∪ Expr(c)Bag(s1 ) then 0 I[[ ( iterate (v1 : τ1 , τ3 ) ]](β, c) = I[[ v2 ]](β, c), if I[[ τ1 ]](β, c) = ∅ 0 I[[ iterate (v1 : τ1 − e1 , τ3 ) ]](β, c), if e1 ∈ I[[ τ1 ]](β, c) with β 00 being a modified assignment β 00 = β{v2 /I[[ τ3 ]](c, β{v1 /e1 })}. In definition vi.(a) we used the notation tail(τ1 ) for an operation that, when applied to a sequence, returns a sequence with its head element removed. We could also use the OCL operation τ1 ->subSequence(2,τ1 ->size) for this purpose. Also, in definition vi.(b) we wrote τ1 − e1 for an operation that removes the element e1 from a set or bag given by the expression τ1 . For sets the corresponding OCL feature is τ1 ->excluding(e1 ). For bags however, we can not use the equally named OCL feature since it removes all occurrences of e1 at once which is not what we want here. An iterate term effectively receives three parameters: a collection S (set, sequence or bag, given by the term τ1 ) containing the elements e1 , . . . , en , an initial value a (given by the term τ2 ), and a binary operation ⊕ (given by τ3 ). The semantics of an iterate term is then given by the expression (((a ⊕ e1 ) ⊕ e2 ) · · · ⊕ en−1 ) ⊕ en . This kind of evaluation is commonly known in functional style programming languages as fold operation. Note however, that we do not restrict application of iterate expressions to collections of type Sequence. For sets and bags we cannot assume any particular order in which elements of the collection are selected for applying ⊕. Therefore, we require that ⊕ has the property (a ⊕ xi ) ⊕ xj = (a ⊕ xj ) ⊕ xi for any two elements xi , xj ∈ S. If ⊕ is associative and commutative then this property is fulfilled. For operations where evaluation order makes a difference (e.g.,, string concatenation) a non-ordered collection first has to be transformed into a sequence before iterate can be applied.

6

Summary and Conclusion

The Object Constraint Language is an important extension to the UML language. Many kinds of constraints that cannot be expressed purely in the more graphically oriented UML notation can be elegantly stated with OCL expressions. We argued that a formal language also should have a formal semantics and presented our approach to developing a precise semantics of OCL.

On Formalizing the UML Object Constraint Language OCL

463

We identified basic syntactic structures and assigned to them a semantics based on set theory. We also tried to solve some of the problems inherent to the current OCL definition. For example, we proposed a solution to the problems involved in the implicit flattening of complex results, and we pointed out some non-obvious properties of operations which are preconditions for an unambiguous interpretation of iterate expressions. With these rules, the example given in the introduction can be classified as being illegal since the operation concat does not have the property (a concat xi ) concat xj = (a concat xj ) concat xi . To correct the query, the iteration target first must be explicitly transformed from a set to an ordered sequence. In our formalization approach, we strictly distinguish between (immutable) data types and object types having mutable state. This distinction also helps in determining expressions which are safe with respect to termination of evaluation and finite results. This is an important aspect if we want to build CASE tools supporting simulation and validation of models. Most OCL expressions refer to elements in concrete UML class models. This requires a well-defined interface to class models and a precise understanding of the semantics of classes, associations, attributes, and generalization. As far as we know, a comprehensive and precise UML semantics is still not available. Therefore we had to introduce a concept for formally representing class models and their intended meaning by the notion of system state. Future work could extend this framework of formalization to other aspects of UML. Acknowledgments. We thank the anonymous referees for their helpful comments.

References 1. M. Aksit and S. Matsuoka, editors. ECOOP’97 — Object-Oriented Programming: 11th European Conference, Jyvaskyla, Finland, June 9–13, 1997: Proceedings, LNCS 1241. Springer-Verlag, 1997. 2. E. Bertino, D. Castelli, and F. Vitale. A Formal Representation for State Diagrams in the OMT Methodology. In K. G. Jeffery, J. Kral, and M. Bartosek, editors, Proc. Seminar Theory and Practice of Informatics (SOFSEM’96), LNCS 1175, pages 327–341. Springer-Verlag, 1996. 3. J. C. Bicarregui, K. Lano, and T. S. E. Maibaum. Objects, Associations and Subsystems: A Hierarchical Approach to Encapsulation. In Aksit and Matsuoka [1], pages 324–343. 4. G. Booch. Object-Oriented Analysis and Design with Applications. Benjamin/Cummings, 1994. 5. R. Bourdeau and B. Cheng. A Formal Semantics for Object Model Diagrams. IEEE Transactions on Software Engineering, 21(10):799–821, 1995. 6. R. Breu, U. Hinkel, C. Hofmann, C. Klein, B. Paech, B. Rumpe, and V. Thurner. Towards a Formalization of the Unified Modeling Language. In Aksit and Matsuoka [1], pages 344–366. 7. R. G. G. Cattell, editor. The Object Database Standard: ODMG 2.0. Morgan Kaufmann Publishers, Inc., 1997.

464

M. Richters and M. Gogolla

8. P. P. Chen. The Entity-Relationship Model – Toward a Unified View of Data. ACM Trans. on Database Systems, 1(1):9–36, 1976. 9. J. Ebert and R. S¨ uttenbach. Integration of Z-Based Semantics of OO-Notations. In Kilov and Rumpe [17], pages 75–81. 10. A. Evans, R. France, K. Lano, and B. Rumpe. Developing the UML as a Formal Modelling Language. In Muller and B´ezivin [20], pages 297–307. 11. R. France, J. M. Bruel, M. Larrondo-Petrie, and M. Shroff. Exploring the Semantics of UML type structures with Z. In H. Bowman and J. Derrick, editors, Proc. 2nd IFIP Conf. Formal Methods for Open Object-Based Distributed Systems (FMOODS’97), pages 247–260. Chapman and Hall, London, 1997. 12. R. France, A. Evans, and K. Lano. The UML as a Formal Modeling Notation. In H. Kilov, B. Rumpe, and I. Simmonds, editors, Proc. OOPSLA’97 Workshop on Object-oriented Behavioral Semantics. TU M¨ unchen, TUM-I9737, 1997. 13. M. Gogolla. An Extended Entity-Relationship Model – Fundamentals and Pragmatics. LNCS 767. Springer-Verlag, Berlin, 1994. 14. M. Gogolla and M. Richters. On Constraints and Queries in UML. In M. Schader and A. Korthaus, editors, The Unified Modeling Language – Technical Aspects and Applications. Physica-Verlag, Heidelberg, 1998. 15. A. Hamie, J. Howse, S. Kent, R. Mitchell, and F. Civello. Reflections on the OCL. In Muller and B´ezivin [20], pages 137–145. ¨ 16. I. Jacobsen, M. Christerson, P. Jonsson, and G. Overgaard. Object-Oriented Software Engineering: A Use Case Driven Approach. Addison-Wesley, 1992. 17. H. Kilov and B. Rumpe, editors. Proc. of the ECOOP’97 Workshop on Precise Semantics for Object–Oriented Modeling Techniques, Jyv¨ askyl¨ a, Finland, 10 June 1997. TU M¨ unchen, TUM-I9725, June 1997. 18. A. Kleppe, J. Warmer, and S. Cook. Informal formality? The Object Constraint Language and its application in the UML metamodel. In Muller and B´ezivin [20], pages 127–136. 19. K. Lano. Enhancing Object-Oriented Methods with Formal Notations. Theory and Practice of Object Systems, 2(4):247–268, 1996. 20. P.-A. Muller and J. B´ezivin, editors. Proc. of UML’98 International Workshop, Mulhouse, France, June 3 - 4, 1998. ¨ 21. G. Overgaard and K. Palmkvist. A Formal Approach to Use Cases and their Relationships. In Muller and B´ezivin [20], pages 309–317. 22. Rational Software Corporation, Unified Modeling Language (UML) version 1.1, http://www.rational.com, 1997. 23. UML Semantics, 1997. (Published as part of [22]). 24. Object Constraint Language Specification, 1997. (Published as part of [22]). 25. J. Rumbaugh, M. Blaha, W. Premerlani, F. Eddy, and W. Lorensen. ObjectOriented Modeling and Design. Prentice-Hall, Englewood Cliffs (NJ), 1991. 26. M. Shro and R. B. France. Towards a Formalization of UML Class Structures in Z. In Proc. 21st Annual Int. Computer Software and Applications Conference (COMPSAC’97), pages 646–651. IEEE, 1997. 27. E. Y. Wang, H. A. Richter, and B. H. C. Cheng. Formalizing and integrating the object and dynamic models within OMT. In Proc. 19th Int. Conf. on Software Engineering (ICSE’97), pages 45–55. ACM Press, 1997. 28. J. Warmer, J. Hogg, S. Cook, and B. Selic. Experience with Formal Specification of CMM and UML. In Kilov and Rumpe [17], pages 167–171.

Derived Horizontal Class Partitioning in OODBs: Design Strategies, Analytical Model, and Evaluation Ladjel Bellatreche1 , Kamalakar Karlapalem1 , and Qing Li2 1

University of Science and Technology Clear Water Bay Kowloon, Hong Kong E-mail: {ladjel, kamal}@cs.ust.hk 2 Hong Kong Polytechnic University, Hung Hom, Kowloon, Hong Kong E-mail: [email protected] Abstract. Object oriented database systems(OODBSs) have been used for advanced applications that require not only additional data modeling capabilities, but also for efficient query processing. In this paper, we apply the derived horizontal class partitioning (DHCP) technique to facilitate efficient execution of a set of queries. We apply the existing algorithms for determining horizontal class partitioning of a class for a given set of queries. Our contribution is in developing strategies for deciding the derived horizontal class partitioning of the classes, developing an analytical cost model for determining the cost of processing queries under horizontal class partitioning, and conducting experiments to evaluate the utility of DHCP in efficiently executing the queries. The experimental results show that DHCP improves the overall efficiency of query execution under different object sizes and cardinality of classes, fan-out of the classes and DHCP of all classes along the class-composition hierarchy gives the best result. Further, we have classified the queries to determine the effect of DHCP on different types of queries. In particular, we find that DHCP has detrimental affect on queries that access only one class. Thus, the results from this work can be used to design object-oriented databases to efficiently process a set of queries.

1

Introduction

In OODBSs, realizing a good performance is the key factor of their success. Most OODBSs provide high-level query language for retrieving and manipulating large number of objects for applications such as, CAM/CAD and multimedia. An important issue related to performance of OODBs concerns reducing the query processing cost [4]. Most of the work on OODBs design concentrated on developing methodologies for determining class-subclass hierarchies, class-composition hierarchies, and method definition. However, not much work has been done to design (at logical level) an OODB for efficiently executing a given set of queries. Our aim in this paper is to study the utility of horizontal class partitioning (HCP) in processing a given set of queries efficiently. In particular, we focus on the effectiveness of derived horizontal class partitioning(DHCP) in reducing the query execution cost. In contrast to indexing and clustering [15,5], HCP is a facility in T.W. Ling, S. Ram, and M.L. Lee (Eds.): ER’98, LNCS 1507, pp. 465–479, 1998. c Springer-Verlag Berlin Heidelberg 1998

466

L. Bellatreche, K. Karlapalem, and Q. Li

OODBs to reduce the number disk accesses for query execution by reducing the accesses to irrelevant objects. Moreover, partitioning is a logical database design technique whereas indexing/clustering is a physical database design. The use of indexing is complementary and orthogonal to the use of partitioning. That is, once the classes are horizontally partitioned, indices can be built to execute the queries more efficiently. Since horizontal class fragments(HCFs) (output of HCP process) are also classes, clustering can be applied to further improve efficient execution of queries. Early work has showed that the HCP is useful to reduce the number of accesses to irrelevant objects [3,9]. In this paper, we study the concept of DHCP which means that a class in the class composition hierarchy can be horizontally partitioned based on the HCFs of another class and show its utility in improving the performance of the queries. In this paper, we consider only the retrieval queries, as showing effectiveness of DHCP in case of retrieval queries is the first step in this work. That is, if DHCP is not recommended for retrieval queries, it cannot be recommended if updates are taken into consideration. Note that updates have a detrimental effect on the effectiveness of HCP due to the migration of objects. But the solution and the methodology developed in this paper does work when updates are taken into consideration after the cost model is suitably amended. Thus, handling updates will be a separate piece of work which is beyond the scope of this paper. If a class Ci in the class composition hierarchy is horizontally partitioned based on the HCFs of another class Cj 1 , such a partitioning of the class Ci is known as derived horizontal partitioning [3,9]. The classes Ci and Cj are called “owner” and “member” classes, respectively. 1.1

Motivation

Though the concept of HCP (and partitioning, in general) has been around for more than ten years, there has been very little work done in quantifying the benefit of using a partitioning technique for processing a set of queries. The main reasons for employing DHCP are: 1. Most queries access multiple objects along a path in a class composition hierarchy, and these queries are costly to process. Even though indexing and clustering can help provide good access support at physical level, the number of irrelevant objects retrieved during query processing can be high. HCP aims to reduce irrelevant object access. 2. HCP is easy to support and define [12] in OODBs. The HCFs of a class are represented as restricted subclasses (they are not specialized further) of the original classes. All the objects of a HCF satisfy a predicate clause. The advantage of HCF shows up when a query needs to access few, but not all the HCFs of a class. The HCP algorithms take into consideration the query semantics and the OODB schema to generate a HCP that does this. 3. Our early work on HCP [2] shows that HCP is not only viable, but it also improves the performance of the queries substantially. But this work only 1

There is a path from the class Ci to the class Cj with length g(g ≥ 1).

Derived Horizontal Class Partitioning in OODBs

467

concentrated on the HCP of a single class without considering the relationship among different classes, and the queries that access multiple classes. In order to address this type of queries we need to consider the concept of HCP across multiple classes. This type of HCP is known as DHCP. 4. The original concept of DHCP in relational databases was developed to efficiently process joins across multiple relations. By eliminating large subsets of irrelevant tuples from join processing, the queries could be executed more efficiently. In a similar manner, DHCP does aim to reduce the irrelevant object access for queries that access objects along a path of a class-composition hierarchy. Unlike relational databases, an added complication is the number of inter relations (or multiple paths) existing in semantically rich OODBs. This not only gives rise to additional complexity, but also provides for multiple solutions for coming up with DHCP of the classes. 5. Though DHCP procedure takes into consideration the overall processing efficiency of a set of queries, not much work has been done to identify the different types of queries and the effect of DHCP on them. In particular, DHCP may not benefit all types of queries. In this paper, we show that DHCP is very much suitable for queries that access multiple classes across paths in class-composition hierarchies, and have slight detrimental affect on queries that access only a single class. The rest of our paper is organized as follows: Sect. 2 presents the horizontal class partitioning in OODBs, Sect. 3 presents an algorithm for DHCP and the problems related in DHCP, Sect. 4 presents a cost model for query execution on horizontally partitioned classes and unpartitioned classes, Sect. 5 presents experiments results, and Sect. 6 presents the conclusions.

2

Derived Horizontal Class Partitioning

Ceri et al. [7] and Ozsu et al [17] defined derived horizontal partitioning in the relational databases as the partitioning of a relation that results from horizontal predicates being defined on another related relation. In the relational databases, the join of two or more relations is the most costly operation. On the other hand, in an OODB, an important requirement of an object oriented query language is the capability of navigating through the OODB. This capability corresponds to the notion of joins in relational language and is known as implicit join in OODBs [6,15]. There is a lot of work done to optimize object-oriented queries involving path expressions. This work involved the definition of new object algebra [18], complex object assembly [16], and cost models [4]. We present an example to give a progressive overview on how DHCP technique can reduce the cost of a query execution. Example 1. Let class Project in Fig. 1 be horizontally partitioned into two HCFs based on the attribute “Location”. Let HCF P roj1 be a HCF with all those projects that are located in “CA”, and let P roj2 be a HCF with all the projects located in “Paris”. Assume that the class Employee has been partitioned based

468

L. Bellatreche, K. Karlapalem, and Q. Li

Employee EId :N Ename : S Esalary: N Dept Own

Department DId : N Dname : S Proj Vehicle VId : N Manufacturer Color : S Model : S

Project PId Pname Duration Location

:N :S :N :S

Company CId :N Cname :S Clocation : S

Fig. 1. The class composition hierarchy of the class Employee

on the HCP of class Project into two HCFs: Emp1 and Emp2 such as Emp1 and Emp2 represent all employees working in a project located in “CA” and “LA”, respectively. Assume that we have to perform the query q1 which consists of retrieving the names of all employees working in a project located at “CA”. We note that this query accesses only the HCF P roj1 and Emp1 , and it does not need to access P roj2 and Emp2 , because these two HCFs do not contribute to the answer of q1 . The HCP of class Employee based on the HCP of class Proj, is known as DHCP of class Employee. We now present an algorithm of DHCP based on HCFs of another class using, for example, the primary algorithms developed in [3,9]. Let Ci be the class to be derived horizontally partition(ed) (DHP) based on the class Cj which is horizontally partitioned using the primary algorithm into m HCFs {M F1 , M F2 , ..., M Fm }, where every HCF M Fj (1 ≤ j ≤ m) is defined by a qualification clause clj . We notice that there is a path P from the class Ci to the class Cj where: P = Ci .A1 .A2 ...Ak , where Ak is either an attribute or a method of class Cj . By definition, a derived HCF DFj (1 ≤ j ≤ m) of the class Ci is defined as: all instances in the class Ci satisfying the qualification clause of the HCF M Fj (1 ≤ j ≤ m) of the class Cj . Example 2. We assume that the class Project in Fig. 1 has been horizontally partitioned using, for example, the primary algorithm [3] into five HCFs {P roj1 , P roj2 , ..., P roj5 } and the class Employee is to be derived partitioned based on the HCFs of class Project. Let Empj (1 ≤ j ≤ 5) be the derived HCFs of class Employee. In order to simplify example, we will define only the Emp1 . The HCF P roj1 is defined by the clause: (Duration ≤ 4) ∧ (Cost()2 > 7000) ∧ (Location = “CA”). We note that P roj1 contains three predicates p11 : Duration ≤ 4, p12 : Cost() > 7000 and p13 : Location = “CA”, then Emp1 is defined as: Emp1 = σ(E1 ∧ E2 ∧ E3 ) (Employee), where: E1 = Employee.Dept.Proj.Duration ≤ 4, E2 = Employee.Dept.Proj.Cost() > $7000) and E3 = Employee.Dept.Proj.Location = “CA”. 2

Cost() is a method defined in the class Project.

Derived Horizontal Class Partitioning in OODBs

469

Similarly, we can express other derived HCFs of Employee, i.e., Emp2 , Emp3 , Emp4 , Emp5 . Internal Representation for HCFs. Let a class member be horizontally partitioned using the primary algorithm into m HCFs, where each HCF can be represented as an isa hierarchy [14], and every HCF is specified as a class. Let us assume that the class owner has been horizontally partitioned into m HCFs. For executing a predicate defined on a path from the owner to the member classes, we do not need to traverse all classes between the owner and member classes, if we have some information to avoid the traversal of all these classes. To do that, for each derived HCF of Ci we add a reference in the form of a stored OID to its corresponding HCF in Cj as shown in Fig. 2. HCF

DHCF

1

1

{ Attributes & Methods} Instances Link: OID

Owner Class { Attributes & Methods}

Class { Attributes & Methods}

.. .

Instances

Direct Link

Instances

Member Class Attributes & Methods} Instances

Attributes & Methods} Instances

.. . HCF

m

DHCF

m

{ Attributes & Methods}

{ Attributes & Methods}

Legend:

Instances

Isa Relationship

Instances PartOf Relationship Link: OID

Direct Link

Fig. 2. Direct link for derived horizontal partitioning

2.1

Multiple Candidates for DHCP Problem

Let Ci be a class in a class composition hierarchy to be derived horizontally partitioned. We assume that there are r(r > 1) member classes (that are already horizontally partitioned). In this case, there are r possible ways to partition the owner class Ci . For example, let the classes Project and Company both be member classes, and suppose we want to perform derived horizontal partitioning of the class Employee (see Fig. 1). In this case, the question which we can ask is: how to select a member class among r classes (Company and Project classes in our example) to be a good candidate for the DHCP of the class Ci (Employee class in our example)?. Approaches for Selecting Possible Candidates for DHCP. We present here four possible candidates, namely, share candidate, frequency candidate, number of fragments candidate and fusion candidate.

470

L. Bellatreche, K. Karlapalem, and Q. Li

Share candidate: For each member class Cj (1 ≤ j ≤ r), we compute share(Ci , Cj ). The best candidate is the member class which satisfies the minimum share with the class Ci . In our example, suppose that the classes Project and Company in Fig. 1 are horizontally partitioned, and we want to DHP the class Employee. First, we calculate the share(Employee, Project) and share(Employee, Company), and after that we take the member class corresponding to the minimum share (for example Project), and then the Employee class will be DHP based on the class Project. Frequency candidate: Suppose that there are m member classes as candidates for derived horizontal partitioning of a class Ci . Note that each member class Cj (1 ≤ j ≤ r) has been horizontally partitioned based on a set of queries Qj (1 ≤ j ≤ r) 3 [3]. For selecting the member class to be used for derived horizontal partitioning, we calculate the sum of access frequencies of its queries on which it was partitioned for every member class Cj . Finally, we select the member class corresponding to the maximum frequency. Number of HCFs candidate: The choice of a member class used in the partitioning of an owner class can be a decision problem addressed during the process of allocation. In this case, the owner class can be derivedly partitioned based on the member class with the smallest number of HCFs. This solution allows us to reduce the complexity of the allocation process. Fusion Solution: Let r be the number of member classes. Let nj (1 ≤ j ≤ r) be the number of HCFs for each member class Cj . The idea of this solution is to first horizontally partition the owner class Ci into n1 ∗ n2 ∗ ... ∗ nr HCFs. The HCFs of the class Ci is accessed by a set of queries. In this case, we merge all these HCFs which are accessed by a set of queries together to reduce the number of HCFs of the class Ci . Advantages and Disadvantages of Above Approaches. The share solution gives us a good decision on the class to be DHP, but it is very difficult to estimate the value of share [4]. The frequency solution is simple to implement and can ameliorate the performance of the system, but if there are some changes in the frequency of queries, or addition/deletion of queries, then repartitioning of the class may be needed [13]. The merge solution is more costly than the above three solutions, because it requires taking into consideration n1 ∗n2 ∗...∗nr HCFs, plus the additional cost of the merge procedure, but it can generate derived HCFs that meet query requirements (best-fit DHCP). This is because the algorithm has all the knowledge about the HCFs of the member classes, as well as that of the query. Need for a Cost-Driven Solution. Note that the utility of the HCP can be measured by the amount of savings in disk accesses for query execution, and according to Karlapalem et al. [14], two factors, I/O operations and data transfer, are the most important for the performance of the applications in distributed database systems. From these assumptions, we define an analytical cost model for executing a set of queries on horizontally partitioned classes. This cost model will be used to identify the candidate among the member classes for derived horizontal partitioning of a given owner class. 3

Each query qh of Qj is executed with a certain frequency fj .

Derived Horizontal Class Partitioning in OODBs

3

471

Cost Model for Horizontal Class Partitioning

In this section, we present an analytical cost model of executing a sample set of k queries {q1 , q2 , ..., qk }, where every query may contains simple or component predicates. The cost is for executing the same queries on horizontally partitioned class(es) (i.e., we assume that there are some class(es) which are horizontally partitioned). The objective of our cost model is to calculate the cost of executing these queries, each of which accesses a set of objects. The cost of a query is directly proportional to the number of pages it accesses. The total costs are estimated in terms of disk page accesses. The total cost for executing a query is given by [2]: Total Cost = IO Cost + CPU Cost + COM Cost where IO Cost is the input/output cost for reading and writing data between main memory and disk, CPU Cost is the cost of executing CPU instructions, (for example, for evaluating predicate), and COM Cost is the cost of network communication among different nodes. In this paper, we concentrate on the IO Cost and disregard the CP U Cost and COM Cost. This is because for very large database applications with large number of data accesses, the CP U Cost contribution towards the Total Cost will not be significant, and the database we consider is centralized, therefore the COM Cost is not taken into account. We now describe in detail the input parameters and then present the cost model for unpartitioned and horizontally partitioned classes. We can classify the parameters for the cost model into three categories: database, query and HCFs. a- Database Parameters: Cardinality of a class Ci (||Ci ||) (i.e., number of objects), total number of pages occupied by the class Ci ( |Ci |), object length or size (in bytes) in the class collection Ci (LCi ), and page size of the file system(in bytes)(PS ). b- Query Parameters : Starting class of a component predicate pl (Csh ), the number of object references for class Ci during the path evaluation process along the class composition hierarchy(REFi ), number of predicates used by all queries(N ), selectivity of query qh (SELh ), the fan-out for the class composition hierarchy from class Ci−1 to Ci (Fan(Ci−1 , Ci )), selectivity of predicate pl (sell ), and the length of output result(Lproj ). c- Horizontal Class Fragments Parameters : Number of HCFs of class Ci (mi ), cardinality of a HCF M Fj (||MFj ||)(i.e., number of objects), total number of pages occupied by the HCF M Fj (|MFj |), cardinality of a HCF DFj (||DFj ||)(i.e., number of objects), total number of pages occupied by the HCF DFj (|DFj |), the number of object references for HCF Fj (of a member or an owner class) during the path evaluation process along the class composition hierarchy (refj ), selectivity of HCF Fj 4 (sel (Fj )).

4

When we use HCF means that we can apply the definition for HCF of a member or an owner class.

472

3.1

L. Bellatreche, K. Karlapalem, and Q. Li

Data Inputs for Horizontal Class Partitioning

Values for input parameters like (||Ci ||, LCi , P S, k, link, Lproj ) may be obtained in several ways [4,8], as in the case of relational databases. They can be provided by the database administrator, or can be automatically acquired by the system through data inspection or sampling. The selectivity can be estimated by using the t echniques described in [4]. The cardinality of every HCF, Fj of the class Ci is calculated as: ||Fj || = sel(Fj ) × ||Ci || for all 1 ≤ j ≤ mi . The number of HCFs of a member class can be obtained by using, for example, the algorithm described in [3]. In this cost model formulation, we consider only queries having only one component predicate. This cost model can easily be extended to consider queries with more that one component predicate, though it is quite cumbersome to put in all the details. In order to develop the cost model we use a function valid; which is has a predicate p and a HCF defined by a clause cl. The evaluation of valid function for two scenarios has been mentioned in [11]: – (p ∧ cl) is unsatisfiable, implying that the HCF defined by the clause cl will not contribute to the answer to the query. Then, the valid(p, cl) function evaluates to false. – cl implies p, implying that the whole HCF will contribute to the answer to the query. In this case, valid(p, cl) evaluates to true. Our model is adequate enough to support conjunctive queries that are the most popular and also assumed by the most of previous researchers [11]. Wei et al. [11] developed an efficient algorithm to handle the above scenarios. 3.2

Cost of Executing a Single Query on Unpartitioned Classes

In order to make it easier for the formulation of the I/O cost model for both unpartitioned and horizontally partitioned classes, we assume that there is only one query qh containing one component predicate pl . In the next subsection, we shall generalize this formulation for k queries. We assume that the starting class of component predicate as Csh and ending class as Cnh . The length of path from Csh to Cnh is nh . Our cost model is based on the same formulation as that in [10]. The cost for the query execution can be divided into 2 components: the cost of evaluating the predicate(CEP), and the cost of building the output result(CBOR) which corresponds to the number of pages to be loaded in order to evaluate a predicate, and the number of pages to be loaded to build the result, respectively. Estimation of the Number of Pages in a Class Collection. Sequential scan and index scan are the two major strategies used for scanning a class collection. The objective of using an index is to obtain faster instances access, while the objective of using horizontal class partitioning is to reduce irrelevant instances access. These two objectives are orthogonal and complementary. We concentrate

Derived Horizontal Class Partitioning in OODBs

473

on the sequential scan strategy; the use of index can also be incorporated into our model naturally as needed. We assume that the objects are smaller than the page size and do not cross page boundaries. The total number of pages occupied by a class collection C is given by: |C| = d b||C|| P S e, where d e and b c are the ceiling and floor functions respectively. c LC

The same formula defined in the above equation can be extended for a HCF F || of a member class M F as follows: |M F | = d ||M e b PS c LC

Estimation of the Number of Pages in a Derived HCF. We can not apply the same formula in the above equation without adding some changes. A derived HCF contains a direct link to its corresponding HCF of the member class (as shown in Fig. 2) which should be incorporated for estimating the number of pages e DF in a derived HCF given by the following equation: |DF | = d ||DFb P||+link S c LC

Estimation of the Number of Page Accesses for Predicate Evaluation. Let qh be a query having a component predicate pl with sell as its selectivity. We first define the number of distinct references involved in the path [10] REFi = (1 − P robi ) × ||Cih ||, where P robi is the probability of an object of collection Cih to be not involved in the path. Therefore, we have: h h P robi = (1 − ||C1h || )(REFi−1 ×sell ×F AN (Ci−1 ,Ci )) , with REF1 = ||Csh ||. i

In order to estimate the number of page accesses in class collection Cih during the evaluation of a predicate, we use the Yao function [19]: given n records uniformly distributed into m blocks (1 ≤ m ≤ n), each contains n/m records. If k records (k ≤ n) are randomly selected from n records, the expected number of Qk 1 page access is given by: Y ao(n, m, k) = m×[1− i=1 n×d−i+1 n−i+1 ], where d = 1− m . We use the Yao’s formula with n = ||Chi ||, m = |Chi | and k = REFi as defined above. Cost Formulation for Query Execution. The total cost of executing the query qh having a component predicate pl defined on unpartitioned classes is given by the following equation: T otal Cost = CEP + CBOR, with: CEP = P nh SELh ×||Csh || h h e i=s Y ao(||Ci ||, |Ci |, REFi ) and CBOR = d b PS c Lproj

Cost of Executing a Set of Queries on Unpartitioned Classes. We now generalize the total cost for executing k queries {q1 , q2 , ..., qk } which is given by the following equation: T otal Cost = CEP + CBOR where: Pk Pk Pnh SEL ×||C h || Y ao(||Cih ||, |Cih |, REFi ) and CBOR = h=1 [d b hP S cs e] CEP = h=1 i=s h Lproj

We note that all these equations are for queries having one component predicate. For a query having more than one predicate, we calculate the I/O cost for each predicate based on above equations, and finally sum up all these I/O costs.

474

3.3

L. Bellatreche, K. Karlapalem, and Q. Li

Cost of Executing a Single Query on HPCs Cjh

Let be a class in the class composition hierarchy which is horizontally partitioned into mj HCFs. Let Cih be a class horizontally partitioned based on the HCFs of the class Cjh . From this assumption, we can conclude the following points: 1) Cih and Cjh have the same number of HCFs (mj ), and 2) if a component predicate pl accesses two classes, we load only the HCFs of both classes that are needed by pl . Let us consider a query qh having a component predicate pl invoking Cih and Cjh and having the classes Csh and Cnh as starting and ending classes, respectively. Before describing the cost on horizontally partitioned classes, we first estimate the number of page accesses in HCF Fj during the evaluation of a predicate. We can apply, due to the representation of HCF as a class, the same formula for unpartitioned class. Then the number of page accesses is given by: Y ao(||Fj ||, |Fj |, refj ) where: refj = (1 − P robj ) × ||Fj || with P robj being the probability of an object of HCF Fj not to be involved in the path. Therefore, we h have: P robj = (1 − ||F1j || )(REFi−1 ×sell ×F AN (Fj ,Ci )) The CEP is the sum of Yao’s function over the class involved by the query, but we need to consider the effect of HCP of the classes Cih and Cjh which is represented by the binary variable valid(pl , Fj ) which has the value 1 if the predicate pl is valid in HCF Fj , and 0 otherwise. That means we load only the HCFs which satisfy the predicate pl defined in the query qh . Pi−1 Pmj CEP = i=s Y ao(||Cih ||, |Cih |, REFi )+ j=1 valid(pl , DFj )×Y ao(||DFj ||, |DFj |, refj ) Pmj Pn + j=1 valid(pl , M Fj )×Y ao(||M Fj ||, |M Fj |, refj )+ i=j+1 Y ao(||Cih ||, |Cih |, REFi )

The first part of the above equation represents the number of pages of the classes h h , ..., Ci−1 to be loaded. The second and the third parts represent the Csh , Cs+1 number of pages of the HCFs of the class Cih and Cjh to be loaded, and the last h till the ending class part represents the number of pages of the classes from Cj+1 h Cn to be loaded. Note that in the above equation we do not load all classes in the class comh h to Cj−1 , because the class Cih is derived partiposition hierarchy from Ci+1 tioned based on the class Cjh and we have a direct link from the HCFs of the class Cih to HCFs of the class Cjh . The cost of building the output result is: CBOR = d

SELh ×||Csh || ce PS b Lproj

Selecting the Best Candidate for DHCP We now show how we can use our cost model to select the good candidate for DHCP. Let S, F Q, N F , and F U be all possible member classes obtained by share, frequency, number of HCFs, and fusion candidates (see Sect. 2.3), S respectively. S S Let M C be the union of all possible candidates (i.e., M C = S F Q N F F U ). The approach is that for each candidate (member class) mci of M C, we derived horizontally partition the owner class based on the HCFs of mci . After that, we calculate the cost for executing a set of queries on these horizontally partitioned classes. Finally, we take the “best candidate” (based on least cost) to perform derived horizontal partitioning of the owner class.

Derived Horizontal Class Partitioning in OODBs

4

475

Evaluation of the Utility of Horizontal Partitioning

In order to show the utility of horizontal partitioning we conduct some analytical experiments here to see the effect of number of horizontal class fragments, the fan-out, and the cardinality on the improvement of performance due to horizontal partitioning. This improvement of performance is characterized by the normalized IO metric defined by [2] as follow: for the horizontally partitioned class collection Normalized IO = # Of#IOs Of IOs for the unpartitioned class collection We note that if the value of normalized IO is less than 1.0, then it implies that HP is beneficial. To present interesting results, while maintaining a control over the number of parameters needed to study the impact of parameter changes, we consider the following eight parameters, which are, cardinality of root class (class Employee), page size (PS), number of objects per page (LC), number of HCFs per class, Fan-out of a class along the class composition hierarchy (Employee → Department → Project), selectivity of predicate, and the size of the direct link. We concentrate on the following cases of HCP: • 0D- only the class Project is horizontally partitioned. • 1D- class Department is derived horizontally partitioned based on the HCFs of the class Project. • 2D- both classes Employee and Department are derived horizontally partitioned based on the HCFs of the class Project. The reason for selecting such a class collection is that it enables us to study both the impact of DHCP on the performance of path execution, and also the effect of the fan-out along the class composition hierarchy.

4.1

Experiment Setup

The parameters settings are as follows: the cardinality of the three classes shown in Fig. 1 are given by ||Employee||, ||Department|| = ||Employee||∗F AN1,2 and ||P roject|| = ||Department|| ∗ F AN2,3 , the length of objects for all the three classes is the same. The class Project is horizontally partitioned as follows: Horizontal Class Fragments of the Class Project P roject1 P roject2 P roject3 P roject4 P roject5

given given given given given

by by by by by

clause clause clause clause clause

cl1 cl2 cl3 cl4 cl5

: : : : :

(Dur ≤ 4) ∧ (Cost( ) ≤ 7000) ∧ (Loc = “CA”) (Dur ≤ 4) ∧ (Cost( ) > 7000) ∧ (Loc =“CA”) (Dur ≤ 4) ∧ (Cost() ≤ 7000) ∧ (Loc = “LA”) (Dur ≤ 4) ∧ (Cost( ) > 7000) ∧ (Loc = “LA”) Dur > 4

The DHCFs of the classes Department and Employee are defined by using the algorithm described in Sect. 2.1. We consider 18 queries for our experiments and they are classified into 6 types as follows:

476

L. Bellatreche, K. Karlapalem, and Q. Li

Type 1: Queries Accessing only Class Employee

Sel

Select Ename From Employee Where Esal ≤ 10000 Select EId, Ename From Employee Where Esal > 28000 Select Esal From Employee Where Ename = “Dupond”

0.05 0.5 0.1

Select Ename From e Employee Where e.Dept.Dname =“CS” Select EId, Ename e From Employee Where Esal ≤ $10000 And e.Dept.Dname =“EE”

0.05

Type 2: Queries Accessing only Class Employee & Department

Type 3: Queries Accessing all 3 Classes

Select Ename From e Employee Where Esal ≤ $10000 And e.Dept.proj.Loc = “CA” Select EId, Ename From e Employee Where Esal ≤ $28000 And e.Dept.Proj.Dur > 4 Select Esal From e Employee Where e.Dept.Proj.Loc = “LA”

Type 4: Queries Accessing only Class Department Select DId From Department Where Dname = “EE” Select DId From Department Where Dname = “CS”

Type 5: Queries Accessing Classes Department & Project

0.5

0.05 0.5 0.1 0.05 0.5

Select Dname From d Department Where d.Proj.Dur ≥ 5 0.05 Select DId From d Department Where Dname = “CS” And d.Proj.Loc = “CA” 0.5

Type 6: Queries Accessing only Class Project

Select Pname From Project Where Loc = “CA” And Cost ≤ $10000 0.05 Select Loc, Pname From Project Where Dur = 5 0.5 Select PId From Project Where Dur = 4 And Cost() = $8000 And Loc = “LA” 0.1

From the queries, we can enumerate all (component) predicates defind on class Project: p1 : Dur ≤ 4, p2 : Cost() ≤ 7000, p3 : Cost() > 7000, p4 : Loc = “CA”, p5 : Loc = “LA”. Effect of Varying the Cardinality. In this experiment, we study the impact of the variations of the cardinality of the class horizontally partitioned on the performance gain. Parameter values for the experiments are shown in Fig. 3 are: cardinality of the class Project is varying from 40000 to 400000, page size = 4064 bytes, the F AN1,2 = F AN2,2 = 1.0, and the length of the link parameter = 4. All the curves are almost horizontal straight lines, showing that the Normalized IO is independent of cardinality of the root class. That is, irrespective of the cardinality of the class we get the same percentage of reduction in number of disk accesses. Effect of Varying The Fan-Out. In this experiment we study the impact of the variations in the fan-out between the class Department and class Project on the performance gain. The main result of this experiment is to show the performance gain for high fan-out values. The parameters values used to produce experimental results as shown in Fig. 4 are: cardinality of class Employee is 10000, page size 4096 bytes, link length is 4 bytes. We observe the following result: the lower bound of Normalized IO is for D2 curves at high fan-out.

Derived Horizontal Class Partitioning in OODBs 1

0.9

477

1

0D 1D 2D

0D 1D 2D

0.9 0.8 0.7

Normalized IO

Normalized IO

0.8

0.7

0.6

0.6 0.5 0.4 0.3

0.5

0.2 0.4 40000

0.1

80000 120000 160000 200000 240000 280000 320000 360000 400000 Cardinality

2

Fig. 3. Plot of Normalized IO vs. Cardinality

1.3 1.2

Type 1 Type 2 Type 3 Type 4 Type 5 Type 6

1

Normalized IO

Normalized IO

1.1 0.9 0.8 0.7 0.6 0.5 0.4 0.3 0.2 0.1 40000

80000 120000 160000 200000 240000 280000 320000 360000 400000 Cardinality

Fig. 5. Plot of Normalized IO vs. Cardinality for each type of query)

6

8

10

12

14 16 Fan-Out

18

20

22

24

26

Fig. 4. Plot of Normalized IO vs. Fan-Out

1.5 1.4

4

1.8 1.7 1.6 1.5 1.4 1.3 1.2 1.1 1 0.9 0.8 0.7 0.6 0.5 0.4 0.3 0.2 0.1

Type 1 Type 2 Type 3 Type 4 Type 5 Type 6

2

4

6

8

10

12

14 16 Fan-Out

18

20

22

24

Fig. 6. Plot of Normalized IO vs. Fan-out for each type of query

Hence, great savings can be obtained by using the DHCP at high fan-out. Note that with high fan-out, the number of object instances in class Project will be quite large compared to ||Employee||. Therefore, the Predicate Evaluation Cost will dominate the total IO Cost. We can thus conclude that the fan-out has a considerable impact on horizontal class partitioning. Normalized IO per Each Type of Query In the previous experiments, we have studied the effect of changing the cardinality of class employee and the fanout along the class composition hierarchy on the total IO cost. We now study the effect of cardinality and fan-out for each type of queries. It is very important to see for which type of queries the DHCP is beneficial. In this experiment, we assume that the class Project is horizontally partitioned and that the classes Employee, Department are DHP. As shown in Fig. 5, the Normalized IO is very low for the queries of Type 3(queries accessing all three classes). For this type of queries the DHCP is recommended. But for the queries of type 1, 2 and 4, the Normalized IO is greater than 1, implying that the derived HP is not good for these types (that is, those queries which access only one class, or some classes of class composition hierarchy). Similar results hold for varying fan-out for each query type as shown in Fig. 6.

26

478

5

L. Bellatreche, K. Karlapalem, and Q. Li

Conclusions and Future Work

Query processing is integral part of OODBs, and any technique for improving the efficiency of query execution warranties a study of its utility. In this paper, we have addressed the concept of derived horizontal class partitioning (DHCP) in the context of OODBs. The main advantages of DHCP are its simple implementation and representation which incorporates the query semantics and has an ability to efficiently execute the queries. We develop an analytical cost model for query execution with and without DHCP, and show that DHCP facilitates higher performance than unpartitioned classes, under different database sizes, and different fan-outs among class composition hierarchy. Further, we categorized the queries into different types to quantify the exact benefit to each type of query. Our results show that DHCP benefits most queries which accesses all the classes of the class composition hierarchy, but can provide worse performance for queries which access only one class. Thus, DHCP is a technique though useful, must be applied with a caution by taking into consideration query mix in an application processing environment. The results developed in this paper are equally applicable to relational database environments.

References 1. M. Atkinson, F. Bancilhon, F. DeWitt, K. Dettrich, D. Maier, and S. Zdonik. The object database system manifesto. in Proceeding of the first International Conference on Deductive, Object-Oriented Databases, pages 40–57, 1989. 2. L. Bellatreche, K. Karlapalem, and G. B. Basak. Query-driven horizontal class partitioning in object-oriented databases. To appear in 9th International Conference on Database and Expert Systems Applications (DEXA’98), August 1998. 3. L. Bellatreche, K. Karlapalem, and A. Simonet. Horizontal class partitioning in object-oriented databases. in 8th International Conference on Database and Expert Systems Applications (DEXA’97), Toulouse, Lecture Notes in Computer Science 1308, pages 58–67, September 1997. 4. E. Bertino. On modeling cost functions for object-oriented databases. IEEE Transactions on Knowledge and Data Engineering, 9(3):500–508, May/June 1997. 5. E. Bertino and C. Guglielmina. Path-index: An approach to the efficient execution of object-oriented queries. Data & Knowledge Engineering, 10:1–27, 1993. 6. E. Bertino, M. Negri, G. Pelagatti, and L. Sbattella. Object-oriented query languages: The notion and the issues. IEEE Transactions on Knowledge and Data Engineering, 4(3):223–237, 1992. 7. S. Ceri, M. Negri, and G. Pelagatti. Horizontal data partitioning in database design. Proceedings of the ACM SIGMOD International Conference on Management of Data. SIGPLAN Notices, pages 128–136, 1982. 8. W. S. Cho, C. M. Park, K. Y Whang, and S. H. So. A new method for estimating the number of objects satisfying an object-oriented query involving partial participation of classes. Information Systems, 21(3):253–267, 1996. 9. C. I. Ezeife and K. Barker. A comprehensive approach to horizontal class fragmentation in distributed object based system. International Journal of Distributed and Parallel Databases, 3(3):247–272, 1995.

Derived Horizontal Class Partitioning in OODBs

479

10. G. Gardarin, J.-R. Gruser, and Z.-H. Tang. A cost-based selection of path expression processing algorithms in object-oriented databases. 22th International Conference on Very Large Data Bases, VLDB’96, pages 390–401, 1996. 11. S. Guo, S. Wei, and M. A. Weiss. On satisfiability, equivalence, and implication problems involving conjunctive queries in database systems. IEEE Transactions on Knowledge and Data Engineering, 8(4):604–612, August 1996. 12. K. Karlapalem and Q. Li. Partitioning schemes for object oriented databases. in Proceeding of the Fifth International Workshop on Research Issues in Data Engineering- Distributed Object Management, RIDE-DOM’95, pages 42–49, March 1995. 13. K. Karlapalem, S. B. Navathe, and M. Ammar. Optimal redesign policies to support dynamic processing of applications on a distributed database system. Information Systems, 21(4):353–367, 1996. 14. K. Karlapalem, S.B. Navathe, and M. M. A. Morsi. Issues in distributed design of object-oriented databases. In Distributed Object Management, pages 148–165. Morgan Kaufman Publishers Inc., 1994. 15. A. Kemper and G. Moerkotte. Physical object management. In Kim, W. (eds), Modern Database Systems: The Object Model, Interoperability, and Beyond, pages 175–202. Addison Wesley/ACM Press, 1995. 16. D. Maier, G. Graefe, L. Shapiro, S. Daniels, T. Keller, and B. Vance. Issues in distributed object assembly. In Distributed Object Management, pages 179–181. Morgan Kaufman Publishers Inc., 1994. ¨ 17. M. T. Ozsu and P. Valduriez. Principles of Distributed Database Systems. Prentice Hall, 1991. ¨ 18. D. D. Straube and M. T Ozsu. Queries and query processing in object oriented database systems. ACM Transactions on Information Systems, 18(4):387–430, October 1990. 19. S. B. Yao. Approximating the number of accesses in database organizations. Communication of the ACM, 20(4):260, April 1977.

482

Author Index

---

Conceptual Modeling ER 2001

Read more

Conceptual Modeling - ER 2002

Read more

Conceptual Modeling ER'99: 18th International Conference on Conceptual Modeling Paris, France, November 15-18, 1999 Proceedings

Read more

Conceptual Modeling - ER '97: 16th International Conference on Conceptual Modeling, Los Angeles, CA, USA, November 3-5, 1997. Proceedings

Read more

Conceptual Modeling - ER 2004: 23rd International Conference on Conceptual Modeling, Shanghai, China, November 8-12, 2004. Proceedings

Read more

Conceptual Modeling - ER 2006: 25th International Conference on Conceptual Modeling, Tucson, AZ, USA, November 6-9, 2006, Proceedings

Read more

Conceptual Modeling

Read more

Conceptual Modeling - ER '96: 15th International Conference on Conceptual Modeling, Cottbus, Germany, October 7 - 10, 1996. Proceedings: 15th

Read more

Conceptual Modeling - ER 2011. 30th International Conference on Conceptual Modeling Brussels. Proceedings (Lecture Notes in Computer Science)

Read more

Conceptual Modeling - ER 2000: 19th International Conference on Conceptual Modeling, Salt Lake City, Utah, USA, October 9-12, 2000 Proceedings

Read more

Advances in Conceptual Modeling: ER '99

Read more

Conceptual Modeling - ER 2007, 26 conf

Read more

Advances in Conceptual Modeling.. ER '99

Read more

Conceptual Modeling - ER 2009: 28th International Conference on Conceptual Modeling, Gramado, Brazil, November 9-12, 2009, Proceedings (Lecture Notes ... Applications, incl. Internet Web, and HCI)

Read more

Conceptual Modeling - ER 2010: 29th International Conference on Conceptual Modeling, Vancouver, BC, Canada, November 1-4, 2010, Proceedings (Lecture ... Applications, incl. Internet Web, and HCI)

Read more

Conceptual Modeling - ER 2004: 23rd International Conference on Conceptual Modeling, Shanghai, China, November 8-12, 2004. Proceedings (Lecture Notes in Computer Science)

Read more

Conceptual Modeling: Foundations and Applications

Read more

Conceptual Modeling of Information Systems

Read more

Advances in Conceptual Modeling - Foundations and Applications, ER 2007

Read more

Conceptual Modeling for Discrete-Event Simulation

Read more

The Evolution of Conceptual Modeling: From a Historical Perspective towards the Future of Conceptual Modeling

Read more

Conceptual Modeling for Discrete-Event Simulation

Read more

Conceptual Structures: Theory, Tools and Applications: 6th International Conference on Conceptual Structures, ICCS'98, Montpellier, France, August,

Read more

Conceptual Structures: Theory, Tools and Applications: 6th International Conference on Conceptual Structures, ICCS'98, Montpellier, France, August,

Read more

Conceptual Modeling for E-Business and the Web: ER 2000 Workshops on Conceptual Modeling Approaches for E-Business and the World Wide Web and

Read more

Advanced Multimedia Content Processing: First International Conference, AMCP'98, Osaka, Japan, November 9-11, 1998, Proceedings

Read more

Conceptual Structures: Knowledge Representations as Interlingua: 4th International Conference on Conceptual Structures, ICCS'96, Sydney, Australia,

Read more

Conceptual Relativism

Read more

Conceptual Relativism

Read more

Conceptual mathematics

Read more

Recommend Documents

Conceptual Modeling ER 2001

Lecture Notes in Computer Science Edited by G. Goos, J. Hartmanis, and J. van Leeuwen 2224 3 Berlin Heidelberg New Y...

---

Conceptual Modeling - ER 2002

Lecture Notes in Computer Science Edited by G. Goos, J. Hartmanis, and J. van Leeuwen 2503 3 Berlin Heidelberg New Y...

---

Conceptual Modeling ER'99: 18th International Conference on Conceptual Modeling Paris, France, November 15-18, 1999 Proceedings

Lecture Notes in Computer Science Edited by G. Goos, J. Hartmanis and J. van Leeuwen 1728 3 Berlin Heidelberg New Yor...

---

Conceptual Modeling - ER '97: 16th International Conference on Conceptual Modeling, Los Angeles, CA, USA, November 3-5, 1997. Proceedings

---

Conceptual Modeling - ER 2004: 23rd International Conference on Conceptual Modeling, Shanghai, China, November 8-12, 2004. Proceedings

Lecture Notes in Computer Science Commenced Publication in 1973 Founding and Former Series Editors: Gerhard Goos, Juris ...

---

Conceptual Modeling - ER 2006: 25th International Conference on Conceptual Modeling, Tucson, AZ, USA, November 6-9, 2006, Proceedings

Lecture Notes in Computer Science Commenced Publication in 1973 Founding and Former Series Editors: Gerhard Goos, Juris ...

---

Conceptual Modeling

Lecture Notes in Computer Science Edited by G. Goos, J. Hartmanis and J. van Leeuwen 1565 3 Berlin Heidelberg New Yor...

---

Conceptual Modeling - ER '96: 15th International Conference on Conceptual Modeling, Cottbus, Germany, October 7 - 10, 1996. Proceedings: 15th

---

Conceptual Modeling - ER 2011. 30th International Conference on Conceptual Modeling Brussels. Proceedings (Lecture Notes in Computer Science)

Lecture Notes in Computer Science Commenced Publication in 1973 Founding and Former Series Editors: Gerhard Goos, Juris...

---

Conceptual Modeling - ER 2000: 19th International Conference on Conceptual Modeling, Salt Lake City, Utah, USA, October 9-12, 2000 Proceedings

Lecture Notes in Computer Science Edited by G. Goos, J. Hartmanis and J. van Leeuwen 1920 3 Berlin Heidelberg New Yor...

---

×

Report "Conceptual Modeling - ER '98: 17th International Conference on Conceptual Modeling, Singapore, November 16-19, 1998, Proceedings: v. 1507"

Your name

Email

Reason -Select Reason- Pornographic Defamatory Illegal/Unlawful Spam Other Terms Of Service Violation File a copyright complaint

Description

Close Send

×

Sign In

Email

Password

Remember me Forgot password?

Sign In

---

Login with Facebook

Our partners will collect data and use cookies for ad personalization and measurement. Learn how we and our ad partner Google, collect and use data. Agree & close