'] of T, we have:
Similarly, quantification is induced by the corresponding operation on formulas. Thus if A = [4>] € Proper/I) (^ x F), so that (j>prop [A,F], then
if F = [xi :CTI, . . . , xn : <7n] say. Finally, for each object F = [x\ : a\ , . . . , xn : crn], the equality property EqT 6 Prop(T x F') can be represented by the formula
where the variables x1 , . . . , x'n are chosen to be distinct from x1 , . . . , xn (and each other). Turning now to the generic model G of Th in Cl(Th), its underlying structure is defined for the types and function symbols of Th as in section 4.2, and for relation symbols R C <, . . . , a n by G[R] = [R(x1, . . . , xn)] € Prop ce(Th) ([x1 : «TI , . . . , xn : an])Extending the properties of G on types, terms and equations given in Lemma 4.5, we also have the following properties for formulas and sequents. Lemma 5.30. The structure G has the following properties. (i) If prop [F], then the interpretation G[0[F]] € Propcl(Tht)(F) of the formula > in the structure G is just the C l ( T h ) -property represented by the formula (ii) $ h i[> [F] is a theorem of Th if and only if
In other words, G satisfies a sequent just if it is a theorem of Th. In particular, G satisfies the axioms of Th and hence is a model of Th.
100
Andrew M. Pitts
Corollary 5.31 (Completeness). A judgement over the signature of a given theory is a theorem if it is satisfied by all models of the theory in prop-categories with finite meets, finite joins, Heyting implications, quantification, and equality. Stronger forms of completeness hold with models of Th restricted to particular classes of prop-category. For example, Kripke's completeness theorem for intuitionistic predicate logic (see [Dummett, 1977, Chapter 5]) can be paraphrased in category-theoretic terms as asserting the completeness of the collection of prop-categories of the kind given in Example 5.5, but with the category C restricted to be a category of set-valued presheaves on a poset. Remark 5.32. The classifying prop-category C l ( T h ) and the generic algebra G of a predicate theory have a universal property analogous to that given for algebraic theories in Theorem 4.6 (but with respect to a suitable notion of morphism of prop-categories). This universal property forms the basis of an equivalence between theories in predicate logic (and translations between them) and suitably structured prop-categories (and morphisms between them)—cf. Section 4.3.
6
Dependent types
This section considers the categorical semantics of type theories involving types that depend upon terms. Martin-Lof's type theory (see [Nordstrom et a/., 1990]) provides an example of such a system, as does the GirardReynolds second-order lambda calculus (see [Girard, 1989]). Here we will just consider a fundamental example, namely the dependently typed analogue of the algebraic theories of section 2. So types and terms will be built up from a signature of function symbols which now may be type-valued as well as term-valued. Once one has explained the categorical framework corresponding to such dependently typed algebraic theories,1 one can use it systematically to describe the categorical structure needed for any particular dependently-typed datatype constructor—much as we did for simple types in section 3. We will just consider a single example of this, namely the categorical semantics of dependent products in section 6.5. Several different, but interconnected, categorical structures have been proposed for interpreting the basic framework of dependent types by Seely [1984], Cartmell [1986], Taylor [1999], Ehrhard [1988], Streicher [1989; 1991], Hyland and Pitts [1989], Obtutowicz [1989], Curien [1989], and Jacobs [1999]. This reflects the fact that the categorical interpretation of dependent types is undoubtedly more complicated than the other varieties of categorical logic explained in this chapter. This is due to the structural 'Cartmell [1986] calls such theories 'generalized algebraic'.
Categorical logic
101
complications implicit in the logic. One complication is that in general one must consider the satisfaction of equations not only between terms (of the same type) but also between types. Another complication is that proofs of well-formedness of expressions cannot be separated from proofs of equality, and are by no means unique. If one assigns meanings to expressions by induction on the derivation of their well-formedness, one therefore has to take care that the meaning is independent of the particular derivation. Early accounts [Seely, 1984] of the categorical semantics of dependent types tended to gloss over this point. Here we will take a more careful, but necessarily more complicated approach using an adaptation of Cartmell's notion of 'category with attributes'—a simplified version of the 'contextual categories' he uses in [Cartmell, 1986]. Categories with attributes are categories equipped with some extra structure for interpreting (dependent) types: accordingly we adopt (in Definition 6.9) the rather more compact terminology of 'type-categories' for the categorical structure we use. In order to motivate the notion of type-category, we will reverse the pattern set by sections 2 and 4: having described the syntax, we will organize it into a category just as in section 4.2, and see how the extra structure needed for a type-category emerges. We then give the interpretation of the syntax in a general type-category. The soundness of the interpretation (Theorem 6.20) relies on some 'strictness' conditions built into the definition of type-category to avoid the potential difficulties mentioned above to do with multiple proofs of well-formedness of judgements. The wide range of examples of type-categories which we give shows that these strictness conditions are not restrictive in practice. Without them one must resolve quite complicated coherence problems: see [Curien, 1990].
6.1
Syntactic considerations
We will continue to make use of the simply typed metalanguage introduced at the beginning of Section 3, with ground arities TYPES and TERMS. We assume given a signature, Sg, consisting of collections of meta-onstants • s : TERMS™ -» TYPES, called n-ary type-valued function symbols • F : TERMS™ -» TERMS, called n-ary term-valued function symbols for each number n. The raw2 expressions of the object language over Sg are then the metaexpressions built up from the symbols in Sg and a fixed, countably infinite set Var = {vi, v-2,. • •} of metavariables of arity TERMS. In particular, the raw types over Sg are metaexpressions of arity TYPES and the raw terms over Sg are metaexpressions of arity TERMS. (The algebraic signatures of section 2.1 also contained typing information for the function symbols. The analogous information here will be given as part of the axioms of a dependently typed algebraic theory, rather than as 2 The adjective 'raw' is used to emphasize that not all expressions will be 'well-formed' in the technical sense to be explained.
102
Andrew M. Pitts
part of its underlying signature.) Throughout section 6, Sg will denote a fixed signature in the above sense. Definition 6.1. As before, a context, F, is a finite list F = [xi : a1,..., xn : an] of (variable, type) pairs, but now subject to the following conditions on variables: for each i = 1,..., n,
var((?i) C {xi,...,Xi-i}
and z,- g {z1;.. .,xt-i}
where var(e) denotes the (finite) set of variables occurring in the expression e. Thus the variables Xi are distinct and each type
called the introductory axiom of F. Once again, x is the list of variables of the context fy, and must have length n when F has arity TERMS™ -» TERMS. FF lists the types of the arguments to which F may be applied, and F gives the type of the result (which may depend upon those arguments). • A collection of judgements of the form M = M' : a [F], called the term-equality axioms of Th. • A collection of judgements of the form a = a' [F], called the typeequality axioms of Th. Given such a theory, the theorems of Th are the judgements which are provable using the rules shown in Figs 9 and 10. In the rules, x is the
Categorical logic
103
Table 1. Forms of judgement (F and F" are contexts, a and a' are raw types, M and M' are raw terms, and y and y' are finite lists of raw terms.) Judgement
Intended meaning
Restriction
Primary forms a type [T]
'a is a well-formed type in context F'
var(a) C var(T)
M : a [F]
'M is a well-formed term of type a in context F'
var(M, a) C var(F)
a = a' [F]
'cr and a1 are equal (well-formed) types in context F'
i»var(cr, a') C t>var(F)
M = M' : «7 [F]
'M and M' are equal (well-formed) terms of type a in context F'
var(M,
M', ) C var(F)
Secondary forms F ctxt 7 : F->T'
T is a well-formed context' '7 is a context morphism var(y) C var(F) from F to F"
F = F'
T and F' are equal (wellformed) contexts'
F and F' of equal length
7 = 7': F->F'
'7 and 7' are equal context morphisms from F to F"
var(7,7') C var(T) and 7, 7', and F' of equal length
104
Andrew M. Pitts
Contexts
Context morphisms
Fig. 9. General rules
Categorical logic
ctxt
(s has introductory axiom s(x) type [Tg])
s(x) type [T aF type [IV]
F(x) : aF
105
(F has introductory axiom F(x) : aF [IV])
M :CT[F] M' : a [F]
M = M' : a [T]
(M = M' : a [F] is a term-equality axiom)
a type [F] a' type [F]
(a = a1 [F] is a type-equality axiom).
Fig. 10. Rules for axioms list of variables of context F and x' is the list of variables of context F'; and supposing 7 = [M\,..., Mn] and x = [x\,..., xn], then a[^/x\ denotes the result of simultaneously substituting the term M, for the variable Xi in the type a. In reading the rules, it should not be forgotten that we only consider contexts which are well-formed, in the sense of satisfying the conditions on variables mentioned in Definition 6.1. Thus for example, in the rule for deriving [T,x : a], it is necessarily the case that x is distinct from the variables in F. The rules in Fig. 9 are general rules for dependently typed equational logic, whereas those in Fig. 10 relate to the particular axioms of Th. Note that these rules for using the axioms of Th do have hypotheses: if the hypotheses of such a rule are never satisfied, then we can never make use of the axiom. Definition 6.3. We will say that a dependently typed algebraic theory Th is well-formed if all of the hypotheses of the rules for introducing axioms are provable from the rules in Figs 9 and 10. (In this case each of the axioms is indeed a theorem of Th.) Remark 6.4. The form of judgement ' 7 : F—»F'' is not usually made explicit in presentations of theories of dependent types. We have chosen to make it a 'first-class' judgement form because it permits a conceptually clear and concise formulation of the substitution rules, and because it will play a key role in the definition of the classifying category of a dependently typed algebraic theory in the next section. However, it is easy to see from the rules in Fig. 9 that this form of judgement, along with the forms 'y = 7' : F—>F" ', T ctxt ' and T = F' ', are all equivalent to finite conjunctions of instances of the forms 'a type [T]', 'M : a [F]', 'a = a' [F]', and 'M = M' :o- [F]'.
106
Andrew M. Pitts
Example 6.5. The theory of categories provides an example of a wellformed, dependently typed algebraic theory. The underlying signature is: obj hom Id Comp
: : : :
TYPES TERMS2->TYPES TERMS^TERMS TERMS5-»TERMS.
The axioms are as follows:
where
It is evident from this example that the formal requirement in the introductory axiom of a function symbol (such as that for Comp) that all variables in the context occur explicitly as arguments of the function is at variance with informal practice. See [Cartmell, 1986, Section 10] for a discussion of this issue. Remark 6.6 (Substitution and Weakening.). A general rule for substituting along a context morphism is derivable from the rules in Figs 9 and 10, viz.
Categorical logic
107
where J is one of the four forms 'e type', 'e = e", 'e : e", or 'e = e' : e"\ and y is the list of variables in T'. A special case of this is a general derived rule for weakening contexts:
The rules for substitution in Fig. 9 also have as special cases forms which correspond more closely to the substitution rule (2.9) for simply typed equational logic, viz:
6.2
Classifying category of a theory
We are now going to construct a category, C l ( T h ) , out of the syntax of a dependently typed algebraic theory Th. In fact the construction is essentially just as in section 4.2. More accurately, the construction in that section is the special case of the one given here when the theory has only 0-ary type-valued function symbols. We will use the following terminology with respect to the judgements which are theorems of Th. Say that F is a Th- context if the judgement [F] ctxt is a theorem of Th; say that Th-contexts F and F' are Th-provably equal if the judgement F = F' is a theorem of Th; say that 7 : F-»F' is a morphism of Th-contexts if the judgement 7 : F-»F' is a theorem of Th; and finally, say that two T7i-context morphisms are Th-provably equal if 7 = 7': F-^F' is a theorem of Th. Objects of C l ( T h ) . The collection of objects of the classifying category is the quotient of the set of Th-contexts by the equivalence relation of being 7h-provably equal. This definition makes sense because the following rules are derivable from those in Figs 9 and 10:
We will tend not to distinguish notationally between a Th-context and the object of Cl( Th) that it determines. Morphisms of C l ( T h ) . Given two objects in Cl(Th), represented by Thcontexts F and F' say, the collection of morphisms in the classifying cat-
108
Andrew M. Pitts
egory from the first object to the second is the quotient of the set of Thcontext morphisms F-»F' by the equivalence relation of being Th-provably equal. This definition makes sense because the following rules are derivable from those in Figs 9 and 10:
We will tend not to distinguish notationally between a Th-context morphism and the morphism of Cl ( Th) which it determines. Composition in Cl(Th).
Given context morphisms 7 : F—>T' and 7' : define tion 7' o 7 : F->T" just as in section 4.2, viz.
The derived rules for substitution mentioned in Remark 6.6 can be used to show that the following derived rules for composition are valid.
From these rules it follows that a well-defined and associative operation of composition is induced on the morphisms of the classifying category. The identity morphisms for this composition are induced by the identity context morphisms idr '• F—>F, which are defined just as in section 4.2:
where x 1 , . . . , xn are the variables listed in F. These context morphisms do induce morphisms in Ct( Th) which are units for composition because the following rules are derivable:
Categorical logic
109
This completes the definition of Cl( Th) as a category. We now move on to examine what extra categorical structure it possesses.
6.3
Type-categories
For a simply typed algebraic theory, we saw in section 4 that the relevant categorical structure on the corresponding classifying category was finite products. Here it will turn out to be the more general3 property of possessing a terminal object and some pullbacks. To explain which pullbacks, we identify a special class of morphisms in the classifying category of Th. Definition 6.7. If F is a Th-context, the collection of T-indexed types in Cl( Th) is defined to be the quotient of the set of types a such that a type [F] is a theorem of Th, with respect to the equivalence relation identifying a and a1 just if a = a1 [F] is a theorem of Th. As usual, we will not make a notational distinction between a and the F-indexed type it determines. Each such F-indexed type has associated with it a projection morphism, represented by the context morphism
Here x1,... ,xn are the variables listed in F, and x is any other variable. Note that the object in C l ( T h ) represented by [F,a; : a] and the morphism represented by a are independent of which particular variable x is chosen. Lemma 6.8. Given a morphism 7 : F' —» F in Cl( Th) and a T-indexed type represented by a, then
is a pullback square in CL( Th) . Proof. Suppose we are given 7' : F"->F' and 7" : F"-»[r, x : a] in CL( Th) satisfying 7 o 7' = na o 7" : F"->F. We have to prove that there is a unique 3
Since finite products can be constructed from pullbacks and a terminal object.
110 morphism
Andrew M. Pitts satisfying
and
Now since , we must have that the list of terms 7" is of the form for some term N for which is a theorem of Th. Now since (where x1 is the list of variables in F'), we get a morphism
satisfying
and
as required. If 6' : F"—>•[!", x' : &[j/x\] were any other such morphism, then from the requirement ' we conclude that the list 6' is of the f o r m ; and then from the requirement we conclude further that
The above lemma motivates the following definition, which is essentially Cartmell's (unpublished) notion of 'category with attributes'. Definition 6.9. A type-category, C, is a category possessing a terminal object, 1, and equipped with the following structure: • For each object X in C, a collection Typec(X), whose elements will be called X -indexed types in C. • For each object X in C, operations assigning, to each X-indexed type A an object X K A, called the total object of A, together with a morphism
led the projection morphism of A. • For each morphism / : Y — » X in C, an operation assigning to each X -indexed type A, a, Y indexed type f*A, called the pullback of A
Categorical logic along X, together with a morphism making the following a pullback square in the category C:
111 A
In addition, the following strictness conditions will be imposed on these operations:
Notation 6.10. If C is a type-category and X is a C-object, then as usual C/X will denote the slice category whose objects are the C-morphisms with codomain X, f : dom(f) —l X, and whose morphisms / —> f are C-morphisms g : dom(f) —> dom(f') satisfying /' o g = /. Identity morphisms and composition in C/X are inherited from those in C. Note that for each A £ Typec(X), the projection morphism -ITA is an object in the slice category C/X. So we can make Typec(X) into a category (equivalent to a full subcategory of C/X) by taking morphisms A —> A' to be C/Xmorphisms VA —>• TT^' (with identities and composition inherited from C/X). Proposition 6.11. The classifying category, C l ( T h ) , of a dependently typed algebraic theory Th is a type-category. Proof. We identified notions of indexed type and associated projection morphisms for C l ( T h ) in Definition 6.7 and verified in Lemma 6.8 the existence of pullback squares of the required form (6.5). The 'strictness' conditions in Definition 6.9 are met because of the usual properties of substitution of terms for variables. Finally, note that Cl( Th) does have a terminal object, namely the (equivalence class of the) empty context []. I Here are some examples of 'naturally occurring' type-categories. Example 6.12 (Sets). The category Set of sets and functions supports the structure of a type-category: for each set X, we can take the ^-indexed types to be set-valued functions with domain X. Given such a function A : X —^ Set, the associated total set is the disjoint union
112
Andrew M. Pitts
with projection function ~KA given by first projection (x, a) t-¥ x. The pullback of A along a function / : F —> X is just the composition def
which indeed results in a pullback square in Set of the required form (6.5) and satisfying the strictness conditions, with equal to the function Note the following property of this example, which is not typical of typecategories in general: up to bijection over X, any function with codomain X, f : Y — > X, can be expressed as the projection function associated with an indexed type:
Example 6.13 (Constant families). Every category C with finite products can be endowed with the structure of a type-category, in which the indexed types are 'constant families'. For each object X in C, one defines Typec(X) to be Objc, the class of all objects of C. The associated operations are
Given / : Y —>• X in C and A € Typec(X) = Objc, the pullback F-indexed type f*A is just the object A itself and the morphism X x A is / x id A- This does yield a pullback square in C, viz.
and these pullbacks do satisfy the strictness conditions in Definition 6.9. When a category with finite products is regarded as a type-category in this way, the categorical semantics of simply typed algebraic theories given in section 2 becomes essentially a special case of the semantics of dependently typed theories to be given below. Example 6.14 (Toposes). The following example is given for readers familiar with the notion of topos (see the references in section 7). One
Categorical logic
113
can make each topos £ into a type-category as follows. First note that £ is indeed a category with a terminal object. For each object X e Obj£, define Type£ (X) to consist of all pairs (A, a) where A 6 Obj£ and a : X x A —» fi. Here fi denotes the codomain of the object classifier of £, T : 1 —> fl. Given such an X-indexed type, the total object X x (A, a) is given by forming a pullback square in E:
brojection
morphism associated to (A, a) is defined by the composition
For any morphism is defined to be the F-indexed type The pullback square (6.5) is obtained as the factorization through (6.6) of the corresponding pullback for a o (/ x id A)Elementary properties of pullback squares ensure that the strictness conditions of Definition 6.9 hold, without the need for any coherence assumptions on the chosen pullback operations for E. Traditionally, dependently typed theories have been interpreted in toposes using the locally cartesian closed structure that they possess, whereby the category Type£(X) of X-indexed types (cf. Notation 6.10) is just the slice category E / X : see [Seely, 1984]. Such a choice for indexed types yields a 'non-strict' type-category structure, in that the strictness conditions of Definition 6.9 have to be weakened by replacing the equalities by coherent isomorphisms. This can lead to complications in the interpretation of type theory: see [Curien, 1990]. The method of regarding a topos as a type-category given here achieves the strictness conditions and side-steps issues of coherence. Given that toposes correspond to theories in intuitionistic higher-order logic (cf. [Bell, 1988], [Mac Lane and Moerdijk, 1992]), this example can form the basis for interpreting dependently typed theories in higher-order predicate logic: see [Jacobs and Melham, 1993]. However, Hofmann [1995] has recently pointed out that using a particular construction of Benabou (for producing a split fibration equivalent to a given fibration), an arbitrary locally cartesian closed category, E, can be endowed with a more involved notion of X-indexed type which makes £ a type-category (supporting the interpretation of product, sum, and identity types) and with T y p e e ( X ) equivalent to E/X (pseudo-naturally in X).
114
Andrew M. Pitts
Example 6.15 (Split fibrations). Let Cat denote the category of small categories and functors. We can turn it into a type-category by decreeing that for each small category, C, the C-indexed types are functors A : Cop —> Cat. The associated total category C x A is given by a construction due to Grothendieck: • An object of C x A is a pair (X, A), with X an object of C and A an object of A.(X). • A morphism in C x A from (X, A) to (X1, A') is a pair (x, a), where x : X —>• X' in C and a : A —> A.(x)(A') in A(X). The composition of two such morphisms (a;, a) : (X, A) —> (X',A') and (a;', a') : (X1, A') —> (X", A") is given by the pair (x1 o x, A(x)(a') o a). The identity morphism for the object (^",^4) is given by (id x, id A)The projection functor TTA : C K A —> C is of course given on both objects and morphisms by projection onto the first coordinate. The pullback of the indexed type A : Cop —> Cat along a functor F : D —> C is just given by composing A with F, regarded as a functor Dop —> Cop. Applying the Grothendieck construction to A and to A o F, one obtains a pullback square in the category Cat of the required form, with F x A : D x (A o F) —> C K A the functor which acts on objects and morphisms by applying F in the first coordinate. Regarding a set as a discrete category, there is an inclusion between the type-category of Example 6.12 and the present one. Unlike the previous example, not every morphism in Cat arises as the first projection of an indexed type. Those that do were characterized by Grothendieck and are known as 'split Grothendieck fibrations'. As the name suggests, these are a special instance of the more general notion of 'Grothendieck fibration'— which would give a more general way of making Cat into a type-category, except that the 'strictness' conditions in Definition 6.9 are not satisfied. See [Jacobs, 1999] for more information on the use of Grothendieck fibrations in the semantics of type theory.
6.4
Categorical semantics
We will now give the definition of the semantics of the types, terms, contexts, and context morphisms of Th in a type-category C. In general, contexts are interpreted as C-objects, context morphisms as C-morphisms, and types as indexed types in C. Terms as interpreted using the following notion of 'global section' of an indexed type. Definition 6.16. Given an object X in a type-category C, the global sections of an X-indexed type A € Typec (X) are the morphisms a : X —> X x A in C satisfying IT A ° a = idx- We will write
Categorical logic
115
to indicate that a is a global section of the X-indexed type A. For any morphism / : Y —> X, using the universal property of the pullback square (6.5) we get fa
is the unique morphism with TT/.^ o /*
and (/ x A) o fa = a o /.
Definition 6.17. Let Sg be a signature as in section 6.1. A structure for Sg in a type-category C is specified by: • For each type-valued function symbol s, a C-object Xs and an indexed type As E Typec(Xs). • For each term-valued function symbol F, a C-object Xp, an indexed type AF £ Typec(Xp), and a global section ap £XF AFSemantics of expressions. Given such a structure, we will define four relations:
where the judgements enclosed by [ J are all well-formed and where X is a C-object, A € Typec(X), a Ex A, and / is a morphism from X' to X in C. These relations are defined inductively by the rules in Fig. 11. In the first rule concerning context morphisms, {) : X—*l denotes the unique morphism from X to the terminal object. The two rules concerning the semantics of variables use the following pairing notation. Notation 6.18. Referring to the pullback square h : Z —> X x A are morphisms satisfying / o g = KA o h, then
and
will denote the unique morphism satisfying A) o (g, h} — h whose existence is guaranteed by the universal property of the pullback square. Thus for example, in this notation the global section fa €y /* A referred to in Definition 6.16 is given by the morphism (idy, &° It is clear from the form of the rules in Fig. 11 that the relations they define are monogenic, in the sense that for a judgement J and categorical structures E and E', if [J] x E and [J] x E', then E = E' in C. Thus given a structure for Sg in C, the assignments
116
Andrew M. Pitts Contexts
Terms
Context morphisms
Fig. 11. Categorical semantics
Categorical logic
117
determine partial functions. We will write to indicate that these partial functions are defined at a particular argument J. Remark 6.19. The case when J is x : T [A] deserves some comment, since this is the most complicated case in the definition of [J]. The wellformedness of x : T [A] guarantees that x is assigned a type in A, i.e. that A is of the form [F, x : a, F']. However, the presence of type equality judgements in the logic means that a is not necessarily syntactically identical to T. This accounts for the presence of the hypotheses concerning r in the two rules for this case in Fig. 11. The first of these two rules deals with the case when F' = [ ], and then the second rule must be applied repeatedly to deal with general F'. Satisfaction of judgements Next we define what it means for the various forms of judgement to be satisfied by a structure for Sg in a type-category, C. • F ctxt is satisfied if and only if [F ctaJtJJJ• a type [F] is satisfied if and only if \a type [F]J||.. • M : a [F] is satisfied if and only if |M : a [F]|-IJ.. • 7 : F->F' is satisfied if and only if [7 : F-»F']JJ.. • F = F' is satisfied if and only if for some (necessarily unique) object X
[F ctxt\ x X
and [F7 ctxt] x X.
• a = a' [F] is satisfied if and only if for some (necessarily unique) object X and ^-indexed type A £ Typec(X) {a type [F]I x A [X]
and
[a' type [F]] x A [X].
• M = M' : a [F] is satisfied if and only if for some (necessarily unique) object X, indexed type A 6 Typec(X), and global section a Ex A IM : a [F]] x a : A [X]
and
[M' : a [F]] x a : A [X].
• 7 = 7' : F-»F' is satisfied if and only if for some (necessarily unique) morphism / : X' —> X in C 17 : F'->r] x / : X'-*X
and [7' : F'->F] x / : X'-tX.
Suppose Th is a dependently typed algebraic theory. A model of Th in a type-category, C, is a structure in C for the underlying signature of
118
Andrew M. Pitts
Th with the property that the collection of judgements satisfied by the structure is closed under the rules for the axioms of Th given in Fig. 10. Theorem 6.20 (Soundness). The collection of judgements that are satisfied by a structure in a type-category, C, is closed under the general rules of dependently typed equational logic given in Fig. 9. Consequently, a model inC of a dependently typed algebraic theory, Th, satisfies all the judgements which are theorems of Th. Much as with the corresponding result for simply typed equational logic, the key tool used in proving the above theorem is a lemma giving the behaviour of substitution in the categorical semantics: Lemma 6.21 (Semantics of substitution). Suppose that Then
The generic model of a well-formed theory. Suppose that Th is a dependently typed algebraic theory that is well-formed, in the sense of Definition 6.3. Then the classifying category C l ( T h ) contains a structure, G, for the underlying signature of Th. G is defined as follows, where we use a notation for the components of the structure as in Definition 6.17. The well-formedness of Th implies that for each type-valued function symbol s, with introductory axiom s(x)type [Fs] say, Fs is a ITi-context and hence determines an object Xs of Cl( Th). Then define the X s -indexed type AS 6 TyPecl(Th)(-^s) to be that represented by s(x) (which is a Fs-indexed type because s(x) type [Fs] is a theorem of Th). For each term-valued function symbol F with introductory axiom F(x) : VF [TF],the well-formedness of Th implies that up type \Tp] is a theorem of Th. Hence F is a Th-context and hence determines an object ofCl(Th). Then define the XF-indexed type AF €T y p e c l ( T h ) ( X F )to be that represented by the Ff-indexed type , and define the global section to be the morphism represented by the Th-context morphism The structure G has the following properties: • For each judgement F ctxt and Cl(Th)-object X, the relation [F] x X holds if and only if F ctxt is a theorem of Th and X is the object of C l ( T h ) represented by F. • For each judgement a type, [F], each object X and each X-indexed type A e Typece^Th-)(X), the relation [a type [F]J x A [X] holds if and only if a type [F] is a theorem of Th, X is the object of C l ( T h ) represented by F, and A is the X"-indexed type represented by a.
Categorical logic
119
• For each judgement M : a [F], each object X and each X-indexed type A € Typec^Th)(X), and each global section a Ex A, the relation [M : a [F]] x a € A [X] holds if and only if X is the object of Ci( Th) represented by F, A is the X-indexed type represented by a, and a is the global section represented by the Th-context morphism • An equality judgement is satisfied by G if and only if it is a theorem of Th. Thus G is a model of Th and a judgement is satisfied by this model if and only if it is a theorem of Th. Remark 6.22. The classifying type-category Cl( Th) and the generic model G of a well-formed, dependently typed algebraic theory Th have a universal property analogous to that given for algebraic theories in Theorem 4.6, with respect to a suitable notion of morphism of type-categories. To go further and develop a correspondence between dependently typed algebraic theories and type-categories along the lines of section 4.3, one has to restrict to the subclass of 'reachable' type-categories. For note that an object X in a type-category C can appear as the interpretation of a context only if for some n > 0 there is a sequence of indexed types
with Call C reachable if there is such a sequence for every C-object. Every classifying category has this property. Conversely if C is reachable, then it is equivalent to Cl( Th) for a theory in a suitable 'internal language' for C (cf. section 2.3).
6.5
Dependent products
The categorical framework corresponding to dependently typed equational logic that we have given can be used to infer the categorical structure needed for various dependently typed datatype constructors—much as we did for simple types in section 3. We will just consider one example here, namely the structure in a type-category corresponding to dependent products. The formation, introduction, elimination, and equality rules for dependent product types are given in Fig. 12. Equality rules for both B- and n-conversion are given. There are also congruence rules for the constructors II, A, and ap, which we omit (cf. Remark 3.3). If C is a type-category, the existence of the pullback squares (6.5) allows us to define a functor between slice categories
given by pullback along if A '• X x A —> X :
120
Andrew M. Pitts
Fig. 12. Rules for dependent product types
(This definition makes use of the pairing notation introduced in Notation 6.18.) Symmetrically, there is a functor
induced by pullback along
Definition 6.23. A type-category C has dependent products if for each there type and a morphism in Typec(X tx. A)
Categorical logic
121
satisfying the following: • Adjointness property. ^n(A,A') '• X x 11(^4, A') — > X is the value of the right adjoint to the pullback functor (6.7) at TTA> '• X x A x A' — > X x A, with counit apA A,. In other words, for any / : Y — » X in C and g : there is a unique morphism in C/X satisfying • Strictness property.
For any morphism
Here /* and (f x .A)* are instances of the pullback functors (6.8). To specify the categorical semantics of dependent product types in such a type-category, we give rules extending the inductively defined relation of section 6.4: Formation
Introduction
Note that the conclusion of this rule is well-formed if the hypotheses are. For if a' &XKA A', then a' so we can apply the adjointness property from Definition 6.23 to form cur(o') : idx — > ^n(A>i')' wmch is, in particular, a global section of H(A,A'). Elimination
If the hypotheses of this rule are well-formed, then a Ex A and b €x H(A,A'), Since
122 we
Andrew M. Pitts can use
the pairing operation of Notation
6.18 to form Since by definition and
a, we get a morphism (a, b) : hence by composition a morphism
pairing operation yields a morphism
whose composition with ' is idx and hence which is a global section of a" A'. Thus the conclusion of the above rule is well-formed when its hypotheses are. With these rules we can extend Theorem 6.20 to include dependent product types. Theorem 6.24 (Soundness). Defining satisfaction of judgements as in section 6.4, it is the case that the collection of judgements satisfied by a structure in a type-category with dependent products is closed under the rules in Fig. 12 (as well as the general rules of Fig. 9). Thus the structure of Definition 6.23 is sufficient for soundly interpreting dependent products. On the other hand it is necessary, in the sense that Proposition 6.11 can be extended as follows: Proposition 6.25. If Th is a theory in dependently typed equational logic with dependent product types, then the classifying category C l ( T h ) constructed in section 6.2 is a type-category with dependent products. Proof. Referring to Definition 6.23, to define (6.9) find judgements
that are theorems of Th and such that X, A, and A' are the equivalence classes of F, a, and o-'(x), respectively. Then take 11(^4,^4') to be the Findexed type determined by . This definition is independent of the choice of representatives. The morphism (6.10) is that represented by the Th-context morphism
Since, this does indeed induce a morphism of the required kind; and once again the definition of apAA, is independent of the choice of representatives. The adjointness property required for 11(^4, ^4') can be deduced from the equality rules in Fig. 12, and the stability property follows from standard properties of substitution. •
Categorical logic
7
123
Further reading
This section lists some important topics in categorical logic which have not been covered in this chapter and gives pointers to the literature on them. Higher-order logic The study of toposes (a categorical abstraction of key properties of categories of set-valued sheaves) and their relationship to set theory and higher order logic has been one of the greatest stimuli of the development of a categorical approach to logic. [Mac Lane and Moerdijk, 1992] provides a very good introduction to topos theory from a mathematical perspective; [Lambek and Scott, 1986, Part II] and [Bell, 1988] give accounts emphasizing the connections with logic. The hyperdoctrine approach to first-order logic outlined in section 5 can be extended to higherorder logic and such higher-order hyperdoctrines can be used to generate toposes: see [Hyland et al, 1980], [Pitts, 1981,1999] and [Hyland and Ong, 1993]. Polymorphic lambda calculus Hyperdoctrines have also been used successfully to model type theories involving type variables and quantification over types. [Crole, 1993, Chapters 5 and 6] provides an introduction to the categorical semantics of such type theories very much in the spirit of this chapter. Categories of relations Much of the development of categorical logic has been stimulated by a process of abstraction from the properties of categories of sets and functions. However, categorical properties of sets and binary relations (under the usual operation of composition of relations) have also been influential. The book by Preyd and Scedrov [1990] provides a wealth of material on categorical logic from this perspective. Categorical proof theory Both Lawvere [1969] and Lambek [1968] put forward the idea that proofs of logical entailment between propositions, may be modelled by morphisms, in categories. If one is only interested in the existence of such proofs, then one might as well only consider categories with at most one morphism between any pair of objects, i.e. only consider pre-ordered sets. This is the point of view taken in section 5. However, to study the structure of proofs one must consider categories rather than just pre-orders. Lambek, in particular, has studied the connection between this categorical view of proofs and the two main styles of proof introduced by Gentzen (natural deduction and sequent calculus), introducing the notion of multicategory to model sequents in which more than one proposition occurs on either side of the turnstile, : see [Lambek, 1989]. This has resulted in applications of proof theory to category theory, for example in the use of cut elimination theorems to prove coherence results: see [Mine, 1977], [Mac Lane, 1982]. In the reverse direction of applications of category theory to proof theory, general categorical machinery, particularly enriched category theory, has provided useful guidelines
124
Andrew M. Pitts
for what constitutes a model of proofs in Girard's linear logic (see [Seely, 1989], [Barr, 1991], [Mackie et al., 1993], [Bierman, 1994]); for example, in [Benton et al., 1993], such considerations facilitated the discovery of a well-behaved natural deduction formulation of intuitionistic linear logic. Categorical combinators The essentially algebraic nature of the category theory corresponding to various kinds of logic and type theory gives rise to variable-free, combinatory presentations of such systems. These have been used as the basis of abstract machines for expression evaluation and type checking: see [Curien, 1993], [Ritter, 1992].
References [Backhouse et al, 1989] R. Backhouse, P. Chisholm, G. Malcolm, and E. Saaman. Do-it-yourself type theory. Formal Aspects of Computing, 1:19-84, 1989. [Barr, 1991] M. Barr. *-autonomous categories and linear logic. Math. Structures in Computer Science, 1:159-178, 1991. [Barr and Wells, 1990] M. Barr and C. Wells. Category Theory for Computing Science. Prentice Hall, 1990. [Bell, 1988] J. L. Bell. Toposes and Local Set Theories. An Introduction, volume 14 of Oxford Logic Guides. Oxford University Press, 1988. [Benton et al., 1993] P. N. Benton, G. M. Bierman, V. C. V. de Paiva, and J. M. E. Hyland. A term calculus for intuitionistic linear logic. In M. Bezem and J. F. Groote, editors. Typed Lambda Calculi and Applications, Lecture Notes in Computer Science 664, pages 75-90, SpringerVerlag, 1993. [Bierman, 1994] G. M. Bierman. On intuitionistic linear logic. PhD thesis, Cambridge Univ, 1994. [Bloom and Esik, 1993] S. L. Bloom and Z. Esik. Iteration Theories. EATCS Monographs on Theoretical Computer Science. Springer-Verlag, 1993. [Cartmell, 1986] J. Cartmell. Generalised algebraic theories and contextual categories. Annals of Pure and Applied Logic, 32:209-243, 1986. [Chang and Keisler, 1973] C. C. Chang and H. J. Keisler. Model Theory. North-Holland, 1973. [Coste, 1979] M. Coste. Localisation, spectra and sheaf representation. In M. P. Fourman, C. J. Mulvey, and D. S. Scott, editors, Applications of Sheaves, Lecture Notes in Mathematics 753, pages 212-238. SpringerVerlag, 1979. [Crole, 1993] R. L. Crole. Categories for Types. Cambridge Univ. Press, 1993.
Categorical logic
125
[Crole and Pitts, 1992] R. L. Crole and A. M. Pitts. New foundations for fixpoint computations: Fix-hyperdoctrines and the fix-logic. Information and Computation, 98:171-210, 1992. [Curien, 1989] P.-L. Curien. Alpha-conversion, conditions on variables and categorical logic. Studia Logica, 48:319-360, 1989. [Curien, 1990] P.-L. Curien. Substitution up to isomorphism. Technical Report LIENS-90-9, Laboratoire d'Informatique, Ecole Normale Superieure, Paris, 1990. [Curien, 1993] P.-L. Curien. Categorical Combinators, Sequential Algorithms, and Functional Programming (2nd edition). Birkhauser, 1993. [Dummett, 1977] M. Dummett. Elements of Intuitionism. Oxford University Press, 1977. [Ehrhard, 1988] Th. Ehrhard. A categorical semantics of constructions. In 3rd Annual Symposium on Logic in Computer Science, pages 264-273. IEEE Computer Society Press, 1988. [Freyd, 1972] P. J. Freyd. Aspects of topoi. Bulletin of the Australian Mathematical Society, 7:1-76 and 467-80, 1972. [Freyd and Scedrov, 1990] P. J. Freyd and A. Scedrov. Categories, Allegories. North-Holland, Amsterdam, 1990. [Girard, 1986] J.-Y. Girard. The system F of variable types, fifteen years later. Theoretical Computer Science, 45:159-192, 1986. [Girard, 1987] J.-Y. Girard. Linear logic. Theoretical Computer Science, 50:1-102, 1987. [Girard, 1989] J.-Y. Girard. Proofs and Types, Cambridge Tracts in Theoretical Computer Science 7. Cambridge University Press, 1989. [Goguen and Meseguer, 1985] J. A. Goguen and J. Meseguer. Initiality, induction and computability. In M. Nivat and J. C. Reynolds, editors, Algebraic Methods in Semantics, pages 459-541, Cambridge University Press, 1985. [Goguen et al, 1977] J. A. Goguen, J. W. Thatcher, E. G. Wagner, and J. B. Wright. Initial algebra semantics and continuous algebras. Journal of the Association for Computing Machinery, 24:68-95, 1977. [Hofmann, 1995] M. Hofmann. On the interpretation of type theory in locally cartesian closed cateories. In L. Pacholski and J. Tiuryn, editors, Computer Science Logic, Kazimierz, Poland, 1994, Lecture Notes in Computer Science 933. Springer-Verlag, 1995. [Hyland, 1988] J. M. E. Hyland. A small complete category. Annals of Pure and Applied Logic, 40:135-165, 1988. [Hyland and Ong, 1993] J. M. E. Hyland and C.-H. L. Ong. Modified realizability toposes and strong normalization proofs. In M. Bezem and J. F. Groote, editors, Typed Lambda Calculi and Applications, Lecture Notes in Computer Science 664, pages 179-194. Springer-Verlag, 1993.
126
Andrew M. Pitts
[Hyland and Pitts, 1989] J. M. E. Hyland and A. M. Pitts. The theory of constructions: categorical semantics and topos-theoretic models. In J. W.Gray and A. Scedrov, editors, Categories in Computer Science and Logic, Contemporary Mathematics 92, pages 137-199. American Mathematical Society, 1989. [Hyland et al, 1980] J. M. E. Hyland, P. T. Johnstone, and A. M. Pitts. Tripos theory. Mathematical Proceedings of the Cambridge Philosophical Society, 88:205-232, 1980. [Jacobs and Melham, 1993] B. Jacobs and T. Melham. Translating dependent type theory into higher order logic. In M. Bezem and J. F. Groote, editors, Typed Lambda Calculi and Applications, Lecture Notes in Computer Science, pages 209-229. Springer-Verlag, 1993. [Jacobs, 1999] B. Jacobs. Categorical Loigc and Type Theory. Elsevier, 1999. [Lambek, 1968] J. Lambek. Deductive systems and categories I. Math. Systems Theory, 2:287-318, 1968. [Lambek, 1988] J. Lambek. On. the unity of algebra and logic. In F. Borceux, editor, Categorical Algebra and its Applications, Lecture Notes in Mathematics 1348. Springer-Verlag, Berlin, 1988. [Lambek, 1989] J. Lambek. Multicategories revisited. In J. W. Gray and A. Scedrov,editors, Categories in Computer Science and Logic, Contemporary Mathematics 92, pages 217-239, American Mathematical Society, 1989. [Lambek and Scott, 1986] J. Lambek and P. J. Scott. Introduction to Higher Order Categorical Logic, Cambridge Studies in Advanced Mathematics 7. Cambridge University Press, 1986. [Lawvere, 1964] F. W. Lawvere. An elementary theory of the category of sets. Proceedings of hte National Academy of Sciences of the USA, 52:1506-1511, 1964. [Lawvere, 1966] F. W. Lawvere. The category of categories as a foundation for mathematics. In S. Eilenberg, D. K. Harrison, S. MacLane and H. Rohrl, editors, Proc. Conference on Categorical Algebra, La Jolla, pages 1-21, Springer-Verlag, 1966. [Lawvere, 1969] F. W. Lawvere. Adjointness in foundations. Dialectica, 23:281-296, 1969. [Lawvere, 1970] F. W. Lawvere. Equality in hyperdoctrines and the comprehension schema as an adjoint functor. In A. Heller, editor, Applications of Categorical Algebra, pages 1-14. American Mathematical Society, 1970. [Mac Lane, 1971] S. Mac Lane. Categories for the Working Mathematician, Graduate Texts in Mathematics 5. Springer-Verlag, Berlin, 1971.
Categorical logic
127
[Mac Lane, 1982] S. Mac Lane. Why commutative diagrams coincide with equivalent proofs. Contemporary Mathematics, 13:387-401, 1982. [Mac Lane and Moerdijk, 1992] S. Mac Lane and I. Moerdijk. Sheaves in Geometry and Logic. A First Introduction to Topos Theory. Universitext. Springer-Verlag, 1992. [Mackie et al., 1993] I. Mackie, L. Roman, and S. Abramsky. An internal language for autonomous categories. Applied Categorical Structures, 1:311-343, 1993. [Makkai and Reyes, 1977] M. Makkai and G. E. Reyes. First Order Categorical Logic, Lecture Notes in Mathematics 61. Springer-Verlag, 1977. [McLarty, 1992] C. McLarty. Elementary Categories, Elementary Toposes, Oxford Logic Guides 21. Oxford University Press, 1992. [Mine, 1977] G. E. Mine. Closed categories and the theory of proofs. Zap. Nauch. Sem. Leningrad. Otdel. Mat. Inst. im. V. A. Steklova (LOMI), 96:83-114,145, 1977. Russian, with English summary. [Mitchell and Scott, 1989] J. C. Mitchell and P. J. Scott. Typed lambda models and cartesian closed categories (preliminary version). In J. W. Gray and A. Scedrov, editors, Typed Lambda Calculi and Applications, Lecture Notes in Computer Science, pages 301-316, Springer-Verlag, 1989. [Moggi, 1991] E. Moggi. Notions of computations and monads. Information and Computation, 93:55-92, 1991. [Nordstrom et al., 1990] B. Nordstrom, K. Petersson, and J. M. Smith. Programming in Martin-Lof's Type Theory, volume 7 of International Series of Monographs on Computer Science. Oxford University Press, 1990. [Obtulowicz, 1989] A. Obtulowicz. Categorical and algebraic aspects of Martin-L6f type theory. Studia Logica, 48:299-318, 1989. [Oles, 1985] F. J. Oles. Types algebras, functor categories and block structure. In M. Nivat and J. C. Reynolds, editors. Algebraic Methods in Semantics pages 543-574. Cambridge University Press, 1985. [Pierce, 1991] B. C. Pierce. Basic Category Theory for Computer Scientists. MIT Press, 1991. [Pitts, 1981] A. M. Pitts. The theory of toposes. PhD thesis, Cambridge Univ., 1981. [Pitts, 1987] A. M. Pitts. Polymorphism is set theoretic, constructively. In D. H. Pitt, A. Poigne, and D. E. Rydeheard, editors, Category Theory and Computer Science, Proc. Edinburgh 1987, Lecture Notes in Computer Science 283, pages 12-39. Springer-Verlag, Berlin, 1987. [Pitts, 1989] A. M. Pitts. Non-trivial power types can't be subtypes of polymorphic types. In 4th Annual Symposium on Logic in Computer Science, pages 6-13. IEEE Computer Society Press, 1989.
128
Andrew M. Pitts
[Pitts, 1999] A. M. Pitts. Tripos theory in retrospect. In L. Birkedal and G. Rosolini, editors, Proceedings of the Tutorial Workshop on Realizabitity Semantics, FLOC '99, Trento, Italy, 1999. Electronic Notes in Theoretical Comptuer Science 23, Elsevier, 1999. [Poigne, 1992] A. Poigne. Basic category theory. Chapter in S. Abramsky, D. M. Gabbay and T. S. E. Maibaum, editors, Handbook of Logic in Computer Science, Vol. 1, Oxford University Press, 1992. [Reynolds, 1983] J. C. Reynolds. Types, abstraction and parametric polymorphism. In R. E. A. Mason, editor, Information Processing 83, pages 513-523. North-Holland, 1983. [Reynolds and Plotkin, 1993] J. C. Reynolds and G. D. Plotkin. On functors expressible in the polymorphic typed lambda calculus. Information and Computation, 105:1-29, 1993. [Ritter, 1992] E. Ritter. Categorical abstract machines for higher-order typed lambda calculi. PhD thesis, Cambridge University, 1992. [Roman, 1989] L. Roman. Cartesian categories with natural numbers object. Journal of Pure and Applied Algebra, 58:267-278, 1989. [Scott, 1980] D. S. Scott. Relating theories of the lambda calculus. In J. P. Seldin and J. R. Hindley, editors, To H. B. Curry: Essays on Combinatory Logic, Lambda Calculus and Formalism, pages 403-450. Academic Press, 1980. [Seely, 1984] R. A. G. Seely. Locally cartesian closed categories and type theories. Mathematical Proceedings of the Cambridge Philosophical Society, 95:33-48, 1984. [Seely, 1987] R. A. G. Seely. Categorical semantics for higher order polymorphic lambda calculus. Journal of Symbolic Logic, 52:969-989, 1987. [Seely, 1989] R. A. G. Seely. Linear logic, *-autonomous categories and cofree algebras. In J. W. Gray and A. Scedrov, editors, Typed Lambda Calculi and Applications, Lecture Notes in Computer Science 664, pages 371-382. Springer-Verlag, 1989. [Streicher, 1989] Th. Streicher. Correctness and completeness of a categorical semantics of the calculus of constructions. PhD thesis, University of Passau, 1989. Tech. Report MIP-8913. [Streicher, 1991] Th. Streicher. Semantics of Type Theory. Birkhauser, 1991. [Taylor, 1999] P. Taylor. Practical Foundations of Mathematics. Cambridge University Press, 1999.
A uniform method for proving lower bounds on the computational complexity of logical theories Kevin J. Compton and C. Ward Henson
Contents 1 2 3 4 5 6 7 8 9 10
Introduction Preliminaries Reductions between formulas Inseparability results for first-order theories Inseparability results for monadic second-order theories . . . . Tools for NTIME lower bounds Tools for linear ATIME lower bounds Applications Upper bounds Open problems
129 135 140 151 158 164 173 180 196 204
1 Introduction In this chapter we present a method for obtaining lower bounds on the computational complexity of logical theories, and give several illustrations of its use. This method is an extension of widely used procedures for proving the recursive undecidability of logical theories. (See Rabin [1965] and Ersov et al. [1965].) One important aspect of this method is that it is based on a family of inseparability results for certain logical problems, closely related to the well-known inseparability result of Trakhtenbrot (as refined by Vaught), that no recursive set separates the logically valid sentences from those which are false in some finite model, as long as the underlying language has at least one non-unary relation symbol. By using these inseparability results as a foundation, we are able to obtain hereditary lower bounds, i.e., bounds which apply uniformly to all subtheories of the theory. The second important aspect of this method is that we use interpretations to transfer lower bounds from one theory to another. By doing this we eliminate the need to code machine computations into the models of the theory being studied. (The coding of computations is done once and for all
130
Kevin J. Compton and C. Ward Henson
in proving the inseparability results.) By using interpretations, attention is centred on much simpler definability considerations, viz., what kinds of binary relations on large finite sets can be defined using short formulas in models of the theory. This is conceptually much simpler than other approaches that have been proposed for obtaining lower bounds, such as the method of bounded concatenations of Fleischmann et al. [1977]. We will deal primarily with theories in first-order logic and monadic second-order logic. Given a set 17 of sentences in a logic L, we will consider the satisfiability problem sat(
)={
L \ a is true in some model of 27}
and the validity problem val(
)= {
L \ a is true in all models of 27}.
A hereditary lower bound for 27 is a bound that holds for sat( ') and val( ') whenever 27' C val( ). If L is a first-order logic, define inv(L) to be the set of sentences in L that are logically invalid, i.e., false in all models. If L is a monadic second-order logic, define inv(L) to be the set of sentences false in all weak models. (See section 2 for definitions.) The complexity classes used here are time-bounded classes for nondeterministic Turing machines and for the more general class of linear alternating Turing machines. In providing reductions between different decision problems, we are always able to give log-lin reductions. That is, our reduction functions can be computed by a deterministic Turing machine which operates simultaneously in log space and linear time. In particular, such functions have the property that the size of a value is bounded uniformly by a constant multiple of the size of the argument. Let L0 denote the first-order logic with a single, binary relation symbol. Let MLo denote the corresponding monadic second-order logic. Let T(n) be a time resource bound which grows at least exponentially in the sense that there exists a constant d, 0 < d < 1, such that T(dn)/T(n) tends to 0 as n tends to . (This condition is satisfied by the iterated exponential functions and other time resource bounds which arise most commonly in connection with the computational complexity of logical theories.) Let satT(Lo) denote the set of sentences in L0 such that a is true in some model on a set of size at most T(| |). (Here \ \ denotes the length of a.) Similarly define satT(ML 0 ) for sentences of monadic second-order logic. The inseparability results which form the cornerstone of our method are as follows: (a) For some c > 0, satT(Lo) and inv(Lo) cannot be separated by any set in NTIME(T(cn)). (b) For some c > 0, satT(MLo) and inv(MLo) cannot be separated by
Proving lower bounds
131
any set in ATIME(T(cn),cn). (ATIME(T(cn),cn) denotes the set of problems recognized by alternating Turing machines in time T(cn) making en alternations along any branch.) In proving (b) we prove another interesting result. Let ML+ be a monadic second-order logic with a ternary relation PLUS and satT(M +) be the set of sentences in this logic true in the model (T(n), +}, where n = \ \ and + is the usual ternary addition relation on the set T(n) = {0,..., T ( n ) — 1}. Then for some c > 0, satT(M +) and inv(ML+) cannot be separated by any set in ATIME(T(cn),cn). We prove and discuss these results in sections 3 and 4 respectively. In fact, we prove more: any problem separating satT(L 0 ) and inv(L0) is a hard problem (under log-lin reductions) for the complexity class NTIME(T(cn)). Any problem separating satT(ML0) and inv(ML0) is a hard problem for the complexity class ATIME(T(cn),cn}. In these results one can see a parallel between first-order logic and NTIME, on the one hand, and monadic second-order logic and linear alternating time, on the other. This parallel persists throughout the lower bounds for logical theories which we discuss here, and we feel that our point of view helps to explain why the complexity of some theories is best measured using NTIME while for others the best measure is linear alternating time. In order to obtain lower bounds from these inseparability results, or to transfer lower bounds from one theory to another, we use interpretations. However, sometimes we require not just a single interpretation, but rather a sequence {In | n 0} of such interpretations. Not only do we require that each In define a sufficiently rich class of models when applied to the models of the theory under study, but also we require that the function taking n (in unary) to In should be log-lin computable. As an example of how such interpretations are used, suppose £ is a theory such that for each n 0, In applied to models of yields all the binary relations of size at most T(n) (and perhaps others), for a given time resource bound T. It follows that for some constant c > 0, sat( ) and inv(L) cannot be separated by any set in NTIME(T(cn)). In general, it follows that has a hereditary NTIME(T(cn)) lower bound. There is a corresponding result for the complexity classes ATIME(T(cn),cn) when the interpretations In interpret monadic second-order logic of binary relations on sets of size at most T(n).
132
Kevin J. Compton and C. Ward Henson
Upon establishing a lower bound for sat(£) in this manner, we may then use interpretations of £ to obtain lower bounds for other theories. Continuing the example above, assume In„ applied to the set of models of ' yields all the binary relations of size at most T(n). In fact, it will not be necessary to apply In to all models of £ to obtain all binary relations of size at most T(n), since there are only finitely many such binary relations. Suppose that Cn is a set of models of £ such that In applied to Cn yields all the binary relations of size at most T(n). If {I'n \ n 0} is another log-lin computable sequence of interpretations such that I'n applied to models of a theory ' in a language L' yields all models of Cn (and perhaps others, some possibly not even models of ), then for some constant c > 0, sat( ') and inv(L') cannot be separated by any set in NTIME(T(cn)). Thus, we have developed a theory for establishing lower bounds of logical theories analogous to the theory for establishing NP -hardness results via polynomial time reductions. The observation that I'n applied to models of £' is allowed to yield models not satisfying £ may seem unimportant, but in practice it results in significant simplifications in interpretations, especially compared to the method of establishing lower bounds by Turing machine encodings. There one must must produce, for a nondeterministic Turing machine M with the appropriate running time, an efficient reduction from strings w to sentences in such a way that when M accepts w, w is true in some model of ', w and when M does not accept w, w is true in no model of '. Ensuring that a sentence is true in no model of £' in the case of nonacceptance can be cumbersome. Inseparability considerations eliminate the need for it. Our method can give short, transparent lower bound results in many cases where Turing machine encodings are far from apparent. An example is the result of Compton et al. [1987] that the theory of almost all finite unary functions is not elementary recursive. The proof there, using the methods of this chapter, is set forth in a short paragraph. A proof by Turing machine encodings would run to many pages, and possibly would never have been discovered at all. It seems likely that our method will have to be used if sharp lower bounds are to be obtained for some of the important, more algebraic (and less directly combinatorial) theories which are known to be decidable. (See, for example, Problems 10.1, 10.2, 10.7, 10.9, and 10.10.) In making use of sequential families of interpretations there are certain technicalities regarding lengths of formulas which must be addressed. They can be illustrated by considering the formula ' which results from a formula when one replaces every occurrence of a certain binary relation symbol P by a given formula . If has many occurrences of P and if the length of is of the same order of magnitude as the length of , then ' may well be extremely long compared to . (This is precisely the kind of operation on formulas used as a reduction function between theories when
Proving lower bounds
133
one uses interpretations to obtain lower bounds.) This difficulty can be overcome if one uses sequences {In} of interpretations in which either all formulas in each In are in prenex form, or they are obtained by a certain kind of iterative process. In practice, the interpretation sequences used to transfer lower bounds from one logical problem to another can always be found satisfying one of these conditions. We have used this approach to give a precise analysis of the computational complexity of various theories of finite trees. For each r let r denote the first-order theory of all finite trees of height r, and let M r denote the corresponding monadic second-order theory. Also let and M , denote the corresponding theories of all finite trees. Let expm(n) be the m-times iterated exponential function (e.g., exp2(n) = 2 2n ) and let exp (n) be the tower of twos function:
Our results concerning the various theories of finite trees can be summarized as follows: (a) For each r 4 there are constants c and d > 0 such that sat(r) is in NTIME (ex.pr_2(dn)) but that sat(r) and val( r) are hereditarily not in NTIME (expr_2(cn)). For r = 3 the upper bound is NTIME(2 d n 2 ) and the hereditary lower bound is NTIME(2cn). (b) There exist constants c and d > 0 such that sat( ) is in NTIME but that sat (
) and val(
(exp(dn))
) are hereditarily not in
NTIME (exPoo (cn)). Hence, these problems are hereditarily not elementary recursive. (c) For each r 1 there are constants c and d > 0 such that sat(MSr) is in A TIME (expr (dn/ log n) , dn) but that sat(MSr) and val(M r) are hereditarily not in ATIME(expr(cn/ logn), en). It is not hard to show that and M are mutually interpretable, and hence have the same complexity. In any case, for such rapidly growing time resource bounds as e x p ( n ) , the difference between NTIME and ATIME has vanished.
134
Kevin J. Compton and C. Ward Henson
In order to test our method for effectiveness and smoothness of use, we have used it to provide proofs of essentially all previously known complexity lower bounds for first-order and monadic second-order theories. These arguments avoid direct coding of machine computations, and are usually much simpler and more conceptual than the original arguments. We present many of these proofs, or at least sketches of them, in section 8. In all cases our lower bounds are hereditary, and are expressed in terms of log-lin hardness for certain complexity classes, providing complete problems for many of these NTIME and linear ATIME classes. In some cases we verify results which had been only announced, no published proof ever having appeared. We do not consider lower bounds for sentences with restricted quantification prefixes, as in Lewis [1980], Scarpellini [1984], Gradel [1988; 1989], Schoning [1997], and Borger et al. [1997]; nor do we consider lower bounds for nonclassical logics such as temporal logics, interval logics, dynamic logics, logics of knowledge, and other modal logics. Many of these logics have decision problems complete in deterministic time classes. See, for example, Chagrov and Zakharyaschev [1997], Fischer and Ladner [1979], Emerson and Halpern [1985], Fagin et al. [1995], Halpern and Moses [1992], Halpern and Vardi [1986], Harel [1984], Kutty et al. [1995], Kozen and Tiuryn [1990], Rabinovich [1998], Ramakrishna et al. [1992; 1996], Stirling [1992], and Vardi [1997]. It is our hope that the appearance of this chapter will stimulate the interests of many computer scientists and mathematicians, and that they will be inspired to investigate the many decidable theories for which no detailed complexity bounds have been found. Even though there has been continued interest in the complexity of decidable thoeries since this material first appeared (see, for example, Cegielski [1996], Cegielski et al. [1996], Maurin [1996; 1997a; 1997b], and Michel [1992]), researchers in the area are sometimes unaware of the tools for obtaining lower bounds, and, in particular, of hereditary lower bounds and inseparability results. The organization of our chapter is as follows. Section 2 contains various definitions and technical conventions. In section 3 we present some technical machinery needed to handle the details about lengths of formulas which arise in complexity arguments. Sections 4 and 5 contain the basic inseparability results for logical problems; these are the analogues of the Trakhtenbrot-Vaught theorem. In sections 6 and 7 we discuss interpretations and set up the procedures by which they are used to obtain lower bounds for logical theories. Here too are proved the lower bounds for the various theories of finite trees which are treated here. Section 8 contains a lengthy series of applications of our method, yielding lower bounds for a wide variety of theories of independent interest. In section 9 we obtain various upper bounds for problems treated here, in order to show that our method is capable of achieving sharp results. We present a selected list of
Proving lower bounds
135
open problems at the end of section 10. In this chapter we have not discussed ways in which our method can be used to obtain lower bounds in terms of SPACE(T(n)) complexity classes. This is because there are so few known cases in which best possible lower bounds for logical theories are expressed in terms of space complexity classes. The exceptions are the PSPACE-complete theories such as those discussed in Stockmeyer [1977] and Grandjean [1983]. These are, in some sense, the least complex theories since Stockmeyer shows implicitly that if £ is a logical theory with has at least one model with at least one nontrivial definable relation (i.e., the relation is true of some elements and false of others), then sat( ) is log-space, polynomial-time hard for PSPACE. (Note that if equality is taken as a basic relation in the language, or is definable, then the hypothesis means simply that has at least one model with two or more elements.) If £ does not have any such model, then it is utterly trivial, and both sat( ) and val( ) are in LOGSPACE. (We assume throughout that the vocabularies of logics are finite.) Possibly methods of this chapter could be extended to obtain polynomial nondeterministic time lower bounds for PSPACE -complete theories. On the other hand, all 'natural' logical theories which are known to be decidable seem actually to be primitive recursive. Furthermore, among theories where a somewhat careful upper bound analysis has been carried out, decidable logical theories seem to fall into NTIME(exp (dn)) for some constant d in all but a few cases. The exceptions have been discovered only recently by Vorobynov [1997; preprint]. We believe that the method presented here is simple and can be mastered quickly. To get an overview of the applications of our method we advise looking over the results in section 8 first. Not only does this give a summary of the main complexity lower bounds now known, but also we have tried to present these applications in such a way as to give an accessible exposition of how our methods are meant to be used, and the main technical points which arise in their use.
2
Preliminaries
In this section we present the definitions and notations used throughout the chapter. All alphabets considered will be finite. The length of a string w is denoted \w\. The empty string is denoted . We use the standard 'big oh' and 'little oh' notations throughout, as well as the 'big omega' notation: write f ( n ) = (g(n)) if f ( n ) kg(n) for some k > 0 on all large n. All theories considered here either first-order or monadic second-order. For convenience we explicitly treat only relational languages in sections 37; functions are handled by using their graphs as relations, and constants
136
Kevin J. Compton and C. Ward Henson
are treated as special unary relations. This restriction makes no difference as far as concerns the lower bounds we obtain: sentences containing function and constant symbols can be transformed into equivalent sentences containing relation symbols with an increase in length of only a constant factor. (It is necessary to 'reuse' variables to accomplish this.) We will assume for convenience that our languages contain equality. However, in many situations our methods work even without equality. We use equality mainly to keep formulas short while substituting other formulas for atomic formulas: equations are used for coding truth values. This can usually be done by other formulas, as long as they satisfy the right conditions of nontriviality. If one is willing to accept polynomial-time reductions, then equality is generally not necessary, but the lower bounds degrade slightly. To specify a logic in this chapter, we need only give its set of relation symbols and their associated arities (the vocabulary of the logic) and indicate whether the logic is first-order or monadic second-order. We formulate all of our logics using finitely many symbols, so that all terms and formulas are strings on a finite alphabet. In particular, a variable is represented by a symbol followed by a subscript in binary notation. Thus, to represent n distinct variables we need strings with total length about n log n. (Logarithms will always have base 2.) To avoid subscripted subscripts we use lower-case Roman letters t,u,x,y,z—possibly with subscripts—as formal variables to denote actual variables Vi. Monadic variables are represented by corresponding upper-case letters. The power of a model is the cardinality of its universe. A weak model for a monadic second-order logic L is a pair (, F}, where is a model for L and F is a collection of subsets of the universe of . The truth value of a formula from L in (, .F) is determined in the usual way except that monadic quantifiers range over the sets in .F rather than the collection of all sets. Throughout the chapter, equivalence of monadic second-order formulas will mean equivalence on weak models. This is stronger than the usual notion of equivalence. The first-order and monadic second-order logics with vocabulary consisting just of a binary relation symbol P are central to our investigation. They will be denoted L0 and MLo, respectively. We also study theories of finite trees; again the vocabulary consists just of one binary relation symbol which, in this case, interprets the successor (or parent-child) relation. Let Lt and MLt denote the first-order and monadic second-order logics with this vocabulary. These are essentially the same as Lo and MLo; they differ only in the binary relation symbol used. However, it will be convenient to have a different notation for these logics when considering trees. When considering finite trees we will often require the notion of a primary subtree. Such a subtree is formed by restricting to a set of vertices
Proving lower bounds
137
consisting of a child of the root and all its descendents. Thus, we may regard a tree as being formed by directing an edge from the root of the tree to the root of each of its primary subtrees. The depth of a vertex in a tree is its distance from the root. The height of a vertex is the maximum distance to a leaf below it. Thus, the height of a tree is the maximum depth of its vertices, which is also the height of the root. As we noted in Section 1, we will consider problems of the form sat(£) and val( ). Prom a computational point of view, these two problems are complementary. That is, a sentence a is in sat( ) exactly when - is not in val(£). Hence, sat(£) is a member of a particular complexity class if and only if val( ) is a member of the corresponding co-complexity class. If £ is a complete theory, then sat(£) = val( ). When we are in a firstorder logic, val( ) is the deductive closure of £ by the Godel completeness theorem. There is no corresponding result for monadic second-order logic. Often a logical theory is specified not by giving a set of axioms , but by giving a class of models C. In this situation we take £ to be the set of sentences true in all members of C. It is easy to verify in this case that val( ) = £ and sat(£) is the set of sentences true in some member of C. If L is a first-order logic we define inv(L) to be the set of sentences false in all models for L. This is just the complement of sat( ). If L is a monadic second-order logic we define inv(L) to be the set of sentences false in all weak models for L. Given a time resource bound T(n), let satr(£) be the set of sentences true in some model of £ of size at most T(| |). Also, write satT(L) for satT( ). Let satpt(£) be the set of prenex sentences true in some model of £ of size at most r(| |). Let be a model for a logic L, m = m1, . . . ,m,k elements of , and ( x i , . . . , x n , y 1 , . . . , y k ) a formula from L. Then (x,m) denotes the n-ary relation defined by
for a = a1,. ..,an M. Interpretations of one class of models in another are fundamental to many parts of logic, and we use them extensively here. For example, to interpret a binary relation ' (i.e., a model for the logic L0) in a theory £ from a logic L, we must produce formulas 6(x, u) and (x, y, u) from L so that for some model of £ and some elements m of , ' is isomorphic to the structure
where we require that (x, y,m) (x,m) x (x,7n). There is also a more general kind of interpretation that is often used, in which the domain of can be a set of k-tuples from (not just elements of ) and in which
138
Kevin J. Compton and C. Ward Henson
' is isomorphic to a quotient of by an equivalence relation definable in . Let be a formula in a logic L and D a unary relation symbol. By D we mean the relativization of to D. This is formed by systematically replacing all subformulas y and y of with y(D(y) ) and y(D(y) ), respectively. If L is a monadic second-order logic, it is not necessary to relativize the set quantifiers since elements have already been restricted to D. The complexity classes we use are defined by time resource bounds. A time resource bound T is a mapping from the nonnegative reals to the nonnegative reals such that for each k > 0, T(kn) is dominated by some fully time-constructible function on the integers; see Hopcroft and Ullman [1979] for definitions. (Readers who need a primer in complexity theory may also wish to consult Stockmeyer [1987].) We will also require that T(n) > n and that for each k 1, T(kn] kT(n). This last condition Machtey and Young [1978] call at least linear. It says that when input length is increased by some factor, the allowed computation time increases by at least the same factor. It is included for technical reasons; we could get by with less, namely, that T be nondecreasing and for every / there should be a k such that T(kn) lT(n). The iterated exponentials and tower of twos functions appear often as time resource bounds in the problems we consider. The iterated exponentials exp m (n), where m is a nonnegative integer, are defined by induction on m. Let exp0(n) = n and expm+1(n) = 2expm(n). The tower of twos function e x p ( n ) is defined to be
Recall that a problem is elementary recursive if it is recognized in time exp m (n) for some m 0. All of our bounds are for nondeterministic or alternating Turing machines. The set of problems recognized by nondeterministic Turing machines in time T(n) is denoted NTIME(T(n)). With alternating Turing machines we will be concerned chiefly with the complexity classes ATIME(T(n), cn), the set of problems recognized by an alternating Turing machine in time T(n) making at most cn alternations. We will assume that alternating Turing machines have four types of states: universal, existential, accepting, and rejecting. See Chandra et al. [1981] for the definition of acceptance by alternating Turing machines and a description of the computation trees associated with these machines. We will sometimes say that a theory has a hereditary NTIME(T(cn)) (or ATIME(T(cn), en)) lower bound. By this we mean that there is a c > 0 such that for all E' , neither sat( ') nor val( ') is in NTIME(T(cn))
Proving lower bounds
139
(or in ATIME(T(cn),cn)). A log-lin reduction is a mapping computable in log space and linear time. In some sources this terminology is used for a log space computable, linearly bounded mapping, which is a weaker notion. (Linearly bounded means that output length is less than some constant multiple of input length.) It is not crucial for the applications presented here that our reductions be quite so restricted: polynomial-time, linearly bounded reductions suffice. However, to obtain some results in the literature, such as the nondeterministic polynomial lower bounds in Grandjean [1983], linear time reductions would be needed. We encounter a technical problem with log-lin reductions: we do not know if they are closed under composition. To overcome this difficulty we define a stronger notion of reset log-lin reduction. A machine performing such reduction is a log space, linear time bounded Turing machine with work tapes, an input tape, and an output tape. It has the capability to reset the input tape head to the initial input cell on k moves during a computation, where k is fixed for all inputs; on all other moves the input tape head remains in place or moves one cell to the right. It writes the output sequentially from left to right. Suppose that M' and M" are two such machines using at most k' and k" resets, respectively. We informally describe a machine M to compute the composition of the reductions computed by M' and M". Imagine that the output tape of M' and the input tape of M" have been removed. Instead, M' sends its output directly to M". As M" computes its output, it calls M' to supply it with a new symbol on those moves when the input head of M" would have moved right. M' has only to resume its computation from the last call to supply this symbol. On those moves where M" would have reset its input head, M' must begin its computation anew. Now the input head of M" would have passed over each input cell at most k" +1 times during the computation, and to supply each symbol the input head of M' passes over each input cell at most k' +1 times. Thus, M resets its input head at most (k1 + l)(k" + 1) - 1 times. Clearly, M is log space bounded. Since the part of M corresponding to M' is forced to begin its computation anew at most k" times, it is easy to see that M is linear time bounded. It is not difficult to show that the prenex formulas of a logic are closed under relativization up to reset log-lin reductions. That is, there is a reset log-lin reduction which takes formulas of the form D, where is a prenex formula with no variable quantified more than once, to equivalent prenex formulas. We use this fact often. Unfortunately, we know of no way to eliminate duplicate quantifications of variables using reset log-lin reductions, but this can be accomplished easily with polynomial-time, linearly bounded reductions. A problem is hard for a complexity class C via reductions from a class S if every problem ' C can be reduced to £ by some / € S. That is, if
140
Kevin J. Campion and C. Ward Henson
A and A' are the alphabets for £ and 17' respectively, then / maps A1* to A* so that w " if and only if f ( w ) . If, in addition, £ € C, we say that 1 is complete for C via reductions from S.
3
Reductions between formulas
One of our goals is to develop effective and easily used methods for transferring lower bounds from one problem to another. Our methods are based on interpretations between theories (or equivalently, between classes of models) and can be seen as an extension of the most widely used methods for proving the undecidability of logical theories; see Ersov et al. [1965] and Rabin [1965] for a discussion of undecidable theories from this point of view. To obtain complexity lower bounds for decidable theories we must use interpretations which have a somewhat more general form than those used in undecidability proofs, and there are certain technicalities about lengths of formulas which must be addressed in this more general setting. In this section we will develop the required machinery. The first-time reader may wish to skip the proofs in this section as they are somewhat tedious and only the statements of results will be used later. A common method for proving that a theory £ in a logic L is undecidable is to show that the theory So of finite binary relations, formulated in the logic L0, can be interpreted in In the simplest case this means that formulas (x,u) and (x,y,u) of L are given so that every finite binary relation can be obtained (up to isomorphism) in the form
for a model of £ and m a sequence of elements of 91. The formulas 6 and are then used to define a reduction from formulas of L0 to formulas of L, as follows: given a formula of L0, replace every occurrence of an atomic formula P(z, t) by the formula (Z, t, u) and relativize every quantifier to the formula . (One must rewrite bound variables to avoid conflicts and make sure that u is a sequence of otherwise unused variables.) Call the resulting formula . The reduction mapping is then used to obtain undecidability results for from corresponding results for E0. This kind of simple interpretation is not adequate for obtaining lower complexity bounds when £ is a decidable theory. One works instead with a parameterized family of formulas { n \ n > 0 } from L0 and uses a sequence of formula pairs { (n(x,u),n(x,y,u)) | n > 0} from L. In reducing the formulas n to L: one proceeds as above, except that n is obtained from using n and n. In complexity lower bound arguments, it is not only n necessary that the function n n should be efficiently computable, but also that \'n\ should be linearly bounded in \n\. If P occurs many times in n and |n| grows without bound as n increases, or if n has many quantifiers and |n| grows without bound as n increases, then the linear
Proving lower bounds
141
boundedness condition may not hold. However, in certain cases there are methods to efficiently replace 'n by an equivalent formula for which the linear boundedness condition does hold. Roughly speaking, we can do this when the formulas n and n are all in prenex form, or are obtained by a certain kind of iterative procedure. The machinery developed here to accomplish this task is implicit in most complexity lower bound arguments for logical problems. In order to describe this machinery, it is convenient to introduce an extension L* of each logic L, in which explicit definitions are allowed. (L* has no more expressive power than L, but properties can sometimes be expressed by shorter formulas in L* than in L.) Continuing the example above, let Dn denote a formula in which all quantifiers of n have been relativized to a new unary relation symbol D. Then the extended language L* in this case would include a formula
whose interpretation is exactly the same as that of 'n, although its length is likely to be more under control. Here the equivalences in brackets are interpreted to mean that P is explicitly defined by n and that D is explicitly defined by n. The general problem, treated below in this section, is to find situations in which certain formulas of the extended language L* can be efficiently reduced to equivalent formulas of L, without a significant increase in the length of the formulas. (In general, it is possible to find for each L* formula of length n an equivalent L formula of length O(n log n); this is not good enough for sharp complexity bounds.) Let L be either a first-order or monadic second-order logic. Define L* as follows. Formulas of L* may contain any of the symbols occurring in formulas of L and, in addition, relation variables Sji for each i, j 0. In each case the arity of Sij is j and the subscript and superscript of S? are expressed in binary notation. (If L is a monadic second-order logic we need two superscripts, the first denoting the arity of element arguments and the second denoting the arity of set arguments.) Subscripts and superscripts of relation variables contribute to the length of formulas in which they occur, just as element variable subscripts do. (However, superscripts may be ignored in asymptotic estimates of formula length because they are dominated in length by their corresponding argument lists.) We define the set of formulas of L* inductively, and at the same time define free( ), the set of free variables in (p. An atomic formula of L* is either an atomic formula of L or a formula P(x1 ,...,£_,•) where P denotes a relation variable—in the former free(( ) is the same as in L; in the latter, free((p) — {P,xi,.. - , X j } . More complex formulas may be constructed using the logical connectives and quantifiers appropriate to L; in these cases free(( ) is defined just as in L. The only other way to construct more complex
142
Kevin J. Campion and C. Ward Henson
formulas is by explicit definition. Let and 0 be formulas in L*, P a, relation variable which does not occur freely in 6, and x = xi,...,Xj a sequence of distinct element variables. Then given by
is also a formula of L* and free The part of within brackets is an explicit definition which defines the interpretation of P in . If 0 is a prenex formula from L we will say that it is a prenex definition. The truth value of is the same as that of the second-order expression
Notice that the truth value is consistent with the definition of free( Notice also that the second-order expression above is equivalent to
).
so that (P(x) is equivalent to [P(x) . If free ( ) = 0, then is a sentence of L*. We will let sat*( ) denote the set of sentences from L* true in some model of S, satT( ) denote the set of sentences from L* true in some model of of size at most T(| |), satT(L) denote the set of sentences from L* true in some model of size at most T(| |), and inv*(L) denote the set of sentences from L* true in no model (or no weak model when L is a monadic second-order logic). Introduction of explicitly defined relations is standard practice in mathematical discourse. Explicitly defined relations are also similar to nonrecursive procedures in programming languages. Explicit definitions can be used to define reductions between satisfiability problems. To provide good lower bounds these reductions must be efficiently computable and linearly bounded. We will show, in fact, that there are reset log-lin reductions, defined on certain subsets of sentences from L*, that take formulas to equivalent formulas in L. (Unfortunately, such reductions probably cannot be defined on the set of all sentences in L*; with a little effort we can produce a polynomial-time reduction which maps sentences in L* of length n to equivalent sentences in L of length O(nlogn).) We inductively define positive and negative occurrences of a relation symbol Q in formulas from L*. Q occurs positively in atomic formulas
Proving lower bounds
143
of the form Q(x). Q occurs positively (negatively) in the formulas and V when it occurs positively (negatively) in either of the formulas ( or ). Q occurs positively (negatively) in the formula if it occurs negatively (positively) in the formula . Q occurs positively (negatively) in the formulas and x(p if it occurs positively (negatively) in the formula . Q occurs positively (negatively) in the formula [P(x) , where P is not Q, if it occurs positively (negatively) in , or if it occurs positively (negatively) in 9 and P occurs positively in , or if it occurs negatively (positively) in 9 and P occurs negatively in . We say that P occurs only positively in a formula if it does not occur negatively (in particular, it may not occur at all). We inductively define an iterative definition [P(x) ]n as follows. The iterative definition [P(x) = 0]0 is equivalent to the explicit definition [P(x) = 1], where L is a sentence false in all models (or all weak models if L is a monadic second-order logic). The iterative definition [P(x) ]n+l is equivalent to We make iterative definitions part of the syntax of L*, but we require that the subscript n be written in unary notation so that the length of an iterative definition is of the same order (O(n) and (n)) as the length of the nested explicit definitions it replaces. We call 9 the operator formula for the iterative definition. We can think of iterative definitions as approximations to implicit definitions. We will not formally define implicit definitions, since they do not figure directly in what follows, but an example should convey the idea. (See Moschovakis [1974] for an account.) Consider a language containing just a binary relation symbol E denoting the edge relation on graphs. The implicit definition
[P(x, y) = (x = yVz
(P(x, z) A E(z, y)))]
defines the path relation in each graph: P(x, y) is the least relation satisfying the equivalence, so it holds precisely when there is a path between x and y. Now consider the related iterative definition [P(x, y) = (x = yVz
(P(x, z)
E(z, y)))] n
which defines a relation P(x,y) which holds precisely when the distance between x and y is at most n — 1 (when n > 1). Notice that this 'approximation' to the implicitly defined relation does not converge very rapidly. The iterative definition
[P(x, y)
(x = yV E(x, y) V z (P(x, z) A P(z,
144
Kevin J. Compton and C. Ward Henson
defines a relation P(x, y) which holds precisely when the distance between x and y is at most 2 n - 1 (for n 1), so this approximation to the path relation converges exponentially 'faster'. For an implicit definition to make sense, = (P) should be monotone in P (i.e., for every structure , if P and P' are relations on 21 with PC.P', then 0 (P) C 0 (P')). Monotonicity can be guaranteed by requiring that P is positive in 6. No such restriction is needed for iterative definitions. In most of our applications P does occur positively and the iterative definitions approximate an implicit definition. Usually, the faster the convergence, the better the lower bounds obtained by our methods. We will see that the positivity of P in 0 does have implications in lower bound results. To show that we can efficiently transform iterative definitions into equivalent explicit definitions, we require the following theorem, which will also be used to show that certain sets of formulas in L* can be efficiently transformed into equivalent formulas from L. Theorem 3.1. Let L be a first-order or monadic second-order logic and let L' be a logic, of the same type, whose vocabulary consists of the vocabulary of L together with relation symbols P1 , . . . , Pm . There is a reset log-lin reduction taking each prenex formula of L' to an equivalent prenex formula of L' having at most one occurrence of each Pj . Proof. The proof follows an argument of Ferrante and Rackoff [1979, pp. 155-157]. We must add some details, however, because they were not interested in obtaining a reset log-lin reduction. We adopt the same assumption they did there: we assume that L has a symbol for equality and that all structures have cardinality at least 2. We could dispense with this assumption at the cost of added complications. We deal explicitly only with the case m = 1. It will be clear from the proof that the procedure can be iterated to treat P1 , P2 , . . . in succession. We describe the action of our algorithm on , a prenex formula from L. First add a 0 bit to the end of every variable index occurring in . This will allow us to introduce variables of odd index without creating a conflict. Now is of the form
where each Qi is a quantifier and
is quantifier- free. Let
be all the subformulas of containing P1 . The idea is to replace each subformula P I ( X i l , . . . ,xil) of with a Boolean variable and stipulate with a formula containing just one occurrence of P1 that each of these Boolean variables has the same truth value
Proving lower bounds
145
as the formula it replaces. Since we have no Boolean variable type, we instead replace each subformula PI xil ,...,xil) with an equation v\ = Vb(i) . We must ensure for each i that b(i) is odd and greater than 1, that b(i) is log-lin computable from PI (xil, . . . ,xil) (with no resets), and that b(i) b(j) when i j. To produce b satisfying these conditions suppose that Xil , . . . , xil are formal variables denoting actual variables with subscripts ji, . . . ,ji respectively. In the string j l #j2# • • • #j1 replace every occurrence of 0 with 01, of 1 with 11, and of # with 10; let the result be b(i). Let be the result of replacing each formula PI (xil,, . . . ,xil) in by the formula v1 = ub(i). Now with a little effort we can see that is equivalent to
where y and y* — y1 , . . . , y1 denote variables with odd indices of odd length. It is not difficult to verify that this formula is reset log-lin computable from (f using two resets. I Remark 3.2. The formula in the proof of Theorem 3.1 uses the symbol . If we require that formulas use only the Boolean connectives A, V, and , we must expand the subformula v1 = y P 1 ( y ) of ' to obtain a formula in which P occurs twice, once positively and once negatively. It is easy to see that this is the best we can do. Suppose that the number of occurrences of P in such a formula could be reduced to one. If this occurrence were positive, then ' would be monotone in P (i.e., truth is preserved when the interpretation of P is expanded). If this occurrence were negative, then would be monotone in P. But it is easy to produce a formula such that neither it nor its negation is monotone in P. However, in the case where P occurs only positively in , we can construct in the proof of Theorem 3.1 using the subformula V1 = y -> P\(y ) in place of v1 = y P 1 ( y ) . For this case the theorem is true even if just the connectives A, V, and are allowed. This is one of the advantages of using positive formulas. Theorem 3.3. Let L be a first-order or monadic second-order logic and I be a fixed positive integer. There is a reset log-lin reduction which taking each iterative definition of the form [P(x) = 0]n, where 0 is a formula from L* of length at most I, to an equivalent explicit definition. Proof. Since there are only finitely many formulas from L* of length at most I, we may, given such a formula 0, find an equivalent prenex formula 0' from L in constant time. Moreover, by Theorem 3.1 we may assume that P occurs in just once, say in a subformula P(y). Define formulas 0n = n(x) by induction on n. Let 0 be a sentence false in all models
146
Kevin J. Compton and C. Ward Henson
(or all weak models if L is a monadic second-order logic). Form i+1 by substituting the variables y for corresponding free variables x in 0i (perhaps changing other variables to avoid conflicts) and substituting the result for P ( y ) in '. It is clear that [P(x) = ]n is equivalent to [P(x) = n]. If the substitution of variables has been done in a systematic way in the construction of n, then it is clear that n can be obtained from [P(x) = 9] by a reset log-lin reduction. I Often we need to make several iterative definitions simultaneously. For example, Fischer and Rabin [1974], in their lower bound proof for the theory of real addition, define sequences of formulas m n ( x , y , z ) and n(x, y,z). Formula m n (x, y,z) holds precisely when a; is a nonnegative integer less than 22n and x • y = z; formula n(x, y, z) holds precisely when x, yx, and z are nonnegative integers less than 22n and yx = z. These definitions are simultaneous: the definition of n+1, for example, depends not only on n, but also on mn. Let us make the notion of simultaneous definition precise. Let L be a first-order or monadic second-order logic and 1 , . . . , k be formulas from L*. A simultaneous iterative definition is denoted
Fix a structure for L and an assignment from to the free variables of this definition (defined in the obvious way). The simultaneous iterative definition assigns a relation from the universe of to each symbol P1,...,P k . We define this assignment by induction on the depth n of the definition. When n = 0 it assigns the empty relation to each symbol. When n > 0 the assignment to 6i is determined by letting the assignment for depth n — 1 interpret the free occurrences of P1,...,P k in 0,. We can use simultaneous iterative definitions to augment the syntax of a logic in the same way we used iterative definitions. In particular, subscripts on definitions are expressed in unary notation. The following theorem shows that simultaneous iterative definitions do not increase the expressiveness of a logic. Moreover, their use does not make for appreciably shorter expressions than use of ordinary iterative definitions. The theorem is proved along the same lines as similar results in Fischer and Rabin [1974] and Ferrante and Rackoff [1979, p. 159]. Moschovakis [1974, p. 12] used similar ideas to prove an analogous theorem for simultaneous implicit definitions. Theorem 3.4. Let L be a first-order or monadic second-order logic and P1,..., Pk be fixed relation variables. There is a reset log-lin reduction taking each formula of the form
Proving lower bounds
147
Pi(x1) = 1
where and 1,... ,0k are formulas from L* whose only free relation variables are P1 ,..., Pk, to an equivalent formula of the form
where ' and ' are formulas from L* whose only free relation variable is P. Moreover, if P1, . . . ,Pk occur only positively in each of the formulas 1,.. . ,0k, then we may arrange that P occurs only positively in . Proof. As before, we assume that L has a symbol for equality and that all structures have cardinality at least 2. Again, we could dispense with these assumptions at the cost of added complications. Without loss of generality, we may assume that the variable sequences x1 , . . . , xk are mutually disjoint. Let z denote a sequence z , Z 1 , . . . , Z k of distinct variables disjoint from x1 , . . . , Xk • The idea of the proof is that one relation P ( z , x 1 , . . . , X k ) will code the relations PI (x1 ) , . . . , Pk (xk ). To be more precise, the relation P(z, x1 , . . . , Xk) is equivalent to
Thus, a particular Pi(Xj) can be extracted by writing
Call this formula 5i(x) (or Si for short). Define ' to be the L* formula
Notice that P is the only free relation variable in
Now we can easily show that
. Let
be the formula
148
Kevin J. Compton and C. Ward Henson
is equivalent to [P(x)
]n
' by induction on n.
Remark 3.5. Notice that in the proof of Theorem 3.4 formula is formed simply by inserting explicit definitions of fixed length before . These definitions may be eliminated by replacing relation variables in with their corresponding definitions. Now if is a prenex formula or a member of a prescribed set of formulas (defined below), it is easy to arrange that is a formula of the same type. We can now say precisely which kinds of definitions are used in the reductions described at the beginning of this section: they are prenex definitions and iterative definitions. It is useful, therefore, to have terminology to describe sets of formulas in L* built up from prenex formulas using prenex and iterative definitions. We must place some restrictions on these sets to be able to efficiently translate them into equivalent formulas from L. Let L be a first-order or monadic second-order logic. Let L' be the logic formed by adding relation variables PI, ..., Pk to the vocabulary of L, and I be a fixed positive integer. A prescribed set of formulas over L is a set of formulas of the form
where is a prenex formula from L', and for each i either n, = 1 and 6i is a prenex formula from L' in which only P1, ..., Pi-1 may occur as free relation variables (i.e., Pi has a prenex definition), or is a formula of length at most l from L* in which only PI ,..., Pi may occur as free relation variables (i.e., Pi has an iterative definition in which the operator formula has bounded length). We place one further restriction on sets of prescribed formulas: each variable is quantified at most once in and in each formula i where Pi has a prenex definition. We impose this condition so that when we relativize all the formulas within a set to a unary relation symbol D, there is a reset log-lin reduction taking resulting formulas to equivalent formulas from another prescribed set of formulas. The condition is easy to satisfy in practice. We now present our fundamental theorem for making reductions between formulas. Theorem 3.6. Let L be a first-order or monadic second-order logic. For each prescribed set of formulas over L there is a reset log-lin reduction taking each formula in the set to an equivalent formula in L.
Proving lower bounds
149
Proof. Fix a prescribed set of formulas over L. There are relation variables P1 , . . . , Pk as in the definition such that all formulas in the set are of the form where is a prenex formula in which only P1,..., Pk, may occur as free relation variables, and for each i either ni = 1 and i is a prenex formula in which only P1,...,P,_i may occur as free relation variables, or , is a formula of length at most l in which only P1,..., Pi may occur as free relation variables. At first glance it may seem that P1 , . . . , Pk are being defined simultaneously, but this is not the case. First P1 is assigned a value by an iterative definition of depth n1 which is substituted in the remaining definitions. Then P2 is assigned a value by the next iterative definition of depth n2 which is substituted in the remaining definitions, and so on. The proof combines this observation with the construction used in Theorem 3.4. As in that theorem, we will code the relations P 1 ( x 1 ) , . . . , Pk(xk) into a single relation P ( y ) equivalent to
where y is the variable sequence z , z 1 , . . . , Z k , x \ , . . . , X k be the formula
As before, let
To construct a formula from L equivalent to we build, inductively, a sequence of formulas (y ) , (y ) • • • , k (y) • Begin by taking 0 to be a sentence false in all models (or weak models, if L is a monadic second-order logic). Suppose now that i-1 is given. Consider the simultaneous definition
This definition simply defines Pi(Xi) to be i and leaves the other relations unchanged. Use the construction in the proof of Lemma 3.4 to produce an
150
Kevin J. Compton and C. Ward Henson
equivalent definition [P(y ) ni(y)]. Hence, 77, is
where
is the formula
We claim that there is a reset log-lin reduction taking ni to an equivalent prenex formula ni with just one subformula P(u) in which P occurs. Whether Pi has a prenex definition, in which case , is in prenex form, or an iterative definition, in which case is of bounded length, there is a simple reset log-lin reduction to convert , into prenex form. Apply the reduction given by Theorem 3.1 to the result to obtain an equivalent formula in which each of the symbols PI , . . . , Pk occurs just once. In this formula, for each j, substitute j(yj) for the subformula Pj(yj)- Convert to prenex form again by a reset log-lin reduction and apply the reduction of Theorem 3.1 one more time to obtain n' as desired. Notice that if Pi has an iterative definition, ni, has length less than some constant determined by / and the arities of P1 , . . . , Pk. If Pi has a prenex definition, form by substituting -\(u) for P(u) in n'i- If Pi has an iterative definition we must make several substitutions. Beginning with i-1, replace free variables with the corresponding variables u and substitute the result for P(u) in n'i. Repeat this operation ni, times. The resulting formula is . In either case it is easy to see that i is obtained by a reset log-lin reduction. Since is in prenex form, we can apply the reset log-lin reduction of Theorem 3.1 to obtain an equivalent prenex formula ' in which each of the symbols P1,...,P k occurs at most once. As before, there is a reset log-lin reduction to convert
into a prenex formula with just one subformula P(u) in which P occurs. Substitute k(u) for this subformula to obtain finally '. Repeated use of closure of reset log-lin reductions under composition shows that the mapping is reset log-lin computable. • Remark 3.7. Scrutiny of the preceding proof reveals two useful facts. First, if all the symbols Pi have prenex definitions we can arrange that is in prenex form. Second, if we wish to restrict to formulas in which the only connectives are A, V, and , the theorem remains true providing Pi
Proving lower bounds
151
occurs only positively in i when Pi, has an iterative definition. To see this, observe that by the remark following Theorem 3.1 we can always ensure that the formulas i each contain at most two occurrences of P. This is not a problem when Pi has a prenex definition because i figures only once in the construction of k and there are a bounded number of such definitions. When Pi has an iterative definition we can ensure, again by the remark following Lemma 3.1, that Pi occurs at most once in i since it occurs only positively in i.
4
Inseparability results for first-order theories
Hereditary lower bound results have proofs similar to the classical hereditary undecidability results. Young [1985], for example, modified techniques used in the proof of the hereditary version of Godel's undecidability theorem, which states that all subtheories of Peano arithmetic are undecidable, to show that all subtheories of Presburger aaithmetic have an NTIME (2 2cn ) lower bound. Our starting point is another classical undecidability result— the Trakhtenbrot-Vaught inseparability theorem. Many hereditary undecidability results have been derived from this theorem. Recall that L0 is the first-order logic whose vocabulary contains just a binary relation symbol P. Let fsat(Lo) be the set of sentences of L0 true in some finite model, and inv(Lo) the set of sentences of L0 true in no model. The Trakhtenbrot-Vaught inseparability theorem states that fsat(Lo) and inv(L0) are recursively inseparable: no recursive set contains one of these sets and is disjoint from the other. Trakhtenbrot [1950] showed this for a first-order logic with sufficiently many binary relations in its vocabulary and Vaught [1960; 1962] reduced the number of binary relations to one. To see how this theorem gives hereditary undecidability results, suppose that for some theory £ in a logic L there is a recursive reduction from the sentences of L0 to the sentences of L that takes fsat(L0) into sat( ) and inv(Lo) into inv(L). Clearly sat( ) is not recursive since it separates the image of fsat(Lo) from the image of inv(L0). Moreover, if ' C val( ), then sat( ) sat( ') and sat( ') inv(L) = 0 so sat( ') is not recursive either. Let T(n) be a time resource bound. Recall that sat T(Lo) is the set of sentences in L0 such that is true in a structure of power at most T(l l). Our analogue of the Trakhtenbrot-Vaught Inseparability Theorem states that for T satisfying certain weak hypotheses, satT(L0) and inv(Lo) are NTIME(T(cn))-inseparable for some c > 0. That is, no set in NTIME(T(cn)) contains one of these sets and is disjoint from the other. We show, in fact, that the result is true if we restrict to prenex sentences in L0. Thus, using the reductions between formulas described in the previous section, we can obtain hereditary NTIME lower bounds for theories in much the same way that we obtain hereditary undecidability results. In
152
Kevin J. Compton and C. Ward Henson
the next section we prove an inseparability theorem which gives hereditary linear ATIME lower bounds. Our result is a consequence of the following theorem. Theorem 4.1. Let T(n) be a time resource bound and A be an alphabet. Given a problem A C A* in U c >o NTIME(T(cn)), there is a reset log-lin reduction taking each w A* to a prenex sentence w of L0 such that if w A, then w satT(Lo), and if w , then w inv(L0). Moreover, each variable occurring in w is quantified just once. Proof. Let M be a T(cn) time-bounded nondeterministic Turing machine that accepts A. We may assume that on all inputs, all runs of M eventually halt since we may incorporate into M a deterministic 'timer' which halts after some number of moves given by a fully time-constructible function dominating T(cn). To simplify notation we assume that M has just one tape. Extending to multitape Turing machines requires only minor modifications. Let m be the number of tape symbols used by M. We assume that one of the tape symbols not in A is a blank symbol, denoted #. The proof has two parts. In the first we translate information about runs of M on input w into formulas 'w in a logic with a vocabulary consisting of m + 3 binary relation symbols so that 'w satisfies the conditions of the theorem. In the second we transform sentences 'w into the desired sentences w by combining m + 3 binary relations into one. Translating Turing machine runs into first-order sentences is an old idea in logic; see Turing [1937], Buchi [1962], and, for a general discussion, Borger [l984a; 1984b; 1985]. Our translation of runs into sentences is standard except for some difficulties that must be overcome to obtain prenex sentences using reset log-lin reductions. First we describe the intended meanings of the m + 3 binary relation symbols constituting the vocabulary of the logic for sentences 'w. The symbol will interpret a discrete linear order with a least element. For convenience we use infix notation with this symbol. We call the least element with respect to this order 0 and denote the successor and predecessor of an element x by x + 1 and x - 1, respectively. In this way we identify elements of a model with consecutive nonnegative integers. We also have relation symbols STATE, HEAD, and SYMa for each a A. STATE(x,t] holds if M is in state x at time t. (States are ordered arbitrarily. We may assume that all models considered have at least as many elements as M has states since we can precede w with enough dummy quantifiers to ensure that T ( ' w ) exceeds the number of states.) HEAD(x,t) holds if the read head of M scans the tape cell at position x at time t. SYMa(x,t) holds if the tape cell at position x contains symbol a at time t. Let 'w be a prenex sentence asserting the following. (a) Relation is a discrete linear order with a least element. (b) Each tape cell contains precisely one symbol at each time. The
Proving lower bounds
153
read head scans precisely one cell at a given time. M is in precisely one state at a given time. (c) If HEAD(x, t) does not hold and SYMa(x, t) holds, SYMa(x, t + 1) also holds. If HEAD(x, t) holds, then the values of SYMa(x,t + 1), HEAD(x ± I, t + 1), and the element z making STATE(z, t + 1) true are determined by the values of SYMa(x, t), and the element y making STATE(y, t) true, in accordance with the transition function of M. (d) The read head initially scans cell 0. M begins in its initial state. (e) If STATE(x,t) holds, then t has a successor if and only if x is a final state. For some t there is a final state x such that STATE(x, t) holds. (f) The input tape initially contains w. Notice that (f) is the only conjunct depending on w; the others are fixed. Thus, if we can express (f) as a prenex sentence obtainable from w by a reset log-lin reduction, it is a simple matter to produce a prenex sentence 'w equivalent to the conjunction of (a)-(f) obtainable from w by a reset log-lin reduction. Suppose that w = a0a1...an-1 where each ai is an input alphabet symbol. We cannot just say that there are positions V 0 , V I , . . . ,vn-1 such that U0 = 0, Ui+i = Vi + 1, and SYMai (vi, 0) for i < n because the combined length of variable indices is (n log n) . We must define two relations LEFT and RIGHT to reduce the number of quantified variables. Intuitively, LEFT and RIGHT interpret the left and right child relations for a binary tree T on the first n elements 0, . . . , n — 1 of the model. It is intended that LEFT(x, y) holds precisely when y = 2x and RIGHT (x, y) holds precisely when y — 2x + 1. Then T has root 0 and 0 is its own left child (so the notion of tree is interpreted somewhat loosely). Using the machinery of the previous section it is easy to give short formulas defining LEFT and RIGHT on long intervals. Let 0 be the formula
Then the relation P(X1, x2, y1, y2) given by the iterative definition
is true when 0 y1—x1 2m—1 and 2(y1 -xt) = y2-x2. Now LEFT(x, y) is equivalent to P(0, 0, x, y) and RIGHT(x, y) is equivalent to P(0, l,x, y) on the interval 0, . . . , 2m + 1, so we take m = [log(n — 1)] to obtain RIGHT and LEFT on the interval 0, . . . , n. By Theorem 3.3 there are first-order formulas 'n and "n defining LEFT and RIGHT; moreover, they are computable from the unary representation of m by a reset Turing machine in
154
Kevin J. Compton and C. Ward Henson
time log n and space log log n. By increasing time to log n log log n we can make 'n and prenex formulas. The height of T is h = [log n] . Now for every i such that 0 i < h and every j < 2h-j define a quantifier-free formula i j ( x 0 , . . . , Xi) by induction on i. Roughly, i,j(xo, ... ,xi) says that if (xj,Xj—1,. .. ,x 0 ) is a path in from vertex j = Xi, then the symbol at position X0 on the input tape is a X 0 . First, 0 , J ( X 0 ) is S Y M a j ( x 0 , 0 ) when j < n and some tautology (say X0 = X0) when n j < 2h. Next, 0i +1,j (xo, . . . , x,+i) is the formula
By induction on i we can show that for j
2h–i
the sentence
is true if and only if for every vertex k < n which is an ith-generation descendant of j, S Y M a k ( k , 0 ) holds. Since 0 is a left child of itself, every vertex in T is an hth-generation descendant of 0, so the sentence
says that SYMa k (k, 0) holds when k < n; that is, it says w is written on the first n cells of the input tape at time 0. Let w be a conjunction of this sentence and a sentence that says SYM#(x, 0) holds for all x > n (that is, that all the tape cells from position n onward are blank at time 0). We must show that there is a reset log-lin reduction taking w to w , from which it follows easily that there is a reset log-lin reduction taking w to
w
We describe the actions of a machine effecting such a reduction. For the conjunct asserting that cells from position n onward are blank this is straightforward, so we need only show it for the conjunct asserting w is written on the first n cells. We will suppose that indices of variables are in unary; thus, the formal variable Xi denotes the actual variable
First, w is read from the input tape while h cells are marked off on a work tape. One way to accomplish this is simply to keep a count on a work tape of the number of input tape cells scanned. Count in binary. Incrementing the count requires changing the low order 1-bits to 0 until encountering a 1, which is changed to 0. The work tape head is then returned to the
Proving lower bounds
155
lowest-order bit to prepare for the next advance of the input head. It is not difficult to show that the time required to read the input tape and do all the increments is O(n). Next the input head is reset. Now a simple algorithm utilizing a stack to keep track of subscripts will generate w. The maximal stack height is h. Formula w was defined in such a way that the information required from the input tape can be read off from left to right as the algorithm proceeds. Variable indices are easily computed from the stack height since they are in unary. This computation clearly uses just log space. We need show that it takes just linear time. The time required is less than a constant multiple of the length of h, o(x0, • • •, Xh)', the length of this formula is in turn less than a constant multiple of the combined lengths of variable indices occurring within it. By induction on i, variable Xk occurs no more than 3 • 2i-k times in ij when k < i and Xi occurs just twice. Hence, the combined lengths of variable indices occurring in h,o amounts to no more than
Thus, the computation requires just linear time. Notice that each variable in w is quantified just once so it is easily arranged that the same is true
of
w-
If necessary we can add dummy quantifiers at the beginning of 'w to make its length at least cn. Thus, if w € A, then 'w is true in some model of size at most T(|w|). If w , then 'w is true in no model. Suppose, on the contrary, that 'w is true in some model . Consider the submodel of obtained by restricting to the elements which can be reached from 0 by finitely many applications of successor. The values of STATE (x,t), HEAD(x,t), and SYMa(x, t) on this submodel describe a run of M on input w. Since we have incorporated a timer which halts M on every run, this submodel is finite. But then the last element t in this submodel has no successor, so the state x for which STATE(x, t) holds is final. Thus, M accepts w, a contradiction. This completes the first part of the proof. We now show how to combine m + 3 binary relations into one. To simplify notation, let us rename the relation symbols P0,P1,..., Pm+2. Suppose ' is a model of 'w. Before describing how to transform 'w into w, we describe the model of w corresponding to '. First form the disjoint union of the interpretations of P0, P1,..., Pm+2 in '. We have then relations R 0 ,R1,...,R m+ 2 on disjoint domains B0, B1,.. .,Bm+2. Their union is a single binary relation R on the domain B = <m+2 Now enlarge R so that R (B0 x Bi) is the natural bijection from B0 to Bi
156
Kevin J. Compton and C. Ward Henson
when 1 i m + 2. Next, enlarge B by adding elements bo,b0,..., and then add the pairs (bi, b) to R for each i m + 2 and b Bi. Define to be (B, R}. We see that if bo, bi,..., bm+2 are known, then the relations RO , R1 , . . . , Rm+2 can be recovered in and in fact we may use the natural isomorphisms from B0 to Bi to define an isomorphic image of ' with domain B0. Relativize the quantifiers of 'w to a unary relation symbol D, thereby forming ('w)D. This sentence contains relation symbols D, Po, P1, . . . , Pm+2. Put the explicit definitions [D(x) P(x0,x)], [P0(x, y) = P(x, y)], and, for 1 i m + 2,
before this formula. Here x0, x1 , . . . , xm+2 are new free variables whose intended interpretations in are bo,b1,..., bm+2. Now existentially quantify X0 , X 1 , . . . , xm+2. There is a reset log-lin reduction that takes the resulting formula to an equivalent prenex formula w . (Since each variable in 'w is quantified just once the relativizations may be pushed inward. Then since all of the explicit definitions are of fixed length, conversion to prenex form is straightforward.) If M accepts w, then 'w is true in some model ' of power at most . It follows that w is true in some model of power at most
We can assume this quantity is less than T( w) by lengthening w with dummy quantifiers if necessary and using the definition of time resource bound to infer that (m + 3)T(n) T((m + 3)n). If M does not accept w, then w has no models and hence neither does w. Finally, it is easy to arrange that every variable in w is quantified just once. I Remark 4.2. Inspection of the construction of sentences w in Theorem 4.1 reveals that for every constant b > 0 there are constants c0 and c\ such that \w\ C1\w\ + C0 whenever 0 < c < b. The constant c1 depends only on the size of the input alphabet A and the constant b. (This will be useful in the proof of Theorem 4.5.) On the other hand, the constant C0 depends on the particular Turing machine M whose runs w describes. Corollary 4.3. Let T1(n) and T2(n) be time resource bounds such that NTIME(T2(n)) - NTIME(T1(n))
0.
Suppose that limn 00T1(n)/n = . Then there is a constant c > 0 such that for each set of satis fiable sentences with satT2(Lo) ,
Proving lower bounds
157
Proof. Let A be an element of NTIME(T2(n)) - NTIME(T1(n)), where A C A*. By Theorem 4.1 there is a reset log-lin reduction taking each w € A* to a sentence w in L0 so that A is mapped into satT2(L0) and A* - A is mapped into inv(L0). Suppose that this reduction takes at most time b\w\. Let c = 1/6 and be a set of satisfiable sentences containing satT2(Lo). Suppose that NTIME(T1 (cn)). To decide if w £ A we will compute w in time b\w\, and then determine if w using a T1(cn) time-bounded nondeterministic Turing machine. Certainly \w\ b\w\ so the composition of these two reductions takes time at most b\w\ + T1(bc\w\). We know that \w\ TI(\W\) so this time is bounded above by (b + l)T1(|w>|). Since limn T1(n)/n = we can apply the linear speed-up theorem (see Hopcroft and Ullman [1979]) to show that A € NTIME(T1(n)), a contradiction. Therefore, NTIME(T1(cn)). Remark 4.4. We see from Corollary 4.3 that application of our results relies on the ability to separate nondeterministic time complexity classes. The strongest result in this direction for time resource bounds in the range bounded above by e x p ( n ) is due to Seiferas et al. [1978]. It says that if T2(n) is a time resource bound and T 1 ( f ( n + 1)) 6 (T 2 (f(n))) for some recursively bounded, strictly increasing function f(n), then NTIME(T2(n)} - NTIME(T1(n))
0.
This theorem has interesting implications for us. Let T(n) be a time resource bound. Take T1(n) = T(dn), where 0 < d < 1, T2(n) = T(n), and f ( n ) = n. The Seiferas-Fischer-Meyer theorem tells us that if T(dn + d) = o(T(n)), then
NTIME(T(n)) - NTIME(T(dn))
0.
By taking a slightly smaller d the hypothesis may be simplified to T(dn) = o(T(n)). By Corollary 4.3 there is a constant c > 0 such that if is a set of satisfiable sentences with satT(L0) , then NTIME(T(cn)). Most time resource bounds that occur as complexities of theories satisfy the hypothesis T ( d n ) — o(T(n)). Among them are the functions exp, (n) when r 1, 2n/ logn , and 2nk . Powers do not satisfy this hypothesis, so when T(n) = nk where k > 1, we can only conclude that NTIME(g(n)) for all g(n) such that g ( n ) = o(nk). By considering just one time resource bound T ( n ) we gain another advantage: we can prove a fully-fledged inseparability theorem. This will allow us to obtain NTIME lower bounds for problems of the form val( ), as well as problems of the form sat( ). Theorem 4.5. If T is a time resource bound such that for some d between
158
Kevin J. Compton and C. Ward Henson
0 and 1, T(dn) = o ( T ( n ) ) , then there is a constant c > 0 such that and inv(L0) are NTIME(T(cn))-inseparable. Proof. Corollary 4.3 and the preceding remark show that there is a c > 0 such that if satT(L0) C and inv(L0) = 0, then NTIME(T(cn)). (The corollary applies since limn T(n)/n = when T(dn) = o(T(n)).) We must show that there is a c' > 0 such that if satT(Lo) and inv(Lo) = 0, then , the set of sentences from L0 not in , is not in NTIME(T(c'n)). Let b be a positive constant (say 1). By Theorem 4.1, if A C A* is in NTIME(T(c'n)) for 0 < c' < b, there is a reset log-lin reduction taking each w 6 A* to a sentence w of L0 such that A is mapped into satT(Lo) and A is mapped into inv(L0). We know also from the remark following Theorem 4.1 that \w\ CI\W\ + c0, where c1 is a constant depending only on A and b, not on d or A. Take c' small enough that c'c1 < c. Now consider a set such that satT(L 0 ) and inv(Lo) 1 = 0. We must show that NTIME(T(c'n)) so suppose the contrary. We can take A in the previous paragraph to be so there is a reset log-lin reduction mapping into satT(Lo) and into inv(Lo) . This reduction takes an input w to a sentence w in time at most b\w\. Thus, we can determine whether w by computing w and determining in nondeterministic time T(c'n) if w € F. Hence, NTIME(bn + T(c'(c0n + c1))) C N T I M E ( T ( c n ) ) a contradiction. Remark 4.6. For simplicity, Theorem 4.5 was stated for sets and inv(Lo). By Theorem 4.1 it remains true if we take instead and the set of prenex formulas in inv(L0). Similarly, Theorem 4.3 holds if we restrict to prenex sentences.
5
Inseparability results for monadic second-order theories
In this section we develop inseparability results for monadic second-order theories analogous to those for first-order theories in section 4. The appropriate complexity classes here are the linear alternating time classes rather than nondeterministic time classes. Because linear alternating time classes are closed under complementation, we do not need a special argument like the one used in Theorem 4.5 to obtain inseparability results. Also, lower bounds are obtained by simple diagonalization, rather than more sophisticated results such as the theorem of Fischer, Meyer, and Seiferas used in the previous section.
Proving lower bounds
159
As before, inseparability results are closely related to satisfiability problems that are hard for certain complexity classes. The classes are of the form
ATIME(T(cn),cn) which is in many ways more natural than
ATIME(T(cn),n). If there is a reset log-lin reduction from a problem to a class of the first form, then we may conclude that is also in the class. We know of no speed-up theorem for alternations, so we cannot make the same claim for classes of the second form. One of the main results of the section is Theorem 5.2, an analogue of Theorem 4.1. We could prove this result along the same lines as Theorem 4.1, but we obtain a somewhat sharper result if we appeal to a result of Lynch [1982] relating nondeterministic time classes to the spectra of monadic second-order sentences. Lynch encodes Turing machine runs in a way different from the classical method used in the last section. Rather than explicitly accounting for symbols at each tape position and time in a machine run, he keeps track of just the symbol changed (not its position), the symbol which replaces it, and the direction of head movement at each time. If the underlying models have enough structure, it is possible to express derivability between instantaneous descriptions of nondeterministic Turing machines with just this information. Lynch shows, in particular, that this is the case if the underlying models have an addition relation PLUS(x, y, z) which holds when x + y = z. We begin, therefore, by considering the monadic second-order logic ML+ whose vocabulary contains just a ternary relation symbol PLUS, and M+, the monadic second-order theory of addition on initial segments of the natural numbers. M+ can be axiomatized by a set of first-order sentences. Explicitly define a relation by
Then M+
says that
is a discrete linear order with least element 0 and
(The immediate predecessor of an element x is denoted z— 1; the immediate successor is denoted x + 1.) Note that even though M+ consists of firstorder sentences, satT(ME+) is the set of monadic second-order sentences true in some model of M+ of size at most
160
Kevin J. Compton and C. Ward Henson
Theorem 5.1. Let T(n) be a time resource bound and A an alphabet. Given a problem A C A* in Uc>o ATIME(T(cn),cn), there is a prescribed set of sentences over ML+, and a reset log-lin reduction taking each w . A* to a sentence w in such that if w € A, then w £ satT (M+), a n di f w , then w inv*(ML+). Proof. Fix c > 0; we may take c to be an integer. Let M be an alternating Turing machine that accepts A in time T(cn) with at most en alternations. As in Theorem 4.1 we may assume that on all inputs, all runs of M eventually halt: incorporate into M a deterministic 'timer' which halts after some number of moves given by a fully time constructible function dominating T(cn). We assume for simplicity that M has just one tape; extending to multitape Turing machines requires only minor modifications. Let a 1 , . . . , am be the tape symbols used by M. Let r = {0,... ,r — 1} be a finite ordinal and + be the usual addition relation on r, so that (r, +) is a model of M+. In this model we represent an instantaneous description of M by a sequence of sets X 1 , . . . , Xm+2 = X, where sets X1,..., Xm partition r and Xm+1 is a singleton set, and Xm+2 is singleton set whose element is one of the states of M. The set of states is identified with an initial interval of r. We intend that the symbol ai is at position x when x Xi, that the head scans position x when x € Xm+i, and that M is in state x when x Xm+2 • We will need to restrict to initial intervals in our models because when we consider truth in weak models at the end of the proof we may not be able to quantify over all subsets, but we can arrange to quantify over subsets of finite initial intervals. Let ID(x, X) be the formula specifying that sets X 1 , . . . , Xm partition the interval [0, x], Xm+1 and Xm+2 are singleton sets contained in this interval, and the element in Xm+z is a state. Recall that each state of M is of one of four types: universal, existential, accepting, and rejecting. Let
be formulas indicating that ID(x, X) holds and the state for the instantaneous description represented by X is of the corresponding type. Lynch [1982] shows that for each nondeterministic Turing machine M' there is a monadic second-order formula nM'(X, y) that holds in (r, +) precisely when X and Y represent instantaneous descriptions for M' and Y can be obtained from X within r or fewer moves of M' by a computation in which the head does not reach a tape position greater than or equal to r. We can regard the alternating Turing machine M as a nondeterministic Turing machine simply by ignoring state types. We also form the nondeterministic Turing machine M' by eliminating transitions out of all states in M except existential states and then ignoring state types, and M" by eliminating
Proving lower bounds
161
transitions out of all states in M except universal states and then ignoring state types. Let (x,X,y) be the formula .
Let n (x,X,y) be the formula
Let
be the formula
That is, (x, X, y) expresses derivability between instantaneous descriptions on the interval [0, x]; ( x , X , y ) ( n ( x , X, y)) expresses the same except that all states, excluding possibly the last, are existential (universal). Notice, in particular, that n ( x , X , X ) holds for all X. Let TERM(x ,X] be the formula and TERM ( x , X ) be the formula
Let
w(x, X) be a prenex sentence asserting the following: (a) Relation is a discrete linear order with a least element and a greatest element; every element other than the greatest has a successor. (b) The relation PLUS(x 0 ,, x1, x2) holds if and only if either x0 = x2 and x\ = 0 or PLUS(xo, x1 — l,x% — 1) holds. (c) There is a set Y with no elements. For every set Y and element y there is a set Y U {y}. (d) If y is an element such that for all Y satisfying ID(y, y), TERM(y, X, y) implies ACC(y, y) or REJ(y, y), then x y. (e) The sequence X represents the instantaneous description for an input tape with w written on it, the head scanning the first position, and M in the initial state. Each of these items can be expressed by a fixed sentence except the part of condition (e) concerning the input tape. The prenex formula expressing this part is constructed in the same way as in Theorem 4.1. As in that construction, it follows that there is a reset log-lin reduction taking w to
162
Kevin J. Compton and C. Ward Henson
Consider any weak model 21 in which conditions (a)—(c) hold. For the moment, let us suppose that the element x from the universe of is finite, i.e., a finite distance from the least element 0. (By condition (a), this is a well-defined notion.) Condition (b) ensures that PLUS restricted to the interval [0,..., x] in is the usual addition relation. Condition (c) ensures that quantification over subsets of {0,..., x} in the weak model is quantification over all subsets of {0,...,a;}. Thus, for finite x, the formulas n ( x , X, y), (x, X, y), and (x, X ,y) have interpretations in corresponding to computations of M as described above. Now in consider x and X satisfying conditions (d) and (e). (We no longer stipulate that x is finite.) Since all runs of M on input w halt, say within k moves, we see that every such x is at most distance k from 0. (Note that this is the case even if there is no y as described in (d).) Thus, such an x will be finite and the observations of the previous paragraph pertain. Let (x, X) be the formula
Let
w
be the sentence
where n = \w\. Clearly, there is a prescribed set of sentences containing every w. It is also clear that if w € A, then w is true in some model of size at most T w because in such models P interprets acceptance by M for x = T(cn). If w , then pw is true in no model. Suppose on the contrary that is true in . Then in there are x and X such that w(x, X) and w P(x, X) hold, where P is given by the implicit definition
By the remarks above, x must be finite, and hence P describes an accepting computation of M on input w, which is a contradiction. I The next theorem, the analogue of Theorem 4.1, follows from the previous theorem and a result of Kaufmann and Shelah [1983]. Theorem 5.2. Let T(n) be a time resource bound and A an alphabet. Given a problem A C A* in c>o ATIME(T(cn),cn), there is a prescribed set of sentences over MLo, and a reset log-lin reduction taking each
Proving lower bounds w A* to a sentence and if w , then w
w
in such that if w € A, then inv*(MLo).
163 w
satT
Proof. By the previous theorem we need only show that there is a formula (x,y, z) from MLo such that for each finite ordinal n = {0, . . . , n — 1} there is a binary relation R on n such that (X, y, z) is an addition relation on n, where = (n, R). Kaufmann and Shelah [1983] prove a much stronger result: there is a formula (x, y, z) such that for almost every binary relation R on n, (a;, y, z) is an addition relation on n, where = (n, R). For the sake of completeness, we sketch a proof of the simpler result that there is a formula that codes an addition relation on 'some binary relation of each finite power. First suppose that the vocabulary for the logic has three binary relation symbols P1, P2, P3, rather than just one, and that they interpret binary relations R1, R2, R3 on m. To simplify the proof we assume that m = r3. It is easy to specify a formula (X) saying the relations RI, R2, R3 restricted to m x X are functions, respectively denoted f1, f2, f3, and that (f1(x),/2(x),/3(x)) ranges over each triple in X3 precisely once as x ranges over m. Thus |X"|= r and we have defined a bijection between m and X3. Since we can quantify over subsets of m, we can quantify over ternary relations on X when (X) holds. Therefore, we can, without much trouble, define an addition relation on X. Also, we can extend this relation to define addition modulo r. But then it is easy to define addition on m using the bijection between m and X3. Using the construction in the proof of Theorem 4.1, the three binary relations R1 , R2 , R3 on a set of size m = r3 can be coded as a single binary relation on a set of size n = 3(m + 1). This set has three disjoint subsets of size m on which addition and addition modulo m can be coded. It is not difficult now to define addition on all of n. Thus, there is a binary relation on n from which an addition relation can be defined when n is of the form 3(r3 + 1). With a little effort this construction can be made to work for arbitrary n. I We now state an analogue of Corollary 4.3. Corollary 5.3. Let T1{n) and T2(n) be time resource bounds such that
ATIME(T2(n),n) - ATIME(T1(n),n)
0.
Suppose that l i m n - + T 1 ( n ) / n =. Then there is a constant c > 0 such that for each set of satisfiable sentences with satT2(MLo) , ATIME(T1(cn),cn). The proof is the same as for Corollary 4.3. Note, however, that we rely on a result that says the linear speed-up theorem applies to alternating
164
Kevin J. Compton and C. Ward Henson
Turing machines. We must also use Theorem 3.6 to obtain a reset loglin reduction from a prescribed set of sentences over ML0 to equivalent sentences in ML0. We also have an analogue for Theorem 4.5. Theorem 5.4. // T is a time resource bound such that for some d between 0 and 1, T(dn) = o(T(n)), then there is a constant c > 0 such that and inv(MLo) are ATIME(T (cn), cn) -inseparable. Proof. The proof is much simpler than that of Theorem 4.5. We can separate ATIME(T(n), n) and ATIME(T(dn), dn) using a straightforward diagonalization, so we do not appeal to the more difficult methods used in separating NTIME classes. Then we use the previous corollary to show that for some c > 0, if satT(ML 0 ) and inv(ML0) = 0 then ATIME(T(cn),cn). Since ATIME(T(cn),cn) is closed under complementation, we have that satT(ML 0 ) and inv(MLo) are ATIME(T(cn),cn)inseparable. • Remark 5.5. By Theorem 5.1, Theorems 5.3 and 5.4 hold with and inv(ML+) in place of satT(MLo) and inv(MLo). It is also important to note that even though we reduced the prescribed sets of sentences in these theorems to equivalent sets of monadic secondorder sentences, it is the prescribed sets which are used to obtain lower bound results. For example, in the proof of Theorem 5.4 we actually showed that there is a prescribed set of sentences over ML0 such that satT(ML0) n and inv*(ML0) are ATIME(T(cn),cn)-inseprable. In sections 6 and 7 we will find lower bounds for various theories £ from logics L by finding a reset log-lin reduction from to ', a prescribed set of sentences over L, so that satT(ML 0 ) is mapped into sat*( ) and inv*(ML0) is mapped into inv*(L) n ' . Thus, for some c > 0 sat*( ) and inv*(L) n are ATIME(T(cn),cn)-inseparable. Then by Theorem 3.6, sat( ) and inv(L) are ATIME(T(cn),cn)-inseparable.
6
Tools for NTIME lower bounds
We present several useful tools for establishing NTIME lower bounds for theories by interpreting models from classes of known complexity. We begin with some definitions regarding interpretations of classes of models and give a general outline of how interpretations are used to obtain lower bounds. Theorem 6.2, a specific instance of the method, follows from the results in section 4. It tells how to obtain lower bounds by interpreting binary relations. We then show in Theorem 6.3 how to interpret binary relations in finite trees of bounded height. As a consequence we obtain hereditary lower bounds for theories of finite trees of bounded height and a tool for obtaining further lower bounds by interpreting classes of these
Proving lower bounds
165
trees in other theories. We obtain similar results for classes of finite trees of unbounded height in Theorem 6.7 and its corollaries. Let £ be a theory in a logic L' and Co, C1,C2, • • • be classes of models for a logic L whose vocabulary consists of relation symbols P1 , . . . , Pk . Let x1 , . . . , Xk, be sequences of distinct variables with the length of xt equal to the arity of Pi. Suppose that there are formulas n(x, u), 1 / n ( X I , U ) , . . ., (xk, u) from L' which are reset log-lin computable from n (expressed in unary notation) so that for each Cn we have a model ' of and elements m in ' with
isomorphic to . The parameter sequence u is allowed to grow as a function of n. The sequence {In | n 0} where
is called an interpretation of the classes Cn in S. The interpretation is a simple interpretation if the formulas n,1/n, . . . , kn are fixed with respect to n; this is the situation traditionally found in logic. The interpretation is a prenex interpretation if the formulas n,1/n, . . . , kn are all in prenex form. The interpretation is an iterative interpretation if the formulas n,1/n . . . , kn* are given by iterative definitions. By this we mean that there are formulas
and integer functions f, g 1 , . . . , 9 k which are reset log-lin computable (using unary notation) such that 5n is the formula given by the iterative definition
and in is the formula given by the iterative definition
as in Theorem 3.3. Notice that we may regard simple interpretations as special cases of either prenex or iterative interpretations. There is a slightly more general point of view toward interpretations which is sometimes useful. Suppose C'0 C'1 C'2 • • • are classes of models of such that for each n 0 and € Cn, the model in the above definition can be found in C'n. Then we will say that we have an interpretation of the classes Cn in the classes C'n. Sometimes we will not mention at all when discussing interpretations in this context. In that
166
Kevin J. Compton and C. Ward Henson
case we must say whether we intend the classes C'n to be models for a firstorder or for a monadic second-order logic. Thus, we will say that there is an interpretation of the classes Cn in the first-order (or monadic second-order) classes C'n. Interpretations, inseparability, and prescribed sets are the cornerstones of our method. Suppose we have an interpretation of classes Cn in classes C'kn for some nonnegative integer k. (In most cases ; is 1 but occasionally we need a larger value.) Suppose also that there is a prescribed set of formulas over L such that {
is realized in some
Cn where
= n}
and inv*(L) are NTIME(T(cn))-inseparable for some c > 0. (A formula is realized in if it is true in for some assignment to its free variables.) Now map each formula in to the formula ' given by
where n = . By adding dummy quantifiers in the right places we can ensure that kn. If is realized in some model in Cn, then is realized in some model in C'kn. If i is true in no model, then ' is true in no model. (There is a minor point which should be addressed here. To be completely rigorous we should require for all models ' that (x) 0 since certain formulas in inv*(L) may become true when relativized to an empty relation. For example, consider the sentence x (x x). We can always meet this requirement by replacing 6n(x) with the formula n(x) V x-n(x) so we can ignore this point in subsequent discussions.) When we have a prenex interpretation or an iterative interpretation and the definitions in ' are replaced by the appropriate iterative definitions, the sentences all belong to some prescribed set of formulas over L'. (If the interpretation is iterative, then by definition the parameter sequence cannot grow with n.) We have now that is realized in some
C'n where
= n}
and inv*(L') are NTIME(T(cn))-inseparable for some c > 0. It follows by Theorem 3.6 that sat( ) and inv(L') are NTIME(T(cn))-inseparable for some c > 0 so £ has a hereditary NTIME(T(cn)} lower bound. Now let us broaden the definition of interpretation to cover instances where the formulas n and 1 n , . . . , kn contain free relation variables which also receive prenex or iterative definitions. For example, if these formulas contain a free unary relation variable Q we would write
Proving lower bounds
167
for , the formulas n having the same sort of restrictions as dn and 1n, ... ,kn. In this case we would have
as the elements of our interpretation. If these formulas have all prenex definitions we could use Theorem 3.1 to rewrite n and 1/n, ... , kn so that Q occurs just once and substitute 9n for occurrences of Q. For iterative definitions, we know of no similar substitution which eliminates Q and keeps the length of linearly bounded in n. Fortunately, such a substitution is not needed and is even undesirable. By taking a top-down approach to the construction of interpretations, building complex relations from simpler relations, we make our task easier and exposition clearer. We extend our definitions of simple, prenex, and iterative interpretation to this more general situation. (In principle, some definitions could be prenex and others iterative, but this does not seem to occur in practice.) We see that hereditary lower bounds are obtained using interpretations to transfer inseparability results from one prescribed set of formulas to another. One of the advantages of this method is that by establishing lower bounds in this manner, we also establish tools for proving further lower bounds. In the situation described above, if we have another prenex or iterative interpretation of classes C'n in classes C'n of models of A, then we can use C'n and in place of Cn and to establish a lower bound for A. Compare with the well-known methods for establishing NP-hardness of a problem by reducing to a problem already known to be NP-hard. After the first lower bound or hardness result has been proved one should never again have to code Turing machines. It is worth noting what happens when the interpretation used to establish a lower bound is not prenex or iterative. In that case we do not know that there is a reset log-lin reduction taking formulas defined above to equivalent formulas in L''. As we mentioned in section 3, the shortest equivalent formula in L' we know how to obtain has length (n log n) in the worst case. We can only conclude that sat( ) and inv(L') are NTIME(T(cn/log n))-inseparable for some c > 0 so £ has a hereditary T(cn/log n) lower bound. Successive applications of such interpretations give increasingly worse bounds. After k interpretations the lower bound would be T(cn/(log n)k) rather than T(cn). In the case where the formulas in are in prenex form already we do not have this loss in the lower bound, but even then, when the interpretation is not prenex or iterative, we cannot use the classes Cn to obtain further lower bounds without a subsequent loss. We have introduced prenex and iterative interpretations to avoid these losses. In our experience prenex and iterative interpretations not only achieve sharp lower bounds, but also are easy to manage and occur quite naturally in applications.
168
Kevin J. Compton and C. Ward Henson
Within this framework we can also accommodate the more general kind of interpretation in which the domain of the interpreted model is not a subset of ', but a set of k-tuples from ', and the equality relation is interpreted by an equivalence relation definable in . We have found this to be necessary for only two theories treated here and so we avoided stating these definitions in the fullest generality. However, it would not have been difficult to introduce these features explicitly. (See Examples 8.12 and 8.14.) Although we have emphasized inseparability results, we should not lose sight of the fact that the starting point for our reductions, Theorem 4.1, is a hardness result: every set -T of satisfiable sentences with satT(Lo) is hard for the complexity class
NTIME(T(cn)) c>0
via reset log-lin reductions and for the class
NTIME(T(nc)) c>0 via polynomial time reductions. Thus, all our inseparability results can be reformulated as hardness results. We summarize the previous discussion and make this point precise in the following theorem: Theorem 6.1. Let Co,C1,C2,... be classes of models such that for some prescribed set of formulas over a first-order logic L, is realized in some
Cn where
= n}
and inv*(L) are NTIME(T(en))-inseparable for some c > 0. Let C'0 C C'1 c'2 • • • be classes of models of a theory in a logic L'. If there is a prenex or iterative interpretation of the classes Cn in the classes C'kn for some nonnegative integer k, then the following are true. (i) The sets sat( ) and inv( ) are NTIME(T(cn))-inseparable for some c> 0. (ii) If for some d between 0 and 1, T(dn) = o(T(n)), then has a hereditary NTIME(T(cn)) lower bound. (iii) For each ' val( ), sat( ') and val( ') are both hard for the complexity class NTIME(T(cn)) c>0 via reset log-lin reductions. (iv) For each val( ), sat( ') and val( ') are both hard for the complexity class
Proving lower bounds
169
NTIME(T(nc)) c>0
via polynomial time reductions, (v) There is a prescribed set of sentences over L' such that '
is realized in some
C'n where
=
and inv*(L') are NTIME(T(cn))-inseparable for some c > 0. Usually when we apply our method we state the result as in (ii) for brevity, but the reader should be aware that all of the conclusions hold. The following result is an immediate consequence of the above theorem and Theorem 4.5. It is one of the most useful tools for establishing NTIME lower bounds. Theorem 6.2. Let T(n) be a time resource bound such that for some d between 0 and I, T(dn) = o(T(n)). Let Cn be the class of binary relations (i.e., structures for L0) on sets of size at most T(n) and a theory in a logic L. If there is an interpretation of the classes Cn in , then £ has a hereditary N T I M E ( T ( c n ) ) lower bound. The first application of this result is to first-order theories of finite trees of bounded height. Recall that we express first-order properties of trees in the logic Lt whose vocabulary contains just a binary relation symbol which interprets the parent-child relation. Let r be the theory of finite trees of height at most r and be the theory of finite trees of arbitrary height. Define MT and M similarly for the monadic second-order logic MLt. Let Tk be the class of finite trees of height k. We inductively define certain restricted classes of finite trees in which the classes Cn in Theorem 6.2 are interpreted. For each m > 0 let Tm0 be the class To (all of whose elements are isomorphic). The class Tmk+1consists of those trees whose primary subtrees are all in km, but such that no more than m primary subtrees may be in the same isomorphism class. Clearly, km C Tk and isomorphism types, then Tmk+1contains at most (m + 1)t isomorphism types. From the following theorem we obtain hereditary lower bounds for theories of finite trees of bounded height and another useful tool for obtaining other lower bounds. Theorem 6.3. Let Cn be the class of binary relations on a set of size exp r _ 2 (n) and m = m(n) be the least integer such that m logm n. Then there is a prenex interpretation of the classes Cn in the first-order classes mr when r > 3 and in the first-order classes T 2m r when r = 3.
Proof. Note that m = O(n/logn).
n}
170
Kevin J. Compton and C. Ward Henson
First, consider the case r > 3. Define m(x,y) to be a formula, with free relation variable Q, which says that for all children t1,...,t m of x, there are children u1 , . . . , um of y such that Q(ti, ui) holds for 1 i < m, and
We wish to write m as a prenex formula which can be computed from n (in unary) by a reset log-lin reduction. Unfortunately, the displayed formula has length (n 2 ) so we replace it with
which is reset log-lin computable from n. Hence, we can write prenex formula which is reset log-lin computable from n. When k > 0, the iterative definition
m
as a
defines an equivalence relation on the vertices of height less than k. For k = 1, Q is an equivalence relation on the leaves, which are precisely the vertices of height 0. This relation makes all leaves equivalent. Increasing k to 2, we must extend the relation to vertices of height 1. These are the vertices adjacent to a leaf and all of their children are leaves. Two such vertices are equivalent if they either have the same number of children or both have at least m children. In general, for larger k we extend the relation to vertices of height k — 1 leaving the relation unchanged for lower heights. Two vertices of height k — 1 are equivalent if for each equivalence class represented among their children (on which the relation has already been defined), they either have the same number of children in the class or both have at least m children in the class. When k = r we have an equivalence relation on the set of all vertices in a tree of height r except the root. By Theorem 3.3 there is a reset log-lin reduction taking the iterative definition
to an equivalent explicit definition
Proving lower bounds
171
Moreover, since r is fixed we can arrange that mr is a prenex formula. We will say that two vertices x and y in a tree of height r have the same mr -type if (x, y) holds. Define n(x) to be a prenex formula that says x is a child of the root, x has at least one child, and no two distinct children of x have the same type. Define nn(x, y) to be a prenex formula that says d(x) and (y) hold and there is a child z of the root coding (x, y). By z coding (x, y) we mean that for every type, if x and y have no child of that type, then neither does z; if x has a child of that type but y does not, then z has precisely two children of that type; if x has no child of that type but y does, then z has precisely three children of that type; and if x and y both have a child of that type, then z has at least four children of that type. (For this coding to work, m must be at least 4, but this is not really a problem because if m < 4, then n < 5 and we can easily formulate interpretations of Cn in 77™ when n < 5.) Both n(x) and n(x, y) are reset log-lin computable from n. Now it is easy to show by induction on k that if x and y are vertices of height at most k, where k < r - 1, and if the two subtrees formed by restricting to x and its descendents, and to y and its descendents, are nonisomorphic trees in T k m, then x and y have different mr types. We know that = (m + l)m+1 >2n and that if |Tkm| = t, then \Tmk+1\ = (m + 1)t so \Tmk\ > expk_1(n) when k 2. Thus, for vertices of height r - 2 there are at least exp r _ 3 (n)mrtypes. If a; is a child of the root satisfying n(x), its children are of height at most r — 2 and it has either 0 or 1 children of each possible mr-type. Thus, it is possible to distinguish between as many as exp r _ 2 (n) vertices x satisfying n(x). Clearly, if 6n(x) and n(y) hold, (x, y) can be coded by some child of the root. It is easy to see that every binary relation on a set of size at most exp r _ 2 (n) is isomorphic to {n(x),7r*(x, y)) for some tree in mr. This concludes the case r > 3. Now we consider the case r = 3. The construction just given shows that every binary relation on a set of size at most 2m = 2°'n/log n' is isomorphic to (n (x) , n (x , y)} for some tree in Tm3 We must work harder to remove the log n denominator in the exponent; to do this we must interpret Cn in T2m3
We begin by specifying formulas m(x, y) and m(x, y) which will define equivalence relations on vertices of height 2. The formula 0'm(x, y) says that for all children t1, . . . , tm of x with at least 1 but no more than m children, there are children u1 , . . . , um of y with at least 1 but no more than m children, such that m2(ti,ui) holds for 1 i m, and ti — tj ui, = uj holds for 1 i,j < m. The formula n' m (x,y) says that for all children t1,...,t m of x with more than m children, there are children u1 , . . . , um of y with more than m children, such that 2m2(ti, ui) holds for 1 i < m,
172
Kevin J. Compton and C. Ward Henson
and ti = tj Ui = Uj holds for 1 i, j m. Using the same argument as above we can say that m(x,y) is a prenex formula which is reset log-lin computable from n and equivalent to 'm(x, y) m(y, x), and similarly for m(x,y). Since m and m define equivalence relations on the set of vertices of height 2 we can speak of the m-type and m-type of a vertex x of this height. Define vi(x) to be the minimum of m and the number of children of x with precisely i children. Define v+i (x) to be the minimum of m and the number of children of x with at least i children. The m-type of x is precisely determined by the values v1(x), . . . , vm(x) and the nm-type by the values v m +1(x), . . . ,v2m-1,v+2m(x).We see then that the m -type and the m -type of a vertex are independent, and that there are mm+1 > 2™ m -types and the same number of m -types. Let n(x) be a prenex formula that says x is a child of the root and V0(X) — 0. Let n (x, y) be a prenex formula that says 6n(x) and 6n(y) hold and there is a child z of the root such that V0(Z) > 1 and m(x,z) and (z,y) hold. It is easy to arrange that n(x) and n(x, y) are reset log-lin computable from n. Each binary relation on a set of size at most 2™ is isomorphic to for some tree in 2m3. Corollary 6.4. Let r 3. r has a hereditary NTIME(expr_2(cn)) lower bound. Corollary 6.5. Let r 3 and be a theory in a logic L. If there is an interpretation of the classes mr log n in , then £ has a hereditary NTIME(exp r _ 2 (cn)) lower bound. Remark 6.6. For each r 3 there is a constant d > 0 such that every tree in trn/log n has at most exp r _ 2 (dn) vertices. Hence, we can view Corollary 6.5 as a significant improvement over Theorem 6.2 for obtaining NTIME(expr_2(cn)) lower bounds: rather than interpreting all binary relations on sets of size exp,,_2(cn) we need only interpret all trees of height r on sets of this size. In applications it is often much more natural to interpret trees than binary relations. See also Theorems 7.5 and 7.9. We next prove results similar to Theorem 6.3 and Corollaries 6.4 and 6.5 for finite trees of unbounded height. Theorem 6.7. Let Cn be the class of binary relations on a set of size exp (n — 3). Then there is a prenex interpretation of the classes Cn in the first-order classes T2n. Proof. Recall from the proof of Theorem 6.3 the formula mr which defines an equivalence relation on the set of all vertices in trees of depth r except the root. In that proof r was fixed, so we could assume that mr was in prenex form, and TO increased with n. In this proof we fix TO = 2 and
Proving lower bounds
173
consider formulas . We see now that has an iterative definition. By induction on k, there can be as many as exp(k;) 2n-types among vertices of height k < n in a tree in 2n We define n and n in essentially the way as in Theorem 6.3. The only difference is that when a vertex z coded a pair (x, y) there, z could have up to four children of the same type. Here, in trees from 2n, we have at most two, so we refer to types of grandchildren rather than children. Thus, we interpret binary relations on sets of size e x p ( n — 3) rather than n - 2). Corollary 6.8.
, has a hereditary NTIME(exp(cn))
lower bound.
Corollary 6.9. // there is an interpretation of the classes T2n in a theory , then £ has a hereditary NTIME(ex.p(cn)) lower bound.
7
Tools for linear ATIME lower bounds
The theorems in this section are counterparts of those in the last section. In order to obtain linear alternating time lower bounds for logical theories we must introduce a stronger form of interpretability which we call monadic interpretability. Theorems 7.2 and 7.3 tell how to obtain lower bounds by monadic interpretation of addition relations and binary relations. We then show, in Theorems 7.4 and 7.7 that binary relations have monadic interpretations in certain classes of trees of bounded height. From these results we obtain useful tools for establishing linear ATIME lower bounds, and lower bounds for monadic second-order theories of trees of bounded height. Suppose £ is a theory in a logic L' and C o , C i , C 2 , . - - are classes of models for a monadic second-order logic ML whose vocabulary consists of relation symbols P1, ... .Pk. Suppose that there are formulas 6n(x, u), reset log-lin computable from n, so that for each Cn there is a model ' of and elements m in ' with isomorphic to
, and the sets
range over all subsets of n (x, m) as p ranges over '. The parameter sequence u is allowed to grow as a function of n but t must remain fixed. The sequence {In | n 0} where In =
is called a monadic interpretation of the classes Cn in . We define simple, prenex, and iterative monadic interpretations similarly to definitions in the
174
Kevin J, Compton and C. Ward Henson
previous section. We also define the notion of monadic interpretation of classes Cn in classes C'n as in the previous section. Evidently, a monadic interpretation of classes Cn is nothing more than an interpretation where the models in Cn are regarded as models for a monadic second-order logic. Note that if we have an interpretation of classes Cn in a theory in some monadic second-order logic L', then we can automatically extend to a monadic interpretation by taking n(x, X) to be the formula x 6 X where X is a new set variable. The framework for obtaining linear alternating time lower bounds is essentially the same as before. Suppose we have a monadic interpretation of classes Cn in classes C'kn for some nonnegative integer k and that there is a prescribed set of formulas over ML such that is realized in some
Cn where
= n}
and inv*(ML) are ATIME(cn),cn)-inseparable for some c > 0. Now given formula in ML we form as follows. Replace each monadic quantification X or X with a quantification tx or tx, where tx is a variable sequence of the same length as t, uniquely determined by X, and not in conflict with the other variables in . (We may need to change other indices to avoid conflicts.) Introduce a relation variable 5 and replace each atomic formula x X by S(x, tx)- We can easily arrange that there be a reset log-lin reduction taking to . Now map each formula in to the formula given by
where n — . As before, it is easy to arrange that kn. If is realized in some model in Cn, then is realized in some model in C'kn; if is true in no weak model, then is true in no model or weak model (depending on whether L' is first-order or monadic second-order). When we have a prenex interpretation or an iterative interpretation and the definitions in ' are replaced by the appropriate iterative definitions, the sentences all belong to some prescribed set of formulas over L'. We have now that is realized in some
C'n where
= n}
and inv*(L') are ATIME (T(cn),cn)-inseparable for some c > 0. Then sat( ) and inv(L') are ATIME(T(cn),cn)-inseparable for some c > 0 so has a hereditary ATIME(T(cn),cn) lower bound. As before we allow a more elaborate definition of monadic interpretation where formulas n, 1n , . . . , kn , and n may contain free relation variables which also receive prenex or iterative definitions. We summarize these remarks in the following theorem which parallels Theorem 6.1:
Proving lower bounds
175
Theorem 7.1. Let Co,C1,C2, • • • be classes of models such that for some prescribed set of formulas over a monadic second-order logic L, is realized in some
Cn where
= n}
and inv*(L) are ATIME(T(cn),cn)-inseparable for some c > 0. Let C'0 C C'0 C'2 C • • • be classes of models of a theory £ in a logic L'. If there is a prenex or iterative monadic interpretation of the classes Cn in the classes C'kn for some nonnegative integer k, then the following are true. (i) The sets sat(£) and inv( ) are ATIME(T(en), en)-inseparable for some c > 0. (ii) If, for some d between 0 and 1, T(dn) = o(T(n)), then £ has a hereditary ATIME(T(cn),cn) lower bound. (iii) For each £' C val(£), sat( ') and val( ) are both hard for the complexity class ATIME(T(nc)nc) via reset log-lin reductions. (iv) For each val( ), sat( ') and val( ') are both hard for the complexity class ATIME(T(nc),nc) c>0 via polynomial time reductions, (v) There is a prescribed set ' of sentences over L' such that ' is realized in some
C'n where
= n}
and inv*(L') are ATIME(T'(cn), cn) -inseparable for some c> 0. The following theorems are immediate consequences of the preceding theorem and Theorems 5.1, 5.2, and 5.4. Theorem 7.2. Let T(n) be a time resource bound such that for some d between 0 and I, T(dn) = o(T(n)). Let Cn be the class of addition relations on sets of size at most T(n) and £ a theory in a logic L. If there is a monadic interpretation of the classes Cn in £, then £ has a hereditary ATIME(T(cn),cn) lower bound. Theorem 7.3. The previous theorem holds with binary relations in place of addition relations. From the following theorem we obtain a useful tool for obtaining lower bounds, but not the best lower bounds for monadic second-order theories of trees of bounded height.
176
Kevin J. Compton and C. Ward Henson
Theorem 7.4. Let Cn be the class of binary relations on a set of size exp r _ 1 (n). Then there is an iterative monadic interpretation of the classes Cn in the monadic second-order classesT2nrwhen r > 2 and in the monadic second-order classesT22nrwhen r = 2. Proof. The proof is very similar to the proof of Theorem 6.3. First, consider the case r > 2. We iteratively define a relation Q'(X, Y) which says that \X\ = \Y\ 2". Write Z = X U Y as an abbreviation for Z = X Y A X Y = 0. Let p(X, Y) be the formula
The iterative definition [Q'(X, Y) = p]n+l gives the desired relation. Now let (x, y) be a formula, with free relation variable Q, which says that if X is a subset of the set of children of x, and Q(XI, x2) holds for all Xi,x2 X, then there is a set Y which is a subset of the set of children of y such that Q'(X, Y) holds, and Q(y1, y2) holds for all y1,y 2 Y, and holds for all x\ 6 X and y1 Y. Consider a tree in72nr. When k > 0, the iterative definition
defines an equivalence relation on the vertices of height less than k. For k = 1, Q is an equivalence relation making all leaves equivalent. Increasing k to 2, two vertices of height 1 are equivalent if they have the same number of children. (In Theorem 6.3 this was true only up to O(n/ log n) children. This is the reason why we get an additional level of exponentiation here.) For larger k extend to vertices of height k-1, leaving the relation unchanged at lower heights. Two vertices of height k - 1 are equivalent if, for each equivalence class represented among their children (on which the relation has already been defined), they either have the same number of children in the class or both have at least 2" children in the class. There is a reset log-lin reduction taking the iterative definition
to an equivalent explicit definition
Moreover, since r is fixed this can be expressed as a simple definition. The rest of the theorem proceeds as in the proof of Theorem 6.3 except that nr is
Proving lower bounds
177
used in place of mr 2nr is used in place of Trm, and we automatically have a monadic interpretation since we are interpreting in a monadic second-order theory. Let us consider the case r = 2. Our formulas will now have parameters U, V, W, and Z. Let 2 n ( x , y , U ) and 2 n ( x , y , V ) be defined as ( x , y ) above except that, rather than comparing the number of children of vertices x and y of height 1, 0n2 (x, y, U) compares the number of children of x and y in U, and 2n(x ,y, V) compares the number of children of x and y in V. Now let n(x) be the formula x Z. Let n(x, y) be a prenex formula that asserts the following: (a) x Z and y € Z. (b) If x there is a t Z such that 2n (x, t) holds and n2n(t, y) holds. (c) If x = y, then x W. Every binary relation on a set of size at most 2nis interpreted in some tree in T22n2. For every pair (x, y) in the relation with x y there is at least one t Z such that m(x,t) and m(t,y) hold. I Corollary 7.5. Let r 2 and be a theory in a logic L. If there is a monadic interpretation of the classes 2nr in , then has a hereditary ATIME(expr-1(cn),cn) lower bound. Remark 7.6. For each r 2 there is a constant d > 0 such that every tree in 2nr has at most expr_1(dn) vertices, so we can view Corollary 7.5 as an improvement over Theorem 7.3 for obtaining ATIME(expr_1(cn),cn) lower bounds. It is instructive to compare this result with Corollary 6.5. It often happens that an interpretation of the classes Tnr/log n in a theory can be modified slightly to obtain a monadic interpretation of the classesT2nr-1 even when the theory is first-order. This explains, in part, why NTIME lower bounds can often be pushed up to linear ATIME lower bounds. We now prove another theorem about monadic interpretations of classes of trees of bounded height. We shall see in section 9 that this theorem gives the best lower bounds for monadic second-order theories of trees of bounded height. Theorem 7.7. Let Cn be the class of binary relations on a set of size exp,,([n/logn]). Then there is a prenex monadic interpretation of the classes Cn in the monadic second-order classes and in the monadic second-order classes Proof. First, consider the case r > 1. Formulas will contain parameters X = X 1 , X - 2 , . . . ,Xm, where m = 1 + [n/log n]. This is the first case we have encountered where the parameter sequence grows with n. Let Q' be
178
Kevin J. Compton and C. Ward Henson
a binary relation variable and are both leaves and
m(x,y)
a prenex formula that says x and y
Q' will be given by the prenex definition [Q'(x,y) = m]. Q'(x,y) is obviously an equivalence relation on the vertices of height 0. Now let (x, y) be a formula (with free relation variable Q) that says x is not a leaf and for every child t of x there is a child u of y such that Q(t,u) holds. Now the iterative definition
defines an equivalence relation on the vertices of heights less than k. For k = 1, Q is identical to the equivalence relation Q'. For larger k we extend the relation to vertices of height k - 1 by specifying that two such vertices are equivalent if precisely the same equivalence classes are represented among their children. When k = r we have an equivalence relation on the set of all vertices in a tree of depth r except the root. The iterative definition
can be converted to a prenex definition
of fixed length since r is constant. By Theorem 3.1 the sequence
of prenex definitions can be replaced by a single prenex definition
where rm(x, y) is reset log-lin computable from n. We will say that two vertices x and y in a tree of height r have the same rm-type if r m ( x , y ) holds. Now it is easy to show by induction on k that there is a tree of height r and sets X1 , . . . , Xm of leaves in this tree such that there are at least 1 + expk+1([n/logn]) rm-types among vertices of height k when k < r: every nonempty set of rm-types of vertices of height k - 1 determines a distinct rm-type for a vertex of height k. More is required to see that there is such a tree in exP2 r([n/log n]) When k = 0 there is no problem Consider the case k = 1. For i = 1,2, . . . ,exp 2 ([n/logn]) let Tt be the tree of height
Proving lower bounds
179
1 with precisely i leaves. Each of these trees is in exp2(m) Without much trouble we can choose, from the leaves of each Ti, sets X1 , . . . , Xm so that the roots of Ti and TJ have different rm-types when i j. Further, for T1, X1, . . . , Xm can be chosen in at least two ways. Thus, there are at least 1 + exp 2 ([n/ logn]) rm-types among vertices of height 1. From the set of trees {Ti, | 1 i < exp2(m)} choose a nonempty subset and form a tree of height 2 by directing edges from a new vertex to the roots of trees in this subset. We can form at least 1 + exp3 ( [n/ log n] ) such trees, each one in Texp2 Moreover, if we carry along the subsets X1,..., Xm in each subtree Ti, each root has a distinct rm-type. Continue for each k < r, forming at least 1 + expk+1([n/logn n) trees in Tkexp 2 (|n/logn]) subsets X 1 , . . . , X m ,so that each root has a distinct The interpretation of binary relations on sets of size exp r ([n/logn]) now uses precisely the same construction used in the proofs of Theorems 6.3 and 7.4. Let us consider the case r = 1. Our formulas will now have parameters X = Xi,...,Xm, Y = Y1,...,Y m , and Z and W. Define m(x, y) as above and n m ( x , y ) in the same way except that Y is used in place of X. These formulas define independent equivalence relations on leaves so we can speak of the m- and m-type of a leaf. The relations are independent and of index at most 2m. The rest of the proof then follows as the in case r = 2 in Theorem 7.4. Corollary 7.8. Let r > 1. M rr has a hereditary ATIME(expr (en/ log n) , en) lower bound. Corollary 7.9. Let r > 2 and be a theory in a logic L. If there is a monadic interpretation of the classes Trexp2 (n/log n) then £ has a hereditary A TIME (expr (cn/ log n) , cn) lower bound. The case r = 1 is worth stating separately. Observe that trees of height 1 do not really have much structure. We can regard them as sets with one distinguished point, the root. Therefore, we state the result in terms of interpretations of sets rather than trees. Corollary 7.10. Let Cn be the class of sets of size at most 2 n /log n and a theory in a logic L. If there is a monadic interpretation of the classes Cn in , then has a hereditary ATIME(2 c n / l o g n, cn) lower bound.
180
Kevin J. Compton and C. Ward Henson
This concludes our survey of tools for establishing lower bounds. The next section contains many examples of their application.
8
Applications
In this section we use the methods developed in earlier sections to give a representative sample of arguments for known lower bounds of theories. We believe that the details given here justify the claim that every known lower bound for a theory can be obtained in this way, with simpler, more conceptual proofs. In particular, there is no further need to code Turing machine computations. Moreover, in almost all cases our approach gives technical improvements on known results: we always obtain hereditary lower bounds; these bounds hold for both sat( ) and val( ), in contrast to most published NTIME lower bounds which are for just sat( ); and the reductions used are reset log-lin reductions rather than polynomial time, linearly bounded reductions. In a few cases we obtain qualitative improvements in the bounds. The most significant improvement is that we always obtain inseparability results. To simplify the statement of results and avoid repetition, when we say a theory £ has a hereditary NTIME(T(cn)) lower bound, we intend each of the statements (i)-(iv) in Theorem 6.1. When we say a theory has a hereditary ATIME(T(cn),cn) lower bound, we intend each of the statements (i)-(iv) in Theorem 7.1. For convenience we will also use functional notation rather than relations in some examples. Example 8.1 (The first-order theory of finite linear orders with an added unary predicate.). We show that this theory has a hereditary lower bound of NTIME(ex P (cn)) by iteratively interpreting the classes T2nand applying Corollary 6.9. In fact, we interpret the classes Tn consisting of the finite trees of height n. We denote the linear order by and the predecessor and successor of an element x by x — 1 and x+1 (when they exist). Let x < y be the formula x y Ax y and LAST(x) the formula asserting that x is the last element in the order. Let P be the unary relation symbol. We can identify each finite model = (m, R) of this theory (where m = {0,...,m — 1}) with a string a 0 a1... am-1 of 0's and 1's. We stipulate that ai = 1 if and only if P(i) holds. Representing a finite tree as a string of O's and 1's is straightforward. Now with each vertex x in a tree of height n associate a string that begins 0n-rl, where r is the depth of x, followed by the strings associated with each child y of x. The tree is represented by the string associated with the root. For example, the tree of Figure 1 is represented by the string 0001001010100100101.
Proving lower bounds
181
Fig. 1. Tree example In general, T may have many representations, since the children of each vertex are ordered arbitrarily. Thus, the tree above is also represented by the string 0001001001010010101. However, T can be easily interpreted within each of its representations . Let n(x) be the formula P(x) (indicating that is a position where 1 occurs). Let 0 ( x , y ) be the following formula with free relation variable Q:
Then [Q(x, y) ]n+1 defines a relation Q(x, y) which holds precisely when the number of consecutive O's preceding position x and the number of consecutive 0's preceding position y are equal and at most n. Let n(x,y) be the formula
Thus, n (x,y) holds precisely when there are 1's at positions x and y, x precedes y, there is exactly one more 0 preceding x than preceding y (but no more than n 0's preceding y), and there is no position between x and y which has as many 0's preceding as x has. Now a tree T of height n represented by a linear order is isomorphic to when Q is given by the iterative definition [Q(x ,y) 0]n+l. Remark 8.2. Note that for some d > 0 each tree in T2n has at most e x p ( d n ) vertices so for some d' > 0 each representation of a tree in this class has at most exp00(d'n) elements. From Theorem 6.1 we see that we have a tool for obtaining further lower bounds. If there is a prenex or iterative interpretation of the classes Cn consisting of linear orders of length at most exp00(n) with added unary predicates in a theory , then £ has a hereditary lower bound of N T I M E ( e x p ( c n ) ) .
182
Kevin J. Compton and C. Ward Henson
Example 8.3 (The first-order theory of all linear orders). We show that this theory has a hereditary lower bound of NTIME(exp(cn)) by a simple interpretation of the models in Example 8.1. Consider a finite linear order (n, ) together with a unary relation R on n interpreting the predicate symbol P. We will represent (n, , R) by a linear order = (S, ) formed by replacing each i € n with a copy of the closed unit interval [0,1] if i is not in R and with a single point followed by a copy of the closed unit interval [0,1] if i is in R; the order is extended in the obvious way. Now we interpret {n, , R) in as follows. Let (x) be the first-order formula saying that x has no immediate successor and x is either the least element or has an immediate predecessor. Clearly, (x) picks out all the left endpoints of the unit intervals in (S, ). There are precisely n elements in (x). Let (x) be the first-order formula saying that x has no immediate successor but does have an immediate predecessor y such that y either has no immediate predecessor or is the least element. Thus, (X) picks out the left endpoints of the unit intervals associated with elements i in R. Thus, (n,<, R ) i s isomorphic t o ( ( x ) , (x,y),A(x)). Remark 8.4. Robertson [1974] and Stockmeyer [1974] showed independently that the first-order theory of finite linear orders with an added unary predicate is not elementary recursive. They obtained an NTIME (exp (c log n)) lower bound, but not in the hereditary form. Stockmeyer obtained the same bound for the first-order theory of linear orders. Stockmeyer shows in the same paper that if (A, ) is any infinite linear order, then the theory of the models (A, ,R), where R is an added unary relation, is not elementary recursive. A simple interpretation of Example 8.1 in this theory shows that it has a hereditary NTIME'(exp (cn)) lower bound. Of course, from Example 8.1 also follows the result of Meyer [1975] that the weak monadic second-order theory of the natural numbers with successor is not elementary recursive and, hence, the same result for other monadic second-order theories, including the monadic second-order theory of two successors studied by Rabin [1969]. In all cases we have hereditary NTIME (exp (cn)) lower bounds. Another result that can be immediately obtained in this way is the following strengthening of a result announced in Meyer [1974]. Example 8.5 (The first-order theory of ({0, l}*,ro,r1, ), the binary tree formed by taking the prefix order on {0,1}*, together with
Proving lower bounds
183
successor functions ro and r1. (Thus, u w precisely when there is a v such that uv = w, and r0(w) = w0, r1(w) = wl)). We show that this theory has a hereditary lower bound of NTIME(ex P (cn)) by a simple interpretation of the classes in Example 8.1. Let = ({0, l}*,r 0 ,r1, ). We produce formulas 6(x,w), and (x, w) such that
X(x,y,w),
includes all models (n, , R) in Example 8.1 as w ranges over {0,1}*. The formula 6(x, w) is x w A x w. The formula \(x, y, w) is
The formula (X, w) is §(x, w) r1 (x)
w.
Example 8.6 (The first-order theory of finite unary functions). A unary function is a model (B, f) where / is a function from B to itself. We show that this theory has a hereditary lower bound of NTIME (exp (en)) by a simple interpretation of the classes n2 and applying Corollary 6.9. In fact, just as in Example 8.1, we interpret the classes Tn consisting of the finite trees of height n. Let (x) be the formula x = x and (X, y) be the formula
Clearly, every finite tree is isomorphic to a model ( ( x ) , ( x , y ) ) for some unary function . is formed from the tree by mapping every vertex other than the root to its parent and the root to itself. Remark 8.7. A lower bound of JVJ7ME(exp00(c log n)) for the first-order theory of unary functions was announced by Meyer [1974], but no proof has ever been published. A simple interpretation in the opposite direction (interpreting unary functions in trees) appears in Korec and Rautenberg [1975/76]. Thus, the theory of finite unary functions and the theory of finite trees have the same complexity up to a constant factor in the argument. Our next example gives another family of useful theories which are not elementary recursive: the theories of pairing functions. Lower bounds
184
Kevin J. Compton and C. Ward Henson
for these theories were first given by Rackoff [1975b] (see Ferrante and Rackoff [1979]). Their treatment shows that these theories are in fact hereditarily not elementary recursive. (To obtain this from the result stated by Ferrante and Rackoff, we must use the fact that the theory of pairing functions is finitely axiomatizable.) Example 8.8 (The theory of any pairing function). A pairing function is a model = (B, f) where / is a one-to-one binary function on B. We show that the theory of any pairing function has a hereditary lower bound of NTIME(ex P (cn)) by iteratively interpreting the classes Cn of linear orders of length e x p ( n ) with an added unary predicate. These classes were discussed in Example 8.1. The idea behind the interpretation is to have elements of represent sequences of length e x p ( n ) . One way to do this, say for an element a, is to find a pair (a0, a1) such that /(a0, a1) = a. (There is at most one such pair, since / is a pairing function.) Then find a quadruple (a00 , a01,a10,a11) such that f(a00 a01) = ao and /(a10,a11) = a1 and repeat until we have a sequence (aw w {0,1}m), where m = e x p ( n — 1). (The order on {0, l}m is the lexicographic order.) We could then say that a represents this sequence. We may not be able to carry out this construction for every element ae because / may not be onto, but certainly every sequence of length e x p ( n ) is represented by some element, and every element represents at most one sequence of length e x p ( n ) . We may regard this construction as giving a labeling of vertices in the full binary ordered tree of height m. The root is labelled a, its left and right children are labelled a and 01, respectively, and so on. Unfortunately, the construction has two limitations that make it unacceptable for our purposes: a branch in the tree may have several vertices with the same label; and two different branches may be labelled identically. We must modify the construction to overcome these difficulties. Rather than taking awo and aw1 so that f ( a w 0 , a w 1 ) = aw, we will take b and c so that f ( b , c) = aw and then take awo and aw1 so that f ( a w o , a w 1 ) — c. That is, awo and aw1 are chosen so that x f ( x , f ( a w 0 , a w 1 ) ) = aw holds. Clearly, aw0 and aw1 are uniquely determined by aw. It is still true that every element represents at most one sequence of length exp00(n), but now a sequence of length e x p ( n ) may be represented by many elements. We claim that with this modified construction, every sequence of length exp (n) is represented by some element such that no branch in the associated tree has duplicate labels and no two branches are labelled identically. Consider a sequence aw w{0,1}
Proving lower bounds
185
where m = exp00(n — 1). We can find elements aw for each w 6 {0,1}* with k < m such that x f ( x , f(a w o,o,a w1 )) = aw always holds; whenever v is a prefix of w, av aw; and aw0 a w1 . The elements are selected in a bottom-up fashion, i.e., working from the leaves of the tree up to the root. Suppose that elements av are already known when w lexicographically precedes v and we wish to find aw. Since / is a pairing function, f ( x , f(awo, a w 1 )) takes distinct values as x ranges over the model. Thus, we can choose x so that aw = f ( x , f ( a W 0 , a w 1 )) is not equal to av when w lexicographically precedes v since there are only finitely many such v. Following this procedure we eventually reach a and all labels in the tree are distinct. Let LEFT(x, y) be the formula (t, u) f ( t , f ( y , u))=x and RIGHT(x, y) be the formula (t, u) f ( t , f ( u , y)) = x. Thus, LEFT(x, y) holds when y is the left child of x in our tree, and similarly for RIGHT(x,y). Let SUCC(x, y) be the formula LEFT(x, y) V RIGHT(x,y). We wish to specify three first-order formulas. Formula n(x,u) should say that x in an element in a sequence of length e x p ( n ) which is represented by u; that the elements of this sequence are distinct; and that no branch in the associated tree has duplicate labels. Formula n(x, y, u) should say that n(x, u) and n(y,u) hold, and that either x = y or in the sequence represented by u, x occurs before y. Formula n(x,x',u,u') should say that n(x,u) and n(x',u') both hold, and that x occupies the same position in the sequence represented by u as x' does in the sequence represented by u'. Now suppose that we already have relations
that hold under the same conditions as n(x, u), n(x, y, u), and n(x, x',u, u'), only for sequences of length exp00(n — 1) rather than e x p ( n ) . (If n = 0, take Qo, Q1, and Q2 to be empty relations.) Suppose n > 0. Using Qo and Q1, we can say, in regard to a sequence of distinct elements of length exp00(n - 1) represented by v, that a particular element is first; that a particular element is last; that one element occurs before another; and that one element occurs immediately before another. Hence, we can say that v represents a branch of length exp (n -1) from an element u. By this we mean v represents a sequence of distinct elements of length exp00(n — 1) such that when x is the first element of the sequence, SUCC(u, x) holds, and when x occurs immediately before y in the sequence, SUCC(x , y) holds. Now define formulas , A, and a in terms of Q0, Q1, and Q2 The formula 6(x, u) says that if Qo is empty, then x = u, and if Qo is not empty, the following hold. (a) There is an element v representing a branch of length exp00(n — 1)
186
Kevin J. Compton and C. Ward Henson
from u with x as the last element. (b) For every element y occurring in a branch of length exp00(n — 1) from u, if y is not the last element in the branch, then there are distinct elements z and z' such that LEFT(y, z) and RIGHT(y, z') hold. This ensures that the binary tree is full and that no two branches in the tree are labelled identically. (c) If v and v' represent different branches of length e x p ( n — 1) from u, then the last elements of these two branches are distinct. (It is easy to tell if v and represent different branches; this happens if and only if there are elements y y such that Q2(y,y',v,v') holds.) The formula (x, y, u) says that if Q1 is empty, then x — y — u, and if Q1 is not empty, then either x — y, or the following hold. (a) There are branches from u, represented by v and v' and with last elements x and y, of length e x p ( n - 1). (b) There is an element z such that if LEFT(z, t) and RIGHT(z, t') hold, then Q2(t,t',v,v') holds. This guarantees that the branch from u to x is to the left of the branch from u to y. The formula (x,x',u, u') says that if Q2 is empty then x — u and x' = u', and if Q2 is not empty, then the following hold. (a) There is a branch from u, represented by v and with last element x. There is another from u', represented by v1 and with last element x', of length e x p ( n — 1). (b) If Q-2(z,z',v,v') holds, z occurs immediately before t in the branch represented by v, and z' occurs immediately before t' in the branch represented by v1, then LEFT(z,t) just in case LEFT(z',t') holds. This says that the same sequence of lefts and rights leads from u to x as from u1 to x'. Clearly, , A, and a are first-order expressible in terms of Q0, Q1, and Q2. The simultaneous iterative definition
gives the desired formulas , and By Theorem 3.4, this simultaneous iterative definition can be replaced by just an iterative definition. For some element is a linear order of length exp00(n). Then as u' ranges over the elements in 21, ( x , u ) (x,u') ranges over all unary predicates on 6 u ( x , u ) . Thus, we can interpret every linear order of length e x p ( n ) with an added unary predicate. This is the first example we have encountered where the relations being iteratively defined do not occur positively in the operator formulas. Hence, according to the remark following Theorem 3.1, we cannot guarantee that
Proving lower bounds
187
the lower bounds obtained here hold if only the connectives A, V, and are allowed in formulas. However, with slightly more effort we can overcome this difficulty and use only formulas where the defined relation symbols occur only positively. We have not done so to simplify exposition. Remark 8.9. It is a long-standing open question whether the first-order theory of the free group Fk on k > 2 generators is decidable. Semenov [1980] observed that this theory is at least not elementary recursive. He showed that it is possible to give a first-order definition of a pairing function on Fk, from which it follows that the theory of Fk has a hereditary NTIME(exp00(cn)) lower bound. This use of pairing functions is a quick way to show that many decidable theories are hereditarily not elementary recursive. For example, consider the first-order theory of the model (N, +,2 X ), where N is the set of nonnegative integers. Semenov [1984] showed that this theory is decidable. Observe that the function
is a pairing function on this model, so the theory has a hereditary exp00(cn) lower bound. Variations on the same argument apply to the other models (N, +,g(x)) treated by Semenov [1984] when g(x) grows rapidly enough. In particular, when g(x + 1) > 2g(x), the argument gives a hereditary exp00(cn) lower bound. This completes our discussion of theories which are not elementary recursive. Note that in all cases treated here, there is an iterative interpretation of the classes T2n in the theory treated. (Sometimes, there is an iterative interpretation of the classes T2n into classes Cn and another iterative interpretation of the classes Cn in the theory; the chain of interpretations may be even longer. Nonetheless, these situations still qualify as interpretations of T2n.) It seems likely that almost all 'natural' theories that are hereditarily not elementary recursive interpret finite trees (in the same sense that almost all 'natural' theories that are undecidable interpret finite binary relations) . Of course, an interpretation of trees can be traced back, in Theorem 6.3, to an interpretation of binary relations on sets of size expoc(n), and thence, in Theorem 4.1, to a coding of Turing machines with a expoo (n) time resource bound. However, the Turing machine encodings lie very far below the surface and are extremely complicated compared to the original interpretation of trees. We now treat various elementary recursive theories for which sharp lower bounds are known. Example 8.10 (The first-order theory of models (N,S,P), where N is the nonnegative integers, S is the successor function on N, and P is an arbitrary unary relation). We show that this theory has
188
Kevin J. Compton and C. Ward Henson
a hereditary lower bound of
by iteratively interpreting the classesT4n/lognand applying Corollary 6.5. We use the same coding of trees into strings of Os and 1s that was used in Example 8.1. Given a tree be a string that represents it and P a unary relation on N such that P(i) holds if and only if either i < m and a, = 1, or i is m or m + 1. We want to recover T in {N, S, P) . The problem is that we have only a successor function rather than the linear order we had in Example 8.1. When we examine the role played by the linear order there, however, we see that we did not need the full linear order. We needed only enough of the order to say x < y when y is a child of x, and that there is no position between x and y which has as many O's preceding as x has when y is a child of x. Moreover, we did not need to make either statement when x is the root because there are no other positions in the coding which have as many O's preceding as x has. Thus, we needed only to determine if x < y when x and y belong to the same primary subtree. For every primary subtree is in It
is
not
difficult
to
see
that
for
some
d,
the
trees
in
T3n/log
n
are
dn
all represented by strings of length at most 2 . Thus, if we can define a relation which holds when x < y and y — x< 2dn we are done. Let 6 be the formula x — y V S(x) = y V 3z(Q(x,z) A Q(z,y)). The iterative definition [Q(x,y) = 0} d n gives the desired relation. Remark 8.11. This proof shows that if there is a prenex or iterative interpretation of the classes Cn consisting of successor relations of length at most exp2(n) with added unary predicates in a theory E, then E has a hereditary lower bound of NTIME (exp2 (cn)). The same argument shows that if there is a prenex or iterative interpretation of the classes C'n consisting of successor relations of length at most 2n with added unary predicates in a theory E, then E has a hereditary lower bound of NTIME(2 c n ). Example 8.12 (The theory of one-to-one unary functions). We
show that this theory has a hereditary lower bound of
by iteratively interpreting successor relations of length at most 2" with an added unary predicate (as discussed in the preceding remark). Let Cn be the class of finite one-to-one unary functions (B, f) such that the cycle decomposition has, for each k with 1 < k < 2n, at least one and not more than two cycles of length k. We regard each such model as
Proving lower bounds
189
coding a unary predicate P on {1, 2, . . . , 2n}, the presence of a single k-cycle indicating that -P(k) holds and the presence of two k-cycles indicating that P(k) holds. Define the distance between elements x and y on the same same cycle to be the least i such that f i ( x ) = y. For elements x, y, and z on the same k-cycle, we say that y is between x and z if the distance from x to z is the sum of the distances from x to y and y to z. Now it is a simple exercise to give a simultaneous iterative definition
such that Q 1 ( x , y , z ) holds precisely when the distance from x to y and the distance from y to z are at most 2n and y is between x and z, and Qz(x, y, x', y') holds precisely when the distance from x to y is the same as the distance from x' to y1 but not more than 2n. Assume that no cycle has length more than 2n. For Q1 and Q2 as given above, let n n ( x , y) be the formula Q 2 ( f ( x ) , x , f ( y ) , y ) . It is easy to see that nn defines an equivalence relation that holds precisely when x and y lie on cycles of the same length. Let n(x) be the formula
Thus, (x) holds precisely when there are at least two cycles with the same length as the cycle containing x. Let an(x,y] be the formula
We see that an(x, y) holds precisely when x lies on a cycle of length one less than the cycle containing y. These formulas interpret successor relations on sets of size at most 2n with an added unary predicate. The interpretation differs from interpretations discussed previously, however, in that we must take a quotient by the equivalence relation nn, rather than interpret the domain as a subset. Remark 8.13. The theories in Examples 8.10 and 8.12 are treated by Ferrante and Rackoff [1979]. They give a matching upper bound for the former and an upper bound of NTIME(2dn2) for the latter. (Example 8.12 first appeared in Ferrante [1974].) The inseparability, hereditary, and hardness results presented here are new. Stern [1988] investigated the complexity of the theory of commuting permutations.
Example 8.14 (The first-order theory of {{0, l}*,ro,r1), the binary tree on {0, 1}* together with successor functions r0 and r1 (described in Example 8.5)). We show that this theory has a hereditary lower bound of
190
Kevin J. Compton and C. Ward Henson
by a monadic interpretation of the classes T2n and applying Corollary 7.5. Each tree in T22n has at most 2dn vertices, for some d > 0; therefore, each such tree can be represented by a linear order of length 2kn with added unary predicate, as in Example 8.1. Thus, we need only give a monadic interpretation of models
in ({0, 1}*,r 0 ,r1). (The unary predicate comes automatically, since we have a monadic interpretation.) Specifying this monadic interpretation is straightforward: an integer l in {1, . . . , 2kn} is represented by the equivalence class of strings of length l; < is given by the prefix order; and subsets of {1, . . . , 2kn} are represented by strings w via {l | for some v of length l, r1(v) is a prefix of w}. To complete the construction it suffices to give iterative definitions for formulas n(u,v) saying that for some l < 2kn, u and v both have length l, and formulas n (u, v) saying that for some l < 2kn, u has length l and is a prefix of v. This is done analogously to the iterative definition of linear order on intervals of length 2dn in Example 8.10. We leave the details to the reader. Remark 8.15. Example 8.14 was first treated by Volger [1985]. He also showed that for some d > 0, ATIME(2dn,n) is an upper bound for this theory. The hereditary lower bound can be derived from Volger's result because the theory is finitely axiomatizable. In Examples 8.12 and 8.14 the equality relation is interpreted by an equivalence relation rather than true equality. Correspondingly, models U are interpreted by quotients of defined substructures, rather than the substructures themselves. This kind of interpretation is common in undecidability proofs (see examples in Rabin [1965] and Ersov et al. [1965]) and requires no changes in the proofs in sections 6 and 7. In Example 8.14 the equivalence relation could be avoided by representing an integer l by an string of l O's. However, in Example 8.12 the use of an equivalence relation in place of equality seems unavoidable. Example 8.16 (Any extension E, with an infinite model, of the the first-order theory of Boolean algebras). We show that E has a hereditary lower bound of
by giving a monadic prenex interpretation of sets of size up to 2cn/ log n and applying Corollary 7.10.
Proving lower bounds
191
Fix B, an infinite model of E. Let m = [n/log n] and I < 2m. Define 8 n ( x , u ) to be a prenex formula with parameters u = u1,...,u m that says x is an atom relative to u. By this we mean that for each i such that 1 < i < m, either x C ui or x D Ui = 0, and x is a maximal element with this property (with respect to the relation C on B). Clearly, we may take 8(x, u) to be reset log-lin computable. By appropriately choosing elements a in B, we can arrange that (8nB (x, a) has precisely l elements. Let o n (x,u,v) be the formula 8 n ( x , u ) A x C v. As b ranges over the elements of ranges over all subsets of S
192
Kevin J. Compton and C. Ward Henson
nonnegative integers less than 22n and yx= z. Prom these formulas we can define directly formulas an(x,u) that hold when x is a nonnegative integer less than 2™, u is a nonnegative integer less than 2 2 ", and the (x+ l)th bit in the binary representation of u is 1. Thus, as a ranges over elements of R, (x, a) includes all subsets of {0,..., 2n — 1}. We thereby obtain a monadic interpretation of ({0,..., 2n — 1}, +}. Remark 8.21. The same argument will work if the theory of real addition is replaced by any extension of the theory of semigroups that has, for each n, a model (B, o) with an element a whose powers a°, a 1 , . . . , an are distinct. Any such theory has an ATIME(2cn, cn) hereditary lower bound. Example 8.22 (Presburger arithmetic, the first-order theory of addition on the natural number). This is the first-order theory of the model (N,+), where N is the set of nonnegative integers. We obtain a hereditary lower bound of
by giving a monadic iterative interpretation of sets of size 22n with an addition relation and applying Theorem 7.2. As in Fischer and Rabin [1974], we use divisibility to push the definition of the multiplication and exponentiation relations u n ( x , y, z) and n ( X , y, z) in the previous example up to intervals of length g(n), where g ( n ) > exp3(dn); this estimate comes from the prime number theorem. We then obtain a monadic interpretation as in the previous example. Remark 8.23. Evidently, the basic ideas used in Examples 8.20 and 8.22 are already found in Fischer and Rabin [1974]. However, we obtain several benefits from our approach. First, by using the machinery of the previous sections we avoid technicalities concerning coding sequences and Turing machine computations which Fischer and Rabin needed to address. Second, by emphasizing inseparability instead of just complexity, we obtain hereditary lower bounds. Finally, by observing that monadic quantification is implicit in the interpretations, we see why the appropriate lower bounds are ATIME bounds rather than an NTIME bounds. The linear alternating time lower bounds in Examples 8.20 and 8.22 were proved by Berman [1980]. A slightly weaker hereditary NTIME(2cn) lower bound for real addition was obtained by Ferrante and Rackoff [1979], and a hereditary NTIME (22 cn) lower bound for Presburger arithmetic was obtained by Young [1985]. Maurin [1996; 1997a] has shown complexity bounds for theories of ordinal addition. The full strength of the prime number theorem is not needed in Example 8.22. A much simpler result, due to Chebyshev, suffices; see Theorem 7 of Hardy and Wright [1964]. This illustrates a common phenomenon
Proving lower bounds
193
in lower bound results for a theories. Crude arguments often suffice. Sophisticated mathematics and an intimate knowledge of the theory under consideration are rarely needed. In this respect, upper bound results may be much more difficult. Note that since we have monadic quantification on sets of size 2cn in a model of real addition (or any extension of the theory of semigroups as described in the remark after Example 8.20) we have a monadic interpretation of the classes from the same representation of trees used Example 8.1. Similarly, since we have monadic quantification on sets of size 22cn in a model of Presburger arithmetic, we have a monadic interpretation of the c l a s s e s . These facts will be useful in the next two examples. Fischer and Rabin [1974] announced two other lower bounds: a lower bound of NTIME(exp3(cn)) for the first-order theory of integer multiplication; and a lower bound of NTIME(22 c n ) for the theory of finite Abelian groups. They did not supply proofs (but mentioned the key idea of encoding sequences of integers as exponents in a prime decomposition). To our knowledge, no proofs have ever been published. In the next two examples we sketch proofs of the stronger ATIME versions of these results. Example 8.24 ( The first-order theory of multiplication on the positive integers). This is the first-order theory of the model (II, •), where I is the set of positive integers. We obtain a hereditary lower bound of
by giving a monadic iterative interpretation of the classes T42n and applying Corollary 7.5. Let Pi be the i-th prime number. Observe that (II,.) is isomorphic to a direct sum of countably many copies of (N, +) by the mapping that takes the sequence (a1,a2,...), where ai is 0 for all large i, to Moreover, each direct summand (i.e., set of powers of some prime Pi) can be defined. But we saw in Example 8.22 and Remark 8.23 that there is a monadic interpretation of the classes in {N, +). The idea here is that we interpret a tree from in by directing edges from a new root to the roots of trees interpreted in each direct summand. Clearly, we can specify a first-order formula a(x,y,z,t) that says t is a prime, x, y, and z are powers of t, and x.y = z. Let 6n(x, u'), n(x,y,u'), and o n (x,u',v') be formulas giving a monadic iterative interpretation of in (N,+). We saw in Example 8.22 that such formulas exist. (If we substitute a(x,y, z,t) for all occurrences of x + y= z we obtain formulas 8' n (x,t,u'), 'n(x,y,t,u'), and o' n (x,t,u',v') in the language of (II,.). For each fixed prime t, these formulas give a monadic iterative interpretation of in (II,.). For S'n(x,t,u') to be satisfied, it is necessary that x and u' be powers of t. Thus, except possibly for 1, which is a power of every
194
Kevin J. Compton and C. Ward Henson
prime, there are no elements common to the interpretations of S'n(x, t,u') and S'n(x, t',u") when t and t' are distinct primes. Let n(t,u',u) be a formula that says t is a prime and u' is the largest power of t dividing u. Define S'n(x,u) to be the formula
. Define n"(x,y,u) to be the formula
The first disjunct gives an edge from 1, the root of the tree, to the root of each primary subtree; the second disjunct gives the edges in the primary subtrees. UsingS"n|and n" we can interpret each tree from by taking a disjoint collection of trees in and 1 as a new root—we need only choose u suitably. Define 0 " n ( x , u , v , w ) to be the formula
By varying v and w we obtain all subsets of S"n (x, u), so we have a monadic interpretation. Remark 8.25. An upper bound of ATIME(exp3(dn),dn) for the firstorder theory of integer multiplication can be obtained from the treatment of this theory in Ferrante and Rackoff [1979]. The original reference for an upper bound on the first-order theory of integer multiplication is Rackoff [l975b; 1976]. Decidability results and complexity bounds for related theories have been given by Maurin [1997b] and Michel [1981; 1992]. As our last example we consider the first-order theory of finite Abelian groups. Lo [1988] has given an extensive treatment of upper bounds for theories of Abelian groups. He states these bounds in terms of the classes SPACE, but it is clear that his analysis gives ATIME(2 2 d n ,dn) upper bounds. We derive a matching lower bound, not just for the theory of finite Abelian groups, but also for the theory of finite cyclic groups. Example 8.26 (The first-order theory of finite cyclic groups). We
obtain a hereditary lower bound of
by giving a monadic iterative interpretation of the classes and applying Theorem 7.5. We use a device similar to the one used in Example 8.24. Now rather than regarding the positive integers with multiplication as a direct sum of copies of (N,+}, we regard a finite cyclic group as a direct sum of cyclic groups whose orders are prime powers.
Proving lower bounds
195
Let C(l) be the cyclic group of order 1. We know from the remarks following Example 8.22 that there is a d > 0 such that when 1 > 2 2dn , there is a monadic iterative interpretation of T2n in C(l). More precisely, there are formulas S n (x,t',u'), n(x,y,t',u'), and an(x,t',u',v) given by iterative definitions such that if u' is a generator of C(l), then each tree in is isomorphic to (Sn (x,t',u'), n (x,y,t',u')) for some t' in C(l), and includes all subsets of Sn (x, t' , u') as v' ranges over elements of C(/). (We need to mention the generator u of C(l) explicitly in these formulas because there is no preferred generator; this necessitates only a minor modification of the formulas in Example 8.20.) Let p1,p2, . . . ,pk be the prime numbers less than 22dn+1 and mi the largest power of pi less than 22dn+1 . Then since pi • mi > 22dn+1 and Pi < mi, we know that m^ > (22dn+1 )1//2 = 22dn so that there is a monadic iterative interpretation of in each C(mi) as described above. By Chebyshev's theorem (Theorem 7 of Hardy and Wright [1964]) we know that k, the number of primes less than 22dn+1 , is at least 22dn for sufficiently large n. By taking d large enough, we can ensure that k is greater than the maximum number of primary subtrees in each tree of . Now take m = m1.m2 • • mk so that C(m) = C(m l )®C(m2)0- • •®C(mk). We see that m > (22dn )k > 22dn . We need to show that we can combine the monadic iterative interpretations of in the direct summands C(mi) to obtain a monadic iterative interpretation of T32n in C(m). To do this, we will show that we can define the decomposition of C(m) into the factor subgroups C(m,i). Let 0(x, y, t, u) be the following formula with free relation variable Q:
Then by induction on n the relation Q(x,y,t,u) given by the iterative definition is true precisely when there is an integer j in the range 0 < j < 22n-1 such that x = jt and y = ju. Notice that we are using an idea first exploited by Fischer and Rabin [1974] in their proof of a lower bound for Example 8.20: each integer j < 22n can be written j = j1J2 +j3 +J4, where j1,j2,J3,J4 < 22n-1. For the remainder of the proof we will assume that Q(x, y, t, u) is given by the iterative definition
196
Kevin J. Compton and C. Ward Henson
Fix a generator u of C(m). We can specify that x = ju for some integer j in the interval by writing Q(x,x,u,u). Since m > 22 , the values ju are distinct for j E I. Let us identify I with the set of elements ju such that j E I. The group operation restricted to 7 defines integer addition. We can also define multiplication: if X1 = j1u and x2 = J2U then y = j1j2u precisely when Q ( y , x 1 , X 2 , u) holds. Therefore, we can say that s € I is prime and also that s E I is a prime power. Let a(x,y,z, s,u) be a first-order formula that says that s E I is the largest power of some prime in I, that x, y, and z are annihilated by s (i.e., if a(x, y, z, s, u) holds, then s = mi for some i < k and x,y,z E C(mi). In other words, a(x, y, z, s, u) defines addition on the direct summands C(mi). Now every element t in C(m) can be uniquely expressed as a sum
where each ti is an element of C(mi). We can specify a formula n(s, t', t, u) that says t1 = ti when s is m,. We say simply that t' is annihilated by s and t — t' is divisible by s:
Thus, we can define the decomposition of C(m) into its factor subgroups. In particular, since u can be expressed as a sum uI + u2 + • • • + Uk , where ui is a generator of (C(mi), the formula n(s,u',u,u) picks a unique generator for each factor subgroup as s ranges over maximal prime powers in I. The rest of the proof proceeds as in Example 8.24. For example, we form S'n(x, s,t',u',u) by substituting a(x,y,z,s,u) for each occurrence of x + y = z in Sn(x,t', u'). ThenS"n£(x,t, u) is the formula
We define n " ( x , y , t , u ) and a"n(x,t,u,v,w) similarly.
9
Upper bounds
In this section we give upper bounds that show that most of the lower bounds obtained in sections 4-7 are the best possible. First we give upper bounds for satT(Lo), satpT(Lo), and sat*T(Lo). Recall from Theorem 4.5 that when T(dn) = o(T(n)) for some d between 0 and 1, these sets are not in NTIME(T(cn)) for some c > 0.
Proving lower bounds
197
Proposition 9.1. Let T be a time resource bound. Then
Proof. To determine whether a sentence from L0 is in satT(Lo), nondeterministically generate a finite binary relation. We give a nondeterministic recursive procedure that determines whether is true in this relation. If € satT(Lo), then it holds in some binary relation a set of size at most T ( n ) . The representation of this relation requires at most T(n)2 bits. We will show that our recursive procedure halts within time cT(n)n+2 on The procedure tests the subformulas of and combines results to produce an answer. We may assume that all negations have been pushed inward so that only atomic formulas are negated. It is clear how the procedure works if is a conjunction or disjunction. If begins with an existential quantifier, an element of is nondeterministically assigned as the value of the quantified variable and the enclosed formula is checked. If begins with a universal quantifier, then each element of is assigned in turn to the quantified variable and the enclosed formula is checked. When an atomic formula is reached it can be determined in time O(T(n) 2 ) whether it is true in for the assignment values at that point. Since, for each of at most n universal quantifiers, T(n) values are generated, the total time is 0(T(n) 2 T(n) n ), as claimed. Suppose is in prenex normal form. For each E > 0, whenever n is sufficiently large there are at most (1 + E)n/logn universal quantifiers in , so determining whether a prenex formula is in s a t ( L o ) ) is in then used, except that when a relation variable is encountered, it is necessary to jump to its definition (this may take n moves), compute its value by calling our recursive procedure, and return. Total time, then, is O(nT(n) 2 T(n)n) because the tree of recursive procedure calls has height at most n and branches at most n times at each vertex; at the leaves, there is a cost of T(n) 2 moves to evaluate atomic formulas; at the vertices corresponding to relation variable references there is a cost of O(n) moves to find the definition. For all three bounds we must use the linear speed-up theorem (see Hopcroft and Ullman [1979]) to eliminate constants in front of the time bounds. I We see that if T(n) n+2 = O(T(dn)) for some d > 0, then satT(L0) € NTIME(T(dn)), so we have essentially the same upper and lower bounds. Similar remarks pertain in the other cases.
198
Kevin J. Compton and C. Ward Henson
Proposition 9.2. Let T be a time resource bound. Then
Proof. Given a sentence of length n in MZ/o, nondeterministically generate a binary relation. We use alternation to determine whether holds in the relation. If E satT(MLo), then it holds in some binary relation on a set of size at most T ( n ) . For each set quantifier encountered it is necessary to generate T(n) bits to assign a value to the quantified variable. There are O(n) such variables so this part of the computation takes time O(nT(n}). This time is dominated by the O(T(n)2) time needed to generate and verify atomic formulas. If is a sentence in ML0,we use the same procedure except that when a subformula with a relation variable is encountered, the value of the subformula is guessed and verified using alternation. I Notice that if
for some c > 0, then, for some so
same upper and lower bounds. We now turn to upper bounds for theories of finite trees. To determine if a sentence of length n is in s a t ( E r ) , we will show that it suffices to nondeterministically generate a tree in Trm, where mlog m > n, and verify that holds in this tree. In fact, we prove a somewhat stronger result: given a tree of height r or less, there is a tree in satisfying precisely the same sentences of length n or less. Our proof uses Ehrenfeucht games. Observe that a sentence of length n can have at most m distinct variables (i.e., variables with different subscripts). Therefore, we will use the formulation of Ehrenfeucht games for logics with a bounded number of variables. These games were introduced for infinitary logics by Barwise [1977] and later used by Immerman [1986] to obtain lower bounds for queries on finite relational structures. Given two structures A and B for a first-order logic L, write A = B to indicate that A and B satisfy precisely the same sentences from L of quantifier rank at most n and with at most m distinct variables. The game used to characterize = is played for n moves between players I and II on a pair of structures A and B. Each player begins with m pebbles. On the first move player I places a pebble on an element of A (or B) and player II responds by placing a corresponding pebble on an element of 23 (or A). On each remaining move player I has two options: he may place an unplayed pebble on an element of A (or B), in which case player II places a corresponding pebble on an element of B (or A) ; or he may remove one of the pebbles on A (or B) and replay it (not necessarily on the same structure), in which case player II removes the corresponding pebble from
Proving lower bounds
199
B (or A) and replays it in response to the move of player I. Player II wins if the mapping from the set of elements of A covered by pebbles at the end of the game to corresponding elements in 53 is an isomorphism between substructures of A and B; otherwise, player I wins. The basic result concerning this game is that player II has a winning strategy if and only if A = B. We use this fact to prove two simple lemmas. We will assume henceforth that in addition to the binary edge relation, trees also have a unary relation that is true only at the root. This is a technical convenience to force player II to pebble a root whenever player I pebbles a root. For a tree A and a vertex x in A, let Ax be the subtree of A whose set of vertices consists of x and all of its descendents. Lemma 9.3. Suppose A is a tree and x is a vertex of A. Let A' be the result of replacing Ax in A by another tree Proof. We know that player II has a winning strategy for the m pebble game of length n on Ax and B. Player II uses this as part of a winning strategy for the m pebble game of length n on A and A'. Whenever player I pebbles an element in A — Ax or A' — 53, player II responds by pebbling the same element; whenever player I pebbles an element in 2lx or B, player II responds by pebbling the element dictated by the strategy for Ax and 53. Notice that if player I pebbles the root of Ax, player II pebbles the root of 53 (and vice versa) so that adjacency to elements in A — Ax will be the same. This is a winning strategy for player II so A = A'. I Lemma 9.4. Let A and B be trees. Suppose that for each isomorphism type, the set of primary subtrees of A and the set of primary subtrees of 53 either contain the same number of trees of that type, or both contain at least m trees of that type. Then A = B for every n > 0. Proof. It is easy to determine a winning strategy for player II. If player I pebbles the root of A (or B), player II pebbles the root of B (or A). If player I pebbles an element in a primary subtree A' of A and no other elements of A' have been pebbled, player II responds by pebbling the corresponding element in a primary subtree B' = A' of B where no other elements have been pebbled. Player II responds similarly if player I pebbles an element in a primary subtree B' of B and no other elements of B' have been pebbled. If player I chooses an element in a primary subtree A' of A where some elements have already been pebbled, these elements correspond to elements already pebbled in some primary subtree B' — A' of B. The isomorphism determines the response of player II. Player II responds similarly if player I chooses from a primary subtree of 53 where elements have already been pebbled. Because no more than m elements in a structure are pebbled at
200
Kevin J. Compton and C. Ward Henson
any time, it is easy to see that this strategy can always be carried out for A and B satisfying the hypotheses of the lemma. I Theorem 9.5. Given a finite tree A of height at most r, there is a tree B E Trm such that A = B for all n>0. Proof. Modify A in the following manner. For each nonleaf vertex x of depth r — 1, consider all children y of x and subtrees Ay; for each isomorphism type, if more than m subtrees Ay are isomorphic, delete enough of them so that there are precisely m. Continue this modification procedure for vertices of depth r — 2, r — 1, and so on, up to the root. Call the resulting tree B. It is clear from the two preceding lemmas that every time we delete subtrees in this process, we obtain a tree in the same =-class as A. Thus, A = B. It is evident that B € T™. Remark 9.6. With slight modifications, this proof shows that for any tree A of height r or less, there is a tree B € T™ such that A =™ B for all m > 0. We have, as an immediate consequence of the preceding theorem, upper bounds for theories of finite trees.
Corollary 9.7.
Proof. When r > 3 and m is the least integer suc
each tree in Trm has at most exp r _ 2 (cn) vertices. Thus, the time required to nondeterministically generate a tree in 77™ and determine whether a sentence of length n holds in this tree is exp r _ 2 (cn) n . This function is dominated by expr_2(dn) for some d > 0 when r > 3 and by expr_2(dn2) when r = 3. I Remark 9.8. This theorem gives matching upper bounds for the lower bounds obtained in Corollary 6.4 except for the case r = 3. There the lower bound is NTIME(2cn) and the upper bound is NTIME(2dn2). Ferrante and Rackoff [1979] get precisely the same bounds for the theory of one-to-one functions (cf. Example 8.12). It is more satisfying to say that sat(E3) is a complete problem for
via polynomial time reductions, according to Theorem 6.1(iv). Both sat(E1) and sat(E2) are in PSPACE since the number of vertices in trees in T1m and 72m is polynomially bounded in n, where m is the
Proving lower bounds
201
least integer such that m log m > n. (Recall that by Theorem 9.5, for every finite tree A of height at most r, there is a tree such that A =™ B.) Every first-order theory with a model of power greater than 1 is hard for PSPACE (via log-space reductions), so we know fairly precisely the complexity of these theories. Write A &% B to indicate that A and 03 satisfy the same monadic second-order sentences of quantifier rank n with at most m variables. To obtain upper bounds for monadic second-order theories of finite trees we must introduce Ehrenfeucht games characterizing the relation mn. In such a game, players I and II play for n moves on structures A and B. Let PI , P2, . . . , Pm be unary relation symbols not in the language of A and B. During each move of the game one of the symbols Pi will be assigned a pair of sets. This pair contains a subset of A and a subset of B. Initially, each of these symbols is assigned the empty set for A and the empty set for 03. On each move player I picks a relation symbol Pi. The previous assignment to Pi, is forgotten. Player I assigns a subset of A (or B). Player II responds by assigning a subset of B (or A) to Pi . Whenever player I picks a singleton set, player II must respond with a singleton set. (Singleton set moves correspond to element quantifiers.) Now suppose that at the end of the game symbols P1 , P2, . . . , Pm are assigned subsets P1 , R2, . . . , Rm of A and subsets S1 , S2, . . . , Sm of 03. If the set
is an isomorphism between substructures of
then player II wins; otherwise, player I wins. The basic result concerning this game is that player II has a winning strategy if and only if A ™ B. We will sometimes say that an Ehrenfeucht game is played on structures (A, RI , . . . , Rm) and (B, S1 , . . . , Sm), where R1 , . . . , Rm are subsets of A and S1, . . . ,Sm are subsets of 03. By this we mean that the symbols P1 , . . . , Pm initially have R1 , . . . , Rm and S1 , . . . , Sm assigned to them. The game then proceeds as before. We will say that (A, R1, . . . , Rm) and (03, S1, . . . ,Sm) are n-equivalent if player II has a winning strategy for games of length n on this pair of structures. Clearly, n-equivalence is an equivalence relation. Using monadic second-order Ehrenfeucht games, we can show that =™ can be replaced by ™ in Lemma 9.3. In fact, we can show more. Lemma 9.9. Suppose that A is a finite tree with subsets RI, . . . , Rm. Let x be a vertex the structure obtained by replacing the substructure by another
202
Kevin J. Compton and C. Ward Henson
structure (B, S1,..., Sm), where B is a tree. If
are n-equivalent, then so are We would also like to prove a result like Lemma 9.4, but this is not so simple. We must prove results like Lemma 9.4 and Theorem 9.5 simultaneously. To do this we need more complicated sets of trees. Definition 9.10. Define functions f(m,n,r) and g(m,n,r) as follows.
and
Functions f and g are defined for all integers m,n,r repeatedly use the last equation to obtain
> 0 because we can
and then replace each factor f(m,i,r + 1) by 2m(g(m,i,r) + l)f( m , i , r ). Now define classes Urm,n of trees of height at most r. Uom,n contains all trees of height 0. consists of all trees whose primary subtrees come f r o m , no more than g(m, n, r) primary subtrees coming from the same isomorphism class. Define classes Vrm,n of structures (A,R1,..., Rm), where A is a tree of height at most r and each Ri is a subset ofA.vom,n contains all structures (A, R1, ..., Rm), where A is a tree of height 0. by restricting to primary subtrees of A all come from Vrm,n, no more than g(m,n,r) such substructures coming from the same isomorphism class. Notice that . Theorem 9.11. Given a finite tree A of height at most r and an integer n>0, there is a tree B € Urm,n such that A mnB. Proof. We prove a more general assertion. Given a structure ( A , R I , ..., Rm}, where A is a tree of height at most r and R1,..., Rm are subsets of A, there is a structure such that and n-equivalent. The proof is by induction on r. For r = 0, the assertion is obvious. Assume that A has height r > 0 and that the assertion holds for trees of lesser
Proving lower bounds
203
height. Consider the substructures of (A, RI ,..., Rm) formed by restricting to primary subtrees. By the induction hypothesis, each such substructure is n-equivalent to some structure in for each n-equivalence class replace the substructures in that class by a structure from in the same class with the provision that if there are more than g(m, n, r — 1) substructures in the class we first eliminate enough of them to make their number precisely g(m,n,r — 1). In this way we form a structure (B,S1,... ,S m ) € V™'™. We show that (A, RI , . . . , Rm) and (B,S1,..., Sm) are n-equivalent. Fix an n-equivalence type T. Let (A', R1,..., R'm) be the union of substructures of (A, R1,... ,Rm) of type T formed by restricting to primary subtrees of A. Define the substructure (B', S'1..., S'm) of (B,S1,..., Sm) similarly. Thus, A' and B' are forests, and each substructure of (A', R'1,..., formed by restricting to a tree in the forest is of type T. Moreover, A' and B' either contain the same number of trees or both contain at least g(m,n,r — 1) trees. We claim that (A', R'1,..., R'm) and (B',S'1,... ,S'm) are n-equivalent. From this claim it follows easily that (A,R1,.. .,Rm) and (B,S1,...,S m ) are n-equivalent because player II can combine the winning strategies on the pairs of substructures. We establish the claim by induction on n. The case n = 0 is clear. (Notice, however, that it is crucial that g(m,0,r — 1) = 2 because S'i must be assigned a nonsingleton set whenever R'i is assigned a nonsingleton set.) Suppose that n > 0 and that the claim is true for smaller values. Player I will begin by assigning a subset of one of the forests—say subset Ri of A'—to a relation symbol Pi. This gives a new structure where when By the induction hypothesis (for the induction on r), every substructure formed by restricting this structure to a tree in A' is (n — l)-equivalent to some structure in there can be at most f(m, n - 1, r — 1 ) - e q u i v a classes represented among such substructures. Player II responds by assigning a subset of Si" of B' to Pi;to obtain a structure (B', S1",..., Sm), where S" = Sj when i = j. She does this in such a way that for every (n - 1)-equivalence type either have the same number of substructures of type T' formed by restricting to trees in A' and B', or both have at least g(m,n — l,r — 1) such substructures of type T'. Player II can always make such a response because A' and B' either have the same number of trees or both have at least g(m, n, r — 1) = f(m, n — 1, r — l)g(m, n — 1, r — 1) trees. By the induction hypothesis (for the induction on n)
are (n - l)-equivalent so (A',R'1,... ,R'm) and (B',S'1,... ,S'm) are n-equivalent. I Theorem 9.12. For each r > 1 there is a d > 0 such that
204
Kevin J. Compton and C. Ward Henson
Proof. It is easy to show by induction that for each r > 1 there is a c > 0 such that
If we let h(m, n, r) be the maximum number of vertices of any structure in see
we
that
For each r > 2 there is a c > 0 such that h(m, n, r) < expr(c(m + logn)). When r > 2 we can determine if a sentence from MLt is in sat(ME r ) by nondeterministically generating a tree in , where m log m > n, and using alternation to verify that holds in this tree. This can be done in ATIME(expr(dn/ logn), n). When
r = 1 we must be a little more careful because a tree in
structure. Suppose we have chosen m subsets R1,..., Rm from a tree A of height 1. Let +Ri, be Ri and — Ri be the complement of Ri. For We only keep track of which sets Ri contain the root of A and the the values \d • R\ for each d E {+, -}. Thus, {A,R1,..., Rm) can be represented in space O(2m logn). Using this kind of representation, the argument above shows that sat
10
Open problems
We close with a list of problems, first presented in Compton and Henson [1990]. They are mostly concerned with lower bounds for theories arising in algebra. The only one on which there has been substantial progress is Problem 10.13. These problems represent only a small number of the decidable theories whose complexities deserve to be investigated. We have not mentioned many of the decidable theories of modules investigated in recent years—Point [1986; 1993], Point and Prest [1993], Prest [l988a; 1988b], for example. Problem 10.1. Determine the complexity of first-order theories of finite fields.
The first-order theory of finite fields, and several related theories, were shown to be decidable by Ax [1968] in a paper which has proved to be
Proving lower bounds
205
of great mathematical influence. Later Fried and Sacerdote [1976] gave a primitive recursive decision procedure, but it is not known if any of these theories is elementary recursive. It is not difficult to show that these theories have a hereditary lower bound of ATIME(exp 2 (cn),cn) The method is to give a monadic interpretation of the classes of binary relations on sets of size at most exp 2 (n), and then apply Theorem 7.3. We give a brief sketch of the argument. Let f be any infinite field which is a model of the theory of finite fields and in which one has the coding of finite sets used by Duret [1980]. That is, given any two disjoint finite sets A, B C F, there is an element w e f such that if a £ A then a + w is a square in f and if b € B then b + w is not a square in f. Construct by iteration formulas an(x,u) such that for each n there is a choice of parameters u so that an(x,u) is true in f of exp 2 (n) values of x. For example, if f has characteristic 0, then an(x) can be constructed as in Fisher and Rabin [1974] so that an(x) holds in f exactly when x is one of the integers 0,... ,exp 2 (n) — 1. Alternatively, one could use formulas an(x,y) asserting that x is an exp 2 (n)th root of y. Now consider the formulas B n ( x , t , u ) given by
For an appropriate choice of t in F, the mapping is oneto-one on . This together with the coding of finite sets gives every binary relation on sets of size at most exp 2 (n). The coding of finite sets also gives every subset of the universe so we have the required monadic interpretation. Problem 10.2. Determine the complexity of the first-order theory of linearly ordered Abelian groups. This theory was shown to be decidable by Gurevich [1965], with later improvements in Gurevich [1977]. There is a simple interpretation of the first-order theory of linear orders in this theory, so it has a hereditary N T I M E ( e x p ( c n ) ) lower bound. (See Example 8.3.) On the other hand, a primitive recursive decision procedure for the theory was given by Gurevich [1977]. It would be interesting to improve either of these bounds. Problem 10.3. Determine the complexity of first-order theories of valued fields. Ax and Kochen [1966] and Ersov [1965] proved the decidability of various first-order theories of valued fields, including some power series fields and the fields of p-adic numbers Qp. Brown [1978] obtained an elementary recursive upper bound for the first-order theory of 'almost all' of the fields Qp—that is, the set of sentences true in Qp for all but finitely many p. Very little is known about lower bounds for this theory or about the other related theories covered by the Ax-Kochen-Ersov work, with the exception
206
Kevin J. Compton and C. Ward Henson
of the theory of p-adic numbers, whose complexity was determined by Egidi [1993] Problem 10.4. Determine the complexity of the first-order theory of Boolean algebras with several distinguished filters. Ersov [1964] proved decidability of the first-order theory of Boolean algebras with a distinguished filter. Touraille [1985] presents some results on the elimination of quantifiers for this theory, but does not show decidability. Rabin [1969] showed decidability of the theory of Boolean algebras with quantification over filters by giving an interpretation in the monadic second-order theory of two successors. This gives an upper bound of NTIME(exp00(dn)), but nothing is known about lower bounds. Problem 10.5. Determine the complexity of the first-order theory of the lattice of closed subsets of the Cantor set. Rabin [1969] proved that this theory is decidable by interpreting it in the monadic second-order theory of two successors. As in the previous problem, this gives an upper bound of NTIME(exp00(dn)). It is not known if this theory is elementary recursive and no nontrivial lower bounds are known. A more explicit analysis of this theory has been given by Gurevich [1982]. Problem 10.6. Determine the complexity of the first-order theory of the ring of bounded sequences of real numbers. Cherlin [1980] gave a very explicit and difficult decision procedure for this theory, but its complexity has not been analysed. It should be possible to extract an upper bound from Cherlin's work. While it seems unlikely to us that this theory is elementary recursive, there are no good lower bounds known. Problem 10.7. Determine the complexity of the theory of pairs of torsionfree Abelian groups, and of the theory of a vector space V with k distinguished subspaces (k = 1,2,3,4). The theory of pairs of torsion-free Abelian groups was proved decidable by Kozlov and Kokorin [1969]; see also Schmitt [1982]. For k > 5, the theory Tk of vector spaces with k distinguished subspaces (over some fixed field f) is undecidable; see Baur [1976] or Slobodskoi and Fridman [1976]. Here the vector space is equipped with + and a family of unary scalar multiplication functions, one for each element of f'. If k < 4, however, the theory Tk was shown to be decidable by Baur [1980]. See Prest [1988a] for a discussion of how these theories are related to the representation theory of finite-dimensional algebras over F. The theories Tk are stable and hence the undecidability of T5 could not be proved by the usual means of interpreting arithmetic or finite graphs. There does not seem to be any corresponding a priori impediment to using the methods of this chapter to
Proving lower bounds obtain lower bounds on the complexity of
207 ,
Problem 10.8. Determine the complexity of the first-order theory of real closed fields and the theory of the first-order theory of algebraic closed fields. These theories are, respectively, the first-order theory of the real numbers and the first-order theory of the complex numbers. Good upper and lower bounds are known for these theories, but the gap has not been completely closed. Berman's ATIME(2 c n ,n) lower bound for real addition is the best bound known for the theory of real closed fields. We discussed this bound in Example 8.20. By the remarks following the example, we have the same lower bound for the theory of algebraic closed fields. The best upper bound at present for the theory of real closed fields is SPACE(2dn); this was proved by Ben-Or et al. [1986]. This bound holds as well for the theory of algebraic closed fields since there is a simple interpretation of the complex numbers in the real numbers. For the same reason, any lower bound for the theory of algebraic closed fields would hold for the theory of real closed fields. Robinson [l959b] showed that if A is the field of real algebraic numbers, then the first-order theory of (R, +, -, A) is also decidable. It would be interesting to know if this theory has a somewhat higher complexity than the theory of real closed fields. Problem 10.9. Determine the complexity of the first-order theory of differentially closed fields. Robinson [l959a] proved the decidability of this theory, but essentially nothing more is known about its complexity. See Wood [1976] for a fuller discussion of this mathematically interesting theory. Problem 10.10. Is elementary recursiveness of a theory preserved under product and sheaf constructions? Decidability of first-order theories is preserved by many general constructions, such as products (Feferman and Vaught [1959]) and sheaf constructions (Macintyre [1973]). Some upper bound results for weak products are presented in Ferrante and Rackoff [1979], where the question is raised whether, for every model A whose first-order theory is elementary recursive, the first-order theory of the weak direct product Aw is also elementary recursive. The same type of question for other product and sheaf constructions is also open and worth investigating. (See Chapter 5 of Ferrante and Rackoff [1979].) Problem 10.11. Give nontrivial lower bounds for mathematically interesting theories whose decidability is still open. Examples include the first-order theories of the field of rational functions over the complex numbers; the real exponential field (R,+,-,exp);
208
Kevin J. Compton and C. Ward Henson
the field of meromorphic functions; and many others. It may be possible to show that some of these theories are not elementary recursive, just as Semenov [1980] did for the theory of free groups. (See the remarks following Example 8.8.) Problem 10.12. Is there a 'natural' decidable theory which is not primitive recursive? Problem 10.13. In [Compton and Henson, 1990], we asked whether there is there a 'natural' decidable theory with a lower bound of the form NTIME(exp00(f(n)) where f ( n ) is not linearly bounded. This was answered affirmatively by Vorobyov [1997; preprint], who showed that a rudimentary version of the theory of finite typed sets investigated by Henkin [1963] and Statman [1979] is decidable, but has a lower bound of
Problem 10.14. Determine the complexities of fragments of theories with given prefix structures. There has been some interesting work done in this area. (See, for example, Robertson [1974], Reddy and Loveland [1978], Furer [1982], and Scarpellini [1984].) In certain cases the methods of this chapter should give results under these restrictions. This is not likely to be true where iterative interpretations are used, since iterative definitions almost always introduce an unbounded number of alternations of quantifiers. However, where prenex interpretations are used in conjunction with Theorem 6.1 and Corollary 6.1, it seems clear that complexity results for sentences with specific limitations on prefix structure can be obtained. Other syntactic limitations can also be imposed on decision problems and have been widely studied in the setting of the decidable/undecidable distinction. For example, in algebraic theories, one may pay attention to the degree and number of variables of occurring polynomials. In decision problems (and, more generally, complexity problems) the refinements mentioned above, especially limitations to a simpler and more intelligible prefix structure, often reflect restriction to mathematically more interesting and significant problems (as has been emphasized to us by G. Kreisel). Thus, the undecidability of Hilbert's tenth problem is far more interesting than the undecidability of arithmetic; the undecidability of the word problem for finitely presented groups is far more interesting than the undecidability of the theory of groups. One can hope for and expect to see a similar kind of increasing maturity in the study of complexity of decidable problems, not only at the level of NP-complete or PSPACEcomplete problems (where it already exists to some extent), but also at
Proving lower bounds
209
higher levels of complexity. Problem 10.15. Characterize the PSPACE-complete theories. We noted in section 1 that every theory with a model of power greater than 1 is PSPACE-haxd. Thus, the PSPACE-complete theories are, in some sense, the simplest nontrivial theories. A number of different theories have been shown to be PSPACE-complete. (See Stockmeyer [1977], Ferrante and Rackoff [1979], and Grandjean [1983].) It would be interesting to have model-theoretic characterization of these theories. Problem 10.16. If one substitutes 'tree' for 'binary relation' in the definitions of satT(Lo) and satT(M£o), do Theorems 6.1 and 7.1 still hold for An affirmative answer would give a generalization of Corollaries 6.5, 7.5, and 7.9. Problem 10.17. Use the techniques of this chapter to derive a lower bound for the emptiness problem for *-free regular expressions. The proof that this problem is not elementary recursive is one of the more difficult results in Stockmeyer [1974]. McNaughton and Papert [1971] show that the first-order theory of finite linear orders with an added unary predicate (Example 8.1) can be reduced to this problem, but it is not clear that an elementary recursive reduction can be found. Furer [1978] gives a lower bound of for the emptiness problem for *-free regular expressions. It would be interesting to know if this could be strengthened to a lower bound of
Acknowledgements The authors wish to thank Elsevier Scientific for permission to publish an updated, revised version of the paper [Compton and Henson, 1990]. We would like to thank the referee of Compton and Henson [1990] for his extraordinarily careful reading of this chapter and helpful comments.
References [Ax, 1968] J. Ax. The elementary theory of finite fields, Ann. Math., 88, 239-371, 1968. [Ax and Kochen, 1966] J. Ax and S. Kochen. Diophantine problems over local fields III: decidable fields, Ann. Math., 83, 437-456, 1966. [Barwise, 1977] J. Barwise. On Moschovakis closure ordinals, J. Symbolic Logic, 42, 292-296, 1977.
210
Kevin J. Compton and C. Ward Henson
[Baur, 1976] W. Baur. Undecidability of the theory of abelian groups with a subgroup, Proc. Amer. Math. Soc., 55, 125-128, 1976. [Baur, 1980] W. Baur. On the elementary theory of quadruples of vector spaces, Ann. Math. Logic, 55, 125-128, 1976. [Ben-Or et al, 1986] M. Ben-Or, D. Kozen, and J. Reif. The complexity of elementary algebra and geometry, J. Comput. System Sci., 32, 251-164, 1986. [Berman, 1980] L. Berman. The complexity of logical theories, Theoret. Comput. Sci., 11, 71-77, 1980. [Borger, 1984a] E. Borger. Decision problems in predicate logic. In Logic Colloquium 1982, G. Lolli, G. Longo, and A. Marcja, eds., pp. 263-301. North-Holland, Amsterdam, 1984. [Borger, 1984b] E. Borger. Spektralproblem and completeness of logical decision problems. In Logics and Machines: Decision Problems and Complexity, E. Borger, G. Hasenjaeger, and D. Rodding, eds., Lecture Notes in Comput. Sci. 171, pp. 333-356. Springer-Verlag, Berlin, 1984. [Borger, 1985] W. Borger. Berechenbarkeit, Komplexitdt, Logik. ViewegVerlag, Wiesbaden, 1985. [Borger et al, 1997] E. Borger, E. Gradel and Y. Gurevich. The Classical Decision Problem. Springer-Verlag, Berlin, 1997. [Brown, 1978] S. S. Brown. Bounds on transfer principles for algebraically closed and discretely valued fields, Mem. Amer. Math. Soc., 204, 1978. [Bruss and Meyer, 1980] A. Bruss and A. Meyer. On time-space classes and their relation to the theory of real addition, Theoret. Comput. Sci., 11, 56-69, 1980. [Biichi, 1962] J. Buchi. Turing machines and the Entscheidungsproblem, Math. Ann., 148, 201-213, 1962. [Cegielski, 1996] P. Cegielski. Definability, decidability, complexity, Ann. Math. Artificial Intelligence, 16, 311-341, 1997. [Cegielski et al., 1996] P. Cegielski, Y. Matiyasevich, and D. Richard. Definability and decidability issues in extensions of the integers with the divisibility predicate, J. Symbolic Logic, 61, 515-540, 1996. [Chagrov and Zakharyaschev, 1997] A. Chagrov and M. Zakharyaschev. Modal Logic. The Clarendon Press, New York, 1997. [Chandra et al, 1981] A. Chandra, D. Kozen, and L. Stockmeyer. Alternation, J. Assoc. Comput. Mach., 28, 114-133, 1981. [Cherlin, 1980] G. Cherlin. Rings of continuous functions: decision problems. In Model Theory of Algebra and Arithmetic, L. Pacholski, J. Wierzejewski, and A. J. Wilkie, eds., Lecture Notes in Math. 834, pp. 44-91. Springer-Verlag, Berlin, 1980. [Compton and Henson, 1990] K. J. Compton and C. W. Henson. A uni-
Proving lower bounds
211
form method for proving lower bounds on the complexity of logical theories, Ann. Pure Appl. Logic, 48, 1-79, 1990. [Compton et al, 1987] K. J. Compton, C. W. Henson, and S. Shelah. Nonconvergence, undecidability, and intractability in asymptotic problems. Ann. Pure Appl. Logic, 36, 207-224, 1987. [Duret, 1980] J.-L. Duret. Les corps pseudo-finis ont la propriete d'independence. C. R. Acad. Sci. Paris Ser. A-B , 290, A981-A983, 1980. [Egidi, 1993] L. Egidi. The complexity of the theory of p-adic numbers. In Proceedings of the 34th Annual Symposium on Foundations of Computer Science, Palo Alto, CA, pp. 412-421. IEEE Computer Society Press, Los Alamitos, CA, 1993. [Emerson and Halpern, 1985] E. A. Emerson and J. Halpern. Decision procedures and expressiveness in the temporal logic of branching time, J. Comput. System Sci., 30, 1-24, 1985. [Ersov, 1964] Y. Ersov. Decidability of the elementary theory of relatively complemented distributive lattices and the theory of filters, Algebra i Logika, 3, 17-38, 1964. (In Russian.) [Ersov, 1965] Y. Ersov. On the elementary theory of maximal normed fields, Soviet Math. Dokl., 6, 1390-1393, 1965. (English translation.) [Ersov et al., 1965] Y. Ersov, I. A. Lavrov, A. D. Taimanov, and M. A. Taitslin. Elementary theories, Russian Math. Surveys, 20, 35-100, 1965. (English translation.) [Fagin et al., 1995] R. Fagin, J. Halpern, Y. Moses, and M. Vardi. Reasoning about Knowledge. MIT Press, Cambridge, MA, 1995. [Feferman and Vaught, 1959] S. Feferman and R. Vaught. The first-order properties of products of algebraic systems, Fund. Math., 47, 57-103, 1959. [Ferrante, 1974] J. Ferrante. Some upper and lower bounds on decision procedures in logic, Doctoral thesis, Massachusetts Institute of Technology, Cambridge, MA, 1974. [Ferrante and Rackoff, 1979] J. Ferrante and C. Rackoff. The Computational Complexity of Logical Theories, Lecture Notes in Math. 718. Springer-Verlag, Berlin, 1979. [Fischer and Ladner, 1979] M. J. Fischer and R. Ladner. Prepositional dynamic logic of regular programs, J. Comput. System Sci., 18, 194-211, 1979. [Fischer and Rabin, 1974] M. J. Fischer and M. Rabin. Super-exponential complexity of Presburger arithmetic. In Complexity of Computation, R. M. Karp, ed. SIAM-AMS Proc., vol VII, pp. 27-42. American Mathematical Society, Providence, RI, 1974. [Fleischmann et al., 1977] K. Fleischmann, B. Mahr, and D. Siefkes.
212
Kevin J. Compton and C. Ward Henson
Bounded concatenation theory as a uniform method for proving lower complexity bounds. In Logic Colloquium 1976, R. Gandy and M. Hyland, eds. pp. 471-490. North-Holland, Amsterdam, 1977. [Fried and Sacerdote, 1976] M. Fried and G. Sacerdote. Solving Diophantine problems over all residue class fields of a number field and all finite fields, Ann. Math., 104, 203-233, 1976. [Furer, 1978] M. Furer. Nicht-elementare untere Schranken in der Automaten-theorie, Doctoral thesis, ETH, Zurich, 1978. [Furer, 1982] M. Furer. The complexity of Presburger arithmetic with bounded quantifier alternation, Theoret. Comp. Sci., 18, 1-5-111, 1982. [Gradel, 1988] E. Gradel. Subclasses of Presburger arithmetic and the polynomial-time hierarchy, Theoret. Comput. Sci., 56, 289-301. [Gradel, 1989] E. Gradel. Dominoes and the complexity of subclasses of logical theories, Ann. Pure Appl. Logic, 43, 1-30, 1989. [Grandjean, 1983] E. Grandjean. Complexity of the first-order theory of almost all finite structures, Inform. Control, 57, 180-204, 1983. [Gurevich, 1965] Y. Gurevich. Elementary properties of ordered Abelian groups, Amer. Math. Soc. Transl, 46, 165-192, 1965. [Gurevich, 1977] Y. Gurevich. Expanded theory of ordered Abelian groups, Ann. Math. Logic, 12, 192-228, 1977. [Gurevich, 1982] Y. Gurevich. Crumbly spaces, In Proc. 1979 Intl. Cong. Logic, Methodology and Philosophy of Science, pp. 179-191. NorthHolland, Amsterdam, 1982. [Halpern and Moses, 1992] J. Halpern and Y. Moses. A guide to completeness and complexity for modal logics of knowledge and belief, Artificial Intelligence, 54, 319-379, 1992. [Halpern and Vardi, 1986] J. Halpern and M. Vardi. The complexity of reasoning about knowledge and time. In Proc. 18th Ann. ACM Symp. on Theory of Computing, pp. 304-315. Association for Computing Machinery, New York, 1986. [Hardy and Wright, 1964] G. H. Hardy and E. M. Wright. Theory of Numbers. Oxford University Press, Oxford, 1964. [Harel, 1984] D. Harel. Dynamic logic. In Handbook of Philosophical Logic, Vol. II, D. M. Gabbay and F. Guenthner, eds., pp 497-604, Reidel, Dordrecht, 1984. [Henkin, 1963] L. Henkin. A theory of prepositional types, Fund. Math., 52, 323 344, 1963. [Hopcroft and Ullman, 1979] J. Hopcroft and J. D. Ullman. Introduction to Automata Theory, Languages and Computation. Addison-Wesley, Reading, MA, 1979. [Immerman, 1986] N. Immerman. Relational queries computable in polynomial time, Inform. Control, 68, 86-104, 1986.
Proving lower bounds
213
[Kaufmann and Shelah, 1983] M. Kaufmann and S. Shelah. On random models of finite power and monadic logic, Discrete Math., 54, 285-293, 1983. [Korec and Rautenberg, 1975/76] I. Korec and W. Rautenberg. Model interpretability into trees and applications, Arch. Math. Logik Grundlag, 17, 97-104, 1975/76. [Kozen, 1980] D. Kozen. Complexity of Boolean algebras, Theoret. Comp. Sci., 10, 221-147, 1980. [Kozen and Tiuryn, 1990] D. Kozen and J. Tiuryn. Logics of programs. In Handbook of Theoretical Computer Science, Vol. B, pp. 789-840, Elsevier, Amsterdam, 1990. [Kozlov and Kokorin, 1969] G. T. Kozlov and A. I. Kokorin. Elementary theory of Abelian groups without torsion, with a predicate selecting a subgroup, Algebra and Logic, 8, 182-190, 1969. (English translation.) [Kutty et al, 1995] G. Kutty, L. Moser, P. Melliar-Smith, Y. Ramakrishna, and L. Dillon. Axiomatizations of interval logics, Fund. Inform., 24, 313-331, 1995. [Lewis, 1980] H. R. Lewis. Complexity results for classes of quantificational formulas, J. Comput. System Sci., 21, 304-315, 1980. [Lo, 1988] Libo Lo. On the computational complexity of the theory of abelian groups, Ann. Pure Appl. Logic, 37, 205-248, 1988. [Lynch, 1982] J. F. Lynch. Complexity classes and finite models, Math. Systems Theory, 15, 127-144, 1982. [Machtey and Young, 1978] M. Machtey and P. Young. Introduction to the General Theory of Algorithms. North-Holland, New York, 1978. [Macintyre, 1973] A. Macintyre. Model completeness for sheaves of structures, Fund. Math., 81, 73-89, 1973. [Maurin, 1996] F. Maurin. Exact complexity bounds for ordinal addition, Theoret. Comp. Sci., 165, 247-273, 1996. [Maurin, 1997a] F. Maurin. Ehrenfeucht games and ordinal addition, Ann. Pure Appl. Logic, 89, 53-73, 1997. [Maurin, 1997b] F. Maurin. The theory of integer multiplication with order restricted to primes is decidable, J. Symbolic Logic, 62, 123-130, 1997. [McNaughton and Papert, 1971] R. McNaughton and S. Papert. Counter Free Automata. MIT Press, Cambridge, MA, 1971. [Meyer, 1974] A. Meyer. The inherent computational complexity of theories of ordered sets. In Proc. 1974 Intl. Cong, of Mathematicians, Vancouver, BC, pp. 477-482, 1974. [Meyer, 1975] A. Meyer. Weak monadic second order theory of successor is not elementary recursive. In Logic Colloquium (Boston 1972-73), R. Parikh, ed., Lecture Notes in Math. 453, pp. 132-154. Springer-Verlag, Berlin, 1975.
214
Kevin J. Compton and C. Ward Henson
[Michel, 1981] P. Michel. Borne superieure de la complexite de la theorie de N muni de la relation de divisibilite. In Model Theory and Arithmetic (Paris, 1979-1980), C. Berline, M. McAloon, and J.-P. Ressayre, eds., Lecture Notes in Math. 890, pp. 242-250. Springer-Verlag, Berlin, 1981. [Michel, 1992] P. Michel. Complexity of logical theories involving coprimality, Theoret. Comp. Sci., 106, 221-241, 1992. [Moschovakis, 1974] Y. Moschovakis. Elementary Induction on Abstract Structures. North-Holland, Amsterdam, 1974. [Point, 1986] F. Point. Problemes de decidabilite pour les theories de modules, Bull. Soc. Math. Belg. Ser. B, 38, 58-74, 1986. [Point, 1993] F. Point. Decidability questions for theories of modules. In Logic Colloquium '90 (Helsinki, 1990), J. M. R. Oikkonen and J. Vaananen, eds., Lecture Notes in Logic 2, pp. 266-280. Springer, Berlin, 1993. [Point and Prest, 1993] F. Point and M. Prest. Decidability for theories of modules, J. London Math. Soc. (2), 38, 193-206, 1988. [Prest, 1988a] M. Prest. Model theory and representation type of algebras. In Logic Colloquium '86 (Hull, 1986), pp. 219-260. North-Holland, 1988. [Prest, 1988b] M. Prest. Model Theory and Modules, Cambridge University Press, Cambridge, 1988. [Rabin, 1965] M. Rabin. A simple method for undecidability proofs and some applications. In Proc. 1964 Intl. Cong. Logic, Methodology and Philosophy of Science, pp. 58-68. North-Holland, Amsterdam, 1964. [Rabin, 1969] M. Rabin. Decidability of second order theories and automata on infinite trees, Trans. Amer. Math. Soc., 141, 1-35, 1969. [Rabinovich, 1998] A. Rabinovich, Non-elementary lower bound for prepositional duration calculus, Inform. Process. Lett., 66, 7-11, 1998. [Rackoff, 1975a] C. Rackoff. The computational complexity of some logical theories, Doctoral thesis, Massachusetts Institute of Technology, Cambridge, MA, 1975. [Rackoff, 1975b] C. Rackoff. The complexity of theories of the monadic predicate calculus, Research Report no. 136, INRIA, Roquencourt, France, 1975. [Rackoff, 1976] C. Rackoff. On the complexity of the theories of weak direct powers, J. Symbolic Logic, 41, 561-573, 1976. [Ramakrishna et al, 1992] Y. Ramakrishna, L. Dillon, L. Moser, P. Melliar-Smith, and G. Kutty. An automata-theoretic decision procedure for future interval logic. In Foundations of Software Technology and Theoretical Computer Science, R. Shyamasundar, ed., Lecture Notes in Comput. Sci. 652, pp. 51-67. Springer-Verlag, Berlin, 1992. [Ramakrishna et al., 1996] Y. Ramakrishna, P. Melliar-Smith, L. Moser, L. Dillon, and G. Kutty. Interval logics and their decision procedures. I.
Proving lower bounds
215
An interval logic, Theoret. Comput. Sci., 166, 1-47. 1996. [Reddy and Loveland, 1978] C. R. Reddy and D. Loveland. Presburger arithmetic with bounded quantifier alternation. In Proc. 10th Ann. ACM Symp. on Theory of Computing, pp. 320-325. ACM, New York, 1978. [Robertson, 1974] E. L. Robertson. Structure of complexity in the weak monadic second-order theories of the natural numbers. In Proc. 6th Ann. ACM Symp. on Theory of Computing, pp. 161-171. ACM, New York, 1974. [Robinson, 1959a] A. Robinson. On the concept of a differentially closed field, Bull. Res. Council Israel, 8F, 113-128, 1959. [Robinson, 1959b] A. Robinson. Solution of a problem of Tarski, Fund. Math., 47, 179-204, 1959. [Scarpellini, 1984] B. Scarpellini. Complexity of subcases of Presburger arithmetic, Trans. Amer. Math. Soc., 284, 203-218, 1984. [Schmitt, 1982] P. Schmitt. The elementary theory of torsion free Abelian groups with a predicate specifying a subgroup, Z. Math. Logik Grundlagen Math., 28, 323-329, 1982. [Schoning, 1997] U. Schoning. Complexity of Presburger arithmetic with fixed quantifier dimension, Theory Comput. Syst., 30, 423-428, 1997. [Seiferas et al., 1978] J. Seiferas, M. Fischer, and A. Meyer. Separating nondeterministic time complexity classes, J. Assoc. Comput. Much., 25, 146-167, 1978. [Semenov, 1980] A. L. Semenov. An interpretation of free algebras in free groups, Soviet Math. Dokl, 21, 952-955, 1980. (English translation.) [Semenov, 1984] A. L. Semenov. Logical theories of one-place functions on the set of natural numbers, Math. USSR-Izv., 22, 587-618, 1984. (English translation.) [Slobodskoi and Fridman, 1976] A. M. Slobodskoi and E. I. Fridman. Theories of abelian groups with predicates specifying a subgroup, Algebra and Logic, 14, 353-355, 1976. (English translation.) [Stern, 1988] J. Stern. Equivalence relations on lattices and the complexity of the theory of permutations which commute. In Methods and Applications of Mathematical Logic (Campinas, 1985), W. Carnielli and L. de Alcantara, eds., pp. 229-240. American Mathematical Society, Providence, RI, 1988. [Statman, 1979] R. Statman. The typed -calculus is not elementary recursive, Theoret. Comput. Sci., 9, 73-81, 1979. [Stirling, 1992] C. Stirling. Modal and temporal logics. In Handbook of Logic in Computer Science, Vol. 2, pp. 477-563. Oxford University Press, Oxford, 1992. [Stockmeyer, 1974] L. Stockmeyer. The complexity of decision problems in
216
Kevin J. Compton and C. Ward Henson
automata and logic, Doctoral thesis, Massachusetts Institute of Technology, Cambridge, MA, 1974. [Stockmeyer, 1977] L. Stockmeyer. The polynomial-time hierarchy, Theoret Comp. Sci., 3, 1-22, 1977. [Stockmeyer, 1987] L. Stockmeyer. Classifying the computational complexity of problems, J. Symbolic Logic , 52, 1-43, 1987. [Tetruashvili, 1984] M. Tetruashvili. The computational complexity of the theory of abelian groups with a given number of generators. In Frege Conference, 1984 (Schwerin, 1984), PP- 371-375. Akademie-Verlag, Berlin, 1984. [Touraille, 1985] A. Touraille. Elimination des quantificateurs dans la theorie elementaire des algebres de Boole munies d'une famille d'ideaux distingues, C. R. Acad. Sci. Paris, Ser. I, 300, 125-128, 1985. [Trakhtenbrot, 1950] B. A. Trakhtenbrot. The impossibility of an algorithm for the decision problem for finite models, Dokl. Akad. Nauk SSSR, 70, 569-572, 1950. [Turing, 1937] A. M. Turing. On computable numbers with an application to the Entscheidungsproblem, Proc. London Math. Soc. (2), 42, 230-265, 1937. Correction, ibid., 43, 544-546, 1937. [Vardi, 1997] M. Vardi. Why is modal logic so robustly decidable? In Descriptive Complexity and Finite Models (Princeton, NJ), pp. 149-183. Amer. Math. Soc., Providence, RI, 1997. [Vaught, 1960] R. Vaught. Sentences true in all constructive models, J. Symbolic Logic, 25, 39-58, 1960. [Vaught, 1962] R. Vaught. On a theorem of Cobham concerning undecidable theories. In Proc. 1960 Intl. Cong. Logic, Phil, and Methodology of Sci., pp. 14-25. Stanford University Press, Stanford, CA, 1962. [Volger, 1985] H. Volger. Turing machines with linear alternation, theories of bounded concatenation and the decision problem of first-order theories, Theoret. Comp. Sci., 23, 333-337, 1983. [Vorobyov, 1997] S. Vorobyov. The 'hardest' natural decidable theory. In Proc. 12thd Ann. IEEE Conf. on Logic in Computer Science, pp. 294305. IEEE Computer Society Press, 1988. [Vorobyov, preprint] S. Vorobyov. The most nonelementary theory, preprint. [Wood, 1976] C. Wood. The model theory of differential fields revisited, Israel J. Math., 25, 331-352, 1976. [Young, 1985] P. Young. Godel theorems, exponential difficulty and undecidability of arithmetic theories: an exposition. In Recursion Theory, A. Nerode and R. Shore, eds., Proc. Symp. Pure Math. 42. American Mathematic Society, Providence, RI, 1985.
Algebraic specification of abstract data types J. Loeckx, H.-D. Ehrich and M. Wolf
Contents 1 2
3
4
5
6
7
Introduction Algebras 2.1 The basic notions 2.2 Homomorphisms and isomorphisms 2.3 Abstract data types 2.4 Subalgebras 2.5 Quotient algebras Terms 3.1 Syntax 3.2 Semantics 3.3 Substitutions 3.4 Properties Generated algebras, term algebras 4.1 Generated algebras 4.2 Freely generated algebras 4.3 Term algebras 4.4 Quotient term algebras Algebras for different signatures 5.1 Signature morphisms 5.2 Reducts 5.3 Extensions Logic 6.1 Definition 6.2 Equational logic 6.3 Conditional equational logic 6.4 Predicate logic Models and logical consequences 7.1 Models 7.2 Logical consequence 7.3 Theories 7.4 Closures
219 220 220 223 224 225 225 228 228 229 230 230 231 231 234 235 236 236 236 238 239 240 240 241 242 242 244 244 245 246 247
218
8
9 10
11
12 13
14
15
J. Loeckx, H.-D. Ehrich and M. Wolf 7.5 Reducts 7.6 Extensions Calculi 8.1 Definitions 8.2 An example 8.3 Comments Specification Loose specifications 10.1 Genuinely loose specifications 10.2 Loose specifications with constructors 10.3 Loose specifications with free constructors Initial specifications 11.1 Initial specifications in equational logic 11.2 Examples 11.3 Properties 11.4 Expressive power of initial specifications 11.5 Proofs 11.6 Term rewriting systems and proofs 11.7 Rapid prototyping 11.8 Initial specifications in conditional equational logic 11.9 Comments Constructive specifications Specification languages 13.1 A simple specification language 13.2 Two further language constructs 13.3 Adding an environment 13.4 Flattening 13.5 Properties and proofs 13.6 Rapid prototyping 13.7 Further language constructs 13.8 Alternative semantics description Modularization and parameterization 14.1 Modularized abstract data types 14.2 Atomic module specifications 14.3 A modularized specification language 14.4 A parameterized specification language 14.5 Comments 14.6 Alternative parameter mechanisms Further topics 15.1 Behavioural abstraction 15.2 Implementation 15.3 Ordered sorts 15.4 Exceptions
249 249 250 250 251 252 253 254 254 256 257 258 258 259 261 261 262 264 266 267 267 268 271 272 275 279 282 283 283 283 284 285 285 286 288 291 295 296 298 298 300 302 303
16
15.5 15.6 15.7 The 16.1 16.2
Dynamic data types Objects Bibliographic notes categorical approach Categories Institutions
305 307 309 310 310 310
1 Introduction It is widely accepted that the quality of software can be improved if its design is systematically based on the principles of modularization and formalization. Modularization consists in replacing a problem by several "smaller" ones. Formalization consists in using a formal language; it obliges the software designer to be precise and principally allows a mechanical treatment. One may distinguish two modularization techniques for the software design. The first technique consists in a modularization on the basis of the control structures. It is used in classical programming languages where it leads to the notion of a procedure. Moreover, it is used in "imperative" specification languages such as VDM [Woodman and Heal, 1993; Andrews and Ince, 1991], Raise [Raise Development Group, 1995], Z [Spivey, 1989] and B [Abrial, 1996]. The second technique consists in a modularization on the basis of the data structures. While modern programming languages such as Ada [Barstow, 1983] and ML [Paulson, 1991] provide facilities for this modularization technique, its systematic use leads to the notion of abstract data types. This technique is particularly interesting in the design of software for non-numerical problems. Compared with the first technique it is more abstract in the sense that algebras are more abstract than algorithms; in fact, control structures are related to algorithms whereas data structures are related to algebras. Formalization leads to the use of logic. The logics used are generally variants of the equational logic or of the first-order predicate logic. The present chapter is concerned with the specification of abstract data types. The theory of abstract data type specification is not trivial, essentially because the objects considered — viz. algebras — have a more complex structure than, say, integers. For more clarity the present chapter treats algebras, logics, specification methods ("specification-in-the-small"), specification languages ("specification-in-the-large") and parameterization separately. In order to be accessible to a large number of readers it makes use of set-theoretical notions only. This contrasts with a large number of publications on the subject that make use of category theory [Ehrig and Mahr, 1985; Ehrich et a/., 1989; Sannella and Tarlecki, 2001]. Our attention is restricted to those topics that are now standard. Proofs of theorems are omitted. For more details the reader is referred to the literature and, in particular, to the textbook [Loeckx et al., 1996]. This
220
J. Loeckx, H.-D. Ehrich and M. Wolf
textbook treats the subject along the same lines as the present chapter, uses the same notation and contains the proofs of most of the theorems. A book treating more advanced issues is [Sannella and Tarlecki, 2001]. A comprehensive state-of-the-art report on recent advances in algebraic foundations of systems specification may be found in [Astesiano et al., 1999b]. A broad survey of the topic together with an annotated bibliography is in [Cerioli et al., 1997]. Sections 2 to 8 describe the fundamental specification tools. More precisely, sections 2 to 5 are devoted to many-sorted algebras and sections 6 to 8 to logic. Section 9 introduces the general notion of a specification. Sections 10 to 12 present three specification methods for specification-inthe-small: loose specifications, initial specifications and constructive specifications. Section 13 presents a simple prototypical specification language for specification-in-the-large and discusses in some detail the language constructs. Section 14 shows how specification languages may be generalized for modularization and parameterization. Section 15 shortly discusses some further issues. Finally, section 16 briefly indicates how the notions of a category and of an institution may be used in the study of abstract data type specifications.
2
Algebras
The algebras to be discussed here are essentially the universal algebras discussed by Meinke and Tucker in [Tucker and Meinke, 1992]. As a difference the present section only treats those aspects that play a major role in specification.
2.1
The basic notions
The first notion introduced is that of a signature. Informally, a signature constitutes the syntax of an algebra by fixing the names and the arities of the operations. Definition 2.1 (Signature). A signature is a pair (S, ) of sets, the elements of which are called sorts and operations, respectively. Each operation consists of a (k + 2)-tuple with n is called the operation name of the operation and its arity; the sorts s1,..., sk are called the argument sorts of the operation and the sort s its target sort. If k = 0 the operation n: —)• s is called a constant of sort s. Note that different operations may have the same operation name. Actually, the equality of two operations implies the equality of their names and the equality of their arities.
Algebraic specification of abstract data types
221
The preceding definition differs from Definition 3.1.1 in [Tucker and Meinke, 1992] essentially by explicitly distinguishing between an operation and its name. This avoids difficulties when several operations have the same name. Example 2.2. The following signature (S, ) is intended for an algebra with Boolean values and natural numbers:
Signatures may be represented graphically. In this representation sorts correspond to labelled nodes and operations to edges pointing from the argument sorts to the target sort. A graphical representation of the signature of Example 2.2 is in Figure 1. While such a representation is easily understandable it may be ambiguous in that it does not fix the order of the argument sorts. For instance, the graphical representations of operations such as n : s x t —> u and n : t x s — u coincide. True
False Succ
Fig. 1. A graphical representation of the signature of Example 2.2. Informally, an algebra provides a meaning to a signature. Definition 2.3 (Algebra). Let E = (S, n) be a signature. An algebra for this signature (or a E -algebra) assigns: • a set A(s) to each sort s € S, called the carrier set of the sort s; the
222
J. Loeckx, H.-D. Ehrich and M. Wolf elements of a carrier set are called carriers; • a total function to each operation. It is understood that A(n: - s) denotes an element of the carrier set A(s).
The class of all E-algebras is denoted Alg(E). Whenever no ambiguity arises one may write A(n) instead of A(n: S1 x . . . x sk —> s). Note that many publications — including [Tucker and Meinke, 1992] — use the notation As and nA instead of A(s) and A(n), respectively. By the way, Alg(E) is in general a class and not merely a set (see [Loeckx et al, 1996] for a proof). Example 2.4. Let E = (S, ) be the signature of Example 2.2. (i) The "classical" algebra for E is the algebra A defined by:
and similarly for the other operations. Note that, for instance, stands for (ii) The following "fancy" algebra B is also a E-algebra:
etc.
While the last example may suggest that a signature allows any algebra provided its functions respect the arities, the following example illustrates the contrary. Example 2.5. Let E = (S, ) be the signature with For any E-algebra A the carrier sets A(s) and not empty. In fact, Further examples of algebras may be found in [Tucker and Meinke, 1992]. It is of course possible to generalize the definition of an algebra by allowing the functions associated with an operation to be partial. Unfortunately, this generalization leads to serious problems with homomorphisms
Algebraic specification of abstract data types
223
(see section 2.2) as well as with the logic (see section 6) and is therefore not further considered.
2.2
Homomorphisms and isomorphisms
Informally, homomorphisms are mappings between algebras or, more precisely, mappings between their carrier sets that "respect" their functions. The following definition is "classical" . Examples of the different notions and the proofs of the theorems may be found in [Tucker and Meinke, 1992] or any textbook on universal algebra. Definition 2.6 (Homomorphism). Let A, B be two E-algebras, E = homomorphism h : A — > B from A to B is a, family (h s )sES of functions such that for any operation the following holds:
say
Equation (2.1) is called the homomorphism condition of the operation u for the homomorphism h. Theorem 2.7. The composition of two homomorphisms yields a homomorphism. More precisely, if E = (S, ) is a signature and if h : A — >• B, g : B — C are two E -homomorphisms then g oh = (gs°hs)ses constitutes a E -homomorphism too. Definition 2.8 (Homomorphic image). The homomorphic image of the E-algebra A under the homomorphism h : A — > B is the E-algebra h(A) defined by:
denotes the restriction of the function B(w) to the set h(A)(s1) x .. . x Definition 2.9 (Isomorphism).
(i) A (E)-isomorphism is a bijective S-homomorphism. (ii) Let E be a signature. Two E-algebras A and B are called isomorphic if there exists a E-isomorphism from A to B. In that case one writes A~B.
Informally, isomorphic algebras are "abstractly" identical in that they differ only by the nature of their carriers. This property is illustrated by
224
J. Loeckx, H.-D. Ehrich and M. Wolf
the following two theorems, the proofs of which are straightforward. Theorem 2.10. The relation of isomorphism between algebras is an equivalence relation. Theorem 2.11. Let A, B, C be algebras of the same signature. If A ~ B then the following facts hold: (i) whenever there exists a homomorphism from B to C, then there exists a homomorphism from A to C; (ii) whenever there exists a homomorphism from C to B, then there exists a homomorphism from C to A. In the following theorem 1A : A —> A and 1B : B —> B stand for the identity homomorphisms in A and B, respectively. Theorem 2.12. A homomorphism h : A —> B is an isomorphism, if and only if there exists a homomorphism g : B —» A such that h o g = 1B and g°h = 1A. The following definition plays an important role in section 11. Definition 2.13 (Initial algebra, final algebra). Let C C Alg(E) be a class of E-algebras for a signature E. (i) An algebra A 6 C is called initial in the class C, if for each B E C there exists exactly one homomorphism from A to B. (ii) An algebra A £ C is called final in the class C, if for each B E C there exists exactly one homomorphism from B to A. Examples may be found in section 4.2 and in [Tucker and Meinke, 1992, Example 5.1.15]. Theorem 2.14. Let C C Alg(E.) be a class of"E-algebras and let A,B € C. (i) Assume A is initial in C. Then B is initial in C if and only if A ~ B. (ii) As (i) with "final" instead of "initial".
2.3
Abstract data types
Informally, an abstract data type is a class of algebras closed under isomorphism (cf. [Tucker and Meinke, 1992]). This definition fits the study of specification well because the logics to be used cannot distinguish between isomorphic algebras. Definition 2.15 (Abstract data type). An abstract data type for a signature E is a class C C Alg(E) satisfying the condition: for any pair of E-algebras A, B. An abstract data type is called monomorphic if all its algebras are isomorphic to each other; otherwise it is called polymorphic.
Algebraic specification of abstract data types
225
2.4 Subalgebras Informally, a subalgebra is an algebra with some carriers deleted and with the functions restricted accordingly. Definition 2.16 (Subalgebra). Let E = (S, ) be a signature and let A and B be E-algebras. The algebra B is called a subalgebra of A if the following two conditions hold:
Examples may be found in [Tucker and Meinke, 1992]. Fact 2.17. Let C be a class of E-algebras, i.e. C C Alg(E). The relation "is a subalgebra of" constitutes a partial order on C. As a result it makes sense to speak, for instance, of a minimal algebra of a class. Subalgebras may be defined by predicates as is now shown. Definition 2.18 (Induced subalgebra). Let E = ( S , ) be a signature, A a E-algebra and P = ( P s ) s e s a family of predicates Ps on A(s), s E S. The subalgebra induced on A by P is the E-algebra B defined by:
To be consistent the definition requires that the following closure condition holds:
A proof of the following theorem may be found in [Tucker and Meinke, 1992]. Theorem 2.19. Let A and B be E-algebras and let h : A — > B be a homomorphism. The homomorphic image h(A) is a subalgebra of B.
2.5
Quotient algebras
Informally, a quotient algebra is an algebra in which some carriers are identified with the help of a "congruence relation" . A congruence relation is an equivalence relation which is "compatible" with the functions of an algebra in that equivalent arguments lead to equivalent function values. Definition 2.20 (Congruence relation). Let E = (S, ) be a signature, A a E-algebra and Q = (Qs)ses a family of equivalence relations Qs on A(s), s E S. If Q satisfies the following substitutivity condition:
226
J. Loeckx, H.-D. Ehrich and M. Wolf
then it is called a congruence relation on A. Definition 2.21 (Quotient algebra). Let E = (S, ) be a signature, A a E-algebra and Q a congruence relation on A. The quotient algebra of A by Q is the E-algebra A/Q defined by:
where, for instance, denotes the equivalence class of a with respect to the equivalence relation Qs. Building a quotient algebra is called factorization. It is not evident that this definition is consistent. In fact, the value of the right-hand side of the second equality in principle depends on the choice of the representative ai for the equivalence class [ai] on the left-hand side. The consistency proof as well as examples of quotient algebras may be found in [Tucker and Meinke, 1992]. The following two definitions and the subsequent theorem show that there exists a one-to-one correspondence between homomorphisms and quotients. Definition 2.22 (Induced congruence relation). Let E = (S, ) be a signature, let A and B be E-algebras and let h : A — > B be a homomorphism. Furthermore, be the family of equivalence relations on A defined by: for all induced by the homomorphism h.
is called the congruence relation
Definition 2.23 (Quotient homomorphism). Let E = (S, ) be a signature, A a E-algebra and Q a congruence relation on A. The family homQ : A — A/Q of mappings defined by: for all a E A(s), s € S, is called the quotient homomorphism of the congruence relation Q. Of course, it has to be proved that the relation of Definition 2.22 is indeed a congruence relation and that the family of functions of Definition 2.23 indeed constitutes a homomorphism. The following fact expresses the "one-to-one correspondence" announced.
Algebraic specification of abstract data types
227
Fact 2.24. With the notation of Definitions 2. 22 and 2.23 one has: = Q. The following theorem is called the first homomorphism theorem [Tucker and Meinke, 1992] and plays an important role in section 11. Theorem 2.25. Let E be a signature and let A and B be two 'E-algebras. Furthermore, let h : A — » B be a homomorphism and the congruence relation induced by h. If the homomorphism h is surjective, then
3
Terms
If an algebra is viewed as a functional programming language, a program in this language consists of a term. This explains why terms play a prominent role in the specification of abstract data types.
3.1
Syntax
First the notion of a variable has to be made precise. A family V = ( V s ) s e s of pairwise disjoint infinite sets is assumed to be associated with any given signature E = (S, ). An element of Vs is called a variable of sort s, s E S. A family X with X C V is called — by abuse of language — a set of variables for the signature £. It is required that the variables of V and the operation names of have different denotations. Informally, V constitutes a "universe" of variables whereas the variables effectively used in a particular application are those of X. The existence of the universe V makes it possible to add "at any time" "new" variables (i.e. variables of V - X) to X — provided V - X is infinite. Definition 3.1 (Syntax of terms). Let E = (S, ) be a signature and X a set of variables for it. The sets constituting the family are defined by simultaneous induction:
An element of A
is called a E(X)-term of sort s or, shortly, a term.
term without variables is called a ground term. One writes
As different operations may have the same operation name, a term may have several sorts. A trivial example is a signature containing the constants n:—> s, n:—t s'; in that case the term n has two sorts, viz. s and s' . In practice terms with different sorts are a nuisance because they are a potential cause of "programming errors". In fact, the introduction of
228
J. Loeckx, H.-D. Ehrich and M. Wolf
typing was a milestone in the development of programming languages such as Algol and Pascal. This consideration justifies the following definition. Definition 3.2 (Strongly typed signature). A signature E is strongly typed if for any set X of variables for E each E(X)-term has a single sort. Clearly, a signature in which all operations have different names is strongly typed. The following theorem provides a less restrictive criterion. Theorem 3.3. Let E = (S, ) be a signature. If for any two different operations following condition holds: then the signature E is strongly typed. Informally, the theorem states that operations with the same name must have a different number of arguments or differ in the sorts of at least one argument. Of course, this still excludes signatures with, for instance, constants n: -> s and n: -» s' — as does a programming language such as Pascal that uses the constants 1 and 1.0 to distinguish between the sorts integer and real. From now onwards all signatures will be assumed to be strongly typed. Infix and mixfix notation allow us to write, for instance, t1 < t2 and if b then t1 else t2 fi instead of < (t1,t2) and ifthenelse(b,t1,t2). Priorities allow us to write, for instance, t1+t2*t3 instead of (t1 + (t2*t3)). While the formal parts of this chapter will stick to Definition 3.1, examples will make use of these notational facilities. For the case of infix and mixfix notation the position of the arguments is classically indicated by underscores in the signature; for instance:
3.2
Semantics
Variables are given a semantics through "assignments". Definition 3.4 (Assignment). Let E — (5, ) be a signature, X a corresponding set of variables and A a E-algebra. An assignment of X for A is a family of (total) functions as • Xs —>• A(s). One writes a : X —> A. The following definition is classical. Definition 3.5 (Semantics of terms). Let E = (5, ) be a signature, X a set of variables for a term, A a E-algebra and a : X —» A an assignment. The value A(a)(t) of t for a and A is defined by induction on the structure of t:
Algebraic specification of abstract data types
229
When t is a ground term the value of t does not depend on a and one may write A(t) instead of A(a)(t).
3.3
Substitutions
Informally, a substitution is a syntactical operation replacing variables by terms. Definition 3.6 (Substitution). Let £ = (5, ) be a signature and X, Y two (not necessarily distinct) sets of variables for S. A family a = (o s )ses of functions is called a substitution a : X — > TE(Y)- The application of this substitution to a term t € TE(x) yields a term ta € ^TE(Y) inductively defined by:
Where a is the identity except for a finite number of variables, say x1, . . . , xm , it is common practice to write: instead of a. If Y is empty one speaks of a ground substitution. Example 3. 7. Consider the signature of Example 2.2. Let X be defined by
Then:
3.4
Properties
The following important theorem relates terms to homomorphisms. Theorem 3.8. Let A and B be two algebras for the same signature E and let h : A — > B be a E-homomorphism. Then the following facts hold:
230
J. Loeckx, H.-D. Ehrich and M. Wolf
The next theorem states that a term keeps its value in a subalgebra. Theorem 3.9. Let A and B be E-algebras for some signature £ = (5, ). // B is a subalgebra of A, then:
The following theorem states that in a quotient algebra the value of a term is replaced by the equivalence class of its value in the original algebra. Theorem 3.10. Let A be a E-algebra for some signature £ and let Q be a congruence relation on A. Then:
where B : X — > A/Q is the assignment defined by
The substitution theorem expresses the semantical effect of a substitution. Theorem 3.11 (Substitution theorem). Let X, Y be sets of variables for a signature E = (S, ) and let a : X — > TE(y) be a substitution. Finally, let A be a E-algebra and B : Y — A an assignment of Y for A. Then for each by
A is the assignment of X for A defined
4 Generated algebras, term algebras 4.1 Generated algebras A carrier of an algebra can be accessed by a user only through a ground term. A carrier that fails to be the value of a ground term is therefore called junk. Generated algebras capture the notion of an algebra without junk. As an additional important feature, generated algebras allow proofs by induction.
Algebraic specification of abstract data types
231
Definition 4.1 (Generated algebra). Let £ = (S,) be a signature, A a E-algebra and a set of operations called constructors. The algebra A is said to be: (i) generated by the set c (or sufficiently complete with respect to c), if for each carrier of A, say a € >A(s), s E S, there exists at least one ground term with a = A(t); (ii) generated (or reachable), if it is generated by The class of all generated E-algebras is denoted Gen(E). Example 4.2. The algebra A of Example 2.4 is generated by {True, False, 0,Succ}. The algebra C identical with A except for C(nat) = Z (= the set of integers) is not generated because no ground term has a negative value. As a generated algebra contains no junk, it is "minimal". This is made precise by the following theorem. Theorem 4.3. A E-algebra is generated if and only if it is minimal in Alg(E) with respect to the partial order "is a subalgebra of". While the above theorem is mainly of theoretical interest, the following properties play an important role in the theory of specification. Theorem 4.4. Let h : A —t B be a *£-homomorphism for some signature E. (i) If B is a generated algebra, then h is surjective. (ii) If A is a generated algebra, then h is the only homomorphism from A to B. Theorem 4.5. Let A and B be E-algebras for some signature £. (i) If A is generated and A ~ B, then B is also generated, (ii) If A and B are generated and there exist a homomorphism from A to B and a homomorphism from B to A, then A ~ B. Theorem 4.6. Let A, B be generated E,-algebras for some signature ( S , ) such that B(s) C TE,5 for each s € 5. If (i) there exists a homomorphism h : A —> B and (ii) A(B(t)) = A(t) for all ground terms t e TE, then A ~ B. The proofs of these different theorems are not very difficult (see,for example, [Loeckx et al., 1996]). During specification one may leave the definition of some sorts pending. Of course, the algebras thus specified fail to be generated because of the "pending" sorts. This suggests a generalization of the concept of a generated algebra which consists in disregarding some sorts. While this idea sounds simple, its realization is less trivial; the reason lies in the fact
232
J. Loeckx, H.-D. Ehrich and M. Wolf
that the syntax of terms has been defined by simultaneous induction on all sorts. Definition 4.7. Let E = (S, ) be a signature, A a E-algebra and Sc C S a set of sorts. The following notation is introduced:
where each ca: -> s is a "new" constant; (ii) Ac is the EcA-algebra defined by:
Informally, contains additional constants but Ac "remains unchanged" . Definition 4.1 may now be generalized. Definition 4.8 (Algebras generated in some sorts). Let E = (S, ), and Ac be as in Definition 4.7. Finally, let be a set of operations with their target sort in Sc . The E-algebra A is said to be: • generated in the sorts of Sc by the set c of constructors if the -algebra Ac is generated by the set • generated in the sorts of Sc if it is generated in the sorts of Sc by the operations of with their target sort in ScExample 4.9. Let E = (S, ) be the signature with
and A a E-algebra with A(el) = a not further defined set of "elements" , A(list) = the set of finite lists of elements, A([]) = the empty list, and where A(Add) adds an element in front of the list.
Moreover, Ac is defined like A with additionally Clearly, Ac is generated by For instance, the list consisting of a single element, say e, is represented by the term Add(ce, []). Hence, the E-algebra A is generated in the sort list by {[], Add}.
Algebraic specification of abstract data types
233
Note that A is not generated in the sense of Definition 4.1, because in the signature E even a list consisting of a single element is not representable by a ground term. It should be clear that a generated algebra (Definition 4.1) is simply an algebra generated in all sorts (Definition 4.8). When an algebra is generated in some sorts, properties of the carrier sets of these sorts may be proved by induction. For instance, to prove that a property P holds for all carriers of the carrier set A(list) of Example 4.9, it is sufficient to prove:
4.2
Freely generated algebras
Definition 4.10 (Freely generated algebra). A freely generated algebra is defined as a generated algebra in Definition 4.1 but with the condition "there exists at least one ground term" replaced by "there exists exactly one ground term". Example 4.11. The algebra A of Example 2.4 is freely generated by
The following theorem is relatively easy to prove: Theorem 4.12. A freely generated E-algebra is initial in the class Alg(E) of all E,-algebras. As a direct consequence freely generated algebras are isomorphic. Moreover, there exists exactly one homomorphism from a freely generated Ealgebra to any other S-algebra. By the way, Theorem 4.14 will show that there effectively exists a freely generated algebra for each signature. The definition of an algebra freely generated in some sorts may be deduced from Definition 4.8 in a similar way to the above. The algebra A of Example 4.9 is freely generated by {[], Add} in the sort list. While generated algebras allow inductive proofs, freely generated algebras also allow inductive definitions of functions. For instance, to define a function / : A(s) —> ... it is sufficient to define the value f ( A ( t ) ) for all ground terms t of sort s. The reason is that (syntactically) different ground terms have (semantically) different values; this guarantees that the function definition is consistent because the function value for given arguments is defined exactly once (see, for example, [Loeckx and Sieber, 1987] for more details).
234
4.3
J. Loeckx, H.-D. Ehrich and M. Wolf
Term algebras
With each signature is associated an algebra called a "term algebra" , the carriers of which consist of ground terms. Definition 4.13 (Term algebra). The term algebra for the signature
At first sight this definition may be confusing. The reason is that ground terms now play a dual role: as elements of the set TE they are syntactical objects; as carriers of the algebra T(E) they are semantical objects. Actually, this dual role is what makes term algebras so interesting. Theorem 4.14. A term algebra is freely generated. By Theorem 4.12 there exists a unique homomorphism h : T(E) — > A for any E-algebra A. This homomorphism is called the evaluation homomorphism of A. Note that h(t) = A(t) for any ground term t.
Example 4.15.
(ii) Let A be the classical E-algebra with A(nat) = N
tion homomorphism h is defined by: Note that in this case the evaluation homomorphism is an isomorphism. The main property of term algebras is the following. Theorem 4.16. Let A be a generated E-algebra for some signature E and let h : T(E) — > A be its evaluation homomorphism. Then (where =h is the congruence relation induced by h according to Definition 2.22). Example 4.17. Let E be as in Example 4.15 but with the additional operation: The set contains additional carriers such as 0 + S'ucc(O) and Succ(0 + 0) + 0. Let A again be the "classical" algebra, with A(+) being addition, and let h be its evaluation homomorphism. Then the carrier
Algebraic specification of abstract data types
4.4
235
Quotient term algebras
Informally, a quotient term algebra is a quotient of the term algebra that possesses the properties common to a given class of algebras. Such an algebra may be viewed as a generalization of the algebra T(E)/=h introduced above. Quotient term algebras play a major role in section 11. Definition 4.18 (Quotient term algebra). Let C C Alg(E} be a nonempty class of E-algebras for some signature E = (S, ). The quotient term algebra of C is the E-algebra T(E,C) defined by:
This definition is consistent because the relation =c may be proved to be a congruence relation on T(E). The following theorem shows the interest of quotient term algebras. Theorem 4.19. Let C C Alg(E,) be a class of E-algebras for some signature S. For each algebra A £ C there exists a unique homomorphism from T(E,C) to A. Hence, T(E, C) has properties similar to those of an initial algebra. It is indeed an initial algebra if it belongs to C. More precisely: Corollary 4.20. LetC C Alg(E) be a class of E-algebras. Jf T(E,C) E C, then it is initial in C.
5
Algebras for different signatures
The study of algebras above was for a given, fixed signature. The relation between algebras with different signatures is investigated in the present section.
5.1
Signature morphisms
Informally, a signature morphism is a mapping of signatures that respects the arity of the operations. Definition 5.1 (Signature morphism). Let E = (S, ) and E' =
(5',
) be two signatures.
236
J. Loeckx, H.-D. Ehrich and M. Wolf such that, for each operation exists an operation name m such that
(ii) A renaming is a bijective signature morphism, i.e. a signature morphism u = (us, u) with us and bijective. If no ambiguities arise one may write u instead of us and Example 5.2.
a possible signature morphism One then speaks of an inclusion and writes
A possible signature morphism
defined by
To extend signature morphisms to terms it is first necessary to extend them to variables. To simplify the definition it is customary to make the following assumption. Let u : E — > E ' be a signature morphism where the associated with E and E', respectively (see section 3.1). It is then assumed Definition 5.3 (Extension of signature morphisms to variables). Let u, : E —)• E' be a signature morphism with E = (50), E' = (S', '). If X is a set of variables for E, then u ( X ) is defined to be the following set of variables for E':
Informally, u ( X ) contains the same variables as X but the sort of a variable may be different. By the way, thanks to the above assumption u , ( X ) is indeed a set of variables for E'. In the absence of this assumption, Definition 5.3 would have had to additionally rename the variables.
Algebraic specification of abstract data types
237
Definition 5.4 (Extension of signature morphisms to terms). Let E — (S, ) and E' be signatures, X a set of variables for E and a signature morphism. For any term the term is defined inductively as follows:
Example 5.5. Let u be the signature morphism of Example 5.2(ii). Then:
Note that in the left-hand side of the equation the variables x and y are of sorts el1 and e/2, respectively; in the right-hand side they are both of sort nat.
5.2 Reducts Reducts constitute a semantical counterpart of (the syntactical notion of) signature morphisms. As a difference the "mapping" is from E'-algebras to E-algebras. Definition 5.6 (Reduct). Let u : E — E' be a signature morphism with E = (S, ) and let A' be a E'-algebra. The u,-reduct of A' is the E-algebra A' \ u defined by:
If u is an inclusion, one may write A'\ E instead of Informally, is a E-algebra with the semantics of A' . In the case of an inclusion the effect of the u-reduct is simply to "forget" the sorts and operations from E' — E. By the way, it is important not to confuse the notions of a subalgebra and of a reduct: a subalgebra is an algebra for the same signature whereas a (non-trivial) reduct is an algebra for a different signature. Example 5.7. Resuming Example 5.2(ii), let A' be the "classical" E'-algebra, i.e. A'(pairnat) = N0 x NO, A' (nat) = NO, A'(Pair)(m,n) = (m,n), A'(First)((m,n)) = m and A' (Second) ((m, n)) = n for all m, n € N0. The reduct A' \ u is defined by:
238
J. Loeckx, H.-D. Ehrich and M. Wolf
Definition 5.8 (Reduct of a homomorphism) . Let E, E' be signatures and a signature morphism. Let A', B' be E'-algebras and h' : A' — B' a E'-homomorphism. The u-reduct of h' is the E-homomorphism
The now following reduct theorem bears strong similarities with Theorem 3.11. Theorem 5.9 (Reduct theorem). Let E = (S, ), E' be signatures and signature morphism. Let X be a set of variables for E and t a term from TE(x). Finally, let A' be a E' -algebra and a' : u ( X ) — > A' be an assignment for A' . Then:
Note that i f t i s a ground term t h e theorem y
i
e
l
d
s
.
5.3 Extensions Contrasting with a reduct, an extension is a "mapping" in the same "direction" as a signature morphism, i.e. from Alg(E) to Alg(E'). Definition 5.10 (Extension). Let u : E — > E' be a signature morphism.
When u is an inclusion one speaks of an extension instead of a u-extension. Clearly, an algebra possesses in general several extensions. Note that the notion of a u-extension of a class is very "liberal" as it only requires that does not contain "additional" algebras. Note also that the term "extension" refers to the signature, not to the "size" of the class.
Algebraic specification of abstract data types
6
239
Logic
Most books on mathematical logic concentrate on first-order predicate logic. In contrast, specification makes use of a variety of logics such as equational logic, conditional equational logic, first-order predicate logic or modal logic. Moreover, depending on the application, the semantical domains may vary from the class of all algebras to the class of algebras generated in some sorts. Hence, it is appropriate to introduce a general notion of logic encompassing these different logics as particular cases. Such a general notion is introduced in the present section. A still more general notion called "institution" is based on categories and will be briefly discussed in section 16.1. The present section together with section 7 presents a satisfaction-based view of logic; section 8 will present a proof-based view of logic (cf. [Ryan and Sadler, 1992]).
6.1 Definition Definition 6.1 (Logic). An algebra logic (logic for short) L consists of: (i) a decidable set L(£) for each signature £; an element of L(E) is called a (E—)formula; (ii) a function uL '• L(E) — L(E') for each signature morphism u: E —>• E'; this function ul is called the formula morphism; (iii) a relation |=E C Alg(E) x L(E) for each signature E; this relation is called the satisfaction relation (for E); if A |= for some E-algebra A and some formula L(E), one says that is valid in A or that A satisfies It is required that the following two conditions are satisfied: (i) (Isomorphism condition.) For any signature £, for any formula E L(E) and for any E-algebras A, B with A ~ B, (ii) (Satisfaction condition.) For any signatures E, E', for any signature morphism u, : E —» £', for any formula € L ( E ) and for any E'-algebra.A', One may write u instead of uL, if no confusion arises. The following definition defines an important subclass of logics. Definition 6.2 (Logic with equality). A logic L is called a logic with equality if, for any signature E, any sort s of this signature and any ground terms t, u € T£,S, there exists a formula, say such that for all E-algebras A.
240
J. Loeckx, H.-D. Ehrich and M. Wolf
Sections 6.2 to 6.4 now present three different "instances" of this general logic. Each of these instances constitutes a logic with equality — as the reader may easily verify.
6.2
Equational logic
Definition 6.3 (Equational logic). According to Definition 6.1, the equational logic EL is defined as follows: • for each signature E,
A formula from EL(E) is also called an equation. An equation V0.t — u is called a ground equation. It is important not to confuse the symbol "=" of an equation with the operation name "=" of an operation such as As a fundamental difference the symbol "=" of an equation is a logical symbol (i.e. a "metasymbol") with a fixed semantics. One may write t = u instead of "X.t = u if each variable of X has at least one occurrence in t or u. Theorem 6.4. Equational logic satisfies the isomorphism and satisfaction conditions of Definition 6.1. Example 6.5. (i) Let E = (5, ) be the signature with 5 = {bool, nat} and Let, moreover, X be a set of variables for E and x,y € Xnat, b € XbooJExamples of S-equations are:
Note that, for instance, x + 1 = 1 + x stands for V{x;}.x + 1 = 1 + x. (ii) Let E = (S, ) be the signature with 5 = {s,t} and Let X be a set of variables for E and nally, let A be a E-algebra with A(s) = 0 and A(a) A(b). Clearly, A = a = b (which stands for A = V0.a = b) does not hold. On
Algebraic specification of abstract data types
241
the other hand A (= V{x}.a = b does hold because there exists no assignment a : {x} — > A due to the fact that A(s) = 0.
6.3
Conditional equational logic
This logic is a mild generalization of equational logic. Its interest stems from the fact that in the frame of specification it is easier to use than equational logic. Definition 6.6 (Conditional equational logic). The conditional equational logic is the logic CEL defined as follows:
The theorem, the remarks and the notational convention of Section 6.2 may be generalized for conditional equational logic. Example 6.7. Let E be the signature of Example 6.5(i). Examples of conditional equations are:
In the same vein a logic PL called predicate logic can be defined by generalizing classical first-order predicate logic for many-sorted algebras. For more transparency the set PL(E), the formula morphisms UPL and the satisfaction relation |=E are defined separately. Definition 6.8 (The set PL(E)). For each signature E the set PL(E) of formulas is defined inductively:
242
J. Loeckx, H.-D. Ehrich and M. Wolf
As usual it is possible to introduce additional logical symbols such as V, D, =, 3 as abbreviations, and to reduce the number of parentheses in formulas by assigning priorities to the logical symbols. Moreover, one may write instead of Vx: s.(p when the sort s of x is known. Definition 6.9 (The formula morphisms UPL). For each signature morphism u, : E — > E' the corresponding formula morphism uPL is defined by:
The definition of the satisfaction relation requires two additional notions introduced in the following definitions. Definition 6.10 (Free occurrence of a variable). Let E be a signature and a formula of predicate logic. The set free((p) of the variables occurring free in is defined by:
Definition 6.11 (Value of a formula). Let E be a signature, a formula of predicate logic, X a set of variables for S with free( A an assignment. The value for a and A is a truth value from {true, false} and is defined by:
The following definition concludes the definition of the logic PL.
Algebraic specification of abstract data types
243
Definition 6.12 (The satisfaction relation of PL). Let S be a signature. The satisfaction relation of first-order predicate logic is defined by: A (=s V O- (A(a)(
7
Models and logical consequences
This section is devoted to the study of the properties of logic.
7.1
Models
Definition 7.1 (Model). Let L be a logic, E a signature and $ C L(E) a set of formulas. (i) A E-algebra A is called a model of $ if A \=% $, i.e. A (=s if for each
(f e $. (ii) A domain (or universe) for E is a class of E-algebras that is closed under isomorphism, i.e. an abstract data type. The class of all algebras of a domain U for S that are models of $ is denoted Modu, ?,($}, or Modu($) for short. From a pragmatic point of view a domain consists of those algebras one is interested in. Practical examples of domains are Alg(E), the class Gen(E) of all generated S-algebras and the class of E-algebras generated in some sorts. One writes Modv($) instead of Mod A/J(E)($) an
(i) Modz(9) = Alg(L). (ii) Mod(PL(S)) = 0. (iii) Let E = ({s}, {0 : -4 s, _ + _: s x s -»• «}) and let x, y, £ be variables. Let $ C PL(E) be the set $ = { x + (y + z) = (x + j/) + z, z + 0 = z, 0 + z = x, Vx: s.3j/: s.(x + j/ =
244
J. Loeckx, H.-D. Ehrich and M. Wolf The reader may easily check that = {A e Alg(E) \ A(s) with A(+) forms a group with neutral element A(0) }, A(s) =
The following theorem illustrates the relation between logic and abstract data types. Theorem 7.3. Let L be a logic, E a signature, $ C £(S) a set of formulas and If a domain for E. Then Modu($) is an abstract data type. A relation between formula sets and their model classes is given in the following theorem: Theorem 7.4. Let L be a logic, E a signature, $,$ C L(S) sets of formulas and U a domain for E. Then (i) Modu(^ U *) = Modu())• The unique homomorphism from T(E, $) to an arbitrary algebra A of Mods(^) is called the initial homomorphism of A. Again, T(E, $) is an abstract data type. The following theorem is a direct consequence of Corollary 4.20. It forms the basis of the initial specification method (see section 11.1) and thus justifies the above appellation "initial homomorphism" . Theorem 7.6. Let L, E, 4> be as in Definition 7.5. If the quotient term algebra T(S,$) is a model o/$, then it is initial in Mod-z($).
7.2
Logical consequence
Definition 7.7 (Logical consequence). Let L be a logic, E a signature, f € £(E) a formula and $ C £(£) a set of formulas. Furthermore, let U be a domain for E. (i) The formula ip is called a logical consequence of $ in U if A |=s if for each A € U with A |=£ $; one then writes $ |=z/,£ - If W = Alg(£) one speaks of a logical consequence of $ and writes $ (=s ip. If U = Gen(E) one speaks of an inductive consequence of $ and writes * N/nd,S ¥>•
(ii) The formula tp is said to be valid in U if it is a logical consequence
Algebraic specification of abstract data types
245
of the empty set of formulas in U\ one then writes |=w,s (p instead of 0 |=M,E ¥>• If W = -4^(S) the formula ? is said to be logically valid and is called a tautology; one then writes [=s is said to be inductively valid and one writes (=/nd,E V3If the signature is known the index £ may be dropped. Example 7.8. (i) Let $ be as in Example 7.2(iii). Then $ |= Vx: s.Vy: s3z: s.z + x = y. (ii) Let E be the signature of Example 4.17 and $ C EL(S) be the set $ = {x + Q = x, (1) x + Succ(y) = Succ(x + y), (2) (a; + j/) + z = z + (j/ + z)}. (3) One may prove that $ (=/n) \=
7.3
Theories
Definition 7.9 (Theory). Let L be a logic and S a signature. (i) The theory of a E-algebra A with respect to L is the set (ii) The theory of a class C of E-algebras with respect to L is ThLf(C) = { > € L(E) | A ^E V for each A e C }. Informally, the theory of C consists of all formulas that express common properties of the algebras of C. Clearly, the theory is "larger" when the logic is more "powerful" — as illustrated in Example 7.11. Fact 7.10. (i) A theory is closed under logical consequence. (ii) A theory contains all tautologies.
246
J. Loeckx, H.-D. Ehrich and M. Wolf
The first of these facts expresses the fact that, for instance, Thi(C) \= (p implies if £ ThL(C). Example 7.11. Let E = ({nat}, {0 : -> nat, Succ : not -4 nat}) and A be the "classical" E-algebra. (i) ThEi(A) contains tautologies only. (ii) Apart from tautologies, ThpL(A) contains formulas such as -i(0 = Swcc(O)) and Vx: nat.By: nat.y = Succ(x). The first part of the following theorem states that theories cannot distinguish between isomorphic algebras. Theorem 7.12. Let L be a logic, E a signature and A, B 'S-algebras. (i) A~B implies ThL(A) = ThL(B). (ii) If L is a logic with equality and if A and B are generated, then ThL(A) = ThL(B) implies A~B. The following theorem is a dual of Theorem 7.4. Theorem 7.13. Let L be a logic, E a signature and C\, C% two classes of 'E-algebras. Then: (i) ThL(d\JC2) = ThL(d) n 17»i(C2); (ii) Ci C C2 implies ThL(C2) C ThL(Ci). To be manageable a theory should be recursively enumerable. Only in that case it is possible to describe it axiomatically. Definition 7.14 (Axiomatizable algebra). (i) A E-algebra A is said to be axiomatizable in a logic L if ThL,z(A) is recursively enumerable. (ii) An axiomatizable class of algebras is defined similarly. Example 7.15. Let E be the signature of Example 2.2. The classical algebra A for this signature (with A(nat) = NO) is called the Pea.no arithmetic. It is well known that this algebra is not axiomatizable.
7.4
Closures
Informally, the closure of a set of formulas is the smallest theory containing this set as a subset. Definition 7.16 (Closure of a set of formulas). Let L be a logic, E a signature, $ C L(E) a set of formulas and 14 a domain for E. The closure of $ with respect to the domain U is the set of formulas If U = Alg(£) one speaks of the closure of $ and writes $£ or 3>* instead of $*Aig<^y If U = Gen(S) one speaks of the inductive closure and writes or $ * instead o f $ -
Algebraic specification of abstract data types
247
The following theorem allows an alternative definition of a closure. Theorem 7.17. With L, E, $ and U as in Definition 7.16 and * C L(S) *£ = ThL(Modu(*)). Closures possess numerous properties as illustrated in the following theorems. Theorem 7.18. With L, S, $ and U as in Definition 7.16 and * C L(S): ft) * C
^ $ C * implies $£, C *^. Theorem 7.19. Let L be a logic, E a signature, U a domain for E and . Tfcen: £ C $£, if and onfy if Modu($) C w C #£, »/ and on/j/ t/ * \=u $. As a corollary of this theorem two sets $ and $ of formulas have the same models if and only if $ is a logical consequence of $ and vice versa. Hence, to prove that Mod^($) — Modu(^f) it is sufficient to prove $ \=u * and * \=u *• A dual notion of the closure of a set of formulas is that of the closure of a class of algebras. Definition 7.20 (Closure of a class of algebras). Let L be a logic, S a signature, U a domain for S and C C U a class of E-algebras. The closure of C with respect to L and U is the subclass C.*LUTi (or C£ w for short) of U: Clearly, C*L
• C C C' implies Q)M C CJ*>M. Moreover, the dual of Theorem 7.19 also holds: Theorem 7.21. Let L be a logic, E a signature, U a domain for E and C,C' C U classes of'E-algebras. Then: ft) Clu Q C'f-u if and only if ThL(C') C fii; C£^ C C'lju if and only ifCL,u C C
248
J. Loeckx, H.-D. Ehrich and M. Wolf
It is possible to show that Modu and Thi constitute a Galois connection; the interested reader may consult, for instance, Exercise 5.7-4 in [Loeckx et al, 1996].
7.5
Reducts
The present section investigates the logical properties of reducts. To simplify the presentation, the domain is assumed to be Alg(H); a generalization for an arbitrary domain U poses no fundamental problems. The following theorem is a straightforward consequence of the satisfaction condition. Theorem 7.22. Let L be a logic and fj. : S —» E' a signature morphism. (i) Let $ C L(E). Then Modv(n($)) = { A' € Alg(E') \ (A1 (ii) LefD' C Alg(E'). Then (V | ft) = { £' a signature morphism and $ C L(£) o set ofS-formulas. Then p(* E ) C (/*(*))jy.
7.6
Extensions
The present section investigates the logical properties of extensions of model classes. Theorem 7.24. Let L be a logic, p, : £ —> E' a signature morphism and $ C L(E), $' C L(E') sets of formulas. (i) Mods' ($') is a ^-extension of Mod^Q) if and only if $' )=£' /*($)• (ii) If Mods' ($') is a. persistent ^-extension of Modj;($), then Definition 7.25 (Extension of a set of formulas). Let L be a logic, H : E —> E' a signature morphism and $ C L(E), $' C L(E') two sets of formulas. The set $' is called a ^-extension of $ if /<(*E) C $^, . The It-extension is called persistent if /u($s) = ^E ^ ^(M^))Informally, a /i-extension $' of $ contains $ as a subset "modulo /i" . If the /^-extension is persistent, $' is identical with $ "modulo //" except for formulas from L(S') - L(^(E)). By the way, L(E') = £(//(£)) whenever |i is surjective. A reader having difficulties in grasping these notions is recommended to first consider the particular case in which jj, is an inclusion. The following theorem links extensions of formula sets to extensions of model classes.
Algebraic specification of abstract data types
249
Theorem 7.26. Let L be a logic, fj, : E —» E' o signature morphism and $ C L(E), $' C L(E') sets of formulas. Then: (i) $' is a /4-extension of $ if and only t/ Mods/ ($') is a ^-extension of Modify; (ii) if Mod-£> ($') is a persistent ^-extension of Modz($), then $' is a persistent ^-extension This theorem allows Theorem 7.24 to be rephrased.
8
Calculi
Informally, a calculus constitutes an inductive definition of a set of strings. In a calculus for a logic these strings are formulas. The goal of such a calculus is to syntactically grasp the notion of logical consequence. Hence, with such a calculus it is in principle possible to mechanically prove statements such as $(=?. Calculi for particular logics have been studied extensively in the frame of mathematical logic and artificial intelligence (e.g. [Ebbinghaus et al., 1984; Bibel, 1993]). The present section restricts itself to some general definitions, an example and a few comments.
8.1
Definitions
Definition 8.1 (Calculus). Let L be a logic and E a signature. A calculus for L and S is a pair K. = (Ax, Inf) where: • Ax is a finite set of decidable subsets of L(E); an element of Ax is called an axiom scheme and an element of an axiom scheme is called an axiom; • Inf is a finite set of inference rules each of which consists of a decidable subset of (£(S))n x L(E), for some n, n > 0. Note that an inference rule { ((ipi ,... ,ipn),tp) \ . . . } is generally written
as V 1 , • • • ,
The "meaning" of a calculus results from the following definition. Definition 8.2 (Derivable formula). Let L be a logic, E a signature and 1C a calculus for L and E. Let $ C L(E) be a set of formulas and ip € £(E) a formula. The formula (p is said to be derivable from $ in 1C if there exists a finite non-empty sequence i,---,
250
J. Loeckx, H.-D. Ehrich and M. Wolf • tpj is an axiom of AC; • there exist indices ii,...,in < j such that (((^ , . . . ,
is an
One then writes $ l-jc • The following definition reflects the fact that the purpose of a calculus is to grasp the notion of logical consequence. Definition 8.3 (Soundness, completeness). Let L be a logic, S a signature and AC a calculus for L and E. Furthermore, let U be a domain for E. (i) AC is called sound with respect to U if, for any set $ C L(£) and any formula
.
8.2 An example The following calculus is a calculus for the equational logic EL of Section 6.2. It constitutes a generalization of the calculus presented in [Tucker and Meinke, 1992] in that it allows empty carrier sets. Definition 8.4 (Equational calculus). Let E be a signature and X, Y sets of variables for E. The following axiom scheme (I) and four inference rules (II) to (V) constitute a calculus for the equational logic EL and the signature E:
(i) vx.t = t (ll)
teT^xy,
^7
t = n,VX.u = v —
. fc — l/
t,u€T^x}; t, u, v 6 T E(JC );
a : X —> TY.(Y\ a substitution;
Algebraic specification of abstract data types
251
=U
VX.va = vr m > 1, ti,Ui € T-E(X) f°r all z with 1 < i < m, v e T S (y), CT,T : Y —> ?£(X) two substitutions satisfying the condition for each y £ Y: either cr(y) = r(y) or (ff(y) = tj and r(y) = Uj for some j, 1 < 3 < m). Note that the calculus takes cares of empty carrier sets. More precisely, assume that Y C X; the calculus allows VX.t = u to be deduced from VY.t = u; on the other hand it allows VY.t = u to be deduced from VX.t = u only if for each sort s with Ys = 0 and Xs ^ 0 the set T^^Y),s is not empty (cf. Example 6.5(ii) with X = {x} and Y = 0). Theorem 8.5. The equational calculus of Definition 8.4 is sound and complete with respect to Alg(E). While the proof of the soundness is straightforward, the proof of the completeness is not trivial (see, for example, [Tucker and Meinke, 1992] or [Loeckx et al., 1996]).
8.3
Comments
The calculus above is of course also sound with respect to any domain for the signature S, for instance with respect to Gen(E). It is not complete with respect to Gen(S) because the inductive consequence "j=/n. If the calculus is sound without being complete, it is not guaranteed that $ I- tp holds.
252
J. Loeckx, H.-D. Ehrich and M. Wolf
The axioms and inference rules of a calculus are generally very "elementary" and — especially in the case of first-order predicate logic—"nonintuitive" . Proving properties of the form $ h ip in a calculus is therefore very tedious. To this end calculi have been implemented in the form of "proof systems" allowing proofs to be carried out automatically or semiautomatically with a computer. Some of these systems allow induction to be "simulated" and thus inductive consequences to be proved. From what has been said above it follows that such systems cannot be complete. Instead, the most sophisticated among them use heuristics; if these fail the user is asked for some "clever" lemma enabling the proof to be carried out.
9
Specification
The goal of this section is to introduce a few general notions on specification and thus constitute an introduction to sections 10 to 14 of this chapter. The fundamental idea of abstract data type specification is to define abstract data types by formulas. As in the case of programs, specifications must have a modular structure lest they get incomprehensible. This leads to the following techniques for the design of "small" and "large" specifications, respectively. Specification in the small consists in drawing up specifications "from scratch". Essentially such a specification consists of a signature and a set of formulas. The abstract data type defined by the specification is in general a subclass of the model class of the set of formulas. In sections 10 to 12 three different specification techniques will be discussed yielding loose specifications, initial specifications and constructive specifications, respectively. They differ from each other by the logic used, by the set of formulas allowed and/or by the subclass defined. Henceforth, a specification obtained by specification-in-the-small is called an atomic specification. Specification in the large consists in drawing up specifications by putting (already designed) specifications together. This is performed with the help of a formalism called specification language. The principles of such languages will be examined in sections 13 and 14. The following—very general—definition reflects the fact that a specification does not necessarily "exactly" capture the intended abstract data type. Definition 9.1 (Adequate specification). Let C be an abstract data type. Furthermore, let sp be a specification and M(sp) its meaning, i.e. the abstract data type sp defines. Then: (i) sp constitutes an adequate specification of C if C C M(sp); (ii) sp constitutes a strictly adequate specification of C if C = Ai(sp). By an abuse of language one also speaks of an adequate or strictly adequate
Algebraic specification of abstract data types
253
specification of an algebra, meaning an adequate or strictly adequate specification of the monomorphic abstract data type containing this algebra. Note that if M(sp) is a monomorphic abstract data type the notions of adequacy and strict adequacy coincide. A specifier or user of a specification may want to prove certain properties of a specification. He may, for instance, want to prove that a given specification sp constitutes an adequate or strictly adequate specification of an abstract data type C. Of course, such a proof presupposes that the abstract data type C has been precisely defined elsewhere. Another interesting question is whether the class M(sp) defined by the specification sp constitutes a monomorphic abstract data type. A further question is whether M(sp) is empty (and hence "useless"). Proofs of such properties may take the form of informal "mathematical" proofs similar to those that may be found in any textbook on theoretical computer science. A particular case is constituted by those properties that may be reduced to logical consequences. Such a property may be written as $ \=u tp, where y is, for instance, a property of the abstract data type and where $ is the set of formulas of an atomic specification. According to section 8.3 such properties may be proved automatically or semi-automatically whenever there exists a sound (and complete) calculus with respect to U. As indicated above, proving the adequacy of a specification presupposes that the abstract data type to be specified is exactly known. Except for "trivial" abstract data types this is generally not the case in practice: the specification itself often constitutes the only precise definition of the abstract data type. Instead, the adequacy may be checked by a testing method called rapid prototyping. Essentially, this testing method consists in mechanically determining the value of some "critical" terms and comparing this value with one's "expectations". This mechanical evaluation of terms presupposes that the specification is in some sense "executable" or "constructive" and hence monomorphic—as will be discussed below. Clearly, being based on testing rapid prototyping may disprove adequacy but cannot prove it; in other words, rapid prototyping may at best improve one's confidence in the adequacy. A specification defining a polymorphic abstract data type is sometimes called a requirement specification. In contrast, a design specification is a monomorphic specification that allows rapid prototyping.
10
Loose specifications
Loose specifications are atomic specifications in the sense of section 9.
10.1 Genuinely loose specifications Definition 10.1 (Loose specification). Let L be a logic.
254
J. Loeckx, H.-D. Ehrich and M. Wolf
(i) (Abstract syntax.) A loose specification in L is a pair sp = (£,$) where S is a signature and $ C L(E) a decidable set of formulas. (ii) (Semantics.) The meaning M(sp) of a loose specification sp = (£, ft) is defined to be the abstract data type M(sp) = Informally, a loose specification stands for the class of all models of its set of formulas. To improve their readability and allow for a mechanical treatment specifications may be written in a concrete syntax. In this syntax a loose specification is a string of, for instance, the following form: loose spec sorts list-of-sorts opns list-of-operations vars list- of -variables axioms list-of- formulas endspec Informally, the sorts and operations fix the signature; an element of the list- of -variables is of the form x: s where x is a variable and s its sort. It is required that only variables of the list-of-variables occur in the formulas. In the examples a list of variables such as x: s, y: s, z: s will be written x,y,z: s. In this chapter examples will be given in concrete syntax while theorems and definitions will refer to the abstract syntax. Example 10.2. The logic used in this specification is the first-order predicate logic PL of section 6.4. Note the similarity with Example 4.9. loose spec sorts boot, el, list opns True : -> bool False : -¥ bool [ } : ->• list Add: el x list -> list . . _ : list x list ->• list Isprefix : list x list —> list vars l,m,n: list, e: el axioms []./ = { Add(e, l).m = Add(e, l.m) (Isprefix (I, m) — True) = (3n.(m = l.n)) endspec The specification is an adequate specification of the "classical" algebra because this algebra constitutes a model of the "axioms" — as one may easily check. It is not a strictly adequate specification due to the existence of models that are not generated in the sorts bool and list. Example 10.3 (Peano axioms). Again the logic considered is PL. loose spec sorts bool, nat opns True : -»• bool
Algebraic specification of abstract data types
255
False: -+ bool 0: -» nat Succ: nat —t nat _ + _: nat x not —>• nat _ * _ : nat x nat —» nat . < _: nat x nat -> feooJ vars m, n: nat axioms -i(7Vue = False) -.(0 = Succ(n)) Succ(n) = Succ(m) D n = m (0 < n) = Thie (Succ(n) < 0) = Fake (Succ(n) < Swcc(m)) = (n < m) n+0=n n + Swcc(rn) = Succ(n + m) n*0 = 0 n * Swcc(m) = n + (n * m) endspec Clearly, the specification is an adequate specification of Peano arithmetic (see Example 7.15). As Peano arithmetic is not axiomatizable, it cannot be a strictly adequate one. The following theorem indicates some limits of the expressive power of loose specifications. Theorem 10.4. Let sp = (E, $) be a loose specification in a logic L. Then: (i) M(sp) = (M(sp)YL; (ii) if L is a logic for which there exists a sound and complete calculus, then M(sp) is axiomatizable in L; (in) if L is one of the logics of sections 6.2 to 6.4 and if M.(sp) contains an algebra with an infinite carrier set, then M.(sp) also contains nongenerated algebras, A variant of Theorem 10.4(iii) is also known as the Lowenheim-Skolem theorem (see, for instance [Ebbinghaus et al, 1984]). Properties of loose specifications may be proved along the lines discussed in section 9. In general, rapid prototyping is not possible for loose specifications.
10.2 Loose specifications with constructors Definition 10.5 (Loose specification with constructors). Let L be a logic. (i) (Abstract syntax.) A loose specification with constructors is a 4-tuple
256
J. Loeckx, H.-D. Ehrich and M. Wolf sp = (S, $, 5 c ,fl c ) where £ = (5, fi) is a signature, $ C L(S) a set of formulas, Sc C. S a set of sorts and flc C fi a set of operations with target sorts in Sc called constructors.
(ii) (Semantics.) The meaning M(sp) of sp is defined to be the abstract data type M(sp) = Modu($) where U
=
{A € AlgCE) \ A is generated in the sorts of Sc by the set flc}.
The concrete syntax for such specifications may be chosen to be identical with that of section 10.1 except that in the list- of- operations each sort of Sc and each operation of fZc is preceded by the keywords generated and constr respectively. Example 10.6. Consider the specification sp of Example 10.2 with generated preceding boot and list and with constr preceding each of the first four operations. By its very definition M(sp) no longer contains algebras that are not generated in the sorts bool and list. Nevertheless, it still fails to be a strictly adequate specification because it contains models for which the carrier set of sort list is a singleton. Example 10.7. Consider the specification of Example 10.3 with generated preceding bool and nat and with constr preceding each of the first four operations. It may be shown that the abstract data type defined by the specification is monomorphic. Hence, it constitutes a strictly adequate specification of Peano arithmetic. By the way, this specification illustrates that the first-order predicate calculus together with the induction principle is in some sense complete with respect to Gen(S) (cf. section 8.3). The properties of loose specifications — including Theorem 10.4 — carry over to loose specifications with constructors. Of course, the effect of the Lowenheim-Skolem theorem (Theorem 10.4(iii)) is void.
10.3
Loose specifications with free constructors
The definition of these specifications is as in Definition 10.5 with "freely generated" instead of "generated" . Similarly, one uses the keyword freely generated instead of generated. Example 10.8. Consider the specification of Example 10.6 with freely generated instead of generated. The specification is now a strictly adequate specification of the "classical" algebra. Note that the abstract data defined is still polymorphic due to the non-generated sort el. Example 10.9. Consider Example 10.7 but with freely generated instead of generated. The first three formulas are now superfluous and may be removed. Example 10.10. Consider the following specification:
Algebraic specification of abstract data types
257
loose spec sorts opns
freely generated not constr 0 : —» not constr Succ : nat —> not Pred : nat -> nat vars n: no£ axioms Pred(Succ(n)) = n
endspec in which Pred is "intended" to be the predecessor function. This polymorphic specification is a strictly adequate specification of the "classical" algebra in which the value Prerf(O) is left pending. More precisely, the abstract data type defined contains non-isomorphic algebras differing from each other only by the value of Pred(0).
11
Initial specifications
Initial specifications are atomic specifications in the sense of section 9. They differ from loose specifications in that they always define monomorphic abstract data types. Furthermore, initial specifications make use of equational logic or conditional equational logic. While sections 11.1 to 11.7 treat the case of equational logic, section 11.8 briefly considers the case of conditional equational logic.
11.1
Initial specifications in equational logic
Definition 11.1 (Initial specifications). Let EL be the equational logic of section 6.2 and let T(£, $) denote the quotient term algebra of Definition 7.5. (i) (Abstract syntax.) An initial specification in equational logic is a pair sp = (£, $) where S is a signature and $ C EL(E) a set of equations. (ii) (Semantics.) The meaning M(sp) of an equational specification sp = (£, 4>) is the monomorphic abstract data type M(sp) = {Ae Alg(E) | A ~ T(E, *) }. The interest of this definition stems from the following fundamental theorem of equational logic: Theorem 11.2. Let S be a signature and $ C ££(£) a set of equations. ThenT(£,) £ Morf s ($). The proof of this theorem is essentially based on the Theorems 3.10 and 3.11. Together with Theorems 7.6 and 2.14, this leads to the following important result: Corollary 11.3. Let sp = (£,$) be an initial specification. Each algebra of M(sp) is initial in
258
J. Loeckx, H.-D. Ehrich and M. Wolf
Prom the properties of algebras and, in particular, of quotient term algebras one easily deduces the following fact: Fact 11.4. Let sp = (£,$) be an initial specification. (i) Each algebra of M (sp) is generated. (ii) For each algebra A 6 Morfs($) there exists a unique homomorphism h : T(S, $) —> A defined by h([t]) = A(i), t € Ts, viz. the initial homomorphism (see Definition 7.5). (iii) For each algebra B € M(sp) there exists a unique and surjective homomorphism h : T(S) —> B defined by h(i) = A concrete syntax for initial specifications is chosen to be identical with that of section 10.1 but with the keywords initial and eqns instead of loose and axioms.
11.2
Examples
Example 11.5. (i) A trivial initial specification is: initial spec sorts not opns 0 : -» not Succ : nat —>• nat _ + _ : nat x nat —> nat vars m,n:nat eqns n + 0 = n n + Succ(m) = Succ(n + m) endspec One may prove that the specification is an adequate specification of the "classical" algebra. Note, in particular, that T(£,$)(nat) consists of the equivalence classes [0], [5«cc(0)],[5«cc(5«cc(0))], etc. Note also that, for instance, [0] = [0 + 0] = [0 + 0 + 0] = . . . [SMcc(O)] = [SMCc(O) + 0] = [0 + S«cc(0)] = [Succ(O) + 0 + 0] = . . . . (ii) The following example is the classical example of an initial specification. Its glamour stems from the fact that there is no equivalent loose specification with free constructors that is equally "abstract" . initial spec sorts nat,, set opns 0 : ->• nat Succ : nat —> nat 0 : -> set Insert : set x nat —> set vars m, n: nat,s: set eqns Insert(Insert(s,n),n) = Insert (s, n)
Algebraic specification of abstract data types
259
Insert(Insert(s,n),m) = Insert(Insert(s,m),n) endspec Informally, the first equation identifies terms in which the same "element" occurs several times; the second equation identifies terms in which the same "elements" occur in different order. The following example illustrates the problems resulting from the fact that initial specifications cannot define polymorphic abstract data types. Example 11.6. Consider the initial specification deduced from the loose specification of Example 10.10 by deleting the keywords freely generated and constr and by replacing loose and axioms by initial and eqns. This specification is not an adequate specification of the "classical" algebra because T(S,)(na£) now contains additional carriers such as [Pred(O)], [Pred(Pred(0))} and [Succ(Pred(Q))]. Correcting non-adequate initial specifications is non-trivial and errorprone, as the following example illustrates. Example 11.7. To correct the non-adequate specification of Example 11.6 the following solutions may be envisaged. One may add the equation Pred(O) = Swccr™'(0) for some fixed n > 0, and thus use [5wc nat and the equations: Fred(O) Pred(Error) Succ(Error)
= Error, = Error, = Error.
.
(11.1) (11.2) (11-3)
This solution leads to an additional carrier, viz. [Error]. A more elegant "solution" consists in adding the equation Succ(Pred(n)) = n. Actually, it still fails to eliminate superfluous carriers, viz. [Pred(O)], [Pred(Pred(G))},.... A combination of the last two solutions that consists in adding the corresponding four equations has a dramatic result: the carrier set is reduced to a singleton because [Error] = [0] = [SWc(O)] = ... — as one can easily check. For more examples of initial specifications the reader is referred to the abundant literature on the subject (e.g. [Ehrig and Mahr, 1985]).
260
11.3
J. Loeckx, H.-D. Ehrich and M. Wolf
Properties
The following property of equational logic directly follows from Theorem 3.8. Theorem 11.8. Let E be a signature, A, B Ti-algebras and h : A —> B a homomorphism. Furthermore, lett = u be a ground equation and VX.v = w an equation of EL(E) . (i) If A\=t- u, then B\=t = u. (ii) If h is surjective and A \= VX.v — w, then B \= VX.v = w. (iii) If h is injective and B |= VX.v = w, then A \= VX.v = w. This leads to the following important property of initial specifications. Note that T(£, $) may be viewed as a representative of the abstract data type defined by the initial specification (S,«J>). Theorem 11.9. Let (£,$) be an initial specification. Let t = u be a ground equation and VX.v = w an arbitrary equation of ££(£). Then: (i) T(£, $)\=t = u if and only if $ I- t = u; (ii) T(E, $) |= t = u if and only if A |= t = u for each T,-algebra A from (iii) T(S, $) f= VX.v -w if and only if $ (=/nd VX.v = w; (iv) r(S, $) |= VX.v — w if and only if A^= VX.v = w for each E-algebra A from Indz($) (i-e. for each generated 'S-algebra from Informally, the theorem states that T(S, $) possesses exactly those properties that are common to all generated models (and that are expressible in equational logic).
11.4
Expressive power of initial specifications
By their very definition initial specifications can only define monomorphic abstract data types consisting of generated algebras. Conversely, any such abstract data type can be defined by an initial specification, the equation set of which consists of a possibly infinite number of ground equations. Remember that the notions of adequacy and strict adequacy coincide for specifications defining a monomorphic abstract data type. Theorem 11.10. Let A be a generated 'S-algebra. The initial specification (E, { t = u € #L(E) | t, u € TS, A(t) = A(u) }) is an adequate specification of A. In practice one is especially interested in specifications with a finite set of equations. Theorem 11.11. There are generated algebras for which there exists an adequate initial specification with a finite set of equations but no adequate initial specification with a finite set of ground equations.
Algebraic specification of abstract data types
261
An example of such an algebra is the S-algebra A with £ = ({not}, {0: -> nat, Succ: nat -4 nat, Double : nat ->• not}) and A(nat) = NO , A(Q) and A(Succ) "classical", A(Double)(n) = 2 * n. Clearly, the initial specification with: 4> = (Double(G) = 0, Double(Succ(n)) = Succ(Succ(Double(n)))} is an adequate specification of A. The basic idea of the proof that a finite number of ground equations does not suffice, is that one needs recursion to express "doubling" with the help of the successor function. Theorem 11.12. There are generated algebras for which there exists no adequate initial specification with a finite set of equations. An example is the E-algebra A with: S = ({nat}, {0: —> nat, Succ: nat —> nat, Square : nat —» nat}) with A(nat), A(0), A(Succ) as usual and A(Square)(n) = n*n. The basic idea of the proof is to show that one needs nested recursion to express squaring with the help of the successor function (in the absence of an "auxiliary" function such as addition). The following theorem shows that the use of such "auxiliary" functions suffices to finitely specify any generated "computable" algebra. The notation ... | E is that of Definition 5.6. Theorem 11.13. Let A be a generated ^-algebra, the functions of which are computable. Let C be the monomorphic abstract data type containing A. Then there exists an initial specification sp = (£',') with S C £', *' C ££(£') and $' finite such that C = (M(ap) \ E). The idea of the proof is to simulate a Turing machine by an initial specification (cf. [Bergstra and Tucker, 1983]).
11.5
Proofs
Properties of initial specifications may be proved along the lines discussed in section 9. In addition, one may make use of the specific properties of these specifications and apply one of the following four principles of proof. The first principle is based on the fact that the algebra T(£, $) is initial in Modz($). Hence, to prove for a given S-algebra A that A ~ T(£, $), it is sufficient to prove that A is a model of $ and that the initial homomorphism of A (viz. h : T(£, $) —> A defined by h([t]) = A(t), t 6 TE) is bijective. Similarly, to prove that the initial specifications (£, 4>) and (E, *) are equivalent, it suffices to prove that $ |= * and that the initial homomorphism h : T(£, $) —>• T(£, *) is injective. A second principle is based on the fact that T(E, $) is generated and on the following property of equational logic. Theorem 11.14. Let E be a signature, VX.t = u £ EL(E) an equation and A a generated 'E-algebra. Then A \= VX.t = u if and only if
262
J. Loeckx, H.-D. Ehrich and M. Wolf
A\=to- = ua for all ground substitutions a : X —» TE . Informally, the theorem states that a variable may be replaced by an arbitrary ground term. Hence, to prove T(£, $) (= VX.t = u one proves T(£, $) |= ta = ua for all ground substitutions a : X —> TE or, according to Theorem 11.9(i), $ (= ta — ua for all ground substitutions a : X —>• T£. This may be proved by induction on the structure of the ground terms that are substituted for the variables, i.e. on the structure of the value cr(x), x 6 X . Example 11.15. Consider the specification of Example 11.5(i). To prove that T(S, $) |= V{n}.0 + n = n, it is sufficient to prove: (i) $ |= 0 + 0 = 0; (ii) if $ (= 0 + t = t for some t € IE, then $ (= 0 + Swcc(£) = Succ(t); (iii) i f $ | = 0 + u = w and $ h 0 + u = w for some u,v £ Ts, then $ (= 0 + (u + l>) = M + V.
Actually, step (iii) of the proof may be shown to be superfluous. In fact, one may show that for each ground term t e TH there exists n > 0 such that $ (= t = Succ^n\0). Informally, this means that 0 and Succ are constructors of the sort nat. A third principle of proof consists in replacing the algebra T(£, 4>) by an isomorphic but "simpler" algebra. It is based on the following theorem which is a direct consequence of Theorem 4.6. Theorem 11.16. Let (S,$) be an initial specification with S = (5,0). Let C be a generated H-algebra with C(s) C TsiS for each s 6 S. If (i) C \= $, and if, moreover, (ii) $ f= C(t) = t for all ground terms t € T%, thenC ~T(£,$). An algebra C satisfying the conditions of the above theorem is called a characteristic term algebra for the initial specification (E,$). The proof technique now consists in finding a characteristic term algebra C for the given specification and replacing the study of T(S, $) by that of C. Example 11.17. Consider again the specification of Example 11.5 (i). Define the S-algebra C by: C(0) = 0, C(Succ)(Succ(n\Q)) = C-(+)(5«cc ( " ) (0),5'wcc (m) (0)) = Succ(n+m)(0). The reader may check that C satisfies conditions (i) and (ii) of Theorem 11.16. Hence, to prove T(S, $) |= V{n}.0 + n = m it is sufficient to prove C \= V{n}.0 + n = n, i.e.
Algebraic specification of abstract data types
263
C(+)(C(Q),at(ri)) = a(n) for all assignments a : {n} —> C or, equivalently, C'(+)(0,5ttcc(n)(0)) = S«cc(n)(0) for all n > 0. Note that C indirectly proves that 0 and Succ are constructors of the sort not (see the remark in Example 11.15). The fourth principle of proof consists in the use of term rewriting systems, to which we now turn.
11.6
Term rewriting systems and proofs
Term rewriting systems are particular reduction systems. Both types of systems are extensively discussed in [Klop, 1992; Avenhaus, 1995]. In order to fix the notation the main definitions are briefly recalled. First the notions related to reduction systems are defined. Definition 11.18 (Reduction system). (i) A reduction system is a pair (R, ->) where jR is a set and "—>•" a relation on R. (ii) A reduction sequence of a reduction system (R, —») is a possibly infinite sequence r\,..., r / t , . . . of elements of R such that FJ -» r,+1 for each i > 1; one then writes r\ ->•* r^ for any k>l. (iii) An equivalence sequence is defined similarly but with r, —» r»+i or n+i ->• Ti for each i > I; one writes r\ ~ r^ for any k > 1. Definition 11.19 (Noetherian, confluent, locally confluent). (i) A reduction system is Noetherian if it possesses no infinite reduction sequence, (ii) A reduction system (jR, —>) is locally confluent if for all r, s, t € R the following holds: if r -> s and r -* t then there exists u € R with s —>•* w and t —>* u. (iii) The notion of a confluent reduction system is defined as in (ii) but with r —t* s and r —>•* t instead of r —» s and r —> t, respectively. Definition 11.20 (Normal form). Let (R, ->) be a reduction system and r & R. A normal form of r is an element s £ R such that r —»* s and there exists no i € .R with s -t t. Theorem 11.21 (Newman's lemma). A Noetherian and locally confluent reduction system is confluent. Theorem 11.22. Let (R, ->) be a Noetherian and confluent reduction system. (i) Each element of R has exactly one normal form. (ii) Let r,s € R. Then r ~ s if and only if r and s have the same normal form. The following definition associates a reduction system with each initial specification.
264
J. Loeckx, H.-D. Ehrich and M. Wolf
Definition 11.23 (Term rewriting system). The term rewriting system of an initial specification (E, $) is the reduction system (T^, —>) with " ->" inductively defined by: (i) ta —> ucr for each equation VX.t = u £ $ and each ground substitution a : X —> Ts; (ii) if i -> u, then v[t/y] -» w[u/y] for all terms w e ?E({»}) containing at least one occurrence of the variable y. Note that an initial specification in which an equation VX.t = u is replaced by VX.u = t leads to a different relation "-»*" but to the same relation "~". The following definition and theorem constitute the basis of a criterion guaranteeing that a term rewriting system is Noetherian: Definition 11.24 (Rewrite ordering, reduction ordering). Let £ be
a signature with Tg)g ^ 0 for each sort s from £. Furthermore, let X be a set of variables for £. An irreflexive partial order ">" on T£(X) is called a rewrite ordering on TE(X) if f°r a^ terms t,u € TE(X) : (i) i > u implies fcr > ucr for all substitutions a : X —> ^£(-f ) > (ii) t > u implies v[t/y] > v[u/y] for all terms v 6 ?s(xu{y}) where j/ is a variable for S, y £ X, which occurs at least once in v. A rewrite ordering is called a reduction ordering if there exists no infinite sequence • • • < h < *2 < ti,
ti € TZ(X) for all i > 1.
Theorem 11.25. Le£ (E, $) 6e an initial specification and X a set of variables for S containing the variables occurring in $. // there exists a reduction ordering < on T%(x), such that u < t for any equation VY.t = u e $, £/«en #je term rewriting system of (£,$) is Noetherian. The main theorem of term rewriting states that the equivalence relation "~" coincides with the validity of ground equations in the initial algebra: Theorem 11.26. Let (£,$) be an initial specification and (Ts,->) its term rewriting system. For all ground terms t, u € TS it is the case that t ~ u if and only if T(E, $) |= t = u. Note that the theorem does not require that the term rewriting system is Noetherian and/or confluent. The previous theorem is the basis of the fourth principle of proof mentioned in section 11.5. Suppose one has to prove T(E, $) |= VX.t = u. It is sufficient to show that for all ground substitutions a : X •—> T^ it is the case that T(£,<J>) |= ta = ua (according to Theorem 11.14) or, equivalently, that ta ~ ua (according to Theorem 11.26),
Algebraic specification of abstract data types
265
Example 11.27. Consider the specification of Example 11.5(i). (i) To prove that T(E, $) \= Vn.n + Succ(Succ(0)) = Succ(Succ(Q)) + n one proves that t+Succ(Succ(0)) ~ Succ(Succ(t)) + Q for an arbitrary *€TE: t + Succ(Succ(G)) -> Succ(t + Succ(0)) -»• Succ(Succ(t) + 0) ->• Succ(Succ(t)) <- Succ(Succ(t)) + 0. (ii) To prove that T(E, $) (= V{n}.0 + n = n one proves that 0 +1 ~ t for an arbitrary i 6 IE- This may be done by induction on t (cf. Example 11.15). Clearly, a term rewriting system has similarities with a calculus for equational logic. As an advantage over the calculus it leads to an efficient proof procedure in the following particular but "frequent" case. When the term rewriting system of an initial specification (S, $) is Noetherian and confluent, the proof of T(E, $) |= VX.t = u may be replaced by a proof of the following property: for all substitutions a : X —» Tg the ground terms to and ua have the same normal form (see Theorem 11.22). The calculation of the normal form of a ground term is a straightforward procedure that allows an efficient implementation. Hence, the (main parts of the) proof of T(£, $) |= VX.t = u may be carried out automatically. Example 11.28. Consider again Example 11.27(1). One easily proves: t + Succ(Succ(<0)) ->* Succ(Succ(t)), Succ(Succ(t)) + 0 ->* Succ(Succ(t)). Hence t + Succ(Succ(0)) and Succ(Succ(t)) + Q have the same normal form, viz. the normal form of Succ(Succ(t)). Reducing proofs to the calculation of normal forms is possible only if the term rewriting system of the specification is Noetherian and confluent. Several criteria have been developed that guarantee each of these properties; they are based on Theorem 11.25 and Theorem 11.21, respectively. The Knuth-Bendix algorithm constitutes a particular implementation of these criteria: it tries to transform an initial specification sp = (S, $), the term rewriting system of which is Noetherian, into an initial specification sp1 = (E,
11.7
Rapid prototyping
Rapid prototyping consists in the evaluation of ground terms. When the term rewriting system of an initial specification is Noetherian and confluent,
266
J. Loeckx, H.-D. Ehrich and M. Wolf
rapid prototyping may be reduced to the calculation of normal forms — as will now be shown. Definition 11.29. Let (E, <J>) be an initial specification, the term rewriting system of which is Noetherian and confluent. One associates with this specification a E-algebra C defined by: C(s) = { t £ TE>S | Ha a normal form }, for each sort s of E; C(ui) = the normal form of the term n, for each operation u> = (n : —> s) of E; C (w) (ti,... , t k ) — the normal form of the term n(ti, .. .,**), for each operation (jj = (n: si x . . . x Sfc -» s) of E. Theorem 11.30. Let (S,*) and C be as in Definition 11.29. Then C is a characteristic term algebra of the initial specification (E, $). Hence Informally, the theorem states that the algebra C of Definition 11.29 may be viewed as a representative of the abstract data type defined by the initial specification (S, 4>). This justifies why the evaluation of ground terms in the algebra C may be defined as rapid prototyping. As this evaluation consists in the calculation of the normal form of these ground terms, it may be performed automatically by the term rewriting tools mentioned in section 11.6. Example 11.31. Consider the specification of Example 11.5(i). Rapid prototyping of the ground term Succ(Succ(0)) + Succ(Q) yields the ground term Succ(Succ(Succ(0))).
11.8
Initial specifications in conditional equational logic
The main definitions and theorems of sections 11.1 to 11.7 may be generalized in a straightforward way for conditional equational logic. In particular, Theorem 11.2 holds for conditional equational logic as well. The use of conditional equational logic often simplifies the design of initial specifications because it allows the expression of implications between equalities instead of equalities only. On the other hand it may be more difficult to grasp the structure of the initial algebra T(E, $) and, in particular, of its carriers; moreover, term rewriting systems and their tools are more complex (see, for example, [Klop, 1992; Padawitz, 1988]).
11.9
Comments
It is tempting to try to generalize initial specifications for logics other than equational and conditional equational logic. The following example illustrates the limits of such a generalization.
Algebraic specification of abstract data types
267
Example 11.32. Let £ be a signature with a single sort and four constants, viz. a, 5, c, d. Let $ be a set consisting of the single formula (of predicate logic): (-.a = 6 A c = d) V (a = b A -ic = d). Then Mod-s($) has no initial algebra. In fact, by Theorem 3.8 an initial algebra A has to satisfy both A(a) / A(b) and A(c) ^ A(d). Hence, it cannot be a model of $.
12
Constructive specifications
While loose specifications and initial specifications have a model-theoretic semantics, constructive specifications have an operational semantics. Hence, constructive specifications allow rapid prototyping by their very definition. The literature proposes several notions of constructive specifications, some of which lack a precise definition. The following general notion encompasses most of them. For more clarity its definition is given in three steps. Definition 12.1 (Constructive specification: abstract syntax). A
constructive specification is a triple sp = (£,4>, fic) where S = (S,fl) is a signature, $ C EL(%) a set of equations and J7C C fi a set of operations called constructors. It is required that the following three constraints are satisfied. In these constraints X is a set of variables for E, Ec = (S, fl c ) and Var(t) denotes the set of variables occurring in a term t. (i) Each equation of $ is of the form n(vt , . . . , V k ) — t with (n: Si x ... x at -> s) £ n — H c , k > 0, *>i € TSc(x),«,. for all i, 1 < i < k, and t € T^X),sMoreover, it must be the case that Var(t) C Var(n(vi,... , r^)). Finally, no variable may occur more than once in n(«i,... ,ut). (ii) For each ground term n(w\,..., Wk) of IE with (n: si x ... x Sk -» s) € ft - ftc, k > 0, u>i € Tzc,si for each i, 1 < i < k, there exists exactly one equation n(v\,... ,Vk) = t 6 $ and exactly one ground substitution a : Var(n(vi,... ,Vk)) —> T^c such that Wi = Via for all i, 1 < i < k. (iii) There exists a reduction ordering "<" such that t < n(v\,..., Vk) for each equation H ( V I , . . . , Vk) = t € $.
Informally, constraints (i) and (ii) make $ look like a (generalized version of a) recursive function definition in the theory of computability. Constraint
268
J. Loeckx, H.-D. Ehrich and M. Wolf
(iii) makes sure that the calculation of the value of a ground term terminates (see Definition 12.3). The concrete syntax for constructive specifications is chosen to be similar to that of an initial specification but with constructive instead of initial and with constr preceding B each constructor in the list- of -operations. Example 12.2. constructive spec sorts nat opns constr 0 : —» nat constr Succ : nat —)• nat - + . : nat x nat —¥ nat vars m, n: nat eqns m + 0 = 0
m + Succ(n) = Succ(m + n) endspec The reader having difficulties checking that constraints (ii) and (iii) of Definition 12.1 are satisfied may wait for the remarks preceding Example 12.6. Definition 12.3 (Constructive specification: semantics). Let sp = (£,$,n c ) be a constructive specification with S = (5, fl). Put Sc = (5, flc). The meaning M.(sp) of sp is the monomorphic abstract data type M(sp) = { A 6 Alg(E) \ A ~ C } where C is the E-algebra defined by: (i) C(s) = T EcjS for each sort s € 5; (ii) C(uj) = n for each constant u> = (n: —> s) € fic; (iii) C(u)(wi,...,Wk) = n(wi,...,Wk) for each u = (n:si x . . . x Sk -> s) € (7C, k > 1, and all tUj € Tsc<Si, (iv)
C(w)(wi,...,wk) for each w = (n:si x . . . xs^ -» s) € fi — n c , k > 0, and all Wi € 7s c>Si , 1 < i < k, where t and a are defined as follows: t is the right-hand side of the uni vocally defined equation n(vi,...,Vk) = t € $ and a : Var(n(vi, . . . ,Vk)) —> TSC is the univocally defined substitution for which vna — Wi, 1 < i < k, guaranteed by constraint (ii) of Definition 12.1.
The semantics M (sp) is constructive in that the specification sp constitutes an explicit definition of the algebra C and, more importantly, because the value C(t) of an arbitrary ground term t € IE may be effectively computed in finite time— as expressed by the following theorem: Theorem 12.4. Let sp = (S,$,fi c ) and C be as in Definition 12.3. For any ground term t 6 TS the computation of the value C(t) is finite and yields a unique result.
Algebraic specification of abstract data types
269
A precise definition of the notion of a computation of the value C(t) and a proof of the theorem may be found in [Loeckx et al., 1996]. Clearly, the proof of finiteness is founded on constraint (iii) of Definition 12.1. A computation resembles term rewriting with the algebra C behaving as a characteristic term algebra. While the semantics of loose specifications is based on model theory and that of initial specification on term algebras and quotients, the semantics of constructive specifications is operational. The following theorem shows a relation between these three semantics. As constructive specifications may be viewed as programs, the theorem also shows that the difference between specifications and programs is not sharp. Theorem 12.5. Let sp = (E,$,n c ) be a constructive specification with E = (5, fi). Call spL the loose specification with free constructors (S, $, S, fic) and spj the initial specification (£,$). Then M(sp) = M(spL) = M(sPl). Of course, due to the constraints of Definition 12.1 constructive specifications are in some sense less abstract than initial specifications. Similarly, loose specification with free constructors allow, for instance, no "abstract" specification of sets (see Examples ll.S.(ii) and Example 13.11). They are less expressive than loose specifications because—similarly to initial specifications—they can only define monomorphic abstract data types. Nevertheless, in concrete "practical" examples the difference between initial and constructive specifications is often purely notational as is illustrated by Examples 11.5(i) and 12.2. A similar remark holds for loose specifications with free constructors where the abstract data type defined is monomorphic (cf. Example 10.9). The design of a constructive specification faces the problem of checking constraints (i) to (iii) of Definition 12.1. Actually, checking constraint (i) is trivial. Constraint (ii) is verified when the left-hand sides of the equations for any operation of fl — Oc constitute a "complete set of constructor patterns". A precise definition of this notion, which is based on the "unfolding" of variables, may, for instance, be found in [Loeckx et al., 1996]. Complete sets of constructor patterns of the operation "+" of Example 12.2 are, for instance:
{n + m}, {n + 0, n + Succ(m)}, {0 + m, Succ(n) + m}, {0 + 0, Succ(n) + 0,0 + Swcc(m), Succ(ri) + Swcc(m)}, {n + Q,n + Succ(0),n + Succ(Succ(m))}. Finally, a sufficient condition for constraint (iii) is the following: the operations of fZ — fic may be written as w i , . . . , wj, I > 0, such that for each equation n(v\,..., Vk) — t of $, u>i — (n : s\ x ... x Sk —> s):
270
J. Loeckx, H.-D. Ehrich and M. Wolf
• for each subterm n(ti,...,tk) of t, each ti is a subterm of Uj, 1 < i < A;; moreover, there exists j, 1 < j• < fc, such that tj is a proper subterm of Wj . The specification of Example 12.2 satisfies these conditions. A more elaborate example is the following: Example 12.6. The example consists of the constructive specification of Example 12.2 with the additional sort bool, the additional constructors True : —> bool, False : -> bool, the additional operations _ < _: nat x nat —> nat, Even : nat —> bool and the additional equations 0 < n = True, Succ(m) < 0 = False, Succ(m) < Succ(n) = m < n, Even(Q) = True, Even(Succ(0)) = False, Even(Succ(Succ(m))) = Even(m). More details on constructive specifications and on the checking of their constraints as well as a generalization for conditional equational logic may be found in [Loeckx et al., 1996; Loeckx, 1997].
13
Specification languages
As already mentioned in section 9, a specification language allows "large" specifications to be constructed out of "small" ones. The basic building blocks in this construction are the specifications obtained by specificationin-the-small, i.e. the atomic specifications designed along the principles presented in sections 10 to 12. The present section is concerned with the principles of specification languages without modularization and parameterization; these concepts will be investigated in section 14. Several more or less tentative specification languages have been proposed in the literature: CLEAR [Burstall and Goguen, 1980], ASL [Wirsing, 1986], ACT ONE [Classen et al, 1993; Classen, 1989], PLUSS [Bidoit, 1989; Bidoit, 1991], Larch [Guttag et al, 1985; Guttag and Horning, 1993], OBJ 3 [Goguen and Winkler, 1988], OBSCURE [Lehmann and Loeckx, 1993], and Spectrum [Broy et al., 1993]. A more detailed overview may be found in [Wirsing, 1995; Sannella and Wirsing, 1999]. A very general language, called CASL, is presently being developed by an international Working Group (see, for instance, [protectSannella, 1999; Mosses, 1999]); the release of the final version of this language scheduled for the end of
Algebraic specification of abstract data types
271
the year 2000. In this section and in section 14 a very simple specification language is described that encompasses the main features of these different languages.
13.1
A simple specification language
The specification language introduced below is a very elementary one and is called SL. Two additional constructs will be added in section 13.2. The specification language is general in that it fixes neither the nature of the atomic specifications nor the logic. According to its abstract syntax, the language is defined as a set of "specifications". With each specification, say sp, is associated a signature S(sp). Definition 13.1 (Abstract syntax of the specification language SL). The set of specifications sp of the specification language SL and their signatures S(sp) are defined inductively: (i) any atomic specification of sections 10 to 12, say at, is a specification; S(at) is defined to be the signature of this atomic specification; (ii) (union) if spi and sp2 are specifications, then
(iii)
(iv)
(v)
(vi)
is a specification with S(spl + sp2) = S(spl) (renaming) if sp is a specification and // : <S(sp) —>• £' a renaming, then (rename sp by n) is a specification with <S((rename sp by //)) = fj,(S(sp)); (forgetting) if sp is a specification, S a set of sorts and fl a set of operations such that (5, 0) C S(sp) and S(sp) — (S, f2) is a signature, then (sp forget (5, fl)) is a specification with S(sp forget (5,17)) = S(sp) — (5, fi); (extension) if sp is a specification, S a set of sorts and ft a set of operations such that S(sp) U (5, H) is a signature, then (extend sp by (5, ft)) is a specification with <S(extend sp by (S, ft)) = S(sp) U (5, H)); (modelling) if sp is a specification and 4> C L(S(sp)) for some logic L, then
(sp model $) is a specification with S(sp model $) = S(sp); (vii) (restriction) if sp is a specification with S(sp) = (5,0), if 5C C 5 and if Qc C fj is a set of operations with target sorts in Sc, then (sp generated in Sc by Qc)
272
J. Loeckx, H.-D. Ehrich and M. Wolf and
(sp freely generated in Sc by flc) are specifications with <S(sp generated in Sc by H c )
= S(sp freely generated in Sc by ttc) = S(sp). It is easy to show that S(sp) is a signature for any specification sp. Additional syntactical conditions may even guarantee that S(sp) is strongly typed (cf. Definition 3.2). It is understood that parentheses may be deleted if no ambiguity arises. Again, in the examples the abstract syntax is replaced by a concrete one. In this concrete syntax a pair (S, ft) consisting of a set of sorts and a set of operations is again written as sorts opns
list-of-sorts list-of-operations
and a set $ of formulas as vars list-of-variables axioms list-of-formulas Finally, a renaming ^ : S —> E' is written sorts si,...,8k opns u > \ , . . . , u j i as sorts s(,..., s'k opns LJ{ ,..., u\ with S i , . . . , s/c and s(,..., s'k sorts of S and E' respectively, k > 0, and with ui,...,u>i and u > [ , . . . ,u[ operations of E and £' respectively, I > 0. The renaming p, = (us, A*fi) defined by this concrete syntax is , ,. _ J s'i if s = Si for some i, 1 < i'. < k, \ s otherwise, j'j
if u = (jjj for some j, I < j < I, if u = (n: Si x ... x sk —>• s) ^ ujj for all j, 1 < j < /.
Note that the concrete syntax has to satisfy several "context conditions" lest /z fails to be a renaming. For instance, the sorts s{,..., s'k have to be pair wise different. The constructs of the specification language are interpreted as operations on classes of algebras. Alternative semantics descriptions will be briefly described in section 13.8. Definition 13.2 (Semantics of the specification language SL). The meaning M. (sp) of a specification sp of the language SL is defined inductively and follows the structure and notation of Definition 13.1:
Algebraic specification of abstract data types
273
(i) the meaning M(at) of an atomic specification at is the abstract data type defined by this atomic specification according to sections 10 to 12; (ii) M(sp! + sp2) = {Ae Alg(S(sPl + sp2)) | (A \ S(sPl}) 6 M(sPl), (A\S(sp2))eM(sp2)}; (ui) X(rename sp by /z) = {A £. Alg(p,(S(sp))) \ (A \ n) 6 M ( s p ) } ; (iv) M(sp forget (5,0)) = (M(sp)) \ (S(sp) - (5,0)); (v) Ai(extend sp by (5,0)) = { A e Alg(S(sp) U (5,0)) | (A \ S(sp)} € M(sp) }; (vi) M(sp model $) = M(sp) n Mods(sp)($)\ (vii) M(sp generated in 5C by Oc) = { A 6 .M(sp) | ^4 is generated in Sc by Oc } .M(sp freely generated in 5C by O c ) = { A € M(sp) I A is freely generated in 5C by Oc }. Informally, the construct "+" builds a persistent extension of M(spl) and M(sp2) (see Definition 5.10(ii)); note that, if S(spl) and S(sp2) are not disjoint, only those algebras of .M(spi) and M(sp2), the common parts of which "coincide", contribute to the result. The construct rename merely performs a renaming of sorts and operations because /j. is bijective. The construct forget drops sorts and operations. The construct extend adds "loose" sorts and operations. The construct model filters out the algebras that are not models of $; a similar remark holds for generated and freely generated. It is easy to show that M(sp) is an abstract data type of signature S(sp) for any specification sp. Prom a user's point of view the construct "+" allows specifications to be put together. The construct rename allows more appropriate names to be introduced or name collisions to be avoided during a subsequent application of the construct "+". The construct forget allows deletion of sorts and operations that are no longer needed. The construct extend together with the constructs model, generated and freely generated allows sorts and operations to be added and the model class to be restricted, respectively. A non-trivial example of a specification is far too lengthy to be presented here. Moreover, the specification language SL is too elementary to allow elegant descriptions. Hence, the goal of the following example is merely to illustrate the concepts introduced. Example 13.3. The atomic specifications used within the following specification are loose ones with free constructors: ((extend (loose spec sorts freely generated bool opns constr True : -4 bool constr False: -> bool
274
J. Loeckx, H.-D. Ehrich and M. Wolf
endspec + loose spec sorts freely generated nat opns constr 0: —> nat constr Succ: nat —> nat endspec) by opns _ < - : nat x nat -» bool) model vars m,n:nat axioms 0 < n = True Succ(m) < 0 = False Succ(m) < Succ(n) = m < n)
13.2
Two further language constructs
The semantics of the constructs extend and model of the specification language introduced above has a loose flavour that is illustrated by the following fact. An atomic loose specification (S, $) has the same semantics as the specification: (extend e by S) model $ where e stands for a specification defining the abstract data type for the empty signature. The present section introduces two additional similar constructs that have an initial rather than loose flavour. It is necessary first to introduce two notions that may be viewed as a generalization of the notion of an initial algebra and of a quotient term algebra, respectively. Their definition is somewhat technical and may be difficult to understand at first reading. A more detailed treatment of the subject may be found in textbooks such as [Loeckx et al., 1996]. Definition 13.4 (Free extension). Let E, E' be signatures with E C S ' . Let A be a E-algebra and C' a class of S'-algebras. (i) A homomorphic extension of A for (E',C') is a pair (B',g) where B' € C' and where g : A —>• B' \ E is a E-homomorphism. (ii) A free extension of A for (E',C') is a homomorphic extension (E',q) of A for (S',C') such that for each homomorphic extension (B',g) of A for (S',C') there exists exactly one E'-homomorphism h' : E' —> B' with g = (h11 E) o q. (iii) If C' = Mods- ($') where $' C L(E') is a set of formulas for some logic L, one speaks of a homomorphic extension and a free extension of A for (£',*')• If the homomorphism q of a free extension (E',q) is known, it is common practice to identify the algebra E' with (E',q) and to speak of the free extension E'.
Algebraic specification of abstract data types
275
E'
ft1 IE
h' B'
Fig. 2. Illustration of Definition 13.4 Informally, a homomorphic extension is an algebra of C' that is related to A by a homomorphism. Prom a free extension both components B1 and g of an extension (B',g) may be "reached" by a unique homomorphism h'. The definition is illustrated by Figure 2. The use of this graphical representation may be helpful in understanding proofs. Clearly, the notion of a free extension for (E',C') coincides with the notion of an initial algebra of C', when the signature E is empty. Definition 13.5 (Quotient term extension). Let £ = (S, fi), E' = (5', fi') be signatures with £ C E'. Let A be a E-algebra and $' C ££(£') a set of equations. Write • 0.4 = { ca : —>• s \ a <E A(s), s 6 5 } where each ca : —>• s is a "new" constant;
• EC = (s, n u oo, E; = (5', n' u oo; • Ac for the Ec-algebra identical with A but with, additionally, Ac(ca '• -* s) = a for each new constant ca : -» s of fi^; • & = $' U Th^Ac) with ThGEL(Ac) = {(t = u)£ ThEi(Ac) \ t,u € TSc } (i.e. Th^^Ac) consists of the ground equations of ThsL^c))The quotient term extension of the E-algebra A for (E', <£') is the S'-algebra A(E', *') defined as
Informally, the quotient term extension ^4(E', $') is similar to the quotient term algebra T(E', $') but possesses in addition all "equational properties" of ^4. This results from the addition of the ground equations of the equational theory of A to $'. This, in its turn, required the introduction of the constants ca allowing "access" to each carrier of A. The following corollary constitutes a generalization of Theorem 11.2. Corollary 13.6. Let E, E' be signatures with E C E'. Let A be a £algebra and let $' C EL(H'). The quotient term extension A(E',$') is a
276
J. Loeckx, H.-D. Ehrich and M. Wolf
model of $'. Example 13.7. Let £ and A be as in Example 4.9. Let 6,61,62: el and /: list be variables and $ = {Add(e,Add(e,l)) = Add(e,l), Add(e1,Add(e-2,l)) = Add(e2,Add(ei,l))}. Consider the quotient term extension A(£, $) of A for (£, $). Then A(£,$) ~ £ with £(e/) = ^(ei), B(/ist) is the set of all finite sets of elements from B(el), B([]) is the empty set and B(Add)(e, s) = sU{e} for all e 6 B(el) and s € B(list). The following theorem shows that all free extensions coincide with the quotient term extension up to isomorphism. It constitutes a generalization of Corollary 11.3. Theorem 13.8. Let £, S' be signatures with £ C £', let A be a ^-algebra and let $' C EL(S'). Let A(Z',&) be the quotient term algebra of A for (£',$') and q : A —» A(E',$') | £ its associated homomorphism. (i) E' ~ A(£', $') for any free extension (E', r) of A for (£', $'). («J IfE' ~ A(£',$') according to the isomorphism i' : A(£',$') —> E', then (E1, (V \ £) o q) is a free extension of A for (£', $'). By the way, the preceding theorem allows a free extension (E',r) to be identified with the algebra E': according to part (ii) of the theorem the homomorphism r is uniquely determined by E1 (cf. the remark at the end of Definition 13.4). Moreover, part (i) of the theorem allows the free extension E' of A for (£', $') to be identified with the algebra -A(£', $')• In building a free extension of an algebra A the carrier sets may be modified by a factorization—as illustrated in Example 13.7. The following property expresses the fact that the carrier sets remain unchanged—up to isomorphism. Definition 13.9 (Persistent free extension). Let £, £' be signatures with £ C £', let A be a £-algebra and let $' C ££(£'). Also let E1 be a free extension of A for (£', $') and i' : A(E', $') —> E' the corresponding isomorphism. Call q : A —> A(£',$') £ the homomorphism associated with the quotient term algebra j4(£',$'). The free extension E' is called persistent when the homomorphism (i1 | £) o q : A —->• E' \ £ is an isomorphism. Finally, it is possible to introduce the "initial counterparts" of the language constructs extend and model. Definition 13.10 (Two further constructs of the specification language SL). The abstract syntax of the specification language of Definition 13.1 is augmented by two cases:
Algebraic specification of abstract data types
277
(viii) if sp is a specification, S a set of sorts and ft a set of operations such that S(sp) U (5, ft) is a signature, then (freely extend sp by (S, ft)) is a specification with S(freely extend sp by (5, ft)) = S(sp) U (5, ft); (ix) if sp is a specification and $ C EL(S(sp)) a set of equations then (sp quotient $) is a specification with S(sp quotient $) = S(sp). The semantics of Definition 13.2 is accordingly augmented by the cases: (viii) M(freely extend sp by (5, ft)) = { £' 6 Alg(Z') | E' is a free extension of A for (S', 0), AeM(sp) } where S' = S(sp) U (5, ft); (ix) M.(sp quotient $) = { E € 4fy(£) | E is a free extension of A for (S, $), where S = Informally, in contrast with the construct extend, the construct freely extend replaces each algebra of M(sp) by its free extensions rather than by all possible extensions. Similarly, while model removes the algebras that fail to be a model, quotient presses all algebras into a mould by factorization. Hence, when no algebras of sp happen to be a model of 4>, (sp model $) is the empty class of algebras while (sp quotient $) consists of algebras with carrier sets consisting of a singleton. More generally, in contrast with the constructs of the specification language, freely extend and quotient may modify the algebras to which they are applied: whenever the free extension is not persistent, freely extend adds carriers and quotient identifies carriers by factorization. Note that an initial (atomic) specification (E, ) has the same semantics as the specification: (freely extend e by (S,ft)) quotient $ where e stands for the specification defining the abstract data type for the empty signature. The following examples illustrate that the construct quotient may also be used to overcome a deficiency of loose specifications with free constructors and of constructive specifications. As already indicated, these specifications fail to lead to an "elegant" or "abstract" description of abstract data types, the carrier sets of which cannot be represented in a "natural
278
J. Loeckx, H.-D. Ehrich and M. Wolf
way" by a term language. The classical example of such a data type is "set" (cf. Example 11.5(ii)). Example 13.11. Informally, the construct quotient makes sets out of lists; the construct rename introduces more appropriate names: rename ((loose spec sorts el, freely generated list opns constr [ ] : — ) • list constr Add: el x list -»• list endspec) quotient (vars 6,61,62: el, I: list
eqnsAdd(e,Add(e,l)) = Add(e,l) Add(ei,Add(e2,l)) = Add(e2,Add(el,l)}))
by sorts list opns [ ]: —» list
Add: el x list -> list as sorts set opns 0 : —» set Add: el x set -» set
13.3
Adding an environment
While a specification language such as the one described above allows specifications to be "structured", a non-trivial specification is a long piece of text. The idea is to cut this text into "small" specifications and to give them names according to a technique that is classical in programming. To this end it is necessary to add an "environment" mapping names into "environment-dependent specifications"—henceforth called "especifications". A specification then consists of an e-specification together with an environment. While this idea is simple, the corresponding definition is somewhat subtle. In fact, as the notions of an "environment" and of an "e-specification" are interrelated, their inductive definition has to be simultaneous. The definition implicitly makes use of a set of names not further defined. Definition 13.12 (Environment, abstract syntax of e-speciflcations). The notions of an environment, of an e-specification, of the signature of an e-specification in an environment and of the relation "is an e-specification for the environment" are defined by simultaneous induction. Note that an environment is defined to be a partial function: (i) an atomic specification, say at, is an e-specification for the everywhere undefined environment; the signature S(at)(env) of this especification in the environment env is the signature of the atomic specification at;
Algebraic specification of abstract data types
279
(ii) if esp1 and esp2 are e-specifications for the environment env, then (espl + esp2) is an e-specification for the environment env; the signature of this e-specification in the environment env is S(espl + esp2)(env) = S(espl)(env)
(JS(esp2)(env);
(iii) to (ix): as (iii) to (ix) of Definition 13.1 and (the first part of) Definition 13.10 but with: • "e-specification for the environment env" instead of "specification" ; • "S(.. .)(env)" instead of "£(...)" where "..." stands for an (e-)specification; (x) if env is an environment and n a name such that env(n) is defined, then n is an e-specification for the environment env; the signature of the e-specification n in the environment env is S(n)(env) = S(env (n)) (env}; (xi) the everywhere undefined function is an environment; (xii) if env is an environment, n a name such that env(ri) is undefined and esp an e-specification for the environment env, then env[esp/n] is an environment (where env[esp/n] is identical with env except for (env[esp/n])(n) •=• esp); (xiii) an e-specification for an environment env is also an e-specification for an environment env1, whenever env C env'. Informally, an e-specification according to this definition is a specification according to Definition 13.1, it being understood that a name is also an e-specification. If esp is an e-specification for the environment env, then env(n) is defined for each name n occurring in esp. Finally, an environment env contains no "recursive calls" in the sense that there exists no sequence HI ,..., rik of names, fc > 1, such that: • env(ni) contains an occurrence of rij + i, 1 < i < k — 1; • env(rik) contains an occurrence of n\. The following definition fixes the meaning of an e-specification in an environment. Definition 13.13 (Semantics of e-specifications). Let esp be an e-
-specification for the environment env. The meaning M.(esp)(env) of esp in env is defined inductively according to cases (i) to (x) of Definition 13.12: (i) M(at)(env) is the abstract data type defined by the atomic specification at; (ii) M(esp1 + espz)(env) = {A 6 Alg(S(esp1 + esp%)(env)) \
280
J. Loeckx, H.-D. Ehrich and M. Wolf
(A | S(esp1)(env)) e M(espl)(env), (A j S(esp2)(env)) € M(esp2)(env) }; (iii) to (ix): similar to (ii) and in accordance with Definition 13.2 and (the second part of) Definition 13.10; (x) Ai(n)(env) = M (env (n)) (env). It is now possible to define a specification language with an environment. This specification language is called e-SL; syntactically it consists of a set of pairs called "specifications" . Definition 13.14 (The specification language e-SL). (i) (Abstract syntax.) A specification of the specification language e-SL is a pair (env, esp) where esp is an e-specification for the environment env. Its signature is S((env, esp)) — S(esp)(env). (ii) (Semantics.) The meaning of a specification (env, esp) of the language e-SL is the abstract data type M((env, esp)) = M(esp)(env). Again, it is possible to introduce a concrete syntax. One then writes decl; esp instead of (env, esp). In this notation decl stands for a string called a "declaration". The set of all declarations decl and the environments £(decl) they describe are defined inductively: (i) the empty string e is a declaration; it describes the environment £(e) = the everywhere undefined function; (ii) if decl is a declaration, if esp is an e-specification for the environment £(decl) and if n is a name for which £(decl)(ri) is undefined, then decl; n is esp is a declaration; it describes the environment £(decl;n is esp) = (£(decl))[esp/n\. Example 13.15. The specification of Example 13.3 may now be written in a more readable way: BOOL is loose spec sorts freely generated bool opns constr True : -> bool constr False : —> bool endspec; NAT is loose spec sorts freely generated nat opns constr 0 : —» nat constr Succ : nat —» nat endspec; extend (BOOL + NAT)
Algebraic specification of abstract data types
281
by opns _ < _ : nat x nat —)• nat model vars m, n: nat axioms 0 < n = True Succ(m) < 0 = False Succ(m) < Succ(n) = m
13.4
Flattening
Flattening consists in "translating" a specification into an atomic specification with the same meaning. The reason for our interest in flattening will become clear in sections 13.5 and 13.6. In the process of flattening, constructs such as rename and forget lead to some minor problems of a syntactical nature. The other constructs of the specification language SL lead to semantical problems, some of which are undecidable. The following comments illustrate some of these problems. For more simplicity they refer to the specification language SL of sections 13.1 and 13.2. Fact 13.16. (i) Let spt = (Si,$!), sp-2 = (£2,^2) be loose specifications (in the sense of Definition 10.1). Then Ai(spl + sp2) = M(sp), where sp is the loose specification ( S i U £ 2 , $ i U $ 2 ) (ii) Let sp = (£,$) be a loose specification in a logic L. Let S be a set of sorts and (7 a set of operations such that £ U (5, fi) is a signature; let \P C £,(£ U (S, Q)) be a set of formulas. Then ^(extend sp by (S, fi) model $) = M(sp') where sp' is the loose specification (£ U (S, ft), * U #). This fact does not hold for initial specifications, loose specifications with (free) constructors or constructive specifications. Fact 13.17. Let sp = (£,$) be an initial specification. Let S and fi be such that £ U (S, H) is a signature and let * C EL(£, U (5, fi)). Then .M(freely extend sp by (5,0) quotient <£) = M(sp') where sp1 is the initial specification (E U (5, fi), $ U *). Again, the fact does not hold for loose specifications or constructive specifications. As flattening has been studied in some detail for initial specifications, additional properties for this particular case can be found in the literature (see, for example, [Ehrig and Mahr, 1985]). A similar approach to deriving so-called "normal forms" of structured specifications can be found in [Wirsing, 1993].
282
J. Loeckx, H.-D. Ehrich and M. Wolf
13.5
Properties and proofs
A first family of properties refers to the constructs of the specification language. Typical properties one may want to prove are: • M(spl + sp2) is a persistent extension of M(spi), i = 1,2 (in the sense of Definition 5.10); • .M(extend sp by (5, fi) model $) is a persistent extension of M(sp); • each algebra of M(freely extend sp by (S, fi) quotient $) is a persistent free extension of an algebra of M(sp) (in the sense of Definition 13.9). While these properties are undecidable in the general case, the latter one has been studied to some extent in the literature on initial specifications (see, for example, [Ehrig and Mahr, 1985]). A further family of properties refers to the abstract data type defined by a specification (cf. section 9). By flattening the specification the proof of such a property is reduced to a proof for an atomic specification. Alternatively, it is possible to develop a calculus for the specification language that reduces the property of a specification to properties of the atomic specifications contained by it. In this way the proof is again reduced to proofs for atomic specifications. An example of such a calculus is given in [Wirsing, 1993; Bidoit et a/., 1999].
13.6
Rapid prototyping
There are two methods that allow the rapid prototyping of a specification. The first method is based on "compilation" and consists in first flattening the specification into an atomic specification that allows rapid prototyping. The second method is based on "interpretation" and makes use of a program that "interprets" the different constructs. This method is particularly simple when the atomic specifications contained by the specification are constructive (see e.g. [Loeckx and Zeyer, 1995]).
13.7
Further language constructs
The specification languages SL or e-SL may be extended in two ways. First, it is possible to introduce additional constructs by defining them as "macros". While these constructs do not increase the expressive power of the language, they may be very useful by simplifying the notation. Second, it is possible to introduce additional language constructs. An example is the construct of "amalgamated union" which constitutes a generalization of the construct "union" (i.e. "+") and which will now be briefly described. Informally, the amalgamated union avoids name clashes through automatic renaming and thus reduces the number of algebras disregarded by "+".
Algebraic specification of abstract data types
283
Definition 13.18 (Amalgamated union: abstract syntax). (i) (Abstract syntax.) If spl and sp2 are specifications and if 5 is a set of sorts and J) a set of operations with (5, 17) C S(spi) C\S(sp2), then (sPi +(s,fi) sPz) is a specification with S(spl +(s,n) sp2) — A*i(«S(sPi)) U M2(<S(«P 2 )); in this definition Hi and /*2 are the surjective signature morphisms univocally defined as follows:
.
/, f*\ - / s if s € S or s PiW - j s> otherwise for all sorts s 6 S(sp1),
s» otherwise for all sorts s e S(sp2), n:m(s1) x ... x if w 6 (7 or u & S(sp2), n':/ii(si) x ... x fj,i(sk) —t/j,i(s) otherwise for all operations w = (n: si x . . . x sfc -> s) e 5(5;?!), fc > 0, • ^(w) defined similarly, where s', s", . .., n', ... are "new", pairwise different sorts and operation names. (ii) (Semantics.) M(sp1 +(s,n) SP2) — and
(
By the way, as "+s,n" may be defined as a macro with the help of "rename" and "+" , it does not increase the expressiveness of the language.
13.8
Alternative semantics description
The semantics of the specification language was defined above as a function M mapping specifications into abstract data types. This method for describing the semantics is called model- oriented. There exist other description methods, two of which will be briefly discussed. Each of these methods naturally leads to a slightly different semantics of some of the language constructs. According to the specification- oriented (also called presentation-oriented) description method the semantics of a specification language is a function AT mapping specifications into atomic specifications defining the same abstract data type. Assume that Af maps specifications into atomic loose specifications or, alternatively, into atomic initial specifications. Two examples of the semantics of the language constructs are: Af(at)
— at for any atomic specification at;
284
J. Loeckx, H.-D. Ehrich and M. Wolf
with E!, £ 2 , *i, $2 defined by ./V(spi) = (£i,*i) and.W(sp 2 ) = (£ 2 ,$ 2 ). An advantage of this description method is that flattening is explicit. A drawback is that the nature of the atomic specifications has to be fixed and, more importantly, the semantics of some constructs may be less transparent. According to the theory-oriented description method the semantics is a function T mapping specifications into theories. Two examples are: T(at) = $* for the atomic specification at = (£, $); T(*PI + sp2) = (T(sPl) U T(sp2))*. A more detailed description of these alternative semantics classically makes use of some additional notions. The following definition is intended to help the reader in consulting the relevant literature. Definition 13.19 (Theory morphism, specification morphism, hierarchical specification). Let L be a logic, // : £ —>• £' a signature morphism and $ C L(E), $' C £(£')• (i) The signature morphism fj, is called a theory morphism and one writes p, : $* —» <J>'* if 4>' is a /^-extension of $ (cf. Definition 7.25). (ii) The signature morphism /z is called a specification morphism and one writes fj, : (£, $) —>• (£', $') if // : $* —» $'* is a theory morphism. (iii) The pair ((£,), (E',$')) is called a hierarchical specification if the signature morphism n is an inclusion and fj, : (E, $) —>• (E', $') is a specification morphism.
14
Modularization and parameterization
The specification languages discussed above have two shortcomings. First, they are concerned with algebras and thus implicitly suggest a bottom-up design of specifications. A more general design would require the possibility of handling incomplete specifications, say "specification pieces". Next, the languages do not support reusability. This would require the possibility of providing "specification pieces" with parameters. Sections 14.1 to 14.3 introduce a notion of "specification piece", henceforth called "module specification" . A facility for parameterization is proposed in section 14.4.
14.1
Modularized abstract data types
Informally, modularized abstract data types are the objects to be specified by the module specifications to be introduced in sections 14.2 and 14.3. Definition 14.1 (Module signature). A module signature is a pair (Si, Se) of signatures; £j and Se are called the import signature and export signature, respectively. A sort or operation from the signature Ej n Se is called inherited.
Algebraic specification of abstract data types
285
A graphical representation of the module signature (E,, E e ) with ,r},{ui,u2}), Ee = ({s},{wi,w3}) is:
The dotted lines characterize the inherited sorts and operations. Definition 14.2 (Modularized abstract data type). (i) A modularized abstract data type for the module signature (Ej,E e ), or (S,, E e )-modwJe for short, is a (total) function F : such that the class F(A) is an abstract data type for each A € Alg(£i)i in this definition Clas9(£e) denotes the "collection" of all classes of Ee-algebras. (ii) A (Sj,£e)-module F is called persistent if for each A e Alg(Si): for each B € It is called consistent if F(^4) ^ 0 for each A € (iii) A (Sj, Ee)-module F is called monomorphic if F(A) is monomorphic for each A € Alg(Ei). Informally, persistency expresses the fact that inherited sorts and operations have the "same" meaning in A and F(A); consistency expresses the fact that the mapping F is "effective" . Clearly, an abstract data type (in the sense of Definition 2.15) may be viewed as a module with an empty import signature.
14.2
Atomic module specifications
The different atomic specifications of sections 10 to 12 are now generalized for module specifications. By way of restriction, only module signatures (Ej, E e ) with Sj C Ee are considered. This restriction is irrelevant because
286
J. Loeckx, H.-D. Ehrich and M. Wolf
the constructs forget and export rename of the specification language of section 14.3 will allow modules with S, £ Se to be obtained. First, the case of loose specifications is treated (cf. Definition 10.1). Definition 14.3 (Loose module specification). Let L be a logic. (i) (Abstract syntax.) A loose module specification in L is a pair msp = ((S,,E e ),$) where (£j,£ e ) is a module signature with Sj C Ee and where $ C L(E e ) is a set of formulas. (ii) (Semantics.) The meaning of the loose module specification msp — ((£;, £ e ), $) is the module M(msp) defined by M(msp)(A) = {B& Alg^e) B \= $ and (B \ £,) ~ A} for each A 6 Alg(Tn). Clearly, the meaning of a loose module specification is persistent but not necessarily consistent. A possible concrete syntax is like that for loose specifications of section 10.1 but with the keyword import preceding each sort and operation of the import signature and with mspec instead of spec. Example 14.4. The following example is similar to Example 10.2: loose mspec sorts list, import bool, import el opns import True: -> bool import False : —>• bool
[ ]: -4 list Add: el x list —>• list - . - : list x list —> list Isprefix: list x list -> list vars l,m,n: list,e: el axioms [ ]./ = / Add(e, l).m — Add(e,l.m) (Isprefix(l,m) — True) = (3n.(m = l.n)) endmspec Hence, S; = ({bool, el}, {True, False}) and Se = ^t\j({list}, {[ ], Add,...}). Informally, the module specification specifies lists. The abstract data types bool and el are to be defined "elsewhere". Clearly, the abstract data type el is intended to be a parameter. Loose module specifications with constructors and/or free constructors may be defined similarly. Next, the case of initial specifications is treated (cf. section 11.1). Definition 14.5 (Initial module specification).
(i) (Abstract syntax.) An initial module specification in equational logic is a pair msp = ((Sj, £«=),$) where (Sj,S e ) is a module signature with EJ C Ee and where $ C EL(T,e) is a set of equations.
Algebraic specification of abstract data types
287
(ii) (Semantics.) The meaning of the initial module specification msp = ((£j,£ e ),$) is the module M(msp) defined by M(msp)(A) — {B^Alg(T,e)\ B is a free extension of A for(E e ,*)} for each A € Alg(Ei) (cf. Definition 13.4). Clearly, the meaning of an initial module specification is consistent; it is persistent only if for each A € Alg(Y,i) the free extension of A for (S e ,$) is persistent (cf. Definition 13.4). Constructive module specifications may be defined similarly. Their meaning is persistent and consistent. By the way, loose module specifications may be simulated by normal specifications in the following sense. If msp = ((Ej,E e ),$), £{ C £ e , is a loose module specification, consider the loose specification sp = (E e , <£). Then M(sp) = {M(map)(A) \ At Alg^}}. In other words, a loose module specification differs from a loose specification only by the fact that the sorts and operations to be specified "elsewhere", i.e. those of £$, are explicitly marked by the keyword import. Note that this remark does not hold for initial or constructive module specifications.
14.3
A modularized specification language
An elementary modularized specification language, called MSL, is presented. Like the specification language SL presented in sections 13.1 and 13.2, it is not designed for practical use but merely tries to capture the main features of the modularized specification languages presented in the literature. Again, the following definition associates with each module specification msp a module signature S(msp). The goal of the different "context conditions" contained in the definition is essentially to avoid name clashes between the export and import signatures and thus preserve the property of persistency. Definition 14.6 (Abstract syntax of the modularized specification language MSL). The set of module specifications msp of the language MSL and their module signatures S(msp) are defined inductively: (i) any atomic module specification atm is a module specification; S(atm) is the module signature of this atomic module specification; (ii) if mspl and msp2 are module specifications with <S(mspj) — (Sit, £i e ), S(msp2) = (£ 2 i,£2e) and if • each sort and each operation of H\e fl S2i is inherited in • each sort and each operation of E 2e n SH is inherited in
J. Loeckx, H.-D. Ehrich and M. Wolf
288
\[^
e U S2e
ji
Se
1l
m^ »P2
/i
Ei.
msj»,
77k!Oo
1
yl
'£1,
i1 E mt'Pi
S2i
I ^i
^'
j
EH US? . (a) Illustration of Wispj + msp^
VI.
(b) Illustration of msp2
Fig. 3. Illustration of the syntax of the constructs "+" and "o" of the modularized specification language MSL. S(msp2), then
(msp1 is a module specification with S(mspl + msp2) = (E^ U £2*, Sie U S2e) (cf- Figure 3(a)); (iii)
if mspi and msp2 are module signatures with t^mspj) = (£j,E), S(msp2) = (E,E e ), then
is a module specification with S(msp2°msp1 ) = (£j, E e ) (cf. Figure
(iv)
if msp is a module specification with // : Se —^ E' is a renaming such that or operation so with ^i(so) ^ so, then
= (Ei,S e ), if Si for each sort
(rename msp by /j.) is a module specification with <S (rename msp by /x) = (Ej, /i(E e )); (v)-(x) the constructs forget, extend, model, generated, freely generated, freely extend and quotient are defined similarly to (iv) (cf. Definition 13.1(iv) to (vii) and Definition IS.lO(viii) to (ix)).
Algebraic specification of abstract data types
289
Definition 14.7 (Semantics of MSL). The semantics M(msp) of a module specification msp is a module (in the sense of Definition 14.2) and is defined inductively according to Definition 14.6: (i)
(ii)
the semantics M(atm) of an atomic module specification aim is the module defined by this atomic module specification (see section 14.2); if 5(mspj) = (£ij, Si e ) and <S(msp2) = (£ 2i , £ 2e ), then M(mspl + msp2)(A) =
(iii)
{ B € Alg(Zle U S2e) | (B \ £ le ) € M(msPl)(A \ E«), (B|£2e)eM(msp2)(.4|£2i)}
for all A £ Alg(Zu U £ 2i ); if 5(msp!) = (£«,£) and 5(msp2) = (£,£ e ) then ./M(msp2 o msp1)(A) =
(iv)
(^J
.M(msp2)(.B)
for all A e Alg(Zi); if S(msp) = (Ej,S e ), then (rename msp by
for all A e (v)-(x) similar to (iv) (cf. Definition 13.2(iv) to (vii) and Definition 13.10(viii) to (ix)). The reader may easily check that these different definitions are consistent. The different language constructs, except for freely extend and quotient, may be shown to preserve persistency (cf. the remark following Definition 13.10). The constructs also preserve consistency except possibly +, model, generated and freely generated; the construct + preserves consistency, when applied to persistent specifications. As announced above, the constructs rename and forget may lead to module specifications, the import signature of which is not a subsignature of the export signature. An environment may be added in a way similar to that of section 13.3. Example 14.8. Let msp denote the atomic module specification of Example 14.4. An example of a module specification with environment is: LIST is msp; BOOL is loose mspec sorts freely generated bool opns constr True : —> bool constr False : —^ bool endmspec; LIST o BOOL
290
J. Loeckx, H.-D. Ehrich and M. Wolf
The import signature of this module specification is empty. Hence, it may be viewed as a specification with signature ({bool, el, list}, {True, False, [ ], Add,...}) defining lists of "elements". Further comments on the modularized specification language may be found in section 14.5.
14.4
A parameterized specification language
The parameter mechanism introduced in this section is a very elementary one. It allows the simulation of most of the parameter mechanisms that are described in the literature (see section 14.6). According to this parameter mechanism, a parameter is a distinguished sort or operation of the import signature of a module specification. In the module specification of Example 14.4 the imported sort el is predestined as a parameter in contrast with the imported sort bool. The reason is that the sort el is concerned with reusability while bool is concerned with modular design. More precisely, the intended meaning of bool is fixed while the meaning of el is intentionally left pending. In fact, it makes sense to use the module specification with different meanings for el, for instance natural numbers, strings or lists of lists of natural numbers, but it is not sensible to use the module with a meaning for bool other than the intended one. By the way, the difference between imported sorts or operations that are parameters and those that are not is similar to that between parameters and global variables in the procedure body of an imperative language. As a result, imported sorts and operations that are parameters and those that are not do not differ in their semantics but merely differ in their intended use. Providing a module specification language with a parameter mechanism may therefore be reduced to the introduction of two additional language constructs called import rename and import model, respectively. The first of these constructs allows parameter passing by renaming the (imported sorts and operations that constitute the) formal parameters into their actual values. In the case of Example 14.4 it allows one to rename the sort el into, for instance, nat or string. The second construct allows one to put semantic constraints on the parameters. For instance, a module specification of "ordered lists" that is parameterized in the sort of its elements requires that the carriers of this sort satisfy the axioms of a partial order—as will be illustrated in Example 14.13. The following two definitions introduce a parameterized specification language called PSL. This language is identical with the module specification language MSL except for the two additional language constructs mentioned above. The definition of the construct import rename is slightly more complex than the above comments may suggest. While the construct introduces
Algebraic specification of abstract data types
291
"new" names for the sorts and operations of the import signature, it generally modifies the export signature too. In fact, the inherited sorts and operations in the export signature have to be renamed accordingly. The same holds for the inherited sorts occurring in the arities of the exported operations. For this reason the signature morphism is defined as a signature morphism fj,: Ej U Se —» E' rather than // : EJ —> E'. Definition 14.9 (Abstract syntax of the parameterized specification language PSL). The set of parameterized specifications psp of the language PSL and their module signatures S(psp) are defined inductively: (i)-(x) as Definition 14.6(i) to (x) but with "parameterized specification" instead of "module specification"; (xi) if psp is a parameterized specification with S(psp) = (Ej,S e ) and if n : Ej U Ee —t E' is a surjective signature morphism satisfying the following four conditions: (a) for each sort s from E e \Ej, n(s) = s, (b) for each operation u> from E e \£j, /x(w) and w have the same operation name, (c) for any two different sorts or operations so\e and so-ze from S e , n(soie) — n(so?,e) implies that both so\e and so^e are inherited, (d) for any sort or operation so, from Ej and soe from S e , l_t(soi) = n(soe) implies that soe is inherited, then (import rename psp by /x) is a parameterized specification with ^(import rename psp by p.) = (/i(Sj),/x(S e )); (xii) if psp is a parameterized specification with S(psp) = (Sj,£ e ) and if $ C L(Ej) is a set of formulas for some logic L, then (psp import model *) is a parameterized specification with S(psp import model $) = S(psp). Informally, conditions (xi)(a) and (xi)(b) express the fact that /j, constitutes a renaming of the import signature. More precisely, condition (xi) (a) expresses the fact that n renames sorts from Se only if they are inherited. Condition (xi)(b) expresses the same property for operations or, at least, for their names; the condition does not extend to their arities: if a noninherited operation u from Se contains imported sorts in its arity, w and n(w) may differ from each other in their arities. Conditions (xi)(c) and (xi)(d) avoid "name clashes". More precisely, condition (xi)(c) expresses
292
J. Loeckx, H.-D. Ehrich and M. Wolf
the fact that p. is injective on the non-inherited sorts and operations from Se. Condition (xi)(d) expresses the fact that p, may identify a sort or operation from £, and a sort or operation from Se only if the latter is inherited. Note that the signature morphism n is not necessarily bijective and hence may fail to constitute a renaming in the sense of Definition 5.1(ii). This is sensible because it must be possible that two different formal parameters get the same actual value (see Example 14.11). In the concrete syntax one uses pspec and endpspec instead of mspec and endrnspec. Definition 14.10 (Semantics of PSL). The meaning M(psp) of a parameterized specification psp is a module (in the sense of Definition 14.2) and is inductively defined according to Definition 14.9: (i) to (x) as in Definition 14.7 but with "parameterized specification" instead of "module specification"; (xi) ifS(psp) = (Si.Se), then M (import rename psp by n}(A) = { B 6 Alg(»(Ee)) | (B | (/z|EJ) € M(psp)(A \ (/i| Ej )) } for each A 6 Alg(fj,(Si)); (xii) if S(psp) = (Ef,E e ), then ... . . , / M(psp)(A) W/n M(psp import model1 A$)(/!) = < „ v (0 for each A E. Alg(£i).
ifA\=$, ,, • otherwise
The meaning of the construct import rename is as expected. The effect of the construct import model is to "eliminate" the imported algebras that fail to satisfy $. The construct import rename preserves persistency and, when applied to persistent specifications, consistency. The construct import model preserves persistency but not necessarily consistency. Again, the addition of an environment presents no problem. Example 14.11. A "declaration" of a parameterized specification is PAIR is loose pspec sorts freely generated pair, import eli, import el? opns constr [_ , _ ]: el\ x el2 -* pair First :pair —>• el\ Second : pair ->• el2 vars e\: eli, e2: el-2 axioms Fz'rsi([e1;e2]) = ei Second([ei,e2]) = e2 endpspec An instantiation of this parameterized specification is
Algebraic specification of abstract data types
293
import rename PAIR by sorts e l i , eli as sorts nat, not The instantiation specifies pairs of natural numbers or, more precisely, of carriers of sort not. Its module signature (£,, S e ) is as expected: £i = ({nat},®), T,e = ({nat,pair}, {[_]: nat x nat ->• pair, First -.pair ->• na£, Second :pair —> nat}). The example illustrates that import renaming must not be injective. The notation for an instantiation is clumsy and requires repetition of the formal parameters. By storing the formal parameters in the environment one may adopt a "sugared" notation similar to that of programming languages: the formal and actual parameters are written between brackets after the name of the parameterized specification. This notation has the additional advantage that it explicitly distinguishes between parameters and imported sorts and operations that are not parameters. Example 14.12. The parameterized specification of Example 14.11 may now be written: PAIR (sorts eli, el2) is loose pspec sorts freely generated pair, import eli, import e/2 opns [_ , _ ] : . . . endpspec Similarly, the instantiation is written: PAIR (sorts nat, nat) The following example illustrates the use of import model for expressing parameter constraints. Example 14.13. The following parameterized specification defines ordered lists. In it the construct import model makes sure that the relation "C" is a partial order. ORDERED-LISTS (sorts el, opns _ C _ : el x el ->• boot) is ((loose pspec sorts freely generated list, import bool, import el opns import True: —> bool import False : -4 bool import _ C _: el x el —t bool constr [ ]: -> list constr Add : el x list -¥ list Is-ordered : list —»• bool vars e,e\,e^: el, I: list axioms Is-ordered([ ]) = True
294
J. Loeckx, H.-D. Ehrich and M. Wolf Is-ordered(Add(e, [ ])) = True (d C 62) = True D Is- ordered ( Add (ei, Add (e^, I))) = Is-ordered(Add(e<2,l)) (e\ C 62) = Fafae D endpspec) import model vars 6,61,62,63: d axioms (e C e) = TVwe (ei C e 2 ) = Trwe A (e2 C e3) = True D (ei C 63) = True (ei C e 2 ) = True A (e2 C ei) = True D ei = e 2 )
Now let NAT is (loose pspec sorts
freely generated boot, freely generated nat opns constr True : —> bool constr False : —> 6ooZ constr 0 : —> nat constr SWcc : nat —> na£ _ < _ : nat x nai —>• bool vars m, n: nai axioms (0 < n) = TVue (5wcc(m) < 0) = Fafee (Succ(m) < Succ(ri)) = (m < n) endpspec)
Then ORDERED-LISTS (sorts not, opns <:natx nat -> 600?) o is a specification of ordered lists of natural numbers. Of course, this specification makes sense only if the module it defines is consistent (in the sense of Definition 14.2). Hence, it is necessary to prove that "<" satisfies the axioms of "C". By the way, if the reader is still irritated by the clumsy notation he should remember that the specification languages presented are not intended for "practical" use.
14.5
Comments
The remarks on flattening, properties and proofs, rapid prototyping and further constructs of sections 13.4 to 13.5 carry over to the specification languages MSL and PSL. By way of a restriction rapid prototyping is not possible as long as the meaning of the imported sorts and operations is not fixed.
Algebraic specification of abstract data types
14.6
295
Alternative parameter mechanisms
The literature contains a large number of proposals for modularized and parameterized specification languages and, in particular, for parameter mechanisms (see, for example, [Ehrig and Mahr, 1985; Ehrig and Mahr, 1990; Ehrich et al., 1989; Burstall and Goguen, 1980; Wirsing, 1986; Broy et al., 1993; Lehmann and Loeckx, 1993]). Most of them differ from the languages presented here by being less "elementary". The interface, for instance, generally consists of signatures or even specifications rather than single sorts and operations. Two parameter mechanisms are now roughly sketched. In the X-calculus approach a parameter consists of a complete specification. More precisely, a parameterized specification is of the form XX:par.sp
(14.1)
where par and sp are specifications of a specification language with environment—such as the specification language PSL (with an environment)—and where X is a name. It is understood that X may occur in the specification sp. Informally, the notation "A": par" indicates that actual parameters have to be of "type" par, i.e. have to belong to the abstract data type defined by par. Somewhat more precisely, (14.1) is equivalent to the following parameterized specification of PSL: • the environment is extended by the declaration
X is ... where ... stands for the loose atomic module specification ((S(par), S(par)), Th(M(par))); informally, this module specification leaves the signature <S(por) unchanged but adds the formulas of the theory of the abstract data type defined by par; • sp is turned into a parameterized specification of PSL by defining the sorts and operations of S(par) as imported sorts and operations or, more precisely, as parameters. Alternatively, one may define ... to stand for the loose atomic module specification ((S(par),S(par)),®) and add Th(M(par)) to the parameterized specification obtained from sp with the help of the construct import model. The following approach is called the pushout approach because the parameter passing mechanism is defined as a pushout in an appropriate category. The approach was first described in [Ehrich and Lohberger, 1979]. It makes use of the notion of a specification morphism (introduced in Definition 13.19(ii)). To simplify the description of the approach it is assumed that all specifications are atomic ones or, alternatively, have been turned into atomic ones by the meaning function J\f of section 13.8. A parameterized specification is now a specification morphism
296
J. Loeckx, H.-D. Ehrich and M. Wolf
par
'«r
7T
''
sp(par)
\ -^ /
Fig. 4. The specification sp(act) is the pushout object of the specification morphisms •K and /z. TT : par '-^ sp(par) that is an inclusion. The specification par = (E p0 rj3> P ar) constitutes the formal parameter; the specification sp(par) = (£ sp ( pa r)>$s P ( P ar)) constitutes the parameterized specification proper. The notation sp(par) is intended to suggest that par is "part of" sp(par) according to the inclusion ir. Note that £ par C £ sp ( par ) and $*paT C $* , ar j because TT is a specification morphism. Parameter passing is defined as a specification morphism : par
act
where act = (Soct^act) is the specification constituting the actual parameter. Note again that (/i($par))* C $*c< because p. is a specification morphism; this forces the actual parameter act to "respect" the requirements 4>par imposed on par. Finally, the effect of parameter passing is a specification sp(act) that is characterized as the pushout object of TT and fi in the category of specifications and specification morphisms (see Figure 4). This pushout object is unique up to isomorphism and its existence is granted in the usual category of specifications and specification morphisms. Roughly speaking, the pushout of TT and fi constitutes the "minimal completion" to a commutative square. It yields the specification morphism p! : sp(par) —>• sp(act) describing the relationship between the specifications before and after replacing par by act, and the specification morphism TT' : act <—> sp(act) describing the embedding of the actual parameter into the resulting specification. The semantic counterpart of the pushout construction is given by the amalgamation lemma ([Ehrig and Mahr, 1985], p. 217). The class M(sp(act)) consists of the £sp(act)-algebras C for which (C | TT') = A and (C | //) = B with A e M(act), B e M(sp(par)) and (A | fj.) = (B | TT). Informally, (A\ fji) = (B \ ?r) requires that the parameter components of A and B coincide; C is the algebra B with the par component of B replaced by A. A similar property holds for the £sp(acj)-homomorphisms. It is not difficult to simulate the pushout approach in the specification
Algebraic specification of abstract data types
297
language PSL when the parameter passing /j, is surjective. The renaming effect of parameter passing may then be simulated by the import rename construct together with the (surjective) signature morphism A*
:
Essentially, this signature morphism performs the renaming of the import signature Epar into E OC ( and of the inherited sorts and operations in the export signature £ sp ( por ). The condition that the different morphisms are specification morphisms may be taken care of by the construct import model — as in the case of the (simulation of the) A-approach. When the parameter passing \i is not surjective the sorts and operations of SaC( — M(Epar) nave to t>e added to the result of the import rename construct as inherited sorts and operations. This may require a preliminary renaming to avoid "name clashes" . Clearly, one of the advantages of the pushout approach with respect to its simulation in PSL is that all renamings are performed implicitly.
15
Further topics
The present section discusses topics the treatment of which has not (yet) reached the level of maturity of the topics treated above. The reader interested in more details is referred to the bibliographic notes of section 15.7.
15.1
Behavioural abstraction
It is a fundamental principle of specification not to distinguish between "similar" algebras. Clearly, isomorphic algebras are "similar" as they possess, for instance, the same theory. For this reason abstract data types were defined as algebra classes closed under isomorphism. The interest in monomorphic abstract data types exhibited throughout this chapter may suggest that, conversely, "similar" algebras are necessarily isomorphic. The following example shows that in some situations it may be sensible to use a notion of "similarity" that is more general than that of isomorphism. Example 15.1. Let £ = (S, fi) be a signature with S = {bool,nat,set} and H = {0 : —» nat,Succ:nat —>• nat, 0 : —» set, Add: nat x set —>• set, . € . : nat x set -> bool}. Let A and B denote the "classical" algebras of respectively sets and lists of natural numbers. More precisely: • A(bool) = B(bool), A(nat) = B(nat), A(0) = 5(0), A(Succ) = B(Succ) are classical; • A(Q) is the empty set, 5(0) is the empty list; • A(Add) adds a number to a set, B(Add) adds a number to the front of a list; checks whether a number is in a set — and similarly for -B(e).
298
J. Loeckx, H.-D. Ehrich and M. Wolf
Clearly, A and B are not isomorphic and even fail to have the same theory. For instance, putting t = Add(0, 0), u = Add(0, Add(0: 0)), one has A(t) = A(u), but B(t) ^ B(u). On the other hand the algebras A and B behave "similarly" with respect to the operation "e". More precisely, it is easy to show that A(t) = B(t) for all ground terms t € Ts^ooi of sort bool. Informally, this property ensues from the fact that membership depends neither on the order of insertion nor on the number of times an element has been inserted. Hence the algebras A and B are "similar" whenever one restricts attention to the membership problem or, put another way, to the effect of set on the "outside world" of bool.
This notion is now defined formally. Definition 15.2 (Behavioural equivalence). Let £ = (S, J7) be a signature and So C S a set of sorts called observable sorts. Two S-algebras A and B are said to be behaviourally equivalent on SQ, if for each sort s € 5o, for all ground terms t, u € TE<S, One then writes A «s0 B. Example 15.3. Let A and B be the algebras of Example 15.1. Let C similarly be the "classical" algebra of multisets (i.e. "sets" in which elements may occur more than once). Then A f^{booi} B ~{&0oJ} C. The following corollary states that behavioural equivalence generalizes isomorphism. Corollary 15.4. Let E = (5, ft) be a signature, SQ C S be a set of sorts and A and B be 'Z-algebras. (i) IfA~B, then A wSo B. (ii) If A «5 B and both A and B are generated algebras, then A ~ B. Definition 15.5 (Behavioural abstraction). Let S = (5, fi) be a signature, So C S & set of sorts and C C Alg(^) an abstract data type. The behavioural abstraction of C with respect to SQ is the class of E-algebras Behav(C, S0) = { A E Alg(S) \ A wSo B for some B e C }. By Corollary 15.4 a behavioural abstraction is again an abstract data type. The following theorem states that a behavioural abstraction of a monomorphic abstract data type contains final algebras (Definition
Algebraic specification of abstract data types
299
Theorem 15.6. Let S = (5,0) be a signature, So C 5 a set of sorts and C C Alg(S) a monomorphic abstract data type. Then Behav(C,Sg) contains final algebras. Example 15.7. Let A, B, C be as in Example 15.3. Put C = { D € Alg(S) \D~A}. Then A, B and C all belong to Behav(C, {bool}). It is possible to prove that A is final in C. Theorem 15.6 allows one to introduce one more specification method. One should remember that final algebras are unique up to isomorphism (Theorem 2.14(ii)). Definition 15.8 (Final specification). (i) (Abstract syntax.) A final (or behavioural) specification is a triple sp = (£, 4>, SQ) where (£, $) is an initial specification with £ = (5, fi) and where 5o C 5. (ii) (Semantics.) The meaning .M(sp) of a final specification sp = (£, $, 5o) consists of the class of final algebras of Behav(C, So) where C is the (monomorphic abstract data type that constitutes the) meaning of the initial specification (£,$). Example 15.9. Let A be the algebra of sets and B the algebra of lists of Example 15.1. If (£, 3>) is an initial specification for B, the final specification (£, $, {boot}) is a specification for A. Informally, this means that the effect of behavioural abstraction is the same as that of the equations in an initial specification
Add(n,Add(n,s)) = Add(n,s) Add(n, Add(m, s)) = Add(m,Add(n,s)) distinguishing sets from lists.
15.2 Implementation Informally, an implementation consists in the design of an efficient program for a given specification. This design includes, in particular, the transformation of a possibly loose non-constructive specification into a constructive one. It generally also includes a change of data types. The implementation of data types by other more "concrete" ones is briefly discussed in the present section. To abstract from specification details, the notions are defined on data types rather than on specifications. Example 15.10. A classical example of an implementation is that of stacks by arrays and integers. Informally, the contents of a stack of depth n are stored in the elements A[l],..., A[n] of an array A; the integer has the value n and thus "points" to the topmost stack element. The operation Push is implemented by writing the new element into A[n + 1] and by adding one to the integer. The operation Pop is implemented simply by subtracting one from the integer.
300
J. Loeckx, H.-D. Ehrich and M. Wolf
Clearly, an implementation of stacks by arrays and integers is "sensible" : the abstract data types array and integer are more "concrete" than the abstract data type stack because they are available in any imperative programming language and thus are "nearer" to a program. The example above illustrates three features of an implementation: (i) a data type is in general implemented by several data types rather than by a single one; in the example stack is implemented by array and integer; (ii) there does not necessarily exist a one-to-one relation between a data type and its implementation; in the example arrays differing only by the elements beyond the element pointed to by the integer represent the same stack; (iii) the implementing data types may get values that do not represent values of the implemented data type; in the example an integer may, for instance, have a negative value. At first glance one may think that the notion of behavioural equivalence introduced in section 15.1 may capture the notion of implementation. Actually, this does not work because of feature (iii) above. The following definition is intended to capture features (ii) and (iii). Definition 15.11 (Realization). Let A and B be S-algebras for some signature E. The algebra B is said to realize A if A is isomorphic to a quotient of a subalgebra of B. Informally, the definition accounts for feature (iii) by means of the subalgebra and for feature (ii) by means of the quotient. The next definition is intended to capture feature (i). The specifications used in it are from a specification language such as SL. Definition 15.12 (Construction term). Let Bi,...,Bm be algebras for signatures E 1 ; . . . , E m , m > 1. Let spl,..., spm be strictly adequate specifications of Bi,...,Bm (cf. Definition 9.1). A construction term of BI, ..., Bm is a monomorphic specification sp that: • contains (occurrences of) the specifications spl,...,spm instead of (occurrences of) atomic specifications; • uses only the constructs "+", rename and forget. Informally, sp is obtained by "putting together" sp1,..., spm. Finally, it is possible to introduce the notion of an implementation of an algebra by other algebras. Definition 15.13 (Implementation). Let A,Bi,...,Bm, m > 1, be algebras and let sp be a construction term of BI, ..., Bm. The specification sp is said to constitute an implementation of A by the basis BI ,..., Bm if each algebra of the abstract data type M(sp) defined by sp realizes A.
Algebraic specification of abstract data types
301
The condition "sp realizes A" in this definition constitutes the correctness criterion of the implementation. The investigation of this criterion is one of the topics in the study of implementations. The notion of implementation introduced above is for algebras. A generalization of this notion for abstract data types and modularized abstract data types is straightforward. This generalization naturally leads to a notion of implementation for specifications and parameterized specifications. An interesting question is whether the realization "is implemented by" is transitive. Another question is whether the relation is compatible with the constructs of a specification language.
15.3
Ordered sorts
It is common in mathematics that some data types are "subtypes" of others. For example, non-zero natural numbers constitute a subtype of natural numbers which in turn constitute a subtype of integers. These subtypes "inherit" functions from their supertypes. More precisely, if / : A —> B is a function, if C C A and if B C D, then the function g : C —»• D defined by 9(c) = f ( c ) f°r all c e C is said to be obtained by inheritance. These notions are now defined formally. Definition 15.14 (Order-sorted signature). An order-sorted signature is a triple E = (S, ft, <) such that the following conditions hold true: (i) (S, ft) is a signature in the sense of Definition 2.1; (ii) < is a (reflexive) partial order on S; (iii) whenever H:SI x ... x Sk -* s 6 fl, k > 0, s[,..., s'k, s' € S, s 'i < s i j • • • > s fc < Sfc, s < s', then n: s[ x ... x s'k —> s' € n too. Note that an order-sorted signature in general fails to be strongly typed (see Definition 3.2). Example 15.15. An example of an order-sorted signature is the triple S = (S, fl, <) with 5 n
= =
{nat, int, bool}, {0:->nai, Succ: nat —>• nat, Succ : int —> int, Pred: int -> int, Root: nat -4 nat, True : -4 bool, False : -y bool, Eq: int x int -y bool } U f l c , < = {(nat, int)} Li {(nat, nat), (int, int), (bool, bool)} where, according to Definition 15.14(iii),
302
J. Loeckx, H.-D. Ehrich and M. Wolf nc
=
{ 0: -> int, Succ: nat —> int, Pred: nat -4 int, Root: nat —>• int, Eq: nat x int —> bool, Eq: int x nat —>• bool, Eq: nat x nat —> bool }.
Definition 15.16 (Order-sorted algebra). Let E = (S, Q, <) be an order-sorted signature. An order-sorted algebra for S, or ^-algebra, is an (5, fi)-algebra (in the sense of Definition 2.3) with the following conditions holding true: (i) (Subtype condition.) A(SI) C Afa) for any si,S2 £ S with si < s 2 . (ii) (Inheritance condition.) whenever <jj = (n: «i x ... x Sk —> s) € ft and w' = (n: s[ x . . . x s'k -> s') € ft with sj < si,..., s'k < Sk, s < s', k>0, then A(u')(ai ,...,ak) = A(ui)(ai,. ..,ak) for all ai 6 A(s'1), ...,ake A(s'k). Example 15.17. Referring to Example 15.15, a E-algebra is the "classical" algebra given by the natural numbers and the integers—both equipped with the number zero as well as the successor, predecessor, square root and equality functions—and the truth values—equipped with the values true and false. Homomorphisms and terms may be defined in the usual way. As the signature is in general not strongly typed, a term may have different sorts. It should also be noted that some terms are no longer correctly typed that one would consider intuitively so. The term Root (Pred (Succ(Q))), for example, is not correctly typed, because the subterm Pred(Succ(0)) only has the sort int. This is unintuitive because in the "classical" algebra the evaluation of this "term" is—intuitively speaking—unproblematic. Logic may be defined in the usual way. By way of an important result, the initiality results of section 11 remain valid. Moreover, the notion of term rewriting in the same section may be easily generalized for ordered sorts while keeping the main properties. An example of application of ordered sorts is given in section 15.4.
15.4
Exceptions
Informally, an exception is a situation in which a computation cannot be pursued. Classical examples of exceptions are division by zero, popping an empty stack, computing the predecessor of zero in N 0 , etc. Imperative programming languages handle exceptions as "emergency exits" by means of a jump statement. The treatment of exceptions in functional programming
Algebraic specification of abstract data types
303
languages is less trivial. Hence it is not surprising that there is no simple solution to the exception problem in algebraic specification. There exist essentially four different solutions to the exception problem in algebraic specifications: a logical solution, a partial algebra solution, an error solution and an order-sorted solution. In the logical solution the treatment of an exception is left pending by the use of loose specifications. Such a specification, for instance, axiomatically fixes the value of the predecessor function only for non-zero arguments (Example 10.10). In the partial algebra solution a function with exceptions is modelled as a partial function. Popping the empty stack, for instance, leads to the undefined value. In the error solution the partial algebra solution is simulated by total algebras. To this end an extra carrier is added to each carrier set: it is called Error and represents the undefined value. This approach leads to specifications with additional formulas fixing the function values in case one or more arguments are Error — as illustrated by equations (11.2) and (11.3) in Example 11.7. If one adopts the strictness convention, a function value is Error whenever one of the arguments is Error and the corresponding formulas need not appear in the description of the specification. Of course, these formulas may not be ignored in proofs or in rapid prototyping. Unfortunately, there exist operations that are not compatible with the strictness convention. An example of such an operation is if-then-els e-fi: if True then ... else Error fi should yield ... as a value, not Error. By way of a conclusion, the error solution leads to essentially larger sets of formulas that harm the transparency of specifications and make proofs more elaborate. The order-sorted solution uses order-sorted algebras. The following example illustrates the idea. Example 15.18. Let £ = (S, ft,<) be the order-sorted signature with S = {el, stack, nestack} (where nestack stands for "non-empty stack"), with fi = {Emptystack : —>• stack, Push : stack x el -> nestack, Pop : nestack —¥ stack} U fJc and with "<" defined by nestack < stack. As in Example 15.15, fic accounts for Definition 15.14(iii); in the present case fic = {Push: nestack x el -> nestack}. An order-sorted E-algebra A is defined by: A(stack) = the set of all stacks with elements from A(el), A(nestack) = A(stack) — {the empty stack}, A(Emptystack) = the empty stack,
304
J. Loeckx, H.-D. Ehrich and M. Wolf A(Push), A(Pop): as expected.
The main difference with the "classical" (partial) algebra of stacks is that A(Pop) is now a total function, its domain being A(nestack) rather than A(stack). A precise treatment of the order-sorted solution is outside the scope of this overview. Let it suffice to note that a sort SQK is added for each "concerned" sort s. This sort is "safe" in that it does not lead to errors. In the above example nestack stands for stack OK •
15.5
Dynamic data types
Data types are "static" structures consisting of sets and functions. Pieces of software, however, nearly always contain "dynamic" parts consisting of processes operating on data. As long as these processes are algorithms, they can conveniently be described in a functional style that is compatible with algebraic data type specification, i.e. the algorithms are specified as functions within data types. However, algebraic specification as described in this chapter reaches its limits when the processes are reactive, i.e. when they have—possibly elaborate—interaction structures with the environment, and when they are concurrent, i.e. when several processes act independently and autonomously. Dynamic data types constitute an extension of "static" data types that is intended to overcome these limits. Several approaches have been proposed in the literature. The ones most widely discussed so far are based on the idea that concurrent processes are simply data. Basically, these approaches introduce an imperative flavour into the functional style of abstract data type specification by introducing sorts for process data—such as states and actions—and process operations—such as state transitions. On the semantic level some of these approaches use infinitary extensions of algebras—such as continuous algebras or projection spaces—and a form of behavioural semantics. In order to get the flavour of the dynamic data type approach, the basic concepts of one of the earliest and most elaborate approaches in this area, viz. SMoLCS [Astesiano and Reggio, 1987], are briefly outlined. Definition 15.19 (Dynamic signature). A dynamic signature is a triple (E,S 0 ,n) where: • S = (5, (7) is a signature in the sense of Definition 2.1, • S0 C 5 is a set of sorts called dynamic sorts, • II is a set of transition predicates of the form n:si x ... x Sfc
where n is a transition name and $1,..., Sk 6 S, k > 1. It is required that for each dynamic sort d € S0:
Algebraic specification of abstract data types
305
(i) there is a sort l-d 6 S — S0 called the label sort of d; (ii) there is a predicate _ -4 _ : dx l-dx d S II. Example tended to language: S So fi
n
15.20. The following dynamic signature ((5, n),5 0 ,II) is inmodel state variables of sort s of an imperative programming = { s,Var[s}}, 1- Var[s] , = { Var[S}} , = {[±]:-+ Vbr[s], [ _ ] : s -4 Var[s], :=_: s -> 1 - Vor[s] }, = { . -^ . : Var[s]xl - Var[s]x Var[s] }.
Informally, the sort s stands for the values to be assigned, the sort Var[s] stands for the variables containing values of sort s and the sort 1 — Vor[s] stands for labels identifying transitions, i.e. "actions" on variables. For instance, [_L] stands for a variable with undefined contents and [a] stands for a variable with contents a. Similarly,
:= a stands for the action of assigning the value a to a variable. Finally, II is used to denote the possible transitions representing the assignments of values to variables. The signature is used in Examples 15.21 and 15.22. A precise definition of the notion of a dynamic algebra is beyond the scope of this overview. Informally, a dynamic algebra for the dynamic signature (S,5 0 ,II) consists of: • a S-algebra (in the sense of Definition 2.3), say A; • for each element of II, say n : s i X . . .xsk, a subset of A ( S I ) X . . .xA(sk). In such an algebra the triples of the subset defined by "_ -^ _" constitute the possible transitions. Example 15.21. A possible dynamic algebra for the dynamic signature of Example 15.20 is defined as follows: • the E-algebra A is as indicated in the comments on Example 15.20; • the meaning of II is defined to be: {[JL] ^ [a] | a e A ( s ) } u { [ a ] ^4 [b] \a,b&A(s)}. The triples of (the meaning of) II represent the possible transitions; each transition consists in the action of assigning a value to a variable. Note that in the example the actions of the transition predicate are deterministic. This need not be so in general. Abstract dynamic data types and dynamic specifications are defined as usual.
306
J. Loeckx, H.-D. Ehrich and M. Wolf
Dynamic algebras may model transition systems of various dynamic sorts. The question remains how dynamic algebras reflecting the isolated and cooperative behaviour of these systems may be specified. In the case of isolated behaviour, specification up to behavioural equivalence is practical and feasible. In the case of cooperative, i.e. concurrent, behaviour, configurations consisting of multisets of local states are introduced, together with global information on the configuration. Global transitions are specified by splitting the specification into several steps; at each step some partial moves are defined using the partial moves at the previous step; at the first step, the partial moves are defined by using the transitions of the components. Contrasting with the approaches to dynamic data types discussed above there exist approaches in which processes are represented as algebras rather than as data. These approaches offer the possibility of providing states with an algebraic structure that can be specified along the lines described in this book. A transition structure on top of the algebras allows one to move from one state algebra to another. More precisely, a transition allows one to remove or add carriers, or to redefine (the functions that constitute) the meaning of the operations.
15.6
Objects
Since the advent of SMALLTALK [Kay and Goldberg, 1976; Kay, 1993], the underlying paradigm of object-orientation has become quite successful, as illustrated by the numerous object-oriented programming languages, database systems, and software development methods. In this paradigm, software systems are considered to be dynamic collections of autonomous objects that interact with each other. Autonomy means that each object encapsulates all features needed to act as an independent computing agent: individual attributes (data), methods (operations), behaviour (process) and communication facilities. Moreover, each object has a unique identity that is immutable throughout its lifetime. Coincidentally, object-orientation comes with an elaborate system of types and classes, facilitating structuring and reuse of software. Object specification combines ideas from algebraic data type specification, conceptual data modelling, behaviour modelling, specification of reactive systems, and concurrency theory. The basic concepts are now illustrated by means of an example, using an ad-hoc notation in algebraic specification style. Example 15.22. The following specification introduces an object class Var[s] of state variables of sort s (cf. Examples 15.20 and 15.21): object class Var[s]; uses s, nat; attributes value:s; actions create; :=s; delete;
Algebraic specification of abstract data types
307
axioms var v,w: s; ©create = > • ( - > > create A >:=v A > delete) U©delete ©delete =>• > create A ->t>:=v A -i> delete Q:=V => value=D U (0:=w V ©delete) end. A variable x of class Var[s] has an attribute x.value denoting its current value. It has actions x. create for creation, x: =s for assigning values of sort s (i.e. an action x:=v for every value v of sort s), and a;.delete for deletion. In the class axioms, the predicate ©a is used for expressing the fact that action a has just occurred, and >a for expressing the fact that action a is enabled, i.e. that it may occur at the current state. The temporal operator (pUip means that, from the current moment on, (p will be true all the time until \j) becomes true for the next time. The latter must happen some time, and (p need not be true any more when this happens. The axioms express the fact that: • after creation, all actions except creation are enabled until deletion; • after deletion, all actions except creation are disabled; • after assignment, the value assigned is kept until the next assignment is made, or until the variable is deleted. Enabling creation after deletion gives the possibility of reusing variables. More generally, an object type is a set of object instances. It is characterized by a data type of object identities along with an object class describing the prototypical behaviour shared by all instances of the object type. A specification may be defined along the lines described in the previous chapters of this book in the following way. As usual, a signature is a pair S = (S, 0) where S is a set of sorts and n is a set of operations. Data and objects are distinguished by assuming that the set S of sorts consists of two disjoint subsets SD of data sorts and So of object sorts. Interpretation of a signature £ is given in a E-algebra A. The intended interpretation of a data sort d e SD in the algebra A is a set A(d) of data elements that constitutes the carrier set of sort d. The intended interpretation of an object sort s 6 So in A is a pair A(s) = A(s)ld .A(s)bv where A(s)'d is a set of identities and A(s)bv is the behaviour of objects of sort s. In other words, A(s) represents a set of object instances i.A(s)bv, each one given by an object identity i e A(s)td, and each one having the same behaviour A(s)bv. The operations are interpreted as usual. In analogy to data types, one may define an object type as the set A(s) of object instances of sort s £ So within a S-algebra A, together with the operations of fi defined on A(s). Hence, an object type is like a data type but with object instances taking the place of data elements.
308
J. Loeckx, H.-D. Ehrich and M. Wolf
The essential difference is that an object instance has a dynamic behaviour whereas a data element has a static behaviour. Formally, all instances of an object type have the same behaviour. It is possible, however, that different parts of this behaviour are relevant for different objects of the type, thus introducing some heterogeneity. Object-orientation comes with a rich structure of object relationships and constructions: inheritance, generalization, aggregation and interaction. A theoretical basis for their study is given by object type morphisms preserving structure and behaviour of object types: there is a map that relates the identity sets, and a behaviour morphism between related behaviours. A crucial point in object specification is how to specify behaviours and their morphisms, i.e. behaviour preserving maps. In order to describe sequential and concurrent behaviour, many logics and models of concurrency can be adopted and integrated. One choice is to use temporal logic and its extensions for specification, and event structures for interpretation. But other choices are possible, based, for instance, on predicate logic and labelled transition systems (cf. section 15.5). Another crucial point is how to specify interaction between objects. Basically, this can be done by constraining the cooperative behaviour of components within an aggregated object, using the specification logic at hand.
15.7
Bibliographic notes
Sections 15.1 to 15.4 are short accounts of corresponding chapters in [Ehrich et al., 1989] where more details can be found. A survey and annotated bibliography of the state of the art is in [?]. Behavioural abstraction is also treated in [Reichel, 1987; Bidoit and Hennicker, 1996; Bidoit and Hennicker, 1998]. A first concept of implementation was published in [Ehrich, 1978]. A survey of implementation and behavioural equivalence may be found in [Orejas et al., 1993]. In [Goguen and Meseguer, 1992; Goguen and Diaconescu, 1993] an alternative approach to order-sorting and exception handling is provided. A further approach to using order-sorting in algebraic specification is given in [Mosses, 1993]; this paper also contains an overview and comparison with related approaches. An approach to exception handling with explicit error values is pursued in [Gogolla et al, 1984]. Exception handling using partial algebras is treated in [Reichel, 1987]. Dynamic data type approaches have been described in, for instance, [Astesiano and Reggio, 1987; Kaplan, 1989; Ehrig et al., 1990]. A standardized specification language and method developed on these grounds is given in [ISO-LOTOS, 1989]. A survey of algebraic specification of concurrency with an emphasis on dynamic data types is given in [Astesiano and Reggio, 1993]; this paper also contains a rich collection of references to the literature. The states-as-algebras approach to dynamic data types
Algebraic specification of abstract data types
309
is pursued in [Dauchy and Gaudel, 1994; Zucca, 1999; Ehrig et al, 1995]. A recent overview of the topic may be found in [Astesiano et al., 1999a]. Object specification as outlined here is based on work by the second author of this chapter and others [Sernadas et al., 1987; Ehrich and Sernadas, 1995; Ehrich, 1999; Ehrich and Hartel, 1996] and on the TROLL language project [Hartmann et al., 1995; Jungclaus et al., 1996; Hartel et al., 1997]. Other noteworthy object specification approaches related to algebraic specification are FOOPS [Goguen and Meseguer, 1987; Goguen and Socorro, 1995] and MAUDE [Meseguer, 1993]: FOOPS is based on the specification language OBJ3 [Goguen and Winkler, 1988]; MAUDE is based on rewriting logic that is a uniform model of concurrency [Meseguer, 1992].
16 16.1
The categorical approach Categories
Many authors use categories to introduce and discuss the topics treated in this chapter (e.g. [Ehrig and Mahr, 1985; Ehrig and Mahr, 1990; Sannella and Tarlecki, 2001]). By its very abstractness category theory can lead to more general results and avoid repeating similar proofs. On the other hand the approach may be difficult to understand for a reader who is not familiar with category theory. The first clue in this approach is the fact that any class of algebras together with their homomorphisms constitutes a category. A monomorphic module (in the sense of Definition 14.2(iii)) may then be viewed as a functor on such categories. Several notions introduced and properties proved in the present chapter may be viewed as notions and properties from category theory. For instance, the notions of an initial algebra and of a reduct correspond to the categorical notions of an initial object and—at least in the case of an inclusion—of a forgetful functor. As another example, the notion of a free extension is related to the categorical notion of a free functor. Another interesting category is constituted by the atomic specifications together with their specification morphisms (see Definition 13.19(ii)). It allows one to define the amalgamated union (Definition 13.18) and the parameter passing mechanism as pushouts (see section 14.6). The reader interested in category theory and its application to the specification of abstract data types may consult [Sannella and Tarlecki, 2001].
16.2
Institutions
An institution constitutes a generalization of the notion of (algebra) logic introduced in Definition 6.1. For instance, signatures together with their signature morphisms are merely supposed to form a category instead of being the concrete objects in Definitions 2.1 and 5.1. While most of the properties of section 7 still hold, institutions allow a large variety of signatures (including, for instance, order-sorted signatures and higher-order
310
J. Loeckx, H.-D. Ehrich and M. Wolf
signatures), of models (including, for instance, error-algebras and partial algebras) and of logics (including temporal logics). Definition 16.1 (Institution). Let SET be the category of sets and CAT the category of all categories. An institution is a 4-tuple (SIG, Sen, Mod, (=) where: (i) SIG is a category; the objects of SIG are called signatures, and its morphisms signature morphisms; (ii) Sen : SIG —> SET is a functor; for any signature S of SIG, the elements of the set Sen(S) are called (Z)- formulas (or: (E)-sentences); (iii) Mod : SIG —>• CATOJ> is a functor where CATop is the opposite category of CAT; for any signature £ of SIG, the objects of the category Mod(S) are called E-models; (iv) |= = (|=E)E6SiG is a family of relations denned for each signature £ of SIG where (=E C |Mod(£)| x Sen(S) and where |Mod(E)| denotes the class of objects of the category Mod(E). It is required that the following condition is satisfied: For any signature morphism /j, : E —» £' of SIG, any formula ip of Sen(E) and any model M' of Mod(S'), Mod(fj.)(M') ^vV&M' The reader interested in more details may consult [Goguen and Burstall, 1992; Sannella and Tarlecki, 2001].
References [Abrial, 1996] J. R. Abrial. The B-book: Assigning Programs to Meaning. Cambridge University Press, 1996. [Andrews and Ince, 1991] D. Andrews and D. Ince. Practical Formal Methods with VDM. McGraw-Hill, 1991. [Astesiano et al, 1999a] E. Astesiano, M. Broy and G. Reggio. Altebraic specification of concurrent systesm. In E. Astesiano, H.-J. Kreowski and B. Krieg-Briickner, editors, Algebraic Foundations of Systems Specification, pages 467-520. Springer- Verlag, 1999. [Astesiano et al, 1999b] E. Astesiano, H.-J. Kreowski and B. KriegBriickner, editors, Algebraic Foundations of Systems Specification, Springer- Verlag, 1999. [Astesiano and Reggio, 1987] E. Astesiano and G. Reggio. An outline of the SMoLCS methodology. In M. V. Zilli, editor, Proc. Advanced School on Mathematical Models for the Semantics of Parallelism, Lecture Notes in Computer Science 280, pages 81-113. Springer- Verlag, 1987. [Astesiano and Reggio, 1993] E. Astesiano and G. Reggio. Algebraic specification of concurrency. In M. Bidoit and C. Choppy, editors, Recent
Algebraic specification of abstract data types
311
Trends in Data Type Specification, Lecture Notes in Computer Science 655, pages 1-39. Springer-Verlag, 1993. [Avenhaus, 1995] J. Avenhaus. Reduktionssysteme. Springer-Verlag, 1995. [Barstow, 1983] D. Barstow, editor. The Programming Language Ada. Reference Manual. Lecture Notes in Computer Science 155, Springer-Verlag, 1983. [Bergstra and Tucker, 1983] J.A. Bergstra and J.V. Tucker. Initial and final algebra semantics: two characterization theorems. SIAM Journal of Computation, 12(2):366-387, 1983. [Bibel, 1993] Wolfgang Bibel. Deduction: Automated Logic. Academic Press, 1993. [Bidoit, 1989] M. Bidoit. PLUSS, un langage pour le developpement de specifications algebriques modulaires. These d'etat, Universite ParisSud, Orsay, 1989. [Bidoit, 1991] M. Bidoit. Development of modular specifications by stepwise refinement using the PLUSS specification language. Rapport de Recherche LIENS-91-9, Ecole Normale Superieure, 1991. [Bidoit and Hennicker, 1995a] M. Bidoit and R. Hennicker. Behavioural theories. In E. Astesiano, G. Reggio, and A. Tarlecki, editors, Recent Trends in Data Type Specification, Lecture notes in Computer Science 906, pages 153-169. Springer-Verlag, 1995. [Bidoit and Hennicker, 1996] M. Bidoit and R. Hennicker. Behavioural' theories and the proof of behavioural properties. Theoretical Computer Science, 165(1): 3-55, 1996. [Bidoit and Hennicker, 1998] M. Bidoit and R. Hennicker. Modular correctness proofs of behavioural implementations. Acta Inforamtica, 35(11):951-1005, 1998. [Bidoit et al., 1999] M. Bidoit, M. Cengarle, R. Hennicker. Proof systems for structured specifications and their refinements. In E. Astesiano, H.J. Kreowski and B. Krieg-Briickner, editors, Algebraic Foundations of Systems Specification. Springer-Verlag, 1999. [Broy et al, 1993] M. Broy, C. Facchi, R. Grosu, R. Hettler, H. Hettler, H. Hussmann, D. Nazareth, F. Regensburger, O. Slotosch, and K. St01en. The requirement and design specification language SPECTRUM: an informal introduction (Version 1.0). Technical report, Technische Universitat Miinchen, 1993. [Burstall and Goguen, 1980] R. M. Burstall and J.A. Goguen. The semantics of CLEAR, a specification language. In E. Bjorner, editor, Proc. Advanced Course on Abstract Software Specifications, Lecture Notes in Computer Science 86, pages 292-332. Springer-Verlag, 1980. [Cerioli et al., 1997] M. Cerioli, M. Gogolla, H. Kirchner, B. Krieg-Briickner, Z. Qian,
312
J. Loeckx, H.-D. Ehrich and M. Wolf
and M. Wolf, editors. Algebraic System Specification and Development: Survey and Annotated Bibliography (Second Edition), Shaker-Verlag, Aachen, 1997. [Classen, 1989] I. Classen. I. Classen. Revised ACT ONE: Categorical cnstructions for an algebraic specification language. In H. Ehrig, H. Herrlich, H.-J. Kreowski and G. Preuss, editors, Categorical Mthods in Computer Science, Lecture Notes in Computer Science 393, pages 435-465. Springer-Verlag, 1989. [Classen et al., 1993] I. Classen, H. Ehrig, and D. Wolz. Algebraic Specification Techniques and Tools for Software Development: The ACT Approach. World Scientific Publishing Co., 1993. [Dauchy and Gaudel, 1994] P. Dauchy and M.-C. Gaudel. Algebraic specifications with implicit state. Technical report, Universite Paris-Sud, 1994. [Ebbinghaus et al., 1984] H.-D. Ebbinghaus, J. Flum, and W. Thomas. Mathematical Logic. Springer-Verlag, 1984. [Ehrich, 1978] H.-D. Ehrich. Extensions and implementations of abstract data type specifications. In J. Winkowski, editor, Mathematical Foundations of Computer Science, Lecture Notes in Computer Science 64, pages 155-164. Springer-Verlag, 1978. [Ehrich, 1999] H.-D. Ehrich. Object specification. In E. Astesiano, H.J. Kreowski and B. Krieg-Briickner, editors, Algebraic Foundations of System Specification, pages 435-465. Springer-Verlag, 1999. To appear. [Ehrich and Hartel, 1996] H.-D. Ehrich and P. Hartel. Temporal specification of information systems. In A. Pnueli and H. Lin, editors, Logic and Software Engineering. Proceedings of the International Workshop in Honor of C. S. Tang, Beijing, pages 43-71. World Scientific, 1996. [Ehrich and Lohberger, 1979] H.-D. Ehrich and V.G. Lohberger. Constructing specifications of abstract data types by replacements. In V. Glaus, H. Ehrig, and G. Rozenberg, editors, Proc. Colloq. on Graph Grammars, Lecture Notes in Computer Science 73, pages 180-191. Springer-Verlag, 1979. [Ehrich and Sernadas, 1995] H.-D. Ehrich and A. Sernadas. Local specification of distributed families of sequential objects. In E. Astesiano, G. Reggio, and A. Tarlecki, editors, Recent Trends in Data Type Specification, Lecture Notes in Computer Science 906, pages 219-235. SpringerVerlag, 1995. [Ehrich et al., 1989] H.-D. Ehrich, M. Gogolla, and U. Lipeck. Algebraische Spezifikation abstrakter Datentypen. Teubner-Verlag, 1989. [Ehrig and Mahr, 1985] H. Ehrig and B. Mahr. Fundamentals of Algebraic Specification 1. Equations and Initial Semantics. Springer-Verlag, 1985. [Ehrig and Mahr, 1990] H. Ehrig and B. Mahr. Fundamentals of Algebraic
Algebraic specification of abstract data types
313
Specification 2. Module Specifications and Constraints. Springer-Verlag, 1990. [Ehrig et al., 1990] H. Ehrig, F. Parisi-Presicce, P. Boehm, C. Rieckhoff, C. Dimitrovici, and M. Grosse-Rhode. Combining data type and recursive process specifications using projection algebras. Theoretical Computer Science, 71:347-380, 1990. [Ehrig et al, 1995] H. Ehrig, M. Lowe, and F. Orejas. Dynamic abstract data types based on algebraic graph transformations. In E. Astesiano, G. Reggio, and A. Tarlecki, editors, Recent Trends in Data Type Specification, Lecture Notes in Computer Science 906, pages 236-254. SpringerVerlag, 1995. [Gogolla et al., 1984] M. Gogolla, K. Drosten, U.W. Lipeck, and H.-D. Ehrich. Algebraic and operational semantics of specifications allowing exceptions and errors. Theoretical Computer Science, 34:289-313, 1984. [Goguen and Burstall, 1992] J.A. Goguen and R.M. Burstall. Institutions: Abstract model theory for specification and programming. Journal of the Association for Computing Machinery, 39(1):95-146, 1992. [Goguen and Diaconescu, 1993] J.A. Goguen and R. Diaconescu. A short Oxford survey of order sorted algebra. In G. Rozenberg and A. Salomaa, editors, Current Trends in Theoretical Computer Science. Essays and Tutorials, pages 209-222. World Scientific Series in Computer Science 40, World Scientific,1993. [Goguen and Meseguer, 1987] J.A. Goguen and J. Meseguer. Unifying functional, object oriented and relational programming with logical semantics. In B. Shriver and P. Wegner, editors, Research Directions in Object-Oriented Programming, pages 417-477. MIT Press, 1987. [Goguen and Meseguer, 1992] J.A. Goguen and J. Meseguer. Ordersorted algebra I: equational deduction for multiple inheritance, overloading, exceptions and partial operations. Theoretical Computer Science, 105(2):217-273, 1992. [Goguen and Socorro, 1995] J.A. Goguen and A. Socorro. Module composition and system design for the object paradigm. Journal of Object Oriented Programming, 7(14), 1995. [Goguen and Winkler, 1988] J.A. Goguen and T. Winkler. Introducing OBJ3. Research report SRI-CSL-88-9, SRI International, 1988. [Guttag and Horning, 1993] J.V. Guttag and J.J. Horning. Larch: Languages and Tools for Formal Specification. Springer-Verlag, 1993. [Guttag et al., 1985] J. Guttag, J. Horning, and J. Wing. Larch in five easy pieces. Internal report SRC TR# 5, Digital Equipment Corporation, 1985. [Hartel et al., 1997] P. Hartel, G. Denker, M. Kowsari, M. Krone and H.-D. Ehrich. Information systems modelling with TROLL—Formal methods at work. Information Systems, 22(2-3):79-99, 1997.
314
J. Loeckx, H.-D. Ehrich and M. Wolf
[Hartmann et al., 1995] T. Hartmann, G. Saake, R. Jungclaus, P. Hartel, and J. Kusch. Revised Version of the Modelling Language TROLL (Version 2.0). Informatik-Bericht 94-03, Technische Universitat Braunschweig, 1995. [ISO-LOTOS, 1989] ISO-LOTOS. A formal description technique based on the temporal ordering of observational behaviour. Technical report IS 8807. International Standards Organization, 1989. [Jungclaus et al, 1996] R. Jungclaus, G. Saake, T. Hartmann, and C. Sernadas. TROLL—a language for object-oriented specification of information systems. ACM Transactions on Information Systems, 14(2): 175211, 1996. [Kaplan, 1989] S. Kaplan. Algebraic specification of concurrent systems. Theoretical Computer Science, 69(1):69-115, 1989. [Kay, 1993] A. Kay. The early history of Smalltalk. ACM SIGPLAN Notices, pages 69-96, March 1993. [Kay and Goldberg, 1976] A. Kay and A. Goldberg. Smalltalk-72 Instruction manual. Technical report, Xerox PARC, March 1976. [Klop, 1992] J.W. Klop. Term rewriting systems. In S. Abramsky, D.M. Gabbay, and T.S.E. Maibaum, editors, Handbook of Logic in Computer Science. Volume 2. Background: Computational Structures, pages 2-117. Clarendon Press, 1992. [Lehmann and Loeckx, 1993] T. Lehmann and J. Loeckx. OBSCURE, a specification language for abstract data types. Acta Informatica, 30(4):303-350, 1993. [Loeckx, 1997] J. Loeckx. Hierarchical constructive specifications and their termination. Internal note, Computer Science Department, University of Saarbrucken, 1997. [Loeckx and Sieber, 1987] J. Loeckx and K. Sieber. The Foundations of Program Verification. Wiley/Teubner, 1987. [Loeckx and Zeyer, 1995] J. Loeckx and J. Zeyer. Experiences with a specification environment. In M. Broy and S. Jahnichen, editors, KORSO: Methods, Languages and Tools for the Construction of Correct Software, Lecture Notes in Computer Science 655,, pages 255-268. Springer LNCS 1009, 1995. [Loeckx et al., 1996] J. Loeckx, H.-D. Ehrich, and M. Wolf. Specification of Abstract Data Types. Wiley/Teubner, 1996. [Meseguer, 1992] J. Meseguer. Conditional rewriting as a unified model of concurrency. Theoretical Computer Science, 96(1):73-156, 1992. [Meseguer, 1993] J. Meseguer. A logical theory of concurrent objects and its realization in the Maude language. In G. Agha, P. Wegener, and A. Yonezawa, editors, Research Directions in Object-Oriented Programming, pages 314-390. MIT Press, 1993.
Algebraic specification of abstract data types
315
[Mosses, 1993] P.D. Mosses. The use of sorts in algebraic specifications. In M. Bidoit and C. Choppy, editors, Recent Trends in Data Type Specification, Lecture Notes in Computer Science 655, pages 66-92. Springer LNCS 655, 1993. [Mosses, 1999] P. Mosses. CASL. A guided tour of its design. In J. Fiadeiro, editor, Recent Trends in Algebraic Development Techniques, Lecture Notes in Computer Science 1589. Springer-Verlag, 1999. [Orejas et al, 1993] F. Orejas, M. Navarro, and A. Sanchez. Implementation and behavioural equivalence: a survey. In M. Bidoit and C. Choppy, editors, Recent Trends in Data Type Specification, Lecture Notes in Computer Science 655, pages 93-125. Springer-Verlag, 1993. [Padawitz, 1988] P. Padawitz. Computing in Horn Clause Theories. Springer-Verlag, 1988. [Paulson, 1991] L.C. Paulson. ML for the Working Programmer. Cambridge University Press, 1991. [Raise Development Group, 1995] The Raise Development Group. The Raise Development Method. BCS Practitioners Series. Prentice Hall International, 1995. [Reichel, 1987] H. Reichel. Initial Computability, Algebraic Specifications and Partial Algebras. Oxford University Press, 1987. [Ryan and Sadler, 1992] M. Ryan and M. Sadler. Valuation systems and consequence relations. In S. Abramsky, D.M. Gabbay, and T.S.E. Maibaum, editors, Handbook of Logic in Computer Science. Volume 1. Background: Computational Structures, pages 1-78. Clarendon Press, 1992. [protectSannella, 1999] D. Sannella. CoFI WG 29432. - Interim Report 1 October 1998 - 30 September 1999. Internal paper, Computer Science Department, University of Edinburgh, 1999. [Sannella and Tarlecki, 2001] D. Sannella and A. Tarlecki. Foundations of Algebraic Specifications and Formal Program Development. Cambridge University Press, 2001. Scheduled to appear in 2001. [Sannella and Wirsing, 1999] D. Sannella and M. Wirsing. Specification languages. In E. Astesiano, H.-J. Kreowski, and B. Krieg-Briickner, editors, Algebraic Foundations of Systems Specifications, pages 243-272. Springer-Verlag, 1999. [Sernadas et al., 1987] A. Sernadas, C. Sernadas, and H.-D. Ehrich. Object-oriented specification of databases: an algebraic approach. In P.M. Stoecker and W. Kent, editors, Proc. 13th Int. Conf. on Very Large Data Bases, pages 107-116. VLDB Endowment Press, 1987. [Spivey, 1989] J.M. Spivey. The Z Notation. Prentice Hall, 1989. [Tucker and Meinke, 1992] J.V. Tucker and K. Meinke. Universal algebra. In S. Abramsky, D.M. Gabbay, and T.S.E. Maibaum, editors, Hand-
316
J. Loeckx, H.-D. Ehrich and M. Wolf
book of Logic in Computer Science. Volume 1, pages 189-411. Clarendon Press, 1992. [Wirsing, 1986] M. Wirsing. Structured algebraic specifications: a kernel language. Theoretical Computer Science, 42(2): 123-250, 1986. [Wirsing, 1993] M. Wirsing. Structured specifications: syntax, semantics and proof calculus. In F.L. Bauer, W. Brauer, and H. Schwichtenberg, editors, Logic and Algebra of Specification, pages 411-442. SpringerVerlag, 1993. [Wirsing, 1995] M. Wirsing. Algebraic specification languages: an overview. In E. Astesiano, G. Reggio, and A. Tarlecki, editors, Recent Trends in Data Type Specification, Lecture Notes in Computer Science 906, pages 81-115. Springer-Verlag, 1995. [Woodman and Heal, 1993] M. Woodman and B. Heal. Introduction to VDM. McGraw-Hill, 1993. [Zucca, 1999] E. Zucca. From static to dynamic data types: an institution transformation. Theoretical Computer Science, 216:109-157, 1999.
Computable functions and semicomputable sets on many-sorted algebras J. V. Tucker and J. I. Zucker
Contents 1
2
3
Introduction 1.1 Computing in algebras 1.2 Examples of computable and non-computable functions . 1.3 Relations with effective algebra 1.4 Historical notes on computable functions on algebras . . 1.5 Objectives and structure of the chapter 1.6 Prerequisites Signatures and algebras 2.1 Signatures 2.2 Terms and subalgebras 2.3 Homomorphisms, isomorphisms and abstract data types 2.4 Adding Booleans: Standard signatures and algebras . . . 2.5 Adding counters: TV-standard signatures and algebras . . 2.6 Adding the unspecified value ui; algebras A" of signature Eu 2.7 Adding arrays: Algebras A*_of signature IT* 2.8 Adding streams: Algebras A of signature £ While computability on standard algebras 3.1 Syntax of While(E) 3.2 States 3.3 Semantics of terms 3.4 Algebraic operational semantics 3.5 Semantics of statements for While(Yi) 3.6 Semantics of procedures 3.7 Homomorphism invariance theorems 3.8 Locality of computation 3.9 The language WhileProc("S) 3.10 Relative While computability 3.11 .For(£) computability 3.12 WhileN and ForN computability
319 322 325 329 335 340 343 344 344 349 350 351 353 355 356 359 360 361 363 363 364 366 368 371 372 374 375 376 377
318
4
5
6
J. V. Tucker and J. I. Zucker 3.13 While* and For* computability 3.14 Remainder set of a statement; snapshots 3.15 £*/£ conservativity for terms Representations of semantic functions; universality 4.1 Godel numbering of syntax 4.2 Representation of states 4.3 Representation of term evaluation 4.4 Representation of the computation step function 4.5 Representation of statement evaluation 4.6 Representation of procedure evaluation 4.7 Computability of semantic representing functions; term evaluation property 4.8 Universal WhileN procedure for While 4.9 Universal WhileN procedure for While* 4.10 Snapshot representing function and sequence 4.11 Order of a tuple of elements 4.12 Locally finite algebras 4.13 Representing functions for specific terms or programs . . Notions of semicomputability 5.1 While semicomputability 5.2 Merging two procedures: Closure theorems 5.3 Projective While semicomputability: semicomputability with search 5.4 WhileN semicomputability 5.5 Projective WhileN semicomputability 5.6 Solvability of the halting problem 5.7 While* semicomputability 5.8 Projective While* semicomputability 5.9 Homomorphism invariance for semicomputable sets . . . 5.10 The computation tree of a While statement 5.11 Engeler's lemma 5.12 Engeler's lemma for While* semicomputability 5.13 S* definability: Input/output and halting formulae . . . 5.14 The projective equivalence theorem 5.15 Halting sets of While procedures with random assignments Examples of semicomputable sets of real and complex numbers 6.1 Computability on E and C 6.2 The algebra of reals; a set which is projectively While semicomputable but not While* semicomputable . . . . 6.3 The ordered algebra of reals; sets of reals which are While semicomputable but not While* computable 6.4 A set which is projectively While* semicomputable but not projectively WhileN semicomputable
378 380 383 387 388 389 389 390 392 393 394 397 401 402 404 405 406 407 408 409 413 414 416 416 420 421 422 423 425 429 431 434 435 438 439 441 443 445
Computable functions on algebras 6.5
7
8
1
Dynamical systems and chaotic systems on E; sets which are WhileN semicomputable but not While* computable 6.6 Dynamical systems and Julia sets on C; sets which are WhileN semicomputable but not While* computable Computation on topological partial algebras 7.1 The problem 7.2 Partial algebras and While computation 7.3 Topological partial algebras 7.4 Discussion: Two models of computation on the reals . . . 7.5 Continuity of computable functions 7.6 Topological characterisation of computable sets in compact algebras 7.7 Metric partial algebra 7.8 Connected domains: computability and explicit definability 7.9 Approximable computability 7.10 Abstract versus concrete models for computing on the real numbers A survey of models of computability 8.1 Computability by function schemes 8.2 Machine models 8.3 High-level programming constructs; program schemes 8.4 Axiomatic methods 8.5 Equational definability 8.6 Inductive definitions and fixed-point methods 8.7 Set recursion 8.8 A generalised Church-Turing thesis for computability . . 8.9 A Church-Turing thesis for specification 8.10 Some other applications
319
447 449 451 452 453 455 458 460 464 465 465 470 475 479 479 484 488 490 490 492 493 493 496 500
Introduction
The theory of the computable functions is a mathematical theory of total and partial functions of the form / : N™ ->• N,
and sets of the form
SCW1
that can be defined by means of algorithms on the set
320
J. V. Tucker and J. I. Zucker
of natural numbers. The theory establishes what can and cannot be computed in an explicit way using finitely many simple operations on numbers. The set of naturals and a selection of these simple operations together form an algebra. A mathematical objective of the theory is to develop, analyse and compare a variety of models of computation and formal systems for defining functions over a range of algebras of natural numbers. Computability theory on N is of importance in science because it establishes the scope and limits of digital computation. The numbers are realised as concrete symbolic objects and the operations on the numbers can be carried out explicitly, in finitely many concrete symbolic steps. More generally, the numbers can be used to represent or code any form of discrete data. However, the question arises: Can we develop theories of functions that can be defined by means of algorithms on other sets of data? The obvious examples of numerical data are the integer, rational, real and complex numbers; and associated with these numbers there are data such as matrices, polynomials, power series and various types of functions. In addition, there are geometric objects that are represented using the real and complex numbers, including algebraic curves and manifolds. Examples of syntactic data are finite and infinite strings, terms, formulae, trees and graphs. For each set of data there are many choices for a collection of operations from which to build algorithms. How specific to the set of data and chosen operations are these computability theories? What properties do the computability theories over different sets of data have in common? The theory of the computable functions on N is stable, rich and useful; will the theory of computable functions on the sets of real and complex numbers, and the other data sets also be so? The theory of computable functions on arbitrary many-sorted algebras will answer these questions. It generalises the theory of functions computable on algebras of natural numbers to a theory of functions computable on any algebra made from any family of sets and operations. The notion of 'computable' here presupposes an algorithm that computes the function in finitely many steps, where a step is an application of a basic operation of the algebra. Since the data are arbitrary, the algorithm's computations are at the same level of abstraction as the data and basic operations of the algebra. For example, this means that computations over the field E of real numbers are exact rather than approximate. Thus, the algorithms and computations on algebras are intimately connected to their algebraic properties; in particular, the computability theory is invariant under isomorphisms. Already we can see that, in the general case, there is likely to be a ramification of computability notions. For example, in the case of computable functions on the set E of real numbers it is also natural to consider
Computable functions on algebras
321
computability in terms of computing approximations to the values of a function. The use of approximations recognises the fact that data like the real numbers are infinite objects and can, or must, be algorithmically approximated. This is the approach of computable analysis. We wll present two approaches to computation on the reals: 'algebraic' and 'topological'. In our algebraic approach we are looking for what can be computed exactly, knowing only what the operations reveal about the reals. The operations may have been chosen to reveal properties that are specific to the reals, of course. In the topological approach we are looking for what can be computed with essentially infinite data on the basis of a finite amount of information. Actually, this is again, at bottom, an algebraic approach, for the performance of approximate computations. In this chapter our objective is to show the following: 1. There is a general theory of computable functions on an arbitrary algebra that possesses generalisations of many of the important results in computability theory on natural numbers. 2. The theory provides technical concepts and results that improve our understanding of the foundations of classical computability and definability theory on N. 3. The theory has a wide range of applications in mathematics and computer science. 4. The theory can be developed using many models of computation that are equivalent in that they define the same class of computable functions. 5. The theory possesses a generalisation of the Church-Turing thesis for functions and sets computed by algorithms on any abstract algebraic structure. 6. The theory generalises other less general but still abstract and algebraic, theories of finite computation, including effective algebra and computable analysis. Computability theories on particular and general classes of algebras address central concerns in mathematics and computer science. Some, such as effective algebra, have a long history and several subfields with deep results, such as the theory of computable rings and fields and the word problem for groups. However, abstract computability theories of the kind we will develop have a short and less eventful history: starting in the late 1950s, with theoretical work on flowcharts, many approaches have been presented that vary in their generality and objectives; indeed, there has been a remarkable amount of reinventing of ideas and results, sometimes with new motivations, such as obtaining results on: the structure of flowcharts; the power of programming constructs; the design of program correctness logics; the development of axiomatic foundations for generalised recursion theories based on ordinals and higher types; and the study of algorithmic aspects of ring and field theory, and of dynamical system theory.
322
J. V. Tucker and J. I. Zucker
In this section we will introduce in a very informal way the model of computation we will use (in section 1.1) and pose some questions about examples of computable functions (in section 1.2). Then, in section 1.3, we will outline the relationship between our model and other models of computation, especially effective algebra and computable analysis. In section 1.4 the history of the theory of computable functions on abstract algebras is sketched. In sections 1.5 and 1.6 the structure of the chapter and its prerequisites are discussed in more detail. The chapter is closely linked scientifically with the chapters in this Handbook on universal algebra (Volume I), computability (Volume I), and effective algebra (Volume IV); it also connects with other subjects (e.g., topology (Volume I) and those on semantics (Volumes III and IV)). Further information on prerequisites is given in section 1.6.
1.1
Computing in algebras
Let us begin with a basic question: Let S be a non-empty set of data and let f : Sn —> S be a total or partial function. How do we compute f ? The methods we have in mind start with postulating an algebra A containing the set 5. The algebra may consist of a finite family of non-empty sets Ai , . . . , Ak called the carriers of the algebra, one of which is the set S and another is the set B of Booleans. The algebra is also equipped with a finite family Cl , . . . , Cp
of elements of the sets, called constants, and a finite family
of functions on the sets called operations; these functions are of the form F : ASl x ...x ASn -» As
and can be total or partial. Among the operations are some standard functions on the Booleans. Such an algebra is called a standard manysorted algebra; we say it is standard because it contains the Booleans and their special operations. An algebra is often written (Ai,... ,Ak, ci,... ,cp,Fi,... ,Fq). A set £ of names for the data set, constants and operations (and their arities) of the algebra A is called a signature.
Computable functions on algebras
323
For most of the time we will use many-sorted algebras with finitely many constants and total operations, but we will need the case of partial operations to discuss the relationship between our computable functions and continuous functions on topological algebras such as algebras of real numbers, and algebras with infinite data streams. The problem is to develop and classify models of computation that describe ways of constructing new functions on the set 5 from the basic operations of the algebra A. In particular, each model of computation M is a method or technique which we use to define the notion that the function / on the carriers of A is computable from the operations on A by means of method M ; and we collect all such functions into the set M-Comp(A) of functions X-computable over the algebra A. There are many useful choices for a model of computation M with which to develop a computability theory— we list several in a survey in section 8. In this chapter we focus on a theory for computing with a simple imperative model, namely the While programming language. In this programming language, basic computations on an algebra A are performed by concurrent assignment statements of the form
where xi, . : . , xra are program variables and t\ , . . . ,tn are terms or expressions built from variables and the operation symbols from the signature of the algebra A; and x^ and ti correspond in their types (1 < i < n). The control and sequencing of the basic computations are performed by three constucts that form new programs from given programs Si, S^ and S, and Boolean test b: (i) the sequential composition construct
(ii) the conditional branching construct if b then Si else 52 fi (Hi) the iteration construct while b do S od. The set of all programs so constructed over the signature E is denoted While(£). The operational semantics of a While program is a function that, given an initial state, enumerates every state of a resulting computation. The input/output (i/o) semantics of a while program is a function that
324
J. V. Tucker and J. I. Zucker
transforms initial states to final states, if they exist. To compute a function on A by means of a While program we formulate a simple class of function procedures based on While programs; a function procedure P has the form P = proc in a out b aux c begin 5 end where a, b, c are lists of input, output and auxiliary variables, respectively, and 5 is a While program, satisfying some simple conditions. The semantics of a procedure P is a function [P]"4 on A whose input and output types are determined by the types of the lists of input and output variables a and b. A function / is While computable on algebra A if there is a While function procedure that computes it, i.e., {P]A = /. All While computable functions on A are collected in the set While(A). A set is defined to be While computable, or decidable, if its characteristic function is While computable. It is While semicomputable, or semidecidable, if it is the domain of a partial While computable function; in other words, if it is the halting set of a While program. A crucial property of an abstract model of computation is that it is designed to apply to any algebra or class of algebras. Two important consequences are the following. Firstly, it is easy to explore uniform computation where programs generate computations over a class of implementations or representations of data types in a uniform way. For example, think of a While program that is intended to implement Euclid's algorithm to calculate greatest common divisors in a way that is uniform over a class of algebras. By such a 'class' we could mean, for example, any of the following: (i) the class of all Euclidean domains, (ii) the isomorphism class of all (ideal, infinite) implementations of the ring of integers, (Hi) the class of (actual, finite) machine implementations of the integers. Secondly, it is easy to employ certain forms of type constructions. Since we can compute on any algebra (possessing the Booleans) using the programming model, we can augment an algebra A to form a new algebra A', by adding new types and operations, and apply the programming model to A'. Adding new data sets and operations is a key activity in theory and practice. In particular, three modest expansions of an algebra A that have significant practical effects and (as we shall show) interesting mathematical theories are: (a) adding the set N of natural numbers and its standard operations to A to make a new algebra AN; (b) adding finite sequences, and appropriate operations, to A to make an algebra A*. (c) adding infinite streams, and appropriate operations, to A to make an algebra A.
Computable functions on algebras
325
We apply the model of computation to form new classes of computable functions, namely: While(AN),
While(A*)
and
WhUe(A).
By this means it is trivial to add constructs like counters, finite arrays and infinite data streams to the theory of computation, though it is not trivial to chart the consequences. In summary, what mechanisms are available for computing in an algebra? The methods of computation are merely: (i) basic operations of the algebra; and (n) sequencing, branching and iterating the operations. Is equality computable? Do we have available unlimited data storage? Can we search the algebra for data? We will see that for any many-sorted algebra A with the Booleans, by adding the naturals, we can add (Hi) any algorithmic construction on a numerical data representation; and, by adding A*, we can add (iv) local search through all elements of the subalgebra generated by given input data; (v) unlimited storage for data in computations. To obtain equality we have to postulate it as a basic operation of the algebra. We will study these models of computation. The most important turns out to be the programming language While*, which consists of While programs with the naturals and finite arrays, and is defined simply by While" (A) = While(A'). This is the fundamental model of imperative programming that yields a full generalisation, to an arbitrary many-sorted algebra A, of the theory of computable functions on the set N of natural numbers, and for which the generalised Church-Turing thesis for computation on A will be formulated and justified.
1.2
Examples of computable and non-computable functions
First, let us look at the raw material of our theory, namely problems concerning computing functions and sets on specific algebraic structures. We will give a list of questions about computing with While programs on different algebras and invite the reader to speculate on their answers; it is not essential that the reader understand or recognise all the concepts in the examples. The idea is to prepare the reader for the role of algebraic structures in the theory of computable functions and sets, and arouse his or her curiosity.
326
J. V. Tucker and J. I. Zucker
Let B be the set of Booleans and let N, Z, Q, R and C be the sets of natural, integer, rational, real and complex numbers, respectively. 1. Are the sets of functions While computable over the following algebras the same as those computable over (N; 0, n + 1)? (N; 0, n + 1, n + m, n • m, n = m) (N; 0, n + 1, n + m, n • m, nm, n — m) (N; 0, 1, n + m, n • m, n = m) (N; 0, n + m, n • m, n = m) (N; 0, n + m, n = m) (N; 0, n • m, n — m) 2. Consider each of the following functions: f(n) /(n) f(n)
3. 4. 5.
6.
7.
= 4 =n =n+1
f(n,m) —n + m f(n,m) = n —m In each case is / € W7iz/e(N; 0, n — 1)? Let B be the set of Booleans and / : B* -»• B. Is / eW7«Je(B;tt,f, and, not)? Let A be a finite set and / : A -> A. Is / 6 While(A; c\,... , cp, FI, ... ,Fq) for any choice of constants c, and operations Fj on Af Consider the algebra (B,N,[N ->B]; tt,ff, and, not,0, n + l,eval) of Booleans expanded by adding the set N of naturals with zero and successor, and the set [N -> B] of infinite sequences, or streams, of Booleans, with the evaluation map eval:[N —> B] x N —» B defined by eval(6, n) = b(n). Are the following functions While computable over this algebra: shift: [N -» B] x N ->• B defined by shift(a,n) = o(n + 1); Shift: [N -> B] -> [N ->• B] defined by Shift(a)(n) = o(n + 1)? Which of the following sets of Boolean streams are (i) While computable, and (n) While semicomputable, over the stream algebra in question 5? { a | for some n, a(n) =tt} { a I for all n, a(n) =tt) { a | for infinitely many n, a(n) = tt} { a | a(0) = tt, ... , a(n) =tt} for some fixed n Consider each of the following functions: f(x) = 4 f ( x ) = floor(^) f ( x ) = x/2 f ( x ) = 1* f(x) = TT f(x) = sin(x) f(x) = x f(x) = cos(a;) f(x) = x5 f(x) = tan(x) /(x) = v/x /(x) = ex
Computable functions on algebras
327
In each case, is / While computable over (E;0, l,x + y,—x, x-y,x'1) 8. Are cos(a;) and tan(x) While computable over (E;0, l,a; + |/, — x, x-y,x~l,sin(x})'? 9. Let / : E2 ->• E be the step function f(^r)
10.
11. 12. 13.
/Oifa;<7= { , ., ^ I 1 if x > r.
Is / While computable over (E; 0, 1, x + y, -x, x • y, x""1)? Which of the following subsets of E are (a) While computable, and (6) While semicomputable, over the field of real numbers? (z) The rational subfield Q of the field of reals (ii) The subfield Q(>/2) of the field of reals generated by Q and \/2 (Hi) The subfield Q(y^ | p prime) of the field of reals generated by Q and the set {^/p \ p prime} (iv) The subfield Q(r) of the field of reals generated by Q and a non-computable real number r (v) The subfield AR of the field of reals containing precisely the real algebraic numbers Is the subalgebra A of (E; 0, 1, x + y, —x, x • y, a;"1, ex) generated by Q While semidecidable? Is there a While program over (E; 0, 1, x + y, -x, x • y, a;"1, \/x) that computes all the real roots of all quadratic equations with real coefficients? Consider the polynomial p(X) = a0 + aiX + a 2 X 2 + ... + anXn
(a 0 ,... , an € E).
(a) Is the set {x 6 M | p(x) = 0} of roots of p While decidable over (E; 0, 1, x + y, -x, x-y, x~l, o 0 ,... ,a n )? (ft) For each n find operations to add to the algebra (E; 0, 1, x + y, —x, x • y, x"1) to calculate the n roots of the polynomial as functions of the coefficients. 14. Consider the algebra (E, B; 0, 1, x + y, —x, x • y, x~l, x = y) which adds equality =: E2 -» B to the field of reals. What new functions / : E* ->• E can be computed? 15. Consider the algebra (E, B; 0, 1, x + y, —x, x-y, x~*, x = y, x < y) which adds ordering <: E2 -*• B to the field of reals. What new functions / : E* —> E can be computed? 16. Is every cyclic subgroup (t) of the circle group S1 While decidable?
328
J. V. Tucker and J. I. Zucker
17. If / : E -> E is While computable on (E; 0, 1, x+y, -x, x-y, x~l), is / continuous? 18. Can any continuous function / : E —» E be approximated (in some suitable metric) by a function While computable over (E; 0, 1, a; + 2/, -x, a;-?/, a;"1)? 19. What is the relationship, for functions on E, between computability in the sense of computable analysis, and While computability on (E; 0, I , x + y, -x, x - y , or1)? 20. Is there an algebra A(E) containing E, for which
Jtec(R) =
While(A(R)),
where JZec(fi) is the set of functions computable on E in the sense of recursive or computable analysis? 21. Consider the many-sorted algebra (E, N, [N -» E]; OR, 1R, x + y, -x, 0 N , n + 1, eval) that is an expansion of the additive Abelian group of reals, made by adding the naturals with zero and successor, infinite sequences or streams of real numbers, and the evaluation map eval : [N —> E] x N —> E defined by eval(o,n) = a(n). Are the following functions While computable on this algebra: add: [N -> M]2 x N ->• E defined by add(a,b,n) = a(n) + b(n); Add: [N -» M]2 ->• [N -> E] defined by Add(a, b)(n) = a(n) + b(n)l 22. Are complex conjugation — z and modulus \z\ of a complex number z While computable over the following algebras? (i) (C; 0, 1, x + y, -x, x • y, x~l) (ii) (C; 0, l,i,x + y, -x, x • y, 2T1) 23. Is the set {i} While decidable over either of the fields listed in question 22? 24. Consider the function f ( x ) = 4x(l — x) on the reals. Is the orbit of /, defined by orb(f,x)
= (fn(x)
| n e N, 0 < x < 1}
While computable over the field of real numbers? 25. Are the fractal subsets of C, such as the Mandelbrot and Julia sets, While decidable over the field of complex numbers? 26. Are the following subsets of C either While decidable, or While semidecidable, over the algebra (C, B; 0, 1, i, z\, x+y, —x, x-y, a;"1, = )? (i) The set {i} (ii) The set of all roots of unity (Hi) The set of all algebraic complex numbers
Computable functions on algebras
329
27. Consider the rings TL\X\,... ,Xn] of all polynomials in n indeterminates over the integers. Is the ideal membership relation q€ (pi,... ,p m ) (in q,pi,... ,pm) While decidable over this ring? 28. Consider the rings F[Xi,... ,Xn] of all polynomials in n indeterminates over a field F. Is the ideal membership relation While decidable over this ring? 29. Consider the algebra T(S, X) of all terms over signature £ in the finite set X of indeterminates. Let Ax be the set of assignments to X in an algebra A. Define the term evaluation function T: TE(E, X) x Ax -» A
by TE(t,a) = t(a). Is TE While computable over the algebra formed by simply combining the algebras T(£,X) and A! 30. (a) Are all first-order definable subsets of natural numbers While computable with respect to the following algebras? (i) (N; 0, n + 1, n + m, n • m) (M) (N; 0,n + l,n + m) (in) (N; 0, n + 1) (6) Are the While semicomputable sets precisely the Ei-definable sets with respect to these algebras? 31. Is any set of complex numbers that is first-order definable over the field of complex numbers While decidable over this field? Is any set of real numbers that is first-order definable over the ordered field of real numbers While decidable over this field?
1.3 Relations with effective algebra In computer science, many-sorted algebras are used to provide a general theory of data and, indeed, of whole computing systems. They have been employed to specify and analyse many new forms of data types; classify data representations; characterise which data types are implementable; model systems; analyse the modularisation of computing systems; formalise the correctness of systems; and reason about systems. A many-sorted algebra models a concrete representation of a data type or system; such representations are compared by homomorphisms, axiomatised by equations and conditional equations, and prototyped by term rewriting methods. There is a considerable theoretical and practical literature
330
J. V. Tucker and J. I. Zucker
available which may be accessed through survey works such as Meseguer and Goguen [1985], Wirsing [1991], Wechler [1992] and Meinke and Tucker [1992]. The theory of computable functions and sets on many-sorted algebras is intended to provide an abstract theory of computing to complement this abstract algebraic theory of data. With this in mind we ask the question: How is the theory of While computable functions and sets on many-sorted algebras related to other theories of computability on such algebras? We have mentioned earlier that there are many models of computation that can be applied to an arbitrary algebra and that turn out to define the same class of functions and sets as the While language; these equivalent models belong to the computability theory and are the subject of section 8. Here we will discuss an important approach to analysing computability on algebras called effective algebra. Effective algebra is concerned with what algebras are computable, or effective, and what functions and sets on these algebras are computable, or effective. The subject is explained in Stoltenberg-Hansen and Tucker [1995; 1999a]. A starting point for the discussion is the theory of the computable functions on the set N = {0,1,2,...} of natural numbers. According to the Church-Turing thesis, the class Comp(N) of computable function on N, defined by any one of a number of models of computation, is precisely the class of functions definable by means of algorithms on the natural numbers. As we have noted, the algorithms are often over some algebraic structure on N. In fact, seen from the algebraic theory of data, the algebras used form a class of concrete representations of the natural numbers that is parameterised by both the choice of operations and the precise nature of the number representations (e.g., binary, decimal and roman). The extent to which the theory of computable functions on N varies over the class of these algebras of numbers is an important question, but one that is not often asked. We expect there to be very little variation in practice (but compare questions 1 and 2 of section 1.2). In general terms, a computability theory consists of (a) a class of algebraic or relational structures to define data and operations; and (b) a class of methods, which we call a model of computation, to define algorithms and computations on the data using the operations. A generalised computability theory is one which can be applied to a structure containing the set N of natural numbers to define the set C7omp(N) of computable functions on Comp(H). An abstract computability theory is a computability theory in which the theory is invariant up to isomorphism (in some appropriate sense). To develop an abstract generalised computability theory for any algebra A, and classify the computable functions and sets on A, one can proceed
Computable functions on algebras
331
in either of the following two directions. One can apply computability theory on N to algebras using maps from sets of natural numbers to algebras called numberings. The long-established theories of decision problems in semigroups, groups, rings and fields, etc. are examples of this approach. Furthermore, the theory of computable functions Comp(R) on the set E of real numbers in computable analysis uses computability theory on Comp(N) to formalise how real number data and functions are approximated effectively. Theories based on these approaches are parts of what we here call effective algebra. Alternatively, one can generalise the computability theory on N to accommodate abstract structures; the theory of computable functions on many-sorted algebras developed in this chapter is an example, of course, and more will be said about equivalent models of computation in section 8. However, there are examples of generalised computation theories that are strictly stronger, such as ordinal recursion theory, set recursion theory, higher type recursion theory and domain theory. Typically these four generalised computability theories allow infinite computations to return outputs. To appreciate the diversity of some of these theories it is necessary to examine closely their original motivations; seen from our simple finitistic algebraic point of view, generalised recursion theories have a surprisingly untidy historical development. Let us focus on the first direction. Effective algebra is a theory that provides answers for questions such as: When is an algebra A computable? What functions on A are computable? What sets on A are decidable or, at least, semidecidable? It attempts to establish the scope and limits of computation by means of algorithms for any set of data, by applying the theory of computation on N to universal algebras containing the set of data using numberings. Thus, it classifies what data can be represented algorithmically, and what sets and functions can be defined by algorithms, in the same terms as those of the Church-Turing thesis for algorithms on N. Assuming such a thesis, we may then use the theory of the recursive functions on N to give precise answers to the above questions about algebras, and to the question: What sets of data and functions on those data can be implemented on a computer in principle? The numberings capture the scope and limits of digital data representation and, thus, effective algebra is a general theory of the digital view of computation. More specifically, in effective algebra we can investigate the consequences of the fact that 1. an algebra is computable; 2. an algebra is effective in some weaker senses; and
332
J. V. Tucker and J. I. Zucker
3. a topological algebra can be approximated by a computable or effective algebra. Among the weaker senses are the concepts of semicomputable, cosemicomputable and, most generally, effective algebras. For an algebra to be effective it must be countable, so that its elements may be enumerated. For an algebra to be effectively approximable it must have a topological structure, so that its elements may be approximated; the bulk of interesting topological algebras are uncountable. A full account of these concepts is given in Stoltenberg-Hansen and Tucker [1995]. At the heart of the theory of effective algebra is the notion of a computable algebra: a computable algebra is an algebra that can be faithfully represented using the natural numbers in a recursive way. Here is the definition for a single-sorted algebra: Definition 1.1. An algebra A = (A; c\ , . . . , cp, F\ , . . . , Fq) is computable if: (?) the data of A can be computably enumerated— there exists a recursive subset J7a C N and a surjection
a: na -» N called a numbering, that lists or enumerates, possibly with repetitions, all the elements of A; (ii) the operations of A are computable in the enumeration—for each operation Fj : An^ —>• A of A there exists a recursive function that tracks the f, in the set $la of numbers, in the sense that for all Xi, . . . , X n j
& ' 'a
(Hi) the equivalence of numerical representations of data in A is decidable — the equivalence relation =a defined by = a(x-2) s recursve. An equivalent formulation, in the algebraic theory of data, is that A is computable if it is the image of a recursive algebra Ha of numbers under a homomorphism a : J7Q —)• A whose kernel =a is decidable. (This simple algebraic characterisation leads to new methods of generalising computability theories: see Stoltenberg-Hansen and Tucker [1995]). What mechanisms are available for computing in a computable algebra? Via the enumeration, the methods include: (i) basic operations of the algebra; (ii) sequencing, branching and iterating the operations;
Computable functions on algebras (Hi) (iv) (v) (vi)
333
any algorithmic construction on the numerical data representation; global search through all elements of the algebra; unlimited storage for data in computations; the equality relation on the algebra via the congruence.
Conditions (i) and (ii) are shared with our While programming model and, indeed, are necessary for an algebraic theory: recall section 1.1. There are also a number of features that extend the methods of our While model, including conditions (Hi) and, more dramatically, (iv). Using the properties of the numbers that represent the data we can perform global searches through the data sets (by means of an ordering on the code set), and store data dynamically without limitations on data storage (by means of a pairing on the code set). Note that condition (vi) is a denning feature of computable algebras and can be relaxed (as in the case of semicomputable or effective algebras, for instance). Note that an algebra A is computable if there exists some computable numbering a for A. The computability of functions and sets over A may depend on the numbering a; thus, to be more precise, we should say that A, its functions and subsets etc. are a-computable. Let us define the computable subsets and functions for such an algebra. Definition 1.2 (Sets and maps). Let A be an algebra of signature S, computable under the numbering a : Oa —» A. (1) A set 5 = Ak is a-decidable, a-semidecidable or a-cosemidecidable if the corresponding set a~l(S) =
{(x1,...,xk)eflc,k\(a(xi),...,a(xk)€S}
of numbers is recursive, recursively enumerable (r.e.) or co-recursively enumerable (co-r.e.) respectively. (2) A function > : A -4 A is an a-computable map if there exists a recursive function / : $la —> HQ such that for all x € f l a , f ( a ( x ) ) = a(f(x)); or, equivalently, / commutes the following diagram:
a]
o
1-
\ o_
Let Compa(A) be the set of all a-computable maps on A. For any computable algebra, there are many computable numberings, some of which may have desirable properties; for example, it is the case that every computable algebra has a bijective numbering with code set N.
334
J. V. Tucker and J. I. Zucker
Let C(A) be the set of all computable numberings of the algebra A. The choice of a numbering a € C(A) suggests that the effectiveness of a subset or function on A may depend on a. To illustrate, let S C A and consider the following questions: Is S decidable for all computable numberings of A; or decidable for some, and undecidable in others; or undecidable for all computable numberings of A? Another question concerns the invariance of computable maps. // A is computable under two numberings a and /? then what is the relation between the sets Compa(A) and Compg(A)? What is Compa(A)t Consider our abstract model based on While programs. We have noted that While(N; 0, n + 1) = Comp(N). The question then arises for our algebras: What is the relationship between While(A) and Compa(A) for an arbitrary computable representation a ? We can prove that if A is computable then While(A) C
P|
Compa(A).
(1.1)
a€C(A)
(In fact this inclusion holds for much weaker hypotheses on A.) The converse inclusion does not hold in general. To see why, consider the algebra (N; 0, n - 1),
and the use of a While program to compute a function / : N™ —>• N. It turns out that for any x\ , . . . , xn € N, f(xi,...,xn)
< max(z1 , . . . , xn)
because assignments can only reduce the value of the inputs. It follows that While(N; 0, n - 1) C
p|
Compa(N; 0, n - 1)
(1.2)
aec(A) because in any numbering the successor function S(o:) = x + 1 can be computed.
Computable functions on algebras
335
More difficult to answer is the question: When is While(A) =
p|
Compa(A)l
Ot€C(A)
Some results in this direction are known. Inequality (1-2) is not a weakness of the abstract theory. Rather it is an indication of the fact that the abstract models provide a more sensitive analysis of finite computations. For example, the abstract theory reveals the special properties of the algebras of numbers that give computability theory on N its special characteristics: the theory of Compa(A) is the same as the theory of While* (A) when A is an algebra finitely generated by constants.
1.4
Historical notes on computable functions on algebras
The generalisation of the theory of computable functions to abstract algebras has a complicated history. On the one hand the connections between computation and algebra are intimate and ancient: algebra grew from problems in computation. However, the fact that it is now necessary to explain how computation theory can be connected or applied to algebra is an aberration, and is the result of interesting intellectual and social mutations in the past. It is a significant task to understand the history of generalisations of computability theory, with questions for research by historians of mathematics, logic and computing, as well as sociologists of science. The story that underlies this work involves the development of algebra; the development of computability theory; interactions between computability theory and algebra; and applications to computing. Some of the connections between computation theory and algebra have been provided in other Handbook chapters: for notes on the histories of effective algebra, see Stoltenberg-Hansen and Tucker [1995]; computable rings and fields, see Stoltenberg-Hansen and Tucker [l999a]; algebraic methods in computer science, see Meinke and Tucker [1992]. In the following notes we discuss the nature of generalisations and point out the earliest work on abstract computability theory. Section 8 is devoted to a fairly detailed survey of the literature. We first list some common-sense reasons for generalising computability theory. A common view is to say that the purpose of a generalisation of computability theory is one or more of the following: (i) to say something new and useful about the original theory; (it) to provide new methods of use in computer science and mathematics; (Hi) to illuminate and increase our understanding of the nature of computation.
336
J. V. Tucker and J. I. Zucker
As will be seen, the theory of computable functions on many-sorted algebras is certainly able to meet the goals (i)-(iii). For a discussion of these and other reasons for generalising computability theory, see Kreisel [1971]. Broadly speaking, it is often the case that a new mathematical generalisation of an old theory focuses on a few basic technical ideas, results or problems in the old theory and makes them primary objects of study in the new theory. If the generalised theory is technically satisfying then a substantial subject can be built on foundations consisting of little more than some modest technical motivations. Recent history records many attempts at generalisations of computability theory that have different, narrower, technical aims than those of the original theory. Indeed, in some cases, if the generalisation can be applied to analyse computation on algebras then its aims need not be particularly useful or meaningful. Generalised computability theories bear witness to the fact that computability theory has several concepts, results and problems that can bear the weighty load of a satisfying generalisation. For instance, on generalising finiteness, and allowing infinite computations, theoretical differences can be found that allow models of computation to cleverly meet goal (i), but not (if) or (in). Computability theory can also support a good axiomatic framework in which deep results can be proved, and generalised computability theories are models. For example, the axiomatic notion of computation theory developed by by Moschovakis and Fenstad elegantly captures basic results and advanced degree theory: see Stoltenberg-Hansen [1979] and Fenstad [1980]. There is usually a good market for general frameworks in theoretical subjects because there is more space in which to seek ideas and show results. Generality is attractive: there are many new technical concepts and the original theory can underwrite their value. Generalisations are developed, gain an audience and reputation, and, like so many other technical discoveries, await an application. Applications can arise in more exotic or commonplace areas than their creators expected. In the case of generalising computability theory, some theories have useful applications (e.g. in set theory), and some languish in the museum of possible models of axiomatic theories of computation. Abstract computability theory developed rather slowly, and owes much to the development of programming languages. A starting point is the notion of the flowchart. The idea was first seen in examples of programs for the ENIAC from 1946, published in Goldstine and von Neumann [1947]. Flowcharts were adapted and used extensively in practical work. For example, standards were provided by the American Standards Association (see American Standards Association [1963] and Chaplin [1970]). To define mathematically the informal idea of a flowchart required a number of papers on flow diagrams, graph schemata and other mod-
Computable functions on algebras
337
els; some commonly remembered papers are: lanov [i960], Peter [1958], Voorhes [1958], Asser [1961], Corn [*196l] and Kaluzhnin [1961]. By the time of the celebrated Bohm and Jacopini [1966] paper on the construction of normal forms for flowcharts, the subject of flowcharts was well established. In some of these papers the underlying data need not be the natural numbers, strings or bits. In particular, in Kaluzhnin [1961] flowcharts are modelled using finite connected directed graphs. These have vertices either with one edge to which are assigned an operation, for computation, or two exit edges to which are assigned a discriminator, for tests. The graph has one vertex with no incoming edge, for input, and one vertex with no outgoing edge, for output. To interpret a so-called graph scheme, a set of functions is used for the operations, and a set of properties is used for the discriminators. Kaluzhnin's work was used in various studies, such as Bigot's early work, and in Thiele [1966], a major study of programming, in which flow diagrams are presented that are not necessarily connected graphs. The semantics of flow diagrams is denned here formally, in terms of the functions El&£(n) = object or data after the nth step in flow diagram A starting at state £, Kl&^(n) = edge in flow diagram A traversed after the n-th step starting at state £. using simultaneous recursions. Thiele's work influenced the formal development of operational semantics as found in the Vienna Definition Language: see Lauer [1967; 1968] and Lucas et al. [1968]. The important point is that predicate calculus with function symbols and equality is extended by adding expressions that correspond with flow diagrams to make an algorithmic language involving graphs. Thus, in the period 1946-66, some of the basic topics of a theory of computation over any set of data had been recognised, including: equivalence of flowcharts; substitution of flowcharts into other flowcharts; transformations and normal forms for flow charts; and logics for reasoning about flowcharts. Flowcharts were not the only abstract model of computation to be developed. Against the background of early work on the principles of programming by A. A. Lyapunov and theoretical work by lanov and others in the former Soviet Union, Ershov [1958] considered computation with any set of operations on any set of data. In Ershov [i960; 1962] the concept of operator algorithms is developed. These are imperative commands made from expressions over a set of operations; the algorithms allow self-modification. The model was used in early work on compilation in the former Soviet Union. See Ershov and Shura-Bura [1980] for information on early programming. Of particular interest is McCarthy [1963], which reviewed the requirements and content of a general mathematical theory of computation. It
338
J. V. Tucker and J. I. Zucker
emphasises the idea that classes of functions can be defined on' arbitrary sets of data. Starting with a (finite) collection F of base functions on some collection of sets, we can define a class C{F} of functions computable in terms of F. The mechanism used is that of recursion equations with an informal operational meaning based on term substitution. An abstract computability theory is an aim—not 'merely' a model of programming structure etc.—and McCarthy writes (p. 63): Our characterisation of C{F} as the set of functions computable in terms of the base functions in F cannot be independently verified in general since there is no other concept with which it can be compared. However it is not hard to show that all partial recursive functions in the sense of Church and Kleene are in C{zero, succ}. This, of course, falls short of a generalised Church-Turing thesis. The paper also mentions functionals and the construction of new sets of data from old, including a product, union and function space construction for two sets, and recursive definition of strings. McCarthy's paper is eloquent, perceptive and an early milestone in the mathematical development of the subject. E. Engeler's innovative work on the subject of abstract computability begins in Engeler [1967]. This contains a mathematically clear account of program schemes whose operations and tests are taken from a first-order language over a single-sorted signature. The programs are lists of labelled conditional and operational instructions of the form k : k:
if 4> then goto p else goto q do ip then goto p
where k, p and q are natural numbers acting as labels for instructions, is a formula of the language and ijj is an assignment of one of the forms x:=c,
x:=y
or
x:=f (y x ,... ,yk)
where x, y , . . . are variables, and c and f are any constant and operation of the signature. Interpretations are given by means of a notion of state, mapping program variables to data in a model. A basic result proved here is this: To each program TT one can associate a formula that is a countable disjunction of open formulae such that for all models, TT terminates on all inputs from A
<=> A |= <j>.
Results involving the definability of the halting sets of programs in terms of computable fragments of infinitary languages will be proved in section 5 and applied in section 6, where we refer to them as versions of Engeler's lemma.
Computable functions on algebras
339
In Engeler [I968a] two new models of computation are given: one is based on a new form of Kleene ^-recursion, the other on a deductive system. The functions computable by the programs and these two models are shown to be equivalent. Engeler's development of the subject in the period 1966-76 addresses original and yet basic questions including the computability of geometrical constructs, exact and approximate computation, and a Galois theory for specifications and programs: see Engeler [1993]. The study of program schemes and their interpretation on abstract structures grew in the early 1970s, along with the theoretical computer science community. Problems concerning program equivalence, decidability and the expressive power of constructs were studied and formed a subject called program schematology (see, for example, Greibach [1975]). The problem of finding decidable properties, and especially finding decidable equivalence results, is work directly influenced by lanov [i960]. The subject of program schemes, and the promise of decidability results on abstract structures, was addressed in the unpublished Luckham and Park [1964], and early undecidability results appeared in Luckham et al. [1970]. Program schematology was part of the response to the need to develop a comprehensive theory of programming languages, joining early work on programming language semantics, program verification and data abstraction also characteristic of the period. We will look at the subject again in section 8. The next milestone is that of Friedman [l971a]. Friedman's paper has received a fine exegesis in Shepherdson [1985] which we recommend. Against a backcloth of growing interest in the generalisations of computability theory by mathematical logicians, and inspired by the work of Moschovakis on computation, Friedman considered the mathematical question (in Shepherdson's words): What becomes of the concepts and results of elementary recursion theory if, instead of considering only computations on natural numbers, we consider computations on data objects from any relational structure? In this he gave four models of computation for an algebra A. The first two were based on register machines, the programs for which were called finite algorithmic procedures (or faps). The third was a generalisation of Turing machines. The fourth was a model based a set of r.e. lists of conditional formulae of the form Riik...tcRtkt -*• ti for i = 1,2,..., where the fly are tests and the ti is a term, called effective definitional schemes. In section 8 we will discuss these models again. All four models compute the partial recursive functions on the natural numbers. Freidman organised some 22 basic theorems of computability theory on the natural numbers N into six groups which were defined by
340
J. V. Tucker and J. I. Zucker
the properties of a structure A that are sufficient to prove the theorems on A. Also noteworthy are results which showed some of these models are not equivalent in the abstract setting. Friedman's paper is technically intense, with several good ideas and results. Looking back at the literature, one wonders why such an indispensable paper had not appeared before. The theory of computation on natural numbers and strings began in mathematical logic, motivated by questions in the foundations of mathematics. However, the theory of computation on arbitrary algebraic and relational structures began and was sustained in computer science, motivated by the need to model programming language constructs. Mathematical logic plays two roles: firstly, it provides knowledge of abstract structures, formal languages and their semantics; and secondly, it provides a deep theory of computation on natural numbers. Both the complicated history of algebra and computability theory mentioned earlier (and sketched in the historical notes of other Handbook chapters), and the development of abstract computability since the 1950s, have much to offer those interested in the historical development of mathematical theories. In the matter of the development of abstract computability these brief notes, coupled with our survey in section 8, suggest some questions a historical analysis might answer. Why have there been many independent attempts at making models of computation but relatively few attempts to show equivalencies, or undertake sustained programmes of theoretical development and applications? Why have there been so many demonstrations of an ability to ignore earlier work and to reinvent ideas and results? Why was the development of the theory so slow and messy? Why was computability theory on the natural numbers not generalised to rings and fields, or even relational structures, before the Second World War? Why did the subject not find a home in mathematical logic? It is clear that computer science played an essential role in creating the theory of computable functions on abstract algebras. One is reminded of McCarthy's [1963] commonly quoted words: It is reasonable to hope that the relationship between computation and mathematical logic will be as fruitful in the next century as that between analysis and physics in the last. The development of this relationship demands a concern for both applications and for mathematical elegance. It also demands patience.
1.5
Objectives and structure of the chapter
Computability theory over algebras can be developed in many directions and can be used in many applications. In this short introduction we have chosen to emphasise computation on general many-sorted algebras. We see algebra as providing a general theory of data that is theoretically satisfying and practically useful. Therefore, theories of what is computable
Computable functions on algebras
341
over algebras are fundamental for a general theory of data. In section 2 we define the basic algebraic notions we will need: algebras with Booleans and naturals, relative homomorphisms, terms and their evaluation, abstract data types, etc. In particular, we look at expanding an algebra A, by adding new types such as finite sequences to make a new algebra A* that models arrays, and adding infinite sequences to make a new algebra A that models infinite streams of data. In section 3 we begin the study of computing on algebras with While programs. For a satisfactory theory, the algebras are required to include the Booleans and standard Boolean operations. Such algebras are called standard algebras. The semantics of While programs on A is given by a new technique called algebraic operational semantics (AOS). This involves axioinatising a function CompA that defines the state CompA (S, o~, t) at time t in the computation by program 5 starting in intial state a. From this we obtain a state transformer semantics in which a program 5, applied to a state o~, may give rise to a final state [S']"4^). Simple but important properties of computations are examined. First, the invariance of computations under homomorphisms and isomorphisms: if algebras A and B are isomorphic, then the semantical interpretations of any While program S on A and B are isomorphic. This result has many consequences; for example, it comfirms that executing a While program on equivalent implementations of a data type results in equivalent computations. The second property is that each computation by a While program S takes place in the subalgebra of A generated by the input. This is a key to understanding the nature of abstract computation: in any algebra computations are local to the input in this sense and, for instance, searches are at best local. Next, in section 4, we consider the universality of computation by While programs. Let A be an algebra with Booleans and naturals. We can code the While programs •So , S\ , 82, • • •
by the natural numbers in A and ask if there exists a universal While program to compute the function Univ"4 on A such that
We prove that the universal function is While computable on A if, and only if, the term evaluation function is While computable on A. The evaluation of terms is not always computable. However, it is While computable in several commonly used algebras such as: semigroups, groups, rings, fields, lattices, Boolean algebras; this because such algebras have computationally efficient normal forms for their terms. For any algebra A, the algebra A" of finite sequences from A has the property
342
J. V. Tucker and J. I. Zucker
that term evaluation is always While computable on it; hence, the model of While* programs is universal. In section 5 we turn our attention to sets. We begin with a study of computable and semicomputable sets. We prove Post's theorem in the present setting. We also study the ideas of projections of computable and semicomputable sets. It turns out that the classes of computable and semicomputable sets are not closed under projection. The notion of projection is very important since it distinguishes clearly between forms of specification and computation. Furthermore, it focuses our attention to the difference between local search and global search in computation. Projections also lead us to consider the relationship between While programming and certain non-deterministic constructs on data. These include: search procedures; initialisation mechanisms; and random assignments. Next, with each While program is associated a computation tree. With this technique, we prove that every semicomputable set is definable by an effective infinite disjunction of Boolean terms over the signature. In section 6 we illustrate the core of the theory with a study of its application to computing sets of real and complex numbers over various many sorted algebras. We include some pleasing examples from dynamical systems. In section 7 we return to the special properties and problems of computation of the reals. More generally, we study computation on topological algebras. A key consideration is the property that if a function is computable then it is continuous. To guarantee a good selection of applications we use partial functions, which raises interesting topological issues. This study of programming over topological algebras contains new material. We also contrast exact versus approximate computation on the reals. The following fact was observed in Shepherdson [1976]. Let / be a function on the reals. Then / is computable in the sense of computable analysis if, and only if, there is a function g which is While computable over the algebra (E, B, N; 0, 1, x + y, x.y, —x,...) such that \f(x)-g(n,x)\<2-n for all n € N and x 6 K. We extend and adapt this result to topological algebras. In section 8 we survey other models of computation and see their relation with While programs. We consider briefly: /^-recursive functions; register machines; flowcharts; axiomatic methods; set recursion; and equational definability. A generalised Church-Turing thesis is discussed. There are many subjects that we have omitted from the discussion, for example: the delicate classification of the power of constucts, including types; computations with streams; program verification; connections with proof theory; connections with model theory; degree theory; and generalised complexity theory. There will be good work by many authors that
Computable functions on algebras
343
we have neglected to mention, from ignorance or forgetfulness. We will be pleased to receive reminders, information and suggestions. Abstract computability theory is a subject that offers its students considerable theoretical scope, many areas of application, and scientific longevity. We hope this chapter provides a first introduction that is satisfying, stimulating and pleasurable. We thank Jan Bergstra (Amsterdam), Martin Davis (NYU and Berkeley), Jens Erik Fenstad (Oslo), Dag Norman (Oslo), Viggo StoltenbergHansen (Uppsala) and Karen Stephenson (Swansea) for useful discussions on aspects of this work. We thank Peter Lauer (McMaster) and Ithel Jones (Swansea) for discussions and information on the history of programming language semantics. We also thank our colleagues and students in Swansea and McMaster for their helpful responses to the material in courses and seminars, especially Jeff Koster, Matthew Poole, Dafydd Rees, Kristian Stewart, Anton Wilder and Ian Woodhouse. Special thanks are due to Sol Feferman, who presented some of the material of this chapter at a graduate course in computation theory at Stanford University in spring 1999, and provided us with valuable feedback. We are particularly grateful to Jane Spurr for her excellent and essential work in producing the final version of the chapter. The second author is grateful for funding by a grant from the Natural Science and Engineering Research Council of Canada.
1.6
Prerequisites
First, we assume the reader is familiar with the theory of the recursive functions on the natural numbers. It is treated in many books such as Rogers [1967], Mal'cev [1973], Cutland [1980] and Machtey and Young [1978]. An introduction to the subject is contained in this Handbook (see Phillips [1992]) and other handbooks (e.g. Enderton [1977]). Secondly, we assume the reader is familiar with the basics of universal algebra. Some mathematical text-books are: Burris and Sankappanavar [1981] and McKenzie et al. [1987]. An introduction to the subject with the needs of computer science in mind is contained in this Handbook (see Meinke and Tucker [1992]) and in Wechler [1992]. The application of universal algebra to the specification of data types is treated in Ehrig and Mahr [1985], Meseguer and Goguen [1985] and Wirsing [1991]. The theory of computable and other effective algebras is covered by Stoltenberg-Hansen and Tucker [1995]. Thirdly, we will need some topology. This is covered in many books, such as Dugundji [1966] and Kelley [1955] and in a chapter in this Handbook (see Smyth [1992]). Finally, we note that the subject connects with other subjects, including term rewriting (see, for example, Klop [1992]) and domain theory (see, for example, Stoltenberg-Hansen et al. [1994]).
344
2
J. V. Tucker and J. I. Zucker
Signatures and algebras
In this section we define some basic algebraic concepts, establish notations and introduce three constructions of many-sorted algebras. We will use many-sorted algebras equipped with Booleans, which we call standard algebras. Sometimes we use algebras with the natural numbers as well, which we call N-standard algebras. All our algebras have total operations, except in section 7, where we compute on topological partial algebras. We are particularly interested in the effects on computations of adding and removing operations in algebras. To keep track of these changes, we use expansions and reducts of algebras, and relative homomorphisms. The constructions of new algebras from old involve adding (i) un