Spectral Theory of Automorphic Functions (Proceedings of the Steklov Institute of Mathematics)

•

(I'; 1) l�jI H .

•

.

I' E 0R .

.

.

•

.

vii

.

.

.

.

.

•

.

.

.

•

.

•

(r; X).

.

•

.

.

.

•

.

.

.


I'M

•

83

88 SS

101

•

(; HCTpUBIIaJIb HMM COIl3MC-

rp�'llllt.l 1', cou:nlcpu;\lOii C rPYIIIIOH .

73

93

(:r; 1')

pa:lJ[();{,CIIIlH {(';IH Ol Ie pa To pa

.

1) lVIH r pyl l ll! 1

orrcpaTop,t '�l (I'; .

•

:lITCI,Pl'TIIOl'O CIll'I'Tpa. CIICI;l'P olI e pa To p u �l (1'; 1) AJIH .

§ G.1 Ilpo6JICMbI TeoplIII

Tl'O1)(>.\1 f,1

.

48

72

Bpii,1J1-Ct',l/bOl'pra.

CC JlhOe pra lIa oom'l' ITllIpoj.;nii

39

69

('.W;(
.

30

61

'

4.
22

48 53 55

CooTHoIIIeHHe l\-Iaaca-Ccm,oppra '.'

18 19 22

. . . . . . . . . . . . BCHThl 9� (s; f; z) n JIOJIyIIJIoC/,onr. Hl' S > o . § 2.3 Co6cTBeuHI)Ie
15

orrepaTopa

2.2 lIHTerpaJIbHOC yp U HHC HIIC (Da,'l;:Wl'fla II aaamITII'ICeTW(' ](PO;�OJIa\cHne PC30JIb­

.

14

•

.

.

'

104 104 109 116 126

§ 6.5
CeJIb6epra i\JIFl KpaeB oii

MHoroyrOJIbHHRe M.

J� IlPJfXJIC

. . . . . . . . . . . . . . . . 3a;laqll

a)
I1a perYJIFlpHO:\1

133 .

.

.

134

.

.

.

§ 6.6 BJIeMeHThl TeopHH )1;3 eTa-
140

3a;�(l'III J�IIPIIXJIC lIa M

147

§ 6.7 Ou;eHRa OCTaTKa B

152

6)



Ha perYJIFlpHOM MHoroyrOJIbHIIKe

r JI a

.

B a

BeiiJIFl )1;JHI •

•

•

•

•

•

•

7. CrreU;lIaJIbHaFl TCOpIIFl B03:11YID;CHIIH t!>YHRU;IIii

§ 7.1 ,l(et!>0pMaD;uFl rpyrrIIf>I r, cnCRTp O Hc p a Top a '0l BeHThl �

(s; f; X)

•

•

•

•

•

•

•

•

•

•

•

•

•

•

•

.

•

.

.

•

. . .

•

OI w paTo pa

cnCKrpa

HeKoTophle CTHMYJIbI II rrepcrrcRTIIBhI pa3BIITlIH MOP
•

J,paeBoii

•

•

•

. . •

'2l (r; Z).

cllcl·apa.lbHoii TCO[lnn aIlT()-

(I'; z) n oeouhle •

.

•

•

•

•

•

•

•

•

T()'IKH

•

•

.

.

pC30.1b-

. § 7.2. HYJIU )1;SeTa- II
3Ha'IemmOrrepaTopa

.

'�l

•

.

(1'.3;

.

.

.

.

.

J)

.

.

.

.

.

.

.

.

.

157

. . .

157 162 169

viii

Correspondence between Trudy Mat. Inst. Steklov. and Proc. Steklov Inst. Math.

"

Russian imprint

Vol.

English imprint

1966

74

1967

1971

107

1976

1980

139

1982

1965

75

1967

1968

108

1971

1976

140

1979

1

1965

76

1967

1971

109

1974

1976

141

1979

2 3

Russian imprint

Vol.

English imprint

Russian imprint

Vol.

English imprint Year Issue

1965

77

1967

1970

110

1972

1976

142

1979

1965

78

1967

1970

111

1972

1977

143

1980

1965

79

1966

1971

112

'l973

1979

144

1980

2

1965

80

1968

1970

113

1972

1980

145

1981

1

1966

81

1968

1970

114

1974

1978

146

1980

3

1966

82

1967

1971

115

1974

1980

147

1981

2

1965

83

1967

1971

116

1973

1978

148

1980

4

1965

84

1968

1972

117

1974

1978

149

1981

3

1966

85

1967

1972

118

1976

1979

150

1981

4

1965

86

1967

1973

119

1976

1980

151

1982

2

1966

87

1967

1974

120

1976

1980

152

1982

3

1967

88

1969

1972

121

1974

1981

153

1982

4

1967

89

1968

1973

122

1975

1967

90

1969

1973

123

1975

1967

91

1969

1976

124

1978,

1966

92

1968

1973

125

1975 1975

1967

93

1970

1973

126

1968

94

1969

1975

127

1977

1968

95

1971

1972

128

1974

1968

96

1970

1973

129

1976

1968

97

1969

1978

130

1979,

1968

98

1971

1974

131

1975

1967

99

1968

1973

132

1975

1971

100

1974

1973

133

1977

1972

101

1975

1975

134

1977

1967

102

1970

1975

135

1978,

1968

103

1970

1975

136

1978,

1968

104

1971

1976

137

1978,

1969

105

1971

1975

138

1977

1969

106

1972

ix

Issue

3

Issue

4

Issue Issue Issue

1 2 4

Correspondence between Trndy Mat. Inst. Steklov. and Proc. Steklov Inst. Math. Russian imprint

Vol.

English imprint

1966

74

1967

1971

107

1976

1980

139

1982

1965

75

1967

1968

108

1971

1976

140

1979

1

1965

76

1967

1971

109

1974

1976

141

1979

2

III

1972

1976

142

1979

3

1972

1977

143

1980

1

Russian imprint

1965

77

1967

1970

1965

78

1967

1970

Vol.

110

English imprint

Russian imprint

Vol.

English imprint Year Issue

1965

79

1966

1971

112

1973

1979

144

1980

2

1965

80

1968

1970

113

1972

1980

145

1981

1

1966

81

1968

1970

114

1974

1978

146

1980

3

1966

82

1967

1971

115

1974

1980

147

1981

2

1965

83

1967

1971

116

1973

1978

148

1980

4

1965

84

1968

1972

117

1974

1978

149

1981

3

1966

85

1967

1972

118

1976

1979

150

1981

4

1965

86

1967

1973

119

1976

1980

151

1982

2

1966

87

1967

1974

120

1976

1980

152

1982

3

1967

88

1969

1972

121

1974

1981

153

1982

4

1975

1967

89

1968

1973

122

1967

90

1969

1973

123

1975

1967

91

1969

1976

124

1978,

1966

92

1968

1973

125

1975 1975

1967

93

1970

1973

126

1968

94

1969

1975

127

1977

1968

9S

1971

1972

128

1974 1976

1968

96

1970

1973

129

1968

97

1969

1978

130

1979,

1968

98

1971

1974

131

1975 1975

1967

99

1968

1973

132

1971

100

1974

1973

133

1977

1972

101

1975

1975

134

1977

1967

102

1970

1975

135

1978,

1968

103

1970

1975

136

1978,

1968

104

1971

1976

137

1978,

1969

105

1971

1975

138

1977

1969

106

1972

ix

Issue

3

Issue

4

Issue Issue Issue

1 2 4

INTRODUCTION

In his seminal paper [5 1 ], Atle Selberg introduced fundamental new ideas into the classical theory of automorphic forms, a theory whose origins lie in the works of Riemann, Klein, and Poincare. These ideas are connected with an extension of the earlier notion of an automorphic function (or form). Instead of an analytic automor­ phic function, Selberg considered a mapping which is automorphic relative to a given finite-dimensional unitary representation of a discrete group and is an eigen­ function for a commutative ring of elliptic differential operators. At that time Hans Maass' article [36] had appeared, containing similar nonanalytic automorphic " wave" functions defined in a special situation; however, it was Selberg who first took a serious look at Maass' work. In order to implement the new ideas, certain new techniques, not normally used in the classical theory of automorphic functions, were invoked: first, methods from the theory of selfadjoint operators in Hilbert space; then, methods from group representation theory over various fields, methods which turned out to be more natural in spaces of rank greater than one. It was the subsequent global development of Selberg's ideas in the setting of the representation theory of Lie groups which determined the true place of the classical theory of automorphic functions-in both its function theoretic and number theoretic aspects -in the new more general theory, and also clarified the interaction between the old and new theories. It is now already possible to speak of the " Selberg theory", although this theory is still in its initial stages of development. The foundation of the theory consists of: 1 ) theorems on expansion in automorphic eigenfunctions of Laplacians; 2) Selberg trace formulas; and 3) the theory of the Selberg zeta-functions. Here one should also include several very important applications of a theoretical nature (some of which we recently examined in the survey article [66]): 4) applications to global problems in modem number theory, in particular, to the arithmetic theory of automorphic forms (the so-called " Langlands philosophy"); 5) C) applications to solving some difficult concrete problems in number theory, for example, the proof of the refined Kummer conjecture on cubic characters (see [ 1 7]); 1980

Mathematics Subject ClassificatiOn.

Primary lOD05, lOD20, lOD40, lOHIO, 32N05, 32N15;

(1 )Further references to articles on these topics in Selberg theory can be found in our survey article [66)

Secondary lOH26, 20HlO, 22E40, 30D05, 30D15, 30F35. and in the works cited there.

INTRODUCTION

6) applications in the theory of geometric and topological invariants of Rieman­ nian manifolds; 7) the Selberg zeta-function from the point of view of analytic number theory, in particular, the Selberg zeta-function and the Riemann zeta-function; 8) Selberg theory and quasiconformal mappings of Riemann surfaces (see [ 1 1 ] and [ 12]); and the -Selberg trace formula from the point of view of the spectral theory of selfadjoint operators in Hilbert space as a model of stationary and nonstationary scattering theories; applications to classical (Dirichlet, Neumann) boundary value problems of mathematical physics. In this monograph we shall primarily be interested in questions concerning the foundations of Selberg theory and those applications which have a function theoretic character, corresponding in the above list to items 1 )-3), 8) and and parts of 6) and 7). We shall call this entire circle of questions the spectral theory of automor­ phic functions. We are then almost forced to choose to work with weakly symmetric space (in Selberg's sense) and the set of discrete groups acting on it: the Lobachevsky plane and Fuchsian groups of the first kind. In fact, it has recently become apparent that the theory of automorphic functions in spaces of rank greater than one is for objective reasons an arithmetic theory, in essence a branch of modern number theory. Conversely, the hyperbolic plane stands out-even among the other spaces of rank one-by the abundance of different discrete transformation groups, among which the arithmetically defined groups occupy a very modest place; this is what makes the spectral theory of automorphic functions particularly significant. The choice of Fuchsian groups of the first kind is dictated by the good spectral properties of the automorphic Laplacian (finite multiplicity of the continuous spectrum), the Selberg zeta-function (continuation onto the entire plane, a functional equation, low order of meromorphicity), and also by certain traditional applications. We now give a more detailed description of the contents. Chapter 1 is introduc­ tory; it contains the necessary notation and several definitions and auxiliary facts. Chapter 2 is devoted to a proof of the theorem on expansion in eigenfunctions of the automorphic Laplacian �(f; X) for an arbitrary Fuchsian group of the first kind f and an arbitrary finite-dimensional unitary representation X of f (such a choice of f and X will be written f E we, X E 9C(f); see § 1 .2). We shall pay particular attention to the most difficult situation f E we 2 ' X E 9C ( f), where, by definition, we 2 is the set of all groups f E we having noncompact fundamental domain, and 9C sCf) is the set of all so-called singular representations X E 9C(f). Whenever f E we 2 and X E 9C ( f ) the spectrum of the operator �(f; X) is not purely discrete; it also contains an absQlutely continuous spectrum of multiplicity equal to the total degree of singularity of the representation X relative to the group f. The proof of the theorem generalizes L. D. Faddeev's proof of a less general theorem on expansion in eigenfunctions of the operator � (f; 1), where X = 1 is the trivial one-dimensional (and thus singular) representation of f E we 2 (see The proof is based on a study of the resolvent 9l(s; f; X) of the operator �(f; X) using methods from the theory of perturbations of the continuous spectrum of selfadjoint operators. This proof includes several steps. The kernel of the resolvent as an integral operator far from the spectrum is studied in §2. 1 . We then construct an

9)

9),

s

s

,

[9]).

INTRODUCTION

3

auxiliary operator � (s; f; X), which is uniquely determined by 9T(s; f; X), and, by means of a certain integral equation, continue it analytically to a neighborhood of the spectrum of m: ( f ; X) (more precisely, to part of the Riemann surface Re s > 0, which is a two-sheeted covering of the spectral plane containing the spectrum; see §1.4). We shall call this integral equation the Faddeev equation, since it was first introduced into the spectral theory of automorphic functions in [9] in the scalar case (dim V = 1, V the space of the representation X) for the trivial representation X. The next steps in the proof of the expansion theorem are meromorphic continuation of the kernel of the resolvent to a neighborhood of the spectrum and the investigation of the singular points of the resolvent '(§2.2). Finally, by determining the eigenfunc­ tions of the continuous spectrum of m: ( f ; X) in terms of its resolvent, finding the scattering matrix and proving certain of its properties, we are able in §2.3 to complete the proof of the theorem on expansion in eigenfunctions of m: ( f ; X). There is another proof of the expansion theorem for m:(f; X) which is also valid for arbitrary f E WC 2 and X E 9Csef). It was published in [46] by Roelcke, who assumed a very essential conjecture concerning meromorphic continuation of Eisen­ stein series. The conjecture was later proved in a famous paper by Langlands [33]. However, we wish to emphasize that our method of proof is preferable in our case (for the Lobachevsky plane and for any space of rank one), since it enables one to obtain additional information concerning the spectral properties of m:(f; X), more precisely, the properties of its resolvent; this information is of great importance in constructing an analog of Artin theory for the Selberg zeta-function (see §6.2), and also in studying the spectral properties of m: (f; X) under a deformation of the group f (see §7.1). Chapter 3 is devoted to a refinement of the theorem on expansion in eigenfunc­ tions of �(f; X) relating to the part of the theorem concerning the continuous spectrum. In §3.1 Eisenstein series are defined for a group f E WC 2 and a representa­ tion X E 9C sef). (These series are the meromorphic continuation of the eigenfunc­ tions of the continuous spectrum of m:(f; X).) We then construct their Fourier expansion relative to parabolic subgroups fa C f. Such expansions have only been considered before in the scalar theory (dim V = 1) (see [50] for X a nontrivial representation, and [28] for X the trivial representation). At the end of §3.1 we prove meromorphicity of Eisenstein series (i.e., the basic hypothesis in Roelcke's paper [46]) and meromorphic continuation of the kernel of the resolvent to the entire Riemann surface which is a double covering of the spectral plane. The proof of these results is based on Chapter 2, which, in particular, gives us meromorphicity of the scattering matrix, a functional equation for the scattering matrix, a functional equation for Eisenstein series, and, finally, a functional equation for the kernel of the resolvent of �(f; X). In §3.2 we give a description, based on [46], of the Maass-Selberg relation. The first part of §3.3 is devoted to a certain intrinsic characterization of the subspace of the continuous spectrum, and, to a lesser extent, the subspace of the discrete *

*Translator's note. The general case was also considered by Polly Moore, Generalized Eisenstein series: incorporation of a nontrivial representation of r, Ph. D. dissertation, Univ. of Washington, Seattle, Wash., 1 979.

INTRODUCTION

spectrum of m(f; X) in a suitable Hilbert space :Je(f; X). The theory developed here uses elements of the spectral theory of Selberg [52], Roelcke [46], Godement [14], Langlands [33], and Kubota [28], with emphasis on the properties of Eisenstein series and the Maass-Selberg relations, and especially uses the theorem on expansion in eigenfunctions of m(f; X), proved in Chapter 2 based on Faddeev' s method in the scalar theory of automorphic functions. Part 2) of Theorem 3.3.2 is a version for resolvents of the well-known theorem of Gel'fand and PjateckiI-Sapiro (see [l3], Chapter I, §6, or [16], Chapter I, §2). In the second part of §3.3 we introduce a fundamental class of integral operators to be considered in the spectral theory, and we prove some properties of these operators. They were first introduced by Selberg in [51]. The idea of studying these operators by means of the resolvent �(s; f; X) arose in the scalar theory in work by L. D. Faddeev, V. L. Kalinin and the author (see [72]). In §3.4 we derive a vector version of the integral equation, which we call the Selberg-Neunhoffer equation. This integral equation was first proposed to Selberg to prove meromorphicity of Eisenstein series in the scalar theory (dim V = 1) (see [50]). Later, it was reconsidered by Neunhoffer in [39], also in the framework of the scalar theory. We note that the study of the Selberg-Neunhoffer equation is the basis for the third of the methods presently known for proving meromorphicity of Eisenstein series for f E 9JC2 and X E 91/f) (see [66], §8). Unlike in [39] and [50], in §3.4 we derive the Selberg-Neunhoffer equation (dim V � 1) from the Faddeev equation, using the information about the resolvent �(s; f; X) of m(f; X) in Chapter 2, for a single purpose-finding an a priori estimate for the order of meromorphicity of the Eisenstein series and the scattering matrix. In §3.4 we finish our treatment of this theorem. §3.5 is devoted to studying the properties of the determinant of the scattering matrix. Here we generalize results of the scalar theory due to Selberg (see [50]). The basic result of the section is Theorem 3.5.5, which gives a special canonical product over the zeros and poles of the determinant which is different from the Weierstrass product. Chapter 4 is concerned with proving the Selberg trace formula in the general situation f E 9JC, X E 91(f). Again we emphasize the most difficult and least well-known case f E 9JC2' X E 91 if). In §4.1 we prove nuc1earity of the operator K(f; X)�o(f; X), where �o(f; X) is the orthogonal projection in the Hilbert space :Je(f; X) onto the subspace :Jeo(f; X) of cusp-vector-functions. The proof of the theorem is based upon ideas from the theory of perturbations of continuous spectra (see [72]) and results from Chapter 2. In §4.2. we justify the spectral trace formula; this reduces to proving uniform convergence of certain integrals. Our method generalizes the method in [72] and the Selberg-Arthur method for justifying the trace formula for arithmetic groups in the rank one case (see [l]). In §4.3 we transform the spectral trace formula in §4.2 to the Selberg trace formula. The following special cases of the Selberg trace formula for f E 9JC, and X E 91(f) are well known and have often been examined in the literature: 1) f E 9JCI and X E 91(f) (see [51], [13], [19] and others); 2) f E 9JC2' X E 91/f) and dim V = 1 (see [50], [28] and [72]); 3) f E 9JC2 and X E 91 ( f) (see [51D; and 4) f is 1

r

.., ,.

INTRODUCTION

5

an arithmetic group in Wl2 and X E 9Cif) (see [23], [7], [ 1 ] and [ 1 8]) (see § 1 .2 for the notation). However, the general case of the Selberg trace formula for f E Wl2 and X E 9C if) has not been considered before, as far as we know, either in the published literature or in Selberg's lectures at Princeton ( 1 952) and Gottingen ( 1954) (see [50]); hence, we shall concentrate our attention in §4.3 on this case. The Selberg trace formula which we obtain for a general group f E Wl2 and a general represen­ tation X E 9Ci f) clearly includes all of the earlier trace formulas 1)-4). At the beginning of §4.4 we give a proof of a vector version of an asymptotic formula which we have referred to as the Weyl-Selberg formula (see [68]). In the scalar theory this formula was first obtained by Selberg (see [50]). In our opinion it is the natural generalization, in the spectral theory of a self-adjoint operator whose spectrum is not in general purely discrete, of Hermann Weyl's classical asymptotic formula for the distribution function of the eigenvalues of an operator with purely discrete spectrum. In §4.4 we also give an a priori estimate for the distribution function of the values of the norms of primitive hyperbolic conjugacy classes in a group f E Wl2 . Using these two results, later in §4.4 we refine the order of meromorphicity of the determinant of the scattering matrix, and, finally, we give an extension of the Selberg trace formula to a broader class of functions than in §4.3. Chapter 5 is devoted to the theory of the Selberg zeta-function and its spectral and geometric applications in the general situation f E Wl2 and X E 9C(f). In §5. l we give the definition and prove the basic properties of the Selberg zeta-function Z(s; f; X). The function Z(s; f; X) is connected with the Selberg trace formula in the same way as the Riemann zeta-function is connected with Weil's "explicit formula" in analytic number theory. Thus, all of the basic properties of Z(s; f; X) are determined by the Selberg trace formula. In §5. 1 we prove a fundamental formula for the logarithmic derivative of the Selberg zeta-function (Theorem 5 . 1 . 1). This formula gives us meromorphicity of Z(s; f; X), a functional equation, and also a complete description of all the zeros and poles of the zeta-function (Theorems 5. 1 .3 and 5 . 1 .4). In [68] we published similar results for the scalar theory of the zeta-func­ tion Z(s; f; X)(dim V = 1 ); the stimulus for all of these investigations was the brief remarks of Selberg at the end of his lectures [50]. §5.2 is devoted to estimating the remainder in the Weyl-Selberg asymptotic formula. More precisely, we construct an asymptotic formula with three principal terms and a remainder term of order OCTjln T) (Theorem 5.2. 1 ); here the justifi­ cation for the first principal term is the content of Theorem 4.4. 1 in §4.4 and was what lead to the Weyl-Selberg formula. The derivation of the formula is based on the theory of the Selberg zeta-function Z(s; f; X), and is a spectral application of that formula. The method of proof generalized a method well known in analytic number theory for constructing an asymptotic formula for the number of nontrivial zeros of the Riemann zeta-function in a "large" rectangle in the critical strip (see, for example, [57]). Hejhal [ 1 9] and Randol [44] obtained the analogous formula for a group f E Wl) and the trivial one-dimensional representation X, and the author [68] did the same for a group f E Wl2 and X E 9Cif) (dim V = 1). The purpose of §5.3 is to derive an asymptotic formula for the distribution function for the values of the norms of primitive hyperbolic conjugacy classes in a

6

INTRODUCTION

given Fuchsian group. This formula should be regarded as the geometric application of the theory of the Selberg zeta-function Z(s; f; X) which we develop in §§5.1 and 5.2. The formula is analogous to the refined asymptotic law for the distribution of prime natural numbers, and it is connected with the Selberg zeta-function in the same way as that asymptotic law is connected with the Riemann zeta-function. In the theory of the Selberg zeta-function Z(s; f; X) for f E WCI, such a formula is apparently due to Selberg and Huber. There are published proofs in papers by Huber [2 1 ] and Hejhal [ 19]. For our type of group f E WC2 the formula was published by A. I. Vinogradov and the author in the note [71] (see also [66]). Chapter 6 is largely concerned with a refinement of the theorem on expansion in eigenfunctions of the operator �(f; X) for f E WC2 and X E 91 if) in the aspect relating to the discrete spectrum of �(f; X). The first part of §6.1 is an introduction to the chapter as a whole. In this section we formulate the basic problems of the theory of the discrete spectrum that are still unsolved. Special attention is accorded the so-called Roe1cke conjecture to the effect that there are infinitely many eigenval­ ues of the discrete spectrum of �(f; X) for arbitrary f E WC2 and X E 91 if) (see [66]). As early as his lectures [50], Selberg indicated that, from a formula of which a more general version is now known as the Weyl-Selberg formula (see Theorem 4.4. 1), one cannot, in general, extract any information concerning the asymptotic behavior of the distribution function for the eigenvalues of the discrete spectrum of �(f; X)(X = I), except in certain cases of arithmetic groups f for which one can explicitly compute the corresponding determinants of the scattering matrices in terms of the Riemann zeta-function and other special functions of analytic number theory. And in second part of §6.1 we consider the examples of arithmetic groups (congruence-subgroups) for which the Weyl-Selberg formula and explicit formulas for the determinants of the scattering matrices imply Roe1cke's conjecture (and in a significantly stronger form). More precisely, in Theorem 6.1 .2 we establish Weyl's formula for the eigenvalues of the discrete spectrum of �(f; 1) in the case when f is a congruence-subgroup fl(m) or fim); and this result is made even stronger in Theorem 6.1 .1 for a congruence-subgroup fo(m). In §6.2 we derive a formula for'the Selberg zeta-function of a compact Riemann surface (Theorem 6.2.3); this formula should be regarded as a transcendental analog of, the Artin-Takagi formula in algebraic number theory. The ground field in this situation corresponds to an arbitrary normal subgroup of finite index in the fundamental group of an arbitrary compact Riemann surface of genus no less than two. Our formula is obtained as a consequence of a more general theory for the resolvent of the operator �(f; X), a th�ory which holds for any group f E we and any representation X E 91(f) (Theorems 6.2. 1 and 6.2.2). Another consequence of this theory is the Roe1cke conjecture. We prove that for every group f E we 2 there exists a subgroup of finite index fl C f such that the distribution function of the eigenvalues of the discrete spectrum of �(fl; 1) is unbounded at infinity. We also give a lower bound with an effective constant for this distribution function in any sufficiently long finite interval. All of these results were first published in [67]. The basic purpose of §6.3 is to deepen the spectral theory of 2f(f; X) in the case of special Fuchsian groups of the first kind f -groups with nontrivial commensura­ bles. The basic results are 1) construction of a simultaneous spectral decomposition

i',

INTRODUCTION

"

.

7

for the operator m(f; X) and the Hecke operator T( g) (Theorems 6.3.3 - 6.3.5), and 2) proof of Roelcke's conjecture for a group f E we 2 with large commensurable (Theorem 6.3.6). These theorems were first published in [64]. Here we shall not consider the theory in its most general form, but shall limit ourselves to the trivial representation X, dim V = 1 . At the end of the section we give many examples of groups with nontrivial commensurables. Among them the set of groups fM occupies an especially important place. Each group fM E we is a subgroup of index two in a group generated by reflections relative to the sides of a regular polygon M in the Lobachevsky plane (see §6.3). We note that the set of all groups commensurable with the groups fM is rather extensive� 'In particular, this set contains all arithmetic subgroups f E we 2 as a small subset. In §6.4 the theory developed in §6.3 is specialized to the case of an arbitrary group f which is commensurable with one of the groups fM' The basic results are a proof of the Roelcke conjecture for f (X = 1) (Theorem 6.4.5), a proof of a still stronger conjecture concerning the distribution function for the eigenvalues of the discrete spectrum of m(f; 1) (Theorem 6.4.7), and, finally, the demonstration of a connection between the spectral theory of automorphic functions for an arbitrary group fM and the Dirichlet and Neumann boundary value problems on M (Theorems 6.4.2-6.4.4 and 6.4.6). All of these theorems were first published in [61 ] and [70]. The proof of the special case of Roelcke's conjecture for Hecke groups and for X = 1 was given earlier by Roelcke himself in his dissertation [45]. §6.5 is devoted to a derivation of the trace formula for Dirichlet's problem from Theorem 6.4.6, which can be naturally regarded as a variant of the classical Selberg trace formula. At the beginning of the section we prove a spectral trace formula for Dirichlet's problem on an arbitrary regular polygon M. We then consider the theory separately for compact M (§6.5 a)) and noncompact M (§6.5 b)). The proof of the Selberg trace formula for Dirichlet's problem is based on an investigation of the relative conjugacy classes {0Y}rM , where E9 is a fixed reflection relative to a side of M. In §6.5 a) we pay particular attention to the nondegenerate classes {E9JrM (tr 0y =1= 0), and in §6.5 b) we look at the degenerate classes (tr E9y = 0). As a simple consequence of our Selberg trace formula for Dirichlet's problem, in both the compact and noncompact cases we consider Weyl's asymptotic formula for the eigenvalues for the operator 1(1- T(0))m(fM; 1) in Dirichlet's problem on M (the spectrum of this operator is purely discrete); this asymptotic formula is also proved in §6.5. We conclude the section by showing that the Selberg trace formula for the von Neumann problem on M is a consequence of the classical trace formula and the Selberg trace formula for Dirichlet's problem. The basic results of §6.5 were first published in the note [65] (see also [60], [62] and [66]). In §6.6 we define the zeta-function ZM(S), which we call the Selberg zeta-function for the Dirichlet boundary value problem on a regular polygon M, and we prove its basic properties. Among them are meromorphicity and a functional equation. We give a complete description of all of the zeros and poles of the function. These results are all obtained from a fundamental representation for the logarithmic derivative of Z M( s) (Theorem 6.6. 1), which, in turn, is a consequence of the Selberg trace formula for Dirichlet's problem in §6.5. The basic theorems of the section were first published in [65].

8

INTRODUCTION

§6.7 is devoted to estimating the remainder in Weyl's asymptotic formula for the operatorH�} T(tG» 9l:(rM ; 1) in Dirichlet's problem on M. More precisely, for the -

first time we construct an asymptotic formula with three principal terms and a remainder term of order OCTjln T ) (Theorem 6.7. 1), where the justification for the first principal term in this formula is the content of the theorems in §6.5 and is what led to the Weyl formula. We note that the formula in Theorem 6.7. 1 is proved for an arbitrary regular polygon M, not necessarily compact. The derivation is based on the theory of the zeta-function Z M ( S ) and is a spectral application of this theory. A similar formula, but with a somewhat worse remainder term of order O( T), was published in [65]. The last chapter, Chapter 7, consists of two sections. In §7. l we prove a theorem on the continuous dependence of the singular points of the resolvent \R(s; 1'; X) of the operator 9l:(r; X) on the so-called regular deformations of the discrete group r E ill( 2. A similar result is well known for groups r E ill( I (see [ 1 1 D, where it lies within the framework of the classical theory of perturbations of a selfadj oint operator with purely discrete spectrum. Theorem 7. 1 . 1 is a much more specialized fact. In it we are dealing with the continuous dependence of the set of poles of the resolvent as an integral operator, which are located on a two-sheeted covering of the spectral plane. The eigenvalues of the discrete spectrum of the operator 9l:( r; X), r E ill( 2' X E 9( sC r), are a subset of this set. At the end of the section we discuss certain applications of Theorem 7. 1 . 1 to the theory of the discrete spectrum of 9l:(r; X). The results in §7.l have not been published before. At the beginning of §7.2 we give a short survey of the facts presently known which attest to the interrrelation between the spectral theory of automorphic functions for discrete arithmetic groups and the theory of the Riemann zeta-function. Most of the section is devoted to a proof of a theorem which enables one to compare the spectral singularities of the resolvents of automorphic Laplacians in two- and three-dimen­ sional Lobachevsky space. Based on this theorem, we state a conjecture on a possible intersection between the set of zeros of the Selberg zeta-function Z(s; r1:; 1) for Re s = lj2 and the set of zeros of the Dedekind zeta-functions of quadratic imaginary fields. At the end of §7.2 we indicate a new direction in the spectral theory of automorphic functions, which is a far-reaching generalization of the analog of Artin theory in §6.2; our conjecture in a natural way stimulates the development of this generalization. Part of the results in the sections were published by us before in [63] and [69]. We now say a few words concerning terminology and what we have adopted as the standard notation. We set i r-r. In asymptotic formulas we use the symbols o and 0 in estimates for the order of the remainder terms; this notation is standard in analytic number theory and function theory (see [25]). We also use the Vinogra­ dov symbol « which is sometimes more convenient than the symbol O. The other notation includes C , IR, 0 and O({J) for the fields of complex numbers, real numbers, rational numbers, and quadratic algebraic numbers: 7L for the ring of integers in 0; 7L({J) for the ring of integers in O({J); GL(2, IR), SL(2, IR) and PSL(2, R) for the usual Lie groups over IR ; and PSL(2, 7L) for the discrete modular group. Some of the more specialized notation is explained in Chapter 1. =

,

-

INTRODUCTION

.

9

In conclusion, I would like to express my sincere gratitude to L. D. Faddeev, who has had a strong influence on my entire mathematical outlook, for his help with this work. I would also like to thank A. I. Vinogradov, N. V. Kuznecov, S. J. Patterson, and M. M. Skriganov for discussions which led me to a better understanding of the theory of Artin series, the asymptotic formulas of spectral theory, and the nature of the spectral singularities of the resolvent on the second sheet.

CHAPTER

1

NOTATION AND AUXILIARY THEOREMS In this chapter we shall give the auxiliary facts that will be needed later. We shall only present the statements of the theorems, with references to places in the literature where their proofs can be found.

§1 . 1 . The Laplace operator for the Poincare metric on the Lobachevsky plane We introduce some basic notation relating to the differential geometry of the Lobachevsky plane. Let H be the Poincare model of the Lobachevsky plane. It is well known that the group G of all motions of the first kind of H coincides with PSL(2, �), and that H is a homogeneous symmetric space, H = G/SO(2). We introduce the Riemann metric ds on H, which is invariant relative to all motions in G, i.e., the Poincare metric. We also introduce the invariant Riemann measure df.L on H which is associated with ds, the corresponding geodesic distance p and the Laplace differential operator L of the metric ds. In the theory below it will be convenient to replace the distance p = p ( z, z') (z, z' E H ) by the following function of the distance: u ( z , z' ) = (cosh p ( z, z ,) - 1 ) / 2

which is a fundamental invariant of a pair of points; here cosh is the hyperbolic cosine. We further suppose that H is realized as the upper half-plane in the complex z-plane (rather than as the interior of the unit circle) with the natural coordinates x, y; H = {z = x + iy 1 y > O}. The group G acts on it by fractional-linear transfor­ mations z � gz, z E H, g E G. The above invariants have the following form in the x, y-coordinates (see [51], §3):

dp. ( z ) = dxdy /y2 , ds2 = ( dx2 + dy2 )/y2 , u ( z, z ,) = 1 z - z' 12 /yy', L = y2 (a2 /ay 2 + a2/ax2 ). We consider the differential equation

-Lf. ( z ) - s (1 - s ) f. ( z )

In §1 of [9] the following theorem is proved.

= f2 ( z ) .

THEOREM 1.1.1. 1) Suppose that the function f2 : H � C is infini tely and bounded on H. There exists a kernel k( z, z'; s), z, z' E H, s E C,

Re s

> 1 the function

f. ( z )

=

1. k ( z , Z' ; s ) f2( z') dp. ( z') H

has the properties off2 given above, and, in addi ti on, satisfi es (1.1.1). 11

(1.1.1 )

differentiable such that for

•.

.

' �

... .

''-In

.nl"",

J-\UJl.lLlAl< Y

THEOREMS

2) The kernel k( z , z'; s ) is determined by the formulas

k(z, z'; s )

=

k( u (z, z') ; s ) ,

1 k( u; s ) = 4 7T fa I [t( l - t )] s-- I ( t + * fl' dt,

where u(z, z ') is the fundamental invariant of a pair of points, and Re s > 0. 3) The function k( u ; s ) is analytic in s and infinitely differentiable in u for Re s > ° and u > 0, and it has the asymptotic behavior k ( u ; s ) = ( l /4 7T ) In u + 0 ( l) U�O -

and the bound

k( u ; s ) = O( u-Re s ) , where In is the natural logarithm.

We let

u-oo,

§1 .2. Fuchsian group of the first kind r, its fundamental domain F, and the representation X. Series over the group r

we denote the set we = we I U we 2' where 9], I

of all Fuchsian groups of the first kind. By definition, is the set of all cocompact groups and we 2 is the set of groups having noncompact fundamental domain. The definition of a discrete group r E we implies that it has a measurable fundamental domain with finite volume relative to the measure dp., I F I < 00. The following results (Theorems 1 .2. 1-1 .2.4) are classical, and are due to Poincare, Fricke, Petersson and Siegel (in this connec­ tion see, for example, [35], Chapters 3, 4 and 7, [37], Chapter I , and [1 3], Chapter 1 , § 1). THEOREM 1 .2. 1 . A Fuchsian group of the first kind r E we is determined by a finite set of generators AI' B I, . . ,Ag, Bg, SI'." ,Sr' R" ... ,R, with a finite set of defining ·

relations

[ AI' BI l ... [ Ag, Bg] SI '" SrRI ... R, =

E,

E .. . Rm, = E where [ , ] denotes the commutator and mj E Z, mj � 0, j = 1 , . . . , I. The elements RI,...,R, are elliptic, SI" " ,Sr are parabolic , AI' B I,...,Ag, Bg are hyperbolic, and E is the identity of the group r. By definition, the r in Theorem 1.2. 1 has signature r re g ; ml, . . . , m ,; r ). A group r with this signature has the property that r E we I if and only if r = 0. THEOREM 1 .2.2. The fundamental domain F for the group r E we can be chosen in the form of a closed (in the geometric sense) polygon in H bounded by a finite even number of geodesics and having well-determined values for its interior angles. If r E we then the fundamental domain F in Theorem 1 .2.2 has a finite Rm, = I

"

.

2'

=

(nonzero) number of zero interior angles. The vertices of these angles are fixed points of the parabolic transformations in r and are called parabolic vertices (or cusps) of F.

,

'

§1 .2. FUCHSIAN GROUPS OF THE FIRST KIND

13

Two points z, z' E H are said to be equivalent relative to f if there exists a transformation y E f such that z = yz'. A similar definition is given for points in the absolute of H. This equivalence is denoted z = z ' (mod f).

1 .2.3. The fundamental domain F for f E 9.)( 2 can be chosen in such a way that all of its vertices are pairwise inequivalent relative to f. THEOREM

In the sequel we shall assume that the domain F is chosen as in Theorems 1.2.2 and 1 .2.3, and that n is the number of cusps. We denote them by z I ' . . . ,z For each vertex Zj we consider the maximal parabolic subgroup fj C f which stabilizes it. The group IJ is generated by a single parabolic element, which we shall denote � . For eachj 1 , . . . , n there exists a transformation gj E G having the properties n.

=

g; l� gj Z = Soo z = Z + 1 , Z E H. THEOREM 1.2.4. The domain F (more precisely, its closure in the topology of H ) can be represented as a union of disjoint sets as follows: g/X)

= Zj '

F = Fo U

n

U Fj ,

j= l

Fo is a compact set with piecewise smooth boundary, and Fj is the image Fj = gil a of the strip ITa = {z E H I 0 .:;;; X .:;;; 1, a .:;;; y < oo } for some sufficiently where

large positive a not depending on j, under the transformation gj ' j = 1 , . . . , n.

Let V be an h-dimensional complex vector space, V = C h . We shall let v denote an arbitrary element of V. The inner product of v, v' E V, which is antilinear in the second argument, and the norm of an element v, will be denoted, respectively, by < v, v') and I v I The norm of a linear operator q: V � V will be denoted by

v.

IqI

v = sup ( I qV I vii v I v ) · vE V

Let X be an arbitrary representation of the group f E 9.)( which acts in the space V and is unitary with respect to our inner product. We suppose that f E 9.)( 2 . For each j 1, . . . , n we have the subspace Vj C V of fixed vectors v of the operator X( � ), i.e., Vj = {v E V I X( � )v = v}, where � is a generator of the parabolic subgroup IJ C f. We let IJ denote the orthogonal (relative to the inner product < , » projection of V onto the subspace Vj, and we let kj = dim Jj denote the dimension of the subspace Vj, j = 1, . . . , no We further set k = kef; X) = � � kj. Here we are emphasizing the dependence of k on f and X. Following Selberg and Roelcke (see [5 1 ] and [46]), we shall call kj the degree of singularity of the representation X relative to the generator Sj of the subgroup fj C f, and shall call kef; X) the degree of singularity of f relative to X. A representation X is singular for � if kj =I=- 0, and regular otherwise. Similarly, we say that X is a singular representation of f if ke f; X) =I=- 0 ; X is a regular representation if kef; X) = o. We shall let g( = g((f) denote the set of all finite-dimensional unitary representa­ tions of an arbitrary group f E 9.)(. If f E 9.)( 2 ' then we also introduce the sets g( r = g( r(f) and g( = g(sCf) of regular and singular representations X E g(( f), respectively; �n = g( r U g( We now give two theorems which characterize the properties of series of functions over a discrete group. The first is a special case of a criterion for convergence of a

=

s

s.

14

I. NOTATION AND AUXILIARY THEOREMS

series over a discrete group that was found by Selberg (see [5 1], §2). The theorem is proved in [9], §1.4. The second theorem is due to L. D. Faddeev (see [9], Lemma 1 .2). THEOREM

bound

1 .2.5. Suppose that the continuous function k: H X H � C satisfies the

I k ( z , Z ') I � { 1 + u { z , z')fa , where u(z, z') is the fundamental invariant of a pair of points, and (J E IR ; then the series � k { z , yz') yEr

converges absolutely and uniformly in z, z' in a compact subregion of H X H, provided that (J > 1 . For y

E

Obviously,

G we shall let c( y ) denote the number in the denominator of

yz = ( az + b ){ cz + d r l • c( y) is determined up to a

+

sign.

1 .2.6. Let Ll be a subset of G such that, for fixed 8 > 0 and for all y E ll, the condition 1 c( y) 1 � 8 > 0 holds, and the series � [ 1 + u { z, yz')r l -e THEOREM

yEA

converges uniformly in z, z' in a compact region for e > O. Then for (J > 2 the following estimate holds uniformly in z and z' in the strip -X � x, x ' � X; y, y' � B > 0 for any fixed X > 0 and B: e,a

y EA

where e can be chosen to be arbitrarily small. We conclude the section with a classical theorem from the theory of Fuchsian groups which is due to Petersson. It is given in [9]. Let g, g' E G be any fixed elements. For f E 9)( 2 we set Ll gfg'. In Ll we look at the subset Ll oo , where, by definition, Ll oo {y E 11 1 c( y) O}. We let Ll' denote the set-theoretic difference Ll '

=

Ll - Ll oo '

THEOREM

=

= =

1 .2.7. There exists a number 8 > 0 such that 1 c( y) 1 � 8 for any y E ll'.

REMARK. For the set Ll ' in Theorem 1 .2.7,one can choose any set of the form g;;l fgp for a =t= {3, a , {3 1 , . . . , n , since c(y) =t= 0 for any y E g;;l fgp, a =t= {3 (see [9], §1 .3).

=

§ 1 .3. Function spaces and integral operators We first introduce some standard notation. C(X; Y ) is the Banach space of all continuous maps f: X � Y; COO( X; Y ) is the vector space of all infinitely differentia­ ble maps f: X � Y; and CO'C X; Y ) is the subspace of COO( X; Y) consisting of functions having compact support on X. Here X is a region in cm, and Y is a

finite-dimensional vector space over C . Further suppose that Y is a space with an

., .

§1.3.

FUNCTION SPACES AND INTEGRAL OPERATORS

15

inner product; and let dv be a measure on X. We let L 2 ( X; Y; dv) denote the Hilbert space obtained by completing the set of elements f in COOO( X; Y) in the norm

(

f) f( x ) It d V ( x )

) 1/2

,

where I I y is the norm in Y. We now define the basic spaces of automorphic functions (maps). Suppose that f E me , X E 91(f) and X acts on the space V. A map f: H V will be called f-automorphic with representation X if f( yz ) X( y )f( z) for any z E H, and y E f. It is clear that an automorphic function. is uniquely determined by its values on a fundamental domain. We choose a fundamental domain F for the group f E me in accordance with Theorems 1 .2.2- 1 .2.4. By analogy with the previous paragraph, we introduce the following notation. C( F; V; X) is the Banach space of f-automorphic functions with representation X which are bounded on F and continuous on H, COO( F; V; X) is the vector subspace of C(F; V; X) consisting of functions which are infinitely differentiable on H, and Co( F; V; X) is the subspace of COO( F; V; X) of functions with compact support on F. We introduce the Hilbert space X ( f ; X) L2 ( F; V; d/-t; X), obtained by completing Co( F; V; X) in the norm II f II F, X where II f II },x ( f, f ) F,x' and the inner product is given by

=

=

=

( fl ' f2 ) F, x =

� '.

�

fF( fl ( z ) , f2( Z )

dp, ( z )

for fl , f2 E X(f; X). We now describe a more special Banach space �v(f; X) of automorphic func­ tions; this space depends on a parameter v E IR, which plays an important role in the spectral theory. In the scalar case (dim V = 1) with the trivial representation X, this space was first introduced by L. D. Faddeev in [9]. By amtlogy with [9], we start by introducing a partition into components for maps and kernels of integral operators defined on F and on F X F, respectively. In accordance with Theorem 1 .2.4, for f: F � V we set f( z )

n

= � t(z ), j =O

where

=

z E Fj , z f/. Fj .

=

=

Furthermore, fo(z ) io(z), and fj(z) t ( gjz) for j l , . . . , n. Each of the func­ tions fj(z) is defined on ITa' We similarly define the components k a{iz, z'), 0 ::s;;; a, f3 ::s;;; n, of the kernel k(z, z'), k: F X F � V. The index a corresponds to the variable z, and the index f3 corresponds to z'. We define the space �v(f; X). It is obtained as the completion in the norm II f II F, x , v of the set of elementsf E Co( F; V; X), where, by definition, n

II f II F, x , v = max l fo ( z ) I v + � z E Fo

z = x + iy, v E IR .

a=

1

max y -v I fa( z ) l v,

z E I1a

.LU

I. NOTATION AND AUXILIARY THEOREMS

We conclude this section with a few words concerning integral operators acting on the space :K(f; X ). Let K be a linear integral operator, K: :K(f; X) X(f; X). Its kernel k ( z z' ) is a map from F X F to the algebra of linear operators on the space V. An important class of completely continuous operators K in what follows will be the class of Hilbert-Schmidt operators, which are characterized by the following property of their kernel: �

,

1F1FI k( z , z') I� d/L ( z ) d/L(z') < 00 . § 1 .4. The operator & ( f; X ) and its resolvent )R (s; f ; X ) This section is devoted to the definition of the automorphic Laplacian and its resolvent, which are the fundamental objects of study in the spectral theory of automorphic functions, and also to an explanation of their simplest properties. The results of this section are well known, and in the generality we need are due to Roelcke (see [46], §3); however, our presentation here is closer in spirit to Faddeev's paper [9]. We shall limit ourselves largely to the statements of the theorems without proof. We introduce the following (nonclosed) vector subspace of :K(f; X ). By defini­ tion, GD(f; X ) = { f E COO( F; V; X ) I f( z ) and Lf( z ) are restricted to F in the norm of V}. We define the operator W(f; X): :K(f; X) � :K(f; X) by the formula W ( f ; x ) f = -Lf , f E GD ( f ; X ) . THEOREM 1 .4. 1 . The operator W (f; X) is symmetric on GD( f; X) and is nonnegative

definite.

We let &(f; X) denote the closure of W(f; X) in :K(f; X )· THEOREM 1 .4.2. The operator &(f; X) is selfadjoint in :K(f; X) and is nonnegative

definite.

./

The resolvent )R(A; f; X ) of the operator &(f; X ) is defined at a regular point A E C in the usual way: )R (A ; f; X ) ( &(f; X ) - A 1r l , where 1 1(f; X ) is the identity operator in :K(f; X). Because of Theorem 1 .4.2, the set of regular points A contains C with the real semiaxis [0, (0) removed. However, anticipating what comes later, we point out that in this theory one can use meromorphic continuation to attach a meaning to the resolvent on a Riemann surface which, roughly speaking, is a two-sheeted covering of the A-plane. The meromorphicity of the resolvent on this Riemann surface is a very important faCt; which, in particular, implies a theorem on expansion in eigenfunctions of &(f; X ). To describe the Riemann surface, in place of the spectral variable A we introduce a variable s by means of the equation A s ( l - s). The A-plane C cut along the semiaxis [0, (0) corresponds to the s-half-plane Re s > 1 /2 cut along the interval 1 /2 .;;;;; s .;;;;; 1 . By analogy with [9], in what follows we shall let )R ( s ; f; X ) denote the resolvent ( &(f; X ) - s(l - S)1)-I, where for the time being we suppose that Re s > 1/2 and s ft. [1/2, 1]. We now show that the kernel of the resolvent )R ( s ; f; X ), considered as an integral operator in the half-plane Re s > 1 , is in some sense given by an explicit formula.

=

=

=

§1 .4. THE OPERATOR �(r; X) AND ITS RESOLVENT

Suppose that

f

X E 9C(f) and Re s > 1. We consider the series r ( z , z ' ; s ; f ; X ) = � X ( y ) k ( z , yz' ; s ) ,

17

E we ,

yEr

where the Green's function k(z,

( 1 .4 . 1 )

z'; s) is as in Theorem 1 . 1 . 1 .

1 .4.3. The series in ( 1 .4.1) converges absolutely for Re s > 1 and uniformly with respect to z and z' in any compact subregion of H outside a neighborhood of z = z' (mod f). THEOREM

PROOF.

Since the representation X is, �nitary, we have

I r ( z , z' ; s ; f ; X ) l v � � I k ( z , yz ' ; s ) l , yEr

z =1= z' (mod f). The theorem now follows from parts 2) and 3) of Theorem 1 . 1 . 1 and Theorem 1 .2.5. This completes the proof. I t is also easy to prove the following fact. THEOREM

1 .4.4.

X) r * ( z , z , ,· s ,· f ,· X ) = r ( z' , z ·, s·, f ,· where r* denotes the operator in V which is adjoint to r relative to < , ), and the bar denotes complex conjugation. ,

1 .4.5. The kernel r(z, z'; s; f; X ) defined by ( 1 .4. 1) coincides with the kernel of the resolvent 9l(s; f; X) in the space :Je(f, X ), considered as an integral operator for Re s > 2. THEOREM

§2. 1 we give an independent proof of the following lemma.

PROOF.

In

LEMMA

1 .4. 1 . For Re s > 2 the integral

is uniformly bounded in z

E

tl r ( z, z'; s ; f ; X ) I vd.u ( z')

F, where F is a fundamental domain for f.

We now consider equation (1 . 1 . 1), where we suppose that f2 E 6])(f; X). Theorem 1 .4.5 follows from the equality k( gz, gz'; s ) = k(z, Z ' ; s), which is true for any g E G and z, z' E H; Lemma 1 .4. 1 ; Theorem 1 . 1 . 1 ; and the chain of equalities

9l ( s ; f ; x ) f2( z )

= j r ( z, z' ; s ; f; x ) f2( z') d.u( z') F

=

j k ( z, z'; s )f2( z') dp,( z' ) = fl ( z ) . H

The proof is complete. In what follows we shall let r(z, z'; s; f; X) denote the kernel of the resolvent 9l(s; f; X) as an integral operator; we shall keep the notation ( 1 .4. 1) for any s in a region where r(z, z'; s; f; X) is a meromorphic function of s.

CHAPTER 2 THEOREM ON EXPANSION IN EIGENFUNCTIONS OF THE OPERATOR m- ( f

; X)

\

... "

In this chapter we shall suppose that the group f belongs to the set 9JC 2 ' and that the representation X belongs to 9C(f) (see §1.2). For cocompact f E 9JC I and for representations X E 9C(f) the theorem on expansion in eigenfunctions of m- ( f ; X) is well known (see [5 1] or [19]). We fix f E 9JC 2 and X E 9C (f). We choose a fundamental domain F for f in accordance with Theorems 1 .2.2- 1.2.4, and we fix such a fundamental domain. In order to improve the appearance of the formulas in Chapter 2, we shall not always show the dependence on f, X and F in the notation for operators, functions and other objects. For example, we shall write m-(f; X) = m- , 9l(s; f; X) = 9l(s). §2.1. Properties of the resolvent 9l ( s; f; X ) far from the spectrum The basic problem solved in this section is to construct an asymptotic expansion for the kernel r(z, z'; s) of the resolvent 9l(s) for z and z' close to the parabolic vertices of the domain F, while s is fixed and satisfies the condition Re s > 2. The results obtained will play a central role in the proof of the theorem on expansion in eigenfunctions of m- and in the description of the structure of the eigenfunctions of the continuous spectrum. The proof of this asymptotic expansion generalizes the proof of the theorem obtained earlier by Faddeev in the scalar theory (dim V 1) for the trivial represent'ltion X (see [9], §2, and also [10], §3). We fix s with Re s > 2. We consider the expression ( 1 .4. 1) for the kernel r( z , z'; s) in the form of a series over the group f. In this series we take out the terms which decrease rapidly as a function of the closeness of z and z ' to the cusps Z l ' . . . , z n. We do this by dividing the kernel r( z , z'; s) into its components (see § 1.3):

=

n

n

r ( z, z'; s ) = a� � raP ( z , z '; s ). =O p =O

(2 . 1 . 1 )

We postpone for a while consideration of the components roo, a I , . . . , n, and consider rap , a, f3 = I , . . . , n , with a =F f3. We have

=

rap( z , z'; s ) =

�

yEr

rO a

and

rao

for

X (y)k(gaz , ygp z'; s )

�

y Eg; l rgp

y ( gaygil) k( z, yz'; s ) ,

z, z' E ITa .

We estimate the kernel rap , taking (2. 1 .2) into account: I rap ( z, z ' ; s ) I v � � I k( z , yz ' ; s ) I . y Eg; l rgp 19

(2 . 1 .2 )

(2 . 1 .3 )

I I . EXPANSION I N EIGENFUNCTIONS OF \1 1 ( r ; X )

The scalar series on the right in (2. 1 .3) has the following estimate, by part 3) of Theorem 1 . 1 . 1 : �

I k(z , yz'; s ) l «

�

[ 1 + u (z , yz , )] - Re .l .

Consequently, by the remark following Theorem 1 .2.7 and by Theorem 1 .2.6, we have

+e e s I ra(J ( z, z' ; s ) I v « ( yy' ) 2 - R for z, z' E IT a ' e > 0, z = X + iy and z' = X ' + iy ' . We similarly obtain the estimates

where z

E Fo , z' E IT a

I r.O a ( z ,

z , '· s ) I v « y ,2+ e - Re s ,

and a = 1, . . . , n ; and

(2. 1 A)

(2. 1 .5)

(2. 1 .6) I ra O ( z , z , '· s ) I v « y 2 + Re s , where z E IT a ' Z' E EQ and a = 1, . . . , n . The behavior of the component roo(z, z' ; s) in a neighborhood of a cusp is E-

obvious. We consider the component raa(z, z'; s), a � 1 . From the point of view of the estimate that interests us, it is natural to represent this component as a sum

where, by definition,

tv a(Z , Z' ; s ) = O a(Z , z' ; s ) =

�

X ( gayg� l )k(z , yz' ; s ) ,

(2 . 1 .7)

�'

X ( gayg� l )k(z, yz' ; s )

(2. 1 .8)

y Eg,.; l r"ga y Eg:; I rga

(see § 1 .2). The prime in the sum (2. 1 .8) means that the summation is over all elements y E g� I rga , except for y E g� I ra ga ' If we again use the argument which led us to the estimates (2. 1 .4)-(2. 1 .6), we obtain

+e e I o a( Z , Z , ; S ) I v « ( yy ,) 2 - R s ,

(2 . 1 .9)

where z, z' E IT a and a = 1 , . . . , n o Thus, it remains to estimate the behavior of the kernel tv a(z, z'; s) for each a = 1 , . . . , n ; z, z' E IT a . First of all, it is convenient to rewrite the kernel in the following form: tv a(Z , Z ' ; s ) = � - 'X ( Sj ) k ( z, z ' + j ; s ) . ( 2 . 1 . 10) jEZ

From (2. 1 . 10) we see that the properties of tv a( Z , Z'; s) are strongly dependent upon the degree of singularity of the representation X of the group r relative to the generator �. In fact, we set + 00

fa(Z , Z'; s ) = � X ( S� ) Pak(z, z' + j ; s ) , j = - oo + 00

ma( z, z'; s) = � X ( Sn( l v - Pa) k(z, z' +j; s ) , j = - oo

l.

(2. 1 . 1 1 )

§2. 1 . PROPERTIES OF THE RESOLVENT FAR FROM THE SPECTRUM; where Pa is the projection onto the subspace Va in V (see §1 .2). Obviously,

C

21

V, and I v is the identity operator

From the definition of Pa we have

for any j E lL. Consequently,

t a( z,

' z ; s)

1 + 00

=

Pa � k ( z, z' + j; s ) . j= - oo

(2 . 1 .12 )

The kernel which is represented by the scalar series on the right in (2. 1 .1 2) has a Fourier expansion whose coefficients have been computed in the literature (see, for example, [9], §2). The Fourier series has the form + 00

� k ( z, z'

j= - oo

+ j ; s ) = t ( y , y'; s )

00

+ 2 � [cos 2 wj( x - x') ] p i y, y'; s ) , j= !

where the kernels t(y, y'; s ) and p j(y, y'; s ) are the Green's functions of the Bessel ordinary differential equation l{; = l{; ( y ) (2 . 1 . 1 3 ) (the kernel t(y, y'; s) corresponds to the valuej = 0 in (2. 1 . 13)). Here t(y, y'; s ) has the form

{

y sy f\ -s , y � y', 1 t ( y , y ' ,. s ) = 2 s 1 l -s s - Y y ' , y ;;;:' y' ,

(2 . 1 . 14)

and p /y, y'; s ) satisfies the inequality

1 p /y, y' ; s ) 1 « :7 exp( - 2wj [ y - y'D

(2. 1 . 1 5 )

X ( Sa) ( l v - Pa) e,( a ) = "'ae , ( a),

( 2 . 1 . 16 )

s

uniformly in y and y' in the interval a � y, y' < 00 (the estimate depends only on s ) . We return to (2. 1 . 10) and (2. 1 . 1 1), and investigate the behavior of the kernel m a(z , z'; s ) . This kernel has not been examined before in the literature; hence, we shall give the estimate for it in more detail. For each fixed a = 1, . . . , n we choose a basis e 1 (a), . . . , eh(a) for V in which the matrix of the operator X(Sa)(1 v - Pa) is diagonal, where we may suppose that we have one of the alternatives

(2 . 1 . 17 ) with the numbers ()'a satisfying the inequalities 0 < ()' a < 1 . The existence of such a basis and the formula for "'a follow from the unitary property of X(Sa) and the definition of the projection Pa.

II. EXPANSION IN EIGENFUNCTIONS OF

22

W ( r ; X)

1 � a � n and 1 � I � h, we consider the

Now, for fixed values of the indices following series (see (2. 1 . 1 1» : 00

m a(z, z'; s ) e,( a) = � X (s1)( I v - Pa)e,(a)k(z, z ' + j; s ) )= - 00

00

= e,( a ) � vlak ( z, z' + j; s ) .

( 2 . 1 . 18)

) = - 00

The last equality in (2.1 . 18) holds because of (2. 1 . 16). Thus, (2.1 .17) shows that the kernel m a(z, z'; s )ela) may turn out to be identically zero if the number Vi a is zero. If this kernel is identically zero for any I, 1 � I � h , this means that, for the given a, the degree of singularity of the representation X for the generator Sa is maximal and equals h. We consider the other alternative Via = exp 2 'lTi O'a ' We have 00

m a(z, z'; s )e,( a) = e,( a) � (exp 2 'lTijO'a)k ( z, z' + j; s ) . )= - 00

We introduce the following notation, where we take the parameters fixed, 0 = 0ta :

( 2. 1 .19)

a and I to be

00

c ( z, z'; s) = � ( exp 2 'lTijO)k(z, z' + j ; s) . ) = - 00

i

We now study the kernel c ( z, z'; s). It has the properties

c (z + j, z'; s ) = ( exp 2 'lTijO ) c ( z , z'; s ) , c (z, z' + j; s ) = (exp - 2 'lTijO ) c ( z, z' ; s ) for any j E 71.. . From part 1) of Theorem 1 . 1 . 1 it follows that c ( z , z'; s) is the Green's function of equation (1.1.1) in the strip II = { z E H I 0 � x � 1 , 0 < y < oo } with certain boundary conditions. More precisely, if f2 (z) is in the space COO(H; C ), is bounded on H, and satisfies the condition

f2 (Z + j ) = (exp 2 'lTijO) f2 ( Z )

for any j E 71.. , then the function

fl (Z) = 1 c ( z, Z'; s )h(z') dp. ( z ) II has the above properties of h ( z), and also satisfies ( 1 . 1 . 1) (Re s > 2). We now explicitly construct the Fourier expansion of the Green's function c(z, z'; s). Since

the " variables separate" in this problem, there is a well-known general procedure for computing the expansion. According to this procedure, we can look for the Green's function in the form 00

c (z, z'; s ) = � (exp [ 2 'lT i( x - x ')( j + O )] ) o)( y , y ' ; s; 0 ) , (2.1 .20) j= - 00

where o/y, y'; s; 0) is the Green's function for the Bessel ordinary differential equation on the semiaxis 0 < y < 00 with the condition that the solution be bounded as y 0 and as y � 00 . The operator for this equation has the form �

btf; ( y ) = - tf; "( y ) - s ( I - s ) tf; ( y ) jy 2 + 4 'lT 2 ( 0 + j)2 tf; ( y ) .

(2.1 .21 )

�,

§2. 1 . PROPERTIES OF THE RESOLVENT FAR FROM THE SPECTRUM

23

The homogeneous equation determined by the operator (2. 1 .2 1 ) has, up to multiplicative constants, two linearly independent solutions, which can be expressed by the modified Bessel functions where Is ( z )

00

z s+2m 12s+2m r s + ( m=O m .

= �

m

+ 1)

,

and f(s) is Euler's gamma-function. These functions have a well-known asymptotic behavior as y -4 00 and as y 0, which leads to the following formulas: 1 l/; \ ( Y ) exp [ - 2 7T 1 0 + j I y ] , y� oo 2 /1 0 + j l -4

�

l/;2 ( y )

1 exp [ 2 7T 1 0 + j I Y ] , y -� 00 2 7T /1 0 + j I �

This asymptotic result shows that the desired Green's function o/y, y'; s; 0 ) has the form y ' ;", y, y' � y ,

since, in addition, the Wronskian of the solutions l/;\ and l/;2 is equal to one. From this one easily obtains the following estimate, which, together with (2. 1 .20), is the final goal of our study of the kernel c ( z, z'; s): 1 . (2. 1 .22) I OJ ( Y ' y' ; s ; 0 ) 1 ;< 1 0 + j exp( - 2 7T I 0 + } I I Y - y ' I ) , I j E 71.. , 0 < 0 < 1 . We now combine these results concerning the structure of the kernel m 0:( z, z'; s). We have found that, for each basis element etC a ) of V, either the kernel m o:(z, z'; s)etC a ) is identically equal to zero, or else m o:( z , z ' ; s )e[( a )

00

= e,( a )

� [ exp 2 7T i ( x - x ' ) ( j + O,o: ) ] o / y , y ' ; s ; 0'0:) '

)= - 00

(2. 1 .23) and each of the Green ' s func­

where 0 < 0'0: < 1 , z = x + iy , z' = x ' + iy' E ITa' tions ° /y, y'; s; 0[0:) satisfies (2. 1 .22) with 0 = 0'0:' We have thereby investigated all of the terms in the sum (2. 1 . 1), in the process obtaining the necessary expression for the kernel of the resolvent r( z, z'; s ). We summarize by introducing some notation and then stating a theorem. We define four integral operators �(s ), :D(s), �(s ) and :t(s). We give these operators by means of the kernels g (z, z'; s), b ( z, z'; s), e(z, z'; s) and t(z, z '; s),

II. EXPANSION IN EIGENFUNCTIONS OF 9! ( r ; X )

24

respectively, and we define each kernel component by component as follows: A oo ( Z , Z ' ;

S ) = roo( z , Z ' ; s ),

g a,B ( Z , Z ' ;

S) =

where the kernel tJ a is defined by (2. 1 .8); b oa( z , z ' ; b oo ( z , z ' ; b aa( z , z ' ;

s)

s)

=

s ) = 0,

0,

00

b ao( z , z ' ;

b a,B( z , z ' ;

= Pa · 2 �

j= 1

ra,B( z , Z ' ;

s)

s ) = 0,

=

s ),

a

=1=

/3 , 1 � a , /3 � n ,

(2 . 1 .24)

0,

a

=1=

/3, 1 � a, /3 �

n,

(2. 1 .25)

cos 2 '1Tj ( x - x ')p / y, y' ; s ) ,

where the kernel p /y, y ' ; s) satisfies (2. 1 . 1 5), Z = x + iy, z ' = x' + iy ' ; 1 � a � n, e Oa( z , Z I ; s ) = 0, e aO ( z , Z I ; s ) = 0, e oo ( z , z ' ; s ) = 0, a =1= /3 , 1 � a , /3 � n , (2. 1 .26) e a,B ( z , Z I ; s ) = 0 , where the kernel m a( Z, Z '; s) is defined by (2. 1 . 1 1); and t Oa( z , z ' ; t OO( Z , Z ' ; S )

=

s ) = 0,

0

,

taO( z , Z I ;

t a,B ( Z , Z ' ; s )

=

s ) = 0,

0

,

1 � a � n,

a =l= /3 , I � a , /3 � n , (2. 1 .27)

s ) = Pa tey , y' ; s ) , where the kernel t(y , y ' ; s) is defined by (2. 1 . 1 4), Z x + iy, z ' = x' + iy ' . t aa( z , Z I ;

=

From the definition of these kernels one easily notices the following equalities, just as in the proof of Theorem 1 .4.4:

s) = g * ( z , Z ' ; n , e ( z', Z ; s ) e*(z, Z'; n, b ( z ' , z ; s ) = b ( z , Z I ; s) , b (z, Zl; s), b (z, z'; n t ( z , z ' ; s ), t (z', z ; s) t ( z , z ' ; s) , t(z, z'; s ) g ( z ' , z;

= = = =

(2. 1 .28)

where the star denotes the adjoint operator on V and the bar denotes complex conjugation. By means of the above argument we hc;tve proved the theorem, discussed at the beginning of the section, which describes an asymptotic expansion of the kernel of the resolvent. THEOREM

2. 1 . 1 . Suppose that s E C and Re s > 2. The resolvent 9l (s) can be

represented as a sum offour integral operators 9l (s) ;{ ( s ) + Gf ( s ) + :D ( s ) +
=

; '

§2. 1 . PROPERTIES OF THE RESOLVENT FAR FROM THE SPECTRUM

25

=

We now make a remark. From the definition of �(s) = �(s; r; X), @(s) @(s ; r; X) and :£l(s) = :£l(s; r; X) it is clear (under our assumption concerning r E ID( 2 ) that each of these operators may vanish (identically) for certain represen­ tations X E W(r). Namely, the operators �(s) and :£l (s) vanish if and only if the representation X is regular, and in this case the operator @(s) must be nonzero. Conversely, if the representation X is singular with maximum possible total singular­ ity degree k(r; X) = n dim V, then the operators �(s ) and :£l(s) are nonzero, while @(s) is zero. (Of course, by "zero" we mean the zero operator.) We now examine some properties of t�e. operators �(s), @(s), :£l(s) and G£(s) considered as operators in X(r; X).

'

2.1.2. 1) The operator G£ (s) is bounded and compact in X for Re s > 2 as a Hilbert-Schmidt operator. 2) The operator �(s) is bounded in Xfor Re s > 1/2. 3) The opp-rators :£l(s) and @(s) are bounded and compact in X for Re s > 1 as Hilbert-Schmidt operators. THEOREM

PROOF .

1) We consider the integral

f f l g ( z , Z ' ; s ) ItdJL ( z ) dJL ( z ') .

(2 . 1 .29)

F F

We express it as a sum, by splitting the kernel g (z, z'; s) into components:

(2.1 .30) The first integral in this sum is finite, since the domain EO is compact, and the kernel g oo(z, z'; s) is continuous in z and z' everywhere on Fo X Fo except for the surface z = z' (mod r, if z is on the boundary of F), as follows from the definition of g oo( z, z'; s), (1.4. 1), and part 3) of Theorem 1 .1 . 1 . Finiteness of the integrals in the other terms of the sum (2. 1.30) is ensured by (2. 1.4)-(2. 1 .6) and (2. 1 .9), and from this we conclude that the integral (2. 1.29) is finite. 2) Proving boundedness of �(s) in X obviously reduces to proving boundedness of the one-dimensional integral operator with kernel t(y, y '; s) in the Hilbert space L2 ([ a, (0), C , dy/y 2 ). The latter assertion is proved in Lemma 2.2 of [9], and we shall not repeat the proof here, except to note that the main point in the lemma is to prove uniform boundedness of the integral

y -l1

f oo I t( y, y' ; a

s)

I y ' 11-

2 dy'

in y for a � y < 00 for any number 1/ < Re s and for Re s > will be useful to us later.

(2 . 1 .3 1 ) 1/2. This is a fact that

I I . EXPANSION I N EIGENFUNCTIONS O F m:(r;

26

3) We begin with the operator

X)

�(s). It suffices to prove finiteness of the integral

Using (2.1 . 15), we obtain

+ Iy - y ' I ) yd� yd�2' ·

00 00 1 00 00 exp ( - 2 w( i j') « � � 7.f

ll( s ) j= j ' = f f I

I 11

a

The expression on the right in verified estimate:

a

( 2 . 1 .32)

(2.1 .32) is finite because of the following easily (2. 1 .33)

where c E IR and b > o. We now prove the assertion concerning the operator

Gf(s). We have

n

= a�= I l l l ma( z, z ' ; s ) l � dJL ( z ) dJL ( z ') . IT a IT a

Because of (2.1 .23) and the obvious inequality h

I ma( z , z ' ; s ) I � � /�= I (ma( z, z ' ; s )e/(a) , e/a) 1 2 , which holds for z, z ' we have 00 1 s ( ) I 12 1 « a�1 I�I j =�oo j ' =�oo I ()/a + i l l ()I a + j' I d X f oo f oo exp [ - 2w (1 ()/a + i 1 + 1 ()/a + j' 1 ) (ly - y ' I ) ] � d�� . ( 2 . 1 .34) y y n

E IT a '

h

a

I

00

a

Here we have also used (2. 1.22). We now prove finiteness of the expression on the right in (2.1 .34) in a manner analogous to the previous case, based on (2.1 .33). The proof is complete. We conclude this section by noting that, in proving Theorems 2. 1.1 and 2. 1 .2 we have also proved Lemma 1.4. 1, which we stated earlier without proof. In fact, uniform boundedness of the integrals

f I g ( z, z ' ; s ) I vdJL ( z') ,

f I b ( z, z ' ; s ) I vdJL ( z ') follows from the estimates for the kernels g(z, z'; s), e(z, z'; s ) and b(z, z'; s), F

F

which we used in the proof of parts boundedness of the integral

1) and 3) of Theorem 2. 1 .2. Uniform

tl t ( z, z ' ; s ) I vdJL ( z') follows from the boundedness of the integral (2. 1 .31) for 11 = O.

i

�,

.§2.2 . §2.2. Faddeev's integral equatiX) toontheandhalanalf-plytianec conti n uati o n of the resol v ent Re ANALYTIC CONTINUATION OF THE RESOLVENT

27

s>0

9T ( s ; r ;

In the last section we examined the kernel of the resolvent 9T(s) in the half-plane Re s > where it is given by the absolutely convergent series and we obtained some estimates for it (see Theorems and The basic problem which we solve here is to extend the series meromorphically to the half-plane Re s > O. The meromorphic continuation is realized by means of a Fredholm integral equation whose operator depends analytically on the variable s. We call the equation a Faddeev equation, because in t4e scalar case (dim V = for the trivial representation X it was first introduced in [9]. We follow Faddeev's techniques but bring in some modifications due to the greater generality of our theory. There is a well-known identity of Hilbert for the resolvent 9T (s ) which is fundamental in the derivation of the desired Faddeev equation. Suppose that for s, s ' E C we have Re s > and s fl. in that case the resolvents 9T ( s ) and 9T (s ') are correctly defined as operators in the space :Je(r; X). The following identity (the Hilbert identity) holds in :Je(r; X):

2,

(1. 4 .1), 2.1.1 2.1. 2 ). (1.4.1) 1)

1/2

(1/2, 1];

(2.2 .1)

9T ( s ) - 9T ( s' ) = [ s ( 1 - s ) - s ' ( I - s ' )] 9T ( s ) 9T ( s ' ) . We choose a point s ' E IR , s ' > and we fix it. For brevity we shall not write the dependence on s'; then we introduce the notation

2,

w = w ( s ) = s ( I - s ) - s' ( I - s ' ) .

Now

(2.2.1) has the form 9T (s ) = 9T + w 9T 9T ( s ) .

... . '

(2. 2 .2)

This identity can be regarded as a linear equation for the operator 9T ( s ), i.e., taking is not immediately suitable for studying solvability 9T to be the unknown. But in the half-plane Re s > 0, for two reasons. In the first place, for those values of s the operators in generally lose their meaning as operators in the Hilbert space :Je(r; X). In the second place, the operator 9T is not compact, because of the continuous spectrum of m. Nevertheless, can be modified, if we make use of the information concerning 9T in Theorem In fact, that theorem contains the equality 9T � + � , where � (;r + :D + � , in which � is a bounded operator and � is a compact operator in :Je(r; X). We shall further prove that the operator 1 - w � , where 1 is the identity operator in :Je(r; X), has an inverse

(2.2.2)

(2.2.2)

=

=

(2.2.1.1. 2.2)

( ) (1 _ W� )- 1 = 1 + wO (s) . The bounded linear integral operator O (s) depends analytically on s, at least in the half-plane Re s > O. Its kernel is given by an explicit formula; it is closely

2.2 .3

connected with the Green's function of a certain ordinary differential equation. Now we modify according to the following procedure: ( 1 - w � ) 9T ( s ) 9T + w m 9T ( s ) ,

(2.2.2)

9T ( s )

=

= ( 1 + w O ( s )) 9T + w ( 1 + w O ( s )) m 9T ( s ) .

(2. 2 .4)

is now suitable for study; however, it can be simplified by the Equation change of the operator variable

(2.2.4)

9T ( s ) = O ( s ) + ( 1 + w O ( s )) � ( s ) ( 1 + w O ( s )) ,

(

2.2 .5

)

II. EXPANSION IN EIGENFUNCTIONS OF

�( ( f ; X )

which leads to an equation (the Faddeev equation) for the operator ?B ( s ) which is equivalent to (2.2.4): ( 2.2.6 ) ?B ( s ) IE + wlE ( g + w fl (s )) ?B ( s ) . Solvability of (2.2.6) is studied not in the Hilbert space X( f ; X ), but rather in the Banach space 0?> _ I( f; X) (see §1 .3). In this space the operator IE + w � fl ( s ) is compact, and it depends analytically on s in the half-plane Re s > 0; this enables us to a large extent to use the standard procedure for studying (2.2.6), as will be discussed in detail below. In this chapter we shall finish by proving that for any s with Re s > 0 there exists a unique solution ?B ( s ) of (2.2.6) which depends analyti­ cally on s everywhere except for a discrete set of singular points. At these singular points the operator ?B ( s ) has poles of finite order, and the poles Sj in the half-plane Re S � 1/2 are uniquely related to the eigenvalues Aj of the discrete spectrum of m , Aj = si l - Sj ) E [ 0, 00). This now gives us the desired meromorphicity of the resolvent 9l (s) in the half-plane Re S > O. Finally, a system of functional equations will give us meromorphicity of ?B(s) and 9l (s ) as integral operators for s E C (see §3. 1). We begin the program described above by defining the operator fl(s) and explaining its properties. Because of the requirement (2.2.3), the defining equation for the operator has the form (2 .2.7 ) where K = s ' E IR , K > 2 is fixed, and w = s(1 - s ) K(1 - K). Equation (2.2.7) can be rewritten as follows in terms of kernels of integral operators: =

-

q ( z, z ' ; s ) = t ( z , z ' ; K )

+w

-

-

f t ( z, z " ; K ) q ( Z ", z ' ; s ) dJl( z ") . F

(2.2.8 )

From the definition of the kernel of �(K) in (2. 1 .27) it follows that (2.2.8) splits into n equations for each component: (2.2.9 )

where t aal z , z ' ; K) = Pat(y, y' ; K), a = 1 , . . . , n. 1t is now not hard to see that (2.2.9) is satisfied by the kernel where the kernel q(y, y ' ; s) was defined by Faddeev in §3 of [9] when he was considering the scalar theory: y, S)y tl -S , y � y', I
{

,

(2 .2. 1 1 )

In fact, q( y, y ' ; s ) satisfies the equation

q( y , y' ; s ) = t( y, y' ; K ) + w

f

a

00

�" t ( y, y " ; K ) q ( y " , y' ; s ) ----; ;:; y

(2.2 . 12)

§2 .2. ANALYTIC CONTINUATION OF THE RESOLVENT

29

for Re s < K, and the projection Pa has the property that a = 1,. . . ,n.

Thus, the integral operator O(s) satisfying (2.2.7) is given component by compo­ nent by the following kernel on F X F:

q oo( z , z' ; s ) = 0,

q aO ( z , z'; s ) = 0, q aP ( Z , z ' ; s ) = o, a

-=!=

1 � a � n,Z = x

{3 , 1 � a, {3 � n , (2.2 . 1 3 )

+ iy, z' =

x

'

+ iy',

where q(y , y'; s) is defined by (2.2. l0). Since the kernel q( y, y'; s) plays an important role not only in the scalar situation, we shall give some of its properties here, with references to [9]. We begin with the function

a d

Iy =a '

which is selfadjoint in Li[ a, (0), C, dy/y 2 ). This problem only has a continuous spectrum. The kernel q(y, y'; s ) for Re s > 1 /2 is the kernel of its resolvent. 'We have the following formulas:

q ( y , y'; s ) = q( y', y; s ) , q ( y , y' ; s ) - q ( y , y ' ; s ') .. "

= [ s ( l - s ) - s' ( 1 - s')]

ja

q ( y , y' ; s ) = q ( y, y ' ; 05 ) , 00

.

q(y, y" ; s ) q ( y", y'; s' )

ja

00

�

d " ' y '2 (2 .2 . 14)

q( y , y ' ; s )

�2 '

d ' y

Re s' < Re s , 1 q ( y , y' ; s ) - q ( y, y'; 1 - s ) = _

1/2. 2) The operator O(s) gives a mapping from the entire space 'BD _ \(f; X ) to the space 'BD \( f; X ) which depends analytically on s in the half-plane Re s > O. Here if f E 'BD _ \ ( f ; X ) , then O( s )f E 'BD \ - Re sC f ; x ) · 3) Equation (2.2.7) holds as a relation between operators in X ( f ; X ) for s with 1 /2 < Re s < K, and as a relation between operators in 0?J �� \(f; X) for s with 0 < Re s < 2.

30

II. EXPANSION IN EIGENFUNCTIONS OF

4) Suppose that Re s � 1 /2, f E iffi _ \(r; X ) and h

asymptotic expansion holds for each component as y � h a(z )

\ -s

m e r ; X)

= O(s)/. Then the following

00 :

= Pa 2; _ 1 1 «p ( y', s )fa(z') dp. ( z') + 0( 1 ) , iy, z ' = x ' + iy ' . Y ......

IT a

00

x+ 5) The operator products O(S):D( K ) and O(S)Gf( K ) are defined in iffi \(r; X), and the products :D( K )O(S) and Gf(K)O(S) are defined in iffi _ \(r; X); all four of these products are the zero operator. 6) The kernel q (z, z'; s) of the operator O(s) has the properties q (z' , z ; s ) = q ( z , z'; s) , q (z, z ' ; s ) = q ( z, z'; s ) . a = 1 , . . . ,n, z =

PROOF. Part 1) is proved in the same way as part 2) of Theorem 2. 1 .2; it is based on the explicit formula for the kernel q(y, y '; s). We shall not give the details here. We proceed to part 2). Suppose that f E iffi _ \(r; X ). Then

h(z) ho(z) We have

= 0,

= O(s )f( z ) ,

z = x + iy ,

a = 1,. . . ,n,

z' = x ' + iy ' .

I h a(z ) I v « 1 1 q(y, y' ; s ) I l fa(z') I v dp. ( z') « OO l q(y, y'; s ) l y ,- 3 dy' s IT a

S

f

a

and this proves 2), since it is easy to verify continuity of the components h a( z ) and analytic dependence of O(s) on s. Part 3) has already been essentially proved when we defined O(s), and 4) is a simple exercise using the definition of q( y , y '; s). We now prove 5). If we use the technique in the proof of part 3) of Theorem 2. 1 .2, in particular (2. 1 .33), it is not hard to show that the operators :D ( K ) and Gf ( K ) are defined in iffi ,l r; X ) for any v E IR , and they map this space to iffi v - 2 (r; X). Hence, the products O(S):D(K) and O(S)Gf( K ) are defined in iffi \(r; X ), since O(s) is defined in iffi _ 1(r; X), by 2). Using an analogous argument, we see that the products :D(K)O(S) and Gf(K)O(S) are defined in iffi _ 1(r; X). We now show that all of these products are the zero operator. We ·consider the kernel �( z, z' ) of the operator

:D ( K )O(S), z, Z' E F: � (z , z') = ! b ( z , z " ; K ) q ( Z" , z' ; s ) dp. ( z") . F

Obviously, the only nonzero components of �( z, Furthermore, from (2. 1 .25) and (2.2. 1 3) we obtain

z') are � aa(z, z'),

a = 1, . . . ,n.

00

� aa(z , z , ) = 2 Pa2 1 � [cos 2 7Tj( X - x ")] P j ( Y" , y'; K )q( y" , y'; s ) dp. ( z"), IT a j =

lllK:, .

I

(2.2 . 1 5 )

§2 .2. ANALYTIC CONTINUATION OF THE RESOLVENT z

=

31

x + iy, z ' = x ' + iy ' , a = 1, . . . , n. The last equality shows that q aa( z, z ') is also

identically zero, just as the other components, since there is no constant term in the Fourier expansion of the first kernel in (2.2. 1 5). We similarly prove that the operator O (S)cn(K) is equal to zero. The products �(K) O (S) and O (S)�(K) are the zero operator for another reason. Multiplying out these integral operators, we easily see that zero factors occur in the resulting kernel, namely,

because of the following property of projections: Pa = P;, a = 1 , . . . , n o Part 6) of Theorem 2.2. 1 follows easily from the definition (2.2. 1 3) of the kernel q (z, z '; s). The proof is complete. In Theorem 2.2. 1 we described several properties of the operators er (K), � (K) and �(K) in the Hilbert space :Je ( r; X). However, we shall be studying solvability of the Faddeev equation (2.2.6) in the space � _ I(r; X), so we shall need the properties of these integral operators in the spaces �v( r; X). The theorem that follows is only stated in enough generality to suffice for our investigation of equation (2.2.6). 2.2.2. Let K be a fixed number, K > 3. 1) The operator er ( K) is a continuous map from � 1 ( r;

THEOREM

X) to 0?J _ 1 -- .s< r; X) for some

X) to � _ I ( r; X). 2) The operator er( K) is a compact map from � - I ( r; X) to itself. 3) The operators �(K) and �(K) map �v( r; X) continuously to �v - i r; X) for any fixed p E IR . 4) The operators �(K) and <.;f(K) map � _ I ( r; X) to itself compactly.

B > 0 and is a compact map from � I ( r;

All the parts of the theorem are proved by a direct verification to the definitions of continuity (boundedness) and compactness of linear maps between Banach spaces, i.e., for continuity one shows that any set in the domain of definition which is bounded in norm (it suffices to consider the unit ball) is taken to another bounded set under the given operator. In the case of compactness one verifies that a bounded set goes to a compact set. In the latter case an essential role is played by the specific nature of the space �/ r; X), since the compact sets in these spaces are characterized by a variant of the well-known Ascoli theorem. The theorem says the following: A set of ( componentwise) equicontinuous functions which is bounded in �v' is compact in �v for p ' < P . We now prove parts 3) and 4) for the operator � (K). It is the presence of this particular operator which brings in the specific features of our more general theory than the theory in [9]. The other parts are proved analogously, and we shall not give those proofs here. 3) First suppose thatf E Co( F; V; X) (see § 1 .3). We set h(z) = � ( K) f(z), and we estimate all the components of h( z ), starting from the definition (2. 1 .26) of the kernel e(z, z'; K): PROOF.

a = 1 ,. . . ,n.

l l . tXPA N SlON IN EIGENFUNCTIONS OF

\l( ( r ;

X)

We have

« Ir;;� :h

j=

00

�

oo

1 8'0:

1

+j

I �I J

exp ( - 2 7T I 8'0: + j l l y - y / I )] 1 I00 ( z ' ) I v d/l ( z') . (2 .2. 1 6)

In (2.2. 16) we have used (2. 1 .22) and (2. 1 .23). The estimate (2. 1 .33) now shows that the function f can be taken from any of the spaces 0"2>/ f; X). In fact, we continue (2.2. 16) as follows: 00

�

h j=

I h o: ( z ) I v « Ir;;�:

« y "-- 2 I max � '� h

oo

1 1 8'0: 00

�_

j=

00

+

jI �

00

[ exp( - 2 7T I 8'0: + j I l y - y / I )] y/II-- 2 dy'

1

I 8'0: + j 1 2

« y "-2 ;

the estimate holds for any a = 1 , . . . , n. Continuity of the components h o:(z) follows from the fact that the kernel m 0:( Z, z/; K ) is a Green's function. Part 3) is proved. 4) By the above criterion for compactness of sets in g?, _ I (f; X), it is sufficient to prove the following claim. The set of images h = CZ ( K )f, where f runs through the unit ball " f " F, x . - I .;;; 1

consists of componentwise equicontinuous functions. We prove this. It is not hard to verify that

(2 .2 . 1 7)

where the constant in the last estimate does not depend on f. Passing to the limit as Z � Zo in (2.2. 1 7), we obtain the desired equicontinuity. The proof is complete. We now proceed to our study of (2.2.6). We let &) (s) denote the operator where

&) (s ) = ?U ( K ) + W ?U ( K ) O ( S ),

( 2.2 . 1 8) > 3 is fixed. From Theorems 2.2 . 1 and

As in Theorem 2.2.2, suppose that K E R , K 2.2.2 it follows that the operator &) ( s}is defined in 'ffi _ 1(f; X) for every s such that ° < Re s < 2 + e, for some e > 0, and in that space it is a bounded compact operator which depends analytically on s. We consider the homogeneous equation

f = w &) ( s ) f,

(2 .2 . 1 9)

Following Faddeev, we shall call points s in the strip ° < Re s < 2 + e singular points if this equation has a nontrivial solution. The general theory of compact operators depending analytically on a parameter tells us that the singular points are discrete. The following two theorems are a formal generalization of Lemmas 3.2 and 3.3 in [9], and we give them here without proof.

..

33

§2.2 . ANALYTIC CONTINUATION OF THE RESOLVENT

2.2.3. 1) If s is a singular point of equation (2.2. 1 9), where Re s � 1 /2, s =F 1 /2, then A = s( 1 - s ) is an eigenvalue of the operator m . 2) Iff is a solution of (2.2. 1 9) for the singular point s, then the vector-valued function v = f w O(s)f belongs to X(f; X), where v is an eigenfunction of m with eigenvalue A = s(l - s ). 3) If s is a singular point of (2.2. l 9) and Re s = 1 /2, then 1 - s is also a singular point for (2.2. 19). 4) Let v E X(f; X) be an eigenfunction for m with eigenvalue A � O. Then the vector-valued function f = v - W � (K)V belongs to <1>_ I(f; X ) and satisfies (2.2. l 9), where the value of s is determined by the condition A = s( 1 - s) uniquely for Re s > 1 /2 and up to complex conjugation for Re s = 1 /2. THEOREM

+

2.2.4. The discrete spectrum of the operator m ( f; X ) consists of eigenval­ ues offinite multiplicity located on the nonnegative part 0 � A < 00 of the real axis and not having any points of accumulation in any finite interval. THEOREM

We rewrite (2.2.6) with the new notation:

SS(s ) = � ( K ) + w�(s) SS ( s ).

( 2.2.20) The kernel of the free term in this equation has a weak singularity on the diagonal, and so is not a componentwise continuous function on F X F. In order to study the kernel of SS(s) by means of (2.2.20), it is natural to iterate the equation once. We introduce the operator The equation for

SS \ ( s ) = SS ( s) - � ( K ) .

( 2.2.21 )

SS \(s), which is equivalent to (2.2.20), has the form SS\ ( s ) = SSo ( s ) + w� SS \(s ),

(2.2.22)

where, by definition,

b(z, z';

( 2 .2.23 )

(2.2.22) is regarded as a Fredholm integral equation for the kernel s; 1 ) of the operator \(s), which is regarded as an unknown function of the variable The other variables are regarded as parameters. If we keep in mind that for fixed E F the kernels of all of the operators in (2.2.22) are operators in the space V, it is natural to consider the following system of h equations (h = dim V):

SS

z, z'

z.

b( z, z' ; s ; l ) e, = b(z, z' ; s ; O) e, z, z "; s )b( z " , z' ; s ; l ) d ( z " ) e 1 = l , . . . , h . ( 2 .2 .24) +wjl)( F Here we have used the following notation: b(z, z' ; s; 0) is the kernel of the operator SSo(s); l)(z , ."" ; s ) is the kernel of �(s); and e, is an element of any fixed p.

"

basis of V. Clearly, if (2.2.22) is written in terms of kernels and using a basis of V, we obtain the system (2.2.24). Furthermore, using Theorems 2. 1 . 1 , 2.2. 1 , and 2.2.2, it is not hard to show that for every I, 1 ,,;;;;; I ,,;;;;; m , the function O)e , of the variable is an element of � _ \(f; X) which depends analytically on s in the strip o < Re < 2 + E for some E > 0 and depends continuously on the parameter is

s

z

b(z, z' ; s ;

z'

each set Fj of the partition of F (see Theorem 1 .2.4). In addition,

I l b . a( · , z'; s; O) e[ II F,x, - 1 �y' - I , s

(2.2 .25)

where z' E IT a ' Z ' = x' + iy', a = 1 , . . . , n. The norm in (2.2.25) is taken with respect to the variable z, and the a indicates that we are dealing with the a-compo­ nent with respect to the variable z'. Thus, (2.2.24) can be regarded as a system of equations in 'ffi _ I (f; X ). The set of functions b(z, z'; s; l ) e[ , 1 < I � h, which forms a solution of (2.2.24), uniquely determines the kernel b(z, z'; s; 1 ) of the integral operator ?8 1 (s) in the strip 0 < Re s < 2 + f everywhere except at the singular points of (2.2 . 1 9). We shall describe the properties of this kernel in Theorem 2.2.5, which we shall state in terms of the kernel of the operator ?8(s) we are actually interested in; this operator is connected to ?8 1 (s) by the relation (2.2.21). We shall not give a rigorous proof of this theorem here, since it is standard (see [9J, §3, or [32J, Chapter 14, § 1 0). THEOREM

ties:

2.2.5.

There exists an integral operator ?8(s) having the following proper­

1)

?8 (s ) = ?8 1 (s) + <;S; ( K ) + en ( K ) + � ( K) , where the kernels of the operators <;s; (K), en ( K ) and � ( K) are defined in Theorem 2. 1 . 1 . As a function of s, the kernel b( z, z'; s; 1 ) of ?8 /s) is analytic in the strip o < Re s < 2 + f for some f > 0, except at the singular points of equation (2.2. 1 9), where it has poles of finite multiplicity. Furthermore, b(z, z'; s; 1) has all components continuous on F X F and satisfies the estimates I boa(z, z' ; s ; 1 ) I v �y' - l , l I baO (z , z'; s ; 1 ) I v � y - , s

s

1 � a , f3 < n ,

z = x + iy, z' = x ' + iy '. Finally, the kernel b(z, z'; s; 1 ) has the Hermitian property b ( z' , z ,· s ,· 1 ) = b* ( z , z' ·" S· 1 ) , where the star denotes the adjoint operator in V. 2) ?8(s) is a bounded operator in the space 'ffi _ 1 (f; X ) for nonsingular s in the strip o < Re s < 2 + f, f > O . 3) ?8 (s) satisfies (2.2.20). We now note that the Hilbert identity (2.2. 1 ) for the resolvent 9T(s) and the formula (2.2.5), which connects ?8( s) with the resolvent, imply the " Hilbert identity" for ?8(s): ?8 ( s ) - ?8 ( s ') = [ s(I - s) - s '(I - s')] [ ?8 (s ) + w ( s ) ?8 ( s ) G ( s )] X [ G ( s ') ?8 (s')w( s ') + ?8 ( s')] ' where w ( s) = s(l - s) -. x(l - K), which holds as an operator identity in the space X(f; X ) for all s' and s for which Re s', Re s > 1/2 and s', s ft ( 1 /2, I J. By means of (2.2.5) the operator ?8 (s) realizes a meromorphic continuation of the resolvent 9T(s) as an integral operator in the half-plane Re s > O.

§2 . 3 . EIGENFUNCTIONS OF THE CONTINUOUS S.PECTRUM

35

We now explain the behavior of the kernels b(z, z'; s) and r(z, z' ; s) for the operator SB (s) and 9l (s), respectively, in a neighborhood of an arbitrary singular point So for which Re So � 1/2. By Theorem 2.2.3, such a point is connected with an eigenvalue A o of the discrete spectrum of 21 , Ao = so(1 - so) · Using the general spectral theorem for the resolvent of any selfadjoint operator and (2.2.5), we obtain

b (z , z'; s )

=

r (z, z ' ; s ) =

2s o Ao

_

_

no

1 s - SO J�-- l f( z; j) (8) f( z' ; j) + b ( z , z' ; s ) , 1

I

1 s( I

_

no

_

, ; s , 2.2.26 ) v ; v ; s ) :j,�= I ( z j ) \8) ( z' j) + r(z z' ) (

where {v(z; j )}jl2, I is a real basis of the eigensubspace of X( f; the n o-fold eigenvalue Ao of the operator 21 ; furthermore,

X) corresponding to

f( z ; j ) = v ( z ; j) - w( so ) � ( K ) v ( z ; j ) ; the kernels 6(z, z'; s) and fez, z ' ; s) are analytic in s in a neighborhood oLso, and (8)

denotes the tensor product of the values of functions in V. Finally, we give a theorem which summarizes some of the results obtained here concerning the kernel of the resolvent ffi(s).

2.2.6. 1) For fixed z and z', z =F z' (mod f), the kernel r( z , z' ; s) of the resolvent of the operator 21 is an analytic function in the half-plane Re s > 0, except for a discrete �et of singular points of equation (2.2.19), where it has poles of finite multiplicity. 2) The poles s for which Re s � 1/2 are all on the line Re s = 1/2 or on the interval 1 /2 :0::;; s :0::;; 1 of the real axis. All of them are simple poles, with the possible exception of the pole at s = 1/2. The principal parts of r( z, Z ' ; s) at these poles are given in (2.2.26). THEOREM

,

.

§2.3.

Eigenfunctiandons spectral of the contidecomposi nuous spectrum of tion

21 ( f ;

X)

The basic goals of this section are to determine the eigenfunctions of the continuous spectrum of the operator 21 in terms of the operator SB (s), and to prove a theorem on expansion in eigenfunctions of m . These results are generalizations of corresponding facts of the scalar theory which were developed by Faddeev in [9]. Let a be a natural number, I :0::;; a :0::;; n. We choose a basis {etC a ) }7= I of V which is compatible with the direct sum decomposition V = Va liB V;- , where V;- is the orthogonal complement of Va; ( Va' V;- > = 0, i.e., we shall suppose that the set of elements e, (O') E V forms a basis of the subspace Va for I :0::;; 1 :0::;; k a (see §1 .2). We introduce the following set of vector-valued functions cp (z; s; a; I) defined on F, 1 :0::;; a :0::;; n, 1 :0::;; 1 :0::;; k a . We define the components of these functions as follows: CPo ( z ; s ; a ; l ) = ° for all 1 :0::;; a :0::;; n , 1 :0::;; I :0::;; k a' (2.3. l ) cpp( z ; s; a ; l ) = Bape, ( O') cp ( y, s ), where the scalar-valued function cp ( y, s) was defined in (2.2.l 1). Next, using the notation of the last two sections, we introduce the integral operator

( 2.3 .2 )

II. EXPANSION IN EIGENFUNCTIONS OF 21 ( I' ;

X)

and the following set of vector-valued functions defined on F:

( 2.3.3 ) l/;( z ; s; a; l ) =

PROOF. Because of Theorem 2.2.5 and (2.2.26), it suffices to prove 3). We now prove this, disregarding for the time being the point s = 1 /2. This point will be considered separately in Theorem 3.2.2. Suppose that So is a singular point for the operator l{3(s) and Re So = 1 /2, So =F 1 /2. It follows from (2.2.26) that in a neighborhood of s = So the nonanalytic part of the kernel b( z, z'; s) is equal to no 1 1 ( 2.3 .4) . f f( 2 so - 1 s So j� l ( z ; j) 0 z'; j) -

From (2.3.4) it follows that in a neighborhood of s = So the principal part of l/;( z; s; a; l) is equal to no 1 1 2: I v ( z ; j) !F ( f(z';j),

s

1

_

no

SO j=2: 1 v ( z ; j ) lITf a ( fa(z';j) , et(a)

( 2 .3 .5 )

where the function v(z ; j ) is defined in the context of (2.2.26). We want to prove that both parts of (2.3.5) are finite in the limit as s � so. To do this it suffices to prove that the following inner product vanishes: ( fa ,l, rp , et ( a)) , ( 2 . 3 .6 ) where, by definition,

fa ,l,rp = 1 fa(z ' ; j)

" ,

( 2 .3 .7 )

In order to prove this, we consider the quadratic form

t( [5 ( K ) V (Z ; j) , V(Z; j )

dp, ( z ) ,

( 2.3.8 )

where the operator [5 ( K ) is defined by (2.2. 1 8). By our choice of K, the operator [5 (K) is selfadjoint. Hence, the integral (2.3.8) is real. On the other hand, by Theorem 2.2.3 we have

( f( z ; j ) , v ( z ; j ) ) = w ( so ) ( [5 ( K ) V ( Z ; j ) , v ( z ; j ) ) .

( 2 .3 .9)

37

§2 .3 . EIGENFUNCTIONS OF. THE CONTINUOUS SPECTRUM

Since w ( so ) is a real number and < fez ; j), fez ; j) is a real function, it follows, because the integral (2.3.8) is real, that

( fa(z ; j ) , Pa fa(z'; j) [q(y, y'; so ) - q(y, y' ; so ) ] dp,(z) dp, ( z') = 0. a�= 1 1ITa 1ITa n

From the last formula in (2.2. 1 4) we obtain the equality n

� I Pa fa , I , 'P I � = 0, a=l and so, in particular, fa , I, 'P E V;- . Hence, Jhe inner product (2.3.6) is equal to zero. The proof is complete. Now each of the vector-valued functions tJ;(z ; s; a; I), 1 � a � n , 1 � I � k a ? which before were defined on some chosen fundamental domain F, can be extended by automorphicity to the entire half-plane H. For any z E H there exist z' E F and y E r such that z' = yz. By definition,

tJ;(z ; s ; a ; I ) = X*( y )tJ;(z'; s; a ; I ) .

Each of the functions tJ;(z ; s ; a; I), 1 � a � n, 1 � I � k a ' has the following properties: 1 ) The following equality holds for any z E H and y E f : tJ;(yz ; s ; a ; I ) = X(y)tJ;(z ; s; a ; I ) . 2) tJ; E C oo ( F; V; X) (see § 1 .3). 3) -LtJ;(z ; s ; a; I) = s(1 - s) tJ;( z; s ; a; I) (see § 1 . 1). 4) The /3-component has the following asymptotic behavior for z E ITa: tJ;� ( z ; s ; a; I ) = 8a� el (a)cp(y, s ) THEOREM

,.

(2.3 . l O)

+

2.3.2.

:�\ i - sp�lITa lITacp(y, s ) b/3a(z, Z'; s ) cp( Y ', s ) dp, ( z ) dp,(z')el (a) + 0( 1 ) . Y -> 00

2

Part 1 ) is a consequence of (2.3. l O), and 4) is a consequence of part 4) of Theorem 2.2. 1 . To prove 2) and 3), we first verify that PROOF.

91 ( K) tJ;( Z ; s ; a ; I ) = ( 1/w( s ))tJ;(z ; s ; a; I ) .

(2 .3 . 1 1 )

In fact, we have the equality of operators

91 (K ) + 91 ( K) � (S ) = ;t (K ) + ;t ( K ) � (S ) + � (K ) + � ( K ) � (S) = ;t (K ) + ;t ( K ) � (S) + ?S ( s ) . (2 .3 . 1 2) We have used Theorem 2. 1 . 1

and (2.2.5).

Next,

w;t (K ) � ( s ) = w 2 [ ;t ( K ) + w;t ( K ) (} ( s )] ?S (s ) = w2 0(s ) ?S ( s ) = � ( s ) - w ?S ( s ) .

From this and from (2.3. 1 2) we obtain

W 91 ( K ) + w 91 ( K) � (S) = W;t(K)

+

Finally, if we use the equality

� ( s ).

W ;t ( K) cp ( Z ; s ; a ; I) = cp(z; s ; a; I ) ,

then from (2.3. 1 3) we obtain the desired assertion (2.3 . 1 1).

(2 .3 . 1 3 )

II . EXPANSION IN EIGENFUNCTIONS OF � ( r ; X )

38

The equality (2.3. 1 1) shows that l/; ( z; s; a; I ) satisfies the equation in 3) in a weak sense. Since the differential operator L is elliptic, it follows from the general regularity theorem (see [32], Appendix 4, §5, Theorem 4) that we first have part 2), and then also part 3) of the theorem. The proof is complete. We now describe a functional equation for the kernel r(z, z'; s) of the resolvent, considered as a function of s. This functional equation relates its values at nonsingu­ lar points s and I s which both belong to the strip 0 < Re s < 2. With this in mind we prove the following lemma. -

LEMMA 2.3.1 . Let q ( z , z' ; s) be the kernel of the operator O (s). Then O ( z , z'; s ) - O ( z, z' , 1

I

n

ka

�

s)

=

2s

-

_

functions.

PROOF. The lemma follows easily from the definition of O (s), the second formula in (2.2. 14), and the equality ka

Pa

= � e/ ( a ) 0 e[ ( a ) .

The proof is complete.

/= 1

THEOREM 2.3.3. Suppose that s E C satisfies the conditions 0 < I - Re s < 2 and o < Re s < 2, and the points s and I s are nonsingular for the operator Q3 (s). Then -

r( z, z' ; s ) - r ( z , z' ; 1 - s ) = 2s

I _

11

ka

1 a�= I /=�I l/; ( z ; s; a ; I ) 0 l/; ( z' ; 1

-

s; a ; I ) . ( 2 .3 . 14)

PROOF. It suffices to prove (2.3. 14) for points z , z ' E F in general position. Suppose that s, s ' E C satisfy the conditions Re s, Re s ' > 1/2 and s, s ' ti ( 1 /2, 1 ]. We consider the Hilbert identity for fft(s) and fft(s '), which holds as an operator identity in :Je(f; X ) (see (2.2.1)): ( 2 .3 . 15 ) � ( s ) - fft ( s ') = [w( s ) - w( s ')] fft ( s ) � ( s '), where, as before, w ( s) = s( 1 s) - 1((1 I( ) , I( > 3 being fixed. On the right side of (2.3. 15) we substitute the expression for fft (s) and fft (s ') in (2.2.5) in terms of the operators Q3 and O . We obtain the equ�lity -

-

91 (s ) - fft ( s ') = [ 1 + w( s ) ( 1 + w( s ) O ( s )) Q3 ( s ) ] X [ O ( s ) - O (s') ] [Q3 ( s ')( O ( s ')w ( s ') + 1 )w( s ') + �f] + ( w ( s ) - w( s ')) [( 1 + w( s ) O (s ) ) Q3 ( s ) O ( s ') + O ( s ) 'B ( s')( O ( s ')w( s ') + 1 ) + ( 1 + w( s ) 0 (s )) Q3 ( s )(1 + w ( s ) O ( s ) + w( s ') O ( s ')) X Q3 (s')( O ( s ')w( s ') + 1 ) ] . ( 2.3 . 16 )

§2.3. EIGENFUNCTIONS OF THE CONTINUOUS SPECTRUM

39

In deriving (2.3.16) we also used the Hilbert identity for £l (s) at the points s and s ' :

£l ( s ) - £l ( s ') = [w ( s ) - w( s ')] £l ( s ) £l ( s ') , an identity which, in turn, follows from (2.2. 14). The right side of (2.3.16), regarded

as a composition of integral operators, permits a meromorphic continuation in s and s ' to the strip 0 < Re s, Re s ' < 2. Now setting s ' 1 - s and using Lemma 2.3.1, we obtain the desired formula (2.3.14). The proof is complete. We proceed to study the properties of the second coefficients in the asymptotic expansion of the vector-valued functions lj;(z; s; a ; I ), 1 � a � n, 1 � 1 � k a ' in a neighborhood of the cusps of F (see part 4) of Theorem 2.3.2). As we shall later see, the set of these coefficients, arranged in a matrix, plays an important role throughout the spectral theory of automorphic functions. Just as in the scalar theory, this matrix corresponds to the scattering operator in the perturbation theory of the continuous spectrum of selfadjoint operators (see [9], §4). In the notation of part 4) of Theorem 2.3.2 we set

=

@5 al,,Bk (S ) = oa,B ( ek ( /3 ) , e/ (a) ) a 2 S- 1 �: � 1 s 2 ) + ;: � 1 l I a II aqJ ( y, s ) ( e k ( /3 ) , b,BA z , z ' ; ( s ) e/ ( a )) ) qJ ( y ', s ) djl ( z ) djl ( z ') , 1 � a � n , 1 � /3 � n , 1 � l � k a ' 1 � k � k,B ' (2.3 . 17 )

ff

Theorem 2.3.1 implies the following theorem.

2.3.4. Each function ® al,{3k ( s) has the following properties: 1) It is analytic in the strip 0 < Re s < 2, except at the singular points of the operator �(s), where it has poles offinite multiplicity. 2) It does not have poles on the line Re s = 1 /2. 3) It has at most simple poles at the real singular points of � (s) for which 1 /2 < s � 1 . THEOREM

We now prove that the set of functions (2.3. 17) has two important properties, which characterize it as a matrix of order kef; X) = 2.� k a (see §1 .2).

2.3.5. The set offunctions ® al, {3k( s) has the following properties: 1 ) ® al, ,B k( S ) = ® {3k , a l ( S ) ; 2) kp n (2.3. 18) � � ® al , {3k (S ) ® {3k , y m ( 1 - s ) = 0ayOlm , ,B= l k = l

THEOREM

where Oay is-the Kronecker symbol, 1 � a � n, 1 � I � k a ' 1 � Y � n and 1 � m � ky o

PROOF. Part 1) follows easily from (2.3. 17) and the Hermitian property of the kernel b(z, z ' ; s ) (see Theorem 2.2.5 and (2.1 .28)).

I I EX PANSION IN EIGEN F U N CTIONS OF )I ( J' ; X )

40

We prove part 2). By (2.3.17), the left side of (2.3. 1 8) has the form

2 ) oayo/m + � � 1 I I I I a
ff

ffff

_

Next, we have the obvious equality k /i

2:

k= 1

( em( y ) , byfj( z, l' ; l - s)ek( /3 » ) ( ek( /3 ) , bfja( z , z'; s)e/(a») kf3

2:

k=1

( em( y ) , byfj(i, l' ; l - s ) ek(/3 ) ® ek(/3)bfja(z, z'; s )e,( a»)

= ( em(y ) , byfj(z, 1' ; 1 - s )Pfjbfja(z , z'; s ) e,(a» ) . We apply this in the quadruple integral in (2.3.1 9) and use the operator identity SE (s) - SE { I - s ) = w 2 ( s ) SE ( s ) [ 0 (s ) - 0 ( 1 - s )] SE ( 1 - s ) , which is a consequence of the Hilbert identity for the operator SE ( s ) (see the remark following Theorem 2.2.5). After combining similar terms, we then obtain the right side of (2.3. 18). The proof is complete.

In the sequel it will sometimes be convenient to use the following natural replacement of the four indices 0'/, 13k in 6 al, fjk( S ) by two indices:

d = kl + k 2 + . . . + k a - I + /( 0') ,

b = kl + k2 + . . . + kfj_ 1 + k ( /3 ) . (2.3.20 ) In this notation { 6 d,b( S ) } is a square matrix of order kef ; X) = "i.\ k a • We shall let 6 ( s ) denote this matrix. Parts 1) and 2) of Theorem 2.3.5 can be rewritten as follows:

I ) 6 ( s ) = 6* ( 8 ) , (2.3.21 ) 2 ) 6 (s) 6 { I - s ) = Ik ' where Ik is the identity matrix of order k = kef; X ), and * is the adjoint matrix. By analogy with part 2) of Theorem 2.3.5 it is not hard to prove the following theorem giving a functional equation for ",,( z ; s; 0'; I). Here we shall limit ourselves

to the statement of the theorem.

"

§2.3 . EIGENFUNCTIONS OF THE CONTINUOUS SPECTRUM

41

2.3.6.

THEOREM

t/; ( z ; 1 - s ; a ; I ) = where 1 :;;;; a :;;;; n and 1

:;;;;

n

kfJ

@5 al , fJ ( 1 - s ) t/;(z ; s; /3 ; k ) , � � k fJ= i k= i

I :;;;; k a .

We conclude this section, and at the same time Chapter 2, with a theorem on expansion in eigenfunctions of the operator 9l. This theorem shows, in particular, that the vector-valued functions t/;(z; s; a; I ) for 1 :;;;; a :;;;; n, 1 :;;;; I :;;;; ka and s = 1 /2 + it, t E IR , form a complete kef; x )-fold system of eigenfunctions for the continuous spectrum of 9l. The corresponding "eigenvalues" of the continuous spectrum fill out the semiaxis 1/4 :;;;; A < 00 . We shall state the theorem in the manner which is customary in the perturbation theory for the continuous spectrum of selfadjoint operators (compare with Theorem 4.1 of [9]). For convenience, in the Hilbert space L2(1R ; C k; dt), k = kef; X ), we introduce the normalized inner product

On a suitable dense set in L2(1R ; the equation

C k;

dt) we define the selfadjoint operator 910 by

9l0 � ( t ) = (1 + t 2 )H t ) , where ffi o(s) denotes the resolvent ffio(s) = ( 910 - s(1 - S )10) - I , and 10 is the identity operator in LilR ; C k; dt).

We let \l5 denote the projection onto the subspace of :Je ( f; X) spanned by all of the eigenfunctions of the discrete spectrum of 9l , . and we let 1 denote the identity operator in :Je(f; X). THEOREM

2.3.7. The map & : f � � defined for f E Co( F; V; X) by the formula

� A t ) = f ( f( z ) , t/;( z ; 1 + it; a ; I ) ) d/L ( z ) , F

where the parameters d, a and I are connected by (2.3.20), d = 1 , . . . , kef; X ), extends to an isometric map U from the space :Je(f; X) onto all of the space L2(1R, c k, dt). Here the following equalities hold: 1) U*U = 1 - \l5 , where U* is the adjoint. 2) UU* = 10' 3) Uh(91 ) = h( 91 )U for any bounded measurable function h: [ 0, 00) � C . We prove 1). Suppose that f E Co( F; V; X ) and ( A , B ) is an interval on the line Re s = 1/2 not containing singular points of Q3 (s). We have PROOF.

ka n B 1 2 t ) 1 dt = 4 � � f dtf d/L ( z ) f d/L ( z ') � � A f I 4 'lT 'lT a = I 1= 1 A F F d= I A X ( f( z ) , t/;( z ; 1 + it; a ; l ) ) ( f( z'), t/;( Z ' ; � + it; a ; l ) ) .

1

k( r ; x )

B

(2.3 .22)

II. EXPANSION IN EIGENFUNCTIONS OF � ( r ; X )

42

We transform the integrand on the right in (2.3.22): n

ka

� �

a = I /= 1

( f(z), tJ; (Z ; ! + it; a; i) ( f(z '), tJ;(Z' ; ! + it ; a; i) n

= � � ( f(z), tJ; (z; ! + it; a; l ) ® tJ;(z' ; ! + it; a; t) f(z ' ) ) = 2it ( [ r ( z, Z' ; ! + it ) - r ( z, Z' ; ! - it )] f( z '), f( z ) ) . (2.3 .23) ka

a = I /= 1

We have used (2.3. 14) and the Hermitian property of the kernel of the resolvent:

r*(z, z' ; s ) = r(z, z ' ; s ) .

We now recall a general formula which relates the quadratic form of the resolvent of an arbitrary selfadjoint operator to its spectral measure: 1 lim (2.3 .24 ) 2 . ...... O In order to use (2.3.24), we let denote the kernel of the resolvent ( ll in terms of the variable Applying (2.3.23) to (2.3.22) and introducing a new variable of integration, we obtain

7T 1 A 1)- 1

1 k(r; X) � 4 d= 1

7T

E

+ i f) - R(A - is)] f, f) d A = jBA d (Ex f, f). jB([R(A A fez, z'; A ) A.

1 f /4 + f f fAB I � A t) 1 2 dt = 2 7Tl. 1 /l 4 + A2B 2d A Fdp,(z) Fdp,(z') x ([r(z, z'; A + iO) - fez, z '; A - iO)] f(z ', fez))) ,

( 2.3.25 )

where

fez, z ' ; A iO) = limf0 (z, z ' ; A is). We now represent the entire line Re s = 1/2 up to a discrete set of points as a -+-

E '''''

±

union of intervals (A, B) not containing singular points of l8 (s). Summing the contributions of these intervals to the integrals (2.3.25), if we use (2.3.24) and the definition of the spectral measure we obtain ( 2 .3 .26 ) which is obviously preserved for any elementf E :Je(f; X). This proves part 1) of the theorem. The proof of 2) is based on the completeness property and the orthogonality of the functions «p(y, 1/2 E .IR (see the comments on (2.2.14» . We shall not give the proof here, since it carries over formally from the scalar theory (see Theorem 4. 1 of [9]). It suffices to prove part 3) of the theorem for resolvents at some common regular point s: U 91 (s) = 91 o(s ) U . But this follows 'from (2.3.1 1) and the definition of the operator U . The proof is complete.

+ it), t

i '.

CHAPTER 3 FIRST REFINEMENT OF THE EXPANSION THEOREM FOR m { f ; THE CONTINU bus SPECTRUM

X).

In this chapter we shall suppose that f E 9)( 2 and X E &J s( f ) (see § 1 .2). We note that in the class of all groups f E 9)( and representations X E 91 ( f ) this assumption is necessary and sufficient for the existence of a continuous spectrum for the operator W ( f; X) (see the remark following Theorem 2. 1 . 1). As in Chapter 2, we shall not always indicate the dependence on f and X in the notation. §3.1. In this section, following Selberg and Roelcke, we shall define Eisenstein series and construct their Fourier expansions relative to parabolic subgroups fa C f. Such expansions are well known only in the scalar theory (dim V = 1 ; see [50] for X a nontrivial representation, and [28] for X the trivial representation). At the end of the section we use our earlier results from Chapter 2 to prove meromorphicity of Eisenstein series (the fundamental hypothesis of Roelcke's paper [46]) and meromor­ phicity of the resolvent m (s; f; X) of W ( f ; X) on C . For every natural number a, I .:;;; a .:;;; n, and for every element v E Va C V (see § 1 .2) we define an Eisenstein series by the formula E{z; s; a ; v; f ; X ) = E{z; s ; a ; v ) = � y S ( g�I.yz ) X*{ y )v, (3 . 1 . 1 )

Eisenstein series

, .

y E r,,\ r

where s E C , y(z) = 1m z, * denotes the adjoint operator, and y runs through the set of co sets faY ' We combine into one theorem several important but easily verified properties of the series (3. 1 . 1 ) (see [46], §1O). THEOREM 3. 1 . 1 . Let I .:;;; a ':;;; n and v E Va' The Eisenstein series E(z; s; a; v) converges absolutely in the region Re s > 1 for any z E H and in that region is an

analytic function of s. In addition, as a function of z it has the following properties: I) E(yz; s; a; v) = X(y)E(z; s; a; v)for all y E f. 2) E (z; s; a; v) E COO( F; V; X) (see § 1 .3). 3) -LE(z; s; a; v) = s(1 - s)E(z; s; a; v) (see § 1 . I ).

We now construct the Fourier expansion for an arbitrary Eisenstein series. It is obviously sufficient to construct the expansion for each series E(z; s; a; et C a» , where et C a) is an element of the basis for the subspace Va C V. Here we have a useful analogy with the construction of the Fourier expansion for the kernel hJ a( Z, z'; s) in §2. 1 . In particular, we shall use the notation in §2. 1 (and also §2.3) for the basis elements el( a), the eigenvalues Pia' and so on. 43

I I I . FIRST REF I N EMENT OF THE EXPA NSION THEOREM

44

Let Re s > 1 . We shall show that each vector-valued function j( z ) = Pp E( gpz ; s ; a ; e, ( a ) )

is periodic in the variable x = Re z , z = x + iy, with period one. In fact, because of the equality gp Sr:x; = Spgp (see § 1 .2), we have j( Sr:x; z )

=

� y\' ( g�l ygp Soo z ) pp x* ( y ) e, ( a )

y ET,, \ f

2: y\ ( g�lygp z )ppX ( S/J x*{ y ) e/ { a )

y E f,,\ f

=

j{ z ) .

If we take part 2) of Theorem 3 . 1 . 1 into account, we have the Fourier expansion j( z )

=

00

2: a/ y, s ) exp { 2 7TijX ) ,

./ = - 00

where, as usual, a./ ( y, s ) = ( lj( z ) exp(- 27TijX ) dx . Jo

We compute the coefficients a/ y, s ) : a/ y, s ) = .

.

11

2: yS( g�l ygp z ) pp x*( y ) e / a ) exp { - 2 7TijX ) dx ,

0 y E f,,\ f

(3 . 1 . 2 )

j E ll. We introduce notation for the matrix elements of the transformation : g� lygpz = ( az + b )/ ( cz + d ) , c = C( g�lygp ) , d = d ( g� l ygp ) . ( 3 . 1 . 3 ) We have singled out the elements c and d , since they play a special role here. According to the remark following Theorem 1 . 2.7, for y E r we have c( g� I ygp ) =1= 0, as long as a =1= {3. Furthermore, if a = {3, then c( g�l yga) = 0, if and only if y E ra' a, {3 = I , . . . , n o In ( 3 . 1 .2) we shall first suppose that a =1= {3. We have aj { y , s ) =

2: PpX * ( YSpm ) e,{ a ) l lys( g� l yS;lgpz ) exp { - 2 7TijX ) dx ,

2:

0

y E f,,\ f/fp m E Z

( 3 . 1 .4 )

where y runs through the set of double cosets raY rp . I t is not hard to verify the equalities Sfgp = gp S::;. Pp X * ( ySpm ) = Pp X*{ y ) , U sing them in the right side of ( 3 . 1 .4) , we obtain 1

a/ y, s ) =

2:

y E r� r/�

Pp X*{ y ) e/ { a )

For the integral in (3. 1 .5) we have

f oo - y S- 00

f oo

- 00

- (

ys( g�lygpz ) exp { -2 7T ijX ) dx . (3 . L 5 )

S d exp {-2 7T ijX ) dx = y exp 2 7Tij 2 c Ici s I cz + d i s -"-

2

) f-+00

00 exp { -27TijX )

(X2 + Y2 r

dx . (3 . 1 . 6 )

§3.1. The integral on the right in (3.1.6) occurred before in the scalar theory. It can be computed in terms of special functions (see [28], Chapter II, §2. 2 ): ySf-00oo 2 I 2f dx = i -Sr;; r(Sf(s)1) , (X + y ( 3.1.7 ) i i= 0, where f(s) is the Euler gamma-function, and Kiz) is the modified Bessel function (see §2.l). Substituting these expressions -in (3.1.5), we obtain the desired value for the coefficient a/ y, s) in the case a {3. We now consider the situation when a = {3. Then (3.1.2) transforms as follows: a/ y, s) = �OjPae,(a)yS + � 10 ys ( g;; l yga z ) Pax*(y)e,(a) exp ( 27T ijx) dx EISENSTEIN SERIES

45

-

2

i=

y E r,,\r y ¥T" E

1

= �Oj e,(a)y S +

�af3

,

.

X

-

�

f-00oo ys( g;; l ygaz )

y E r"\ I'/r,, y * r" E

Pax*(y)e,(a)

( 27T ijX ) dx,

exp -

E

(3.1 .8)

f.

where is the Kronecker symbol and is the identity of the group The second equality in is obtained in the same way as If we now substitute the values of the integrals in the right side of we find the desired value for the coefficient in the case {3 as well. We have proved the following theorem.

(3.1.8) (3.1.7) a/y, s)

(3.1.4). (3.1.8),

a=

The following Fourier expansion holds for the parameter values I ::;;;;; a, /3 ::;;;;; n and I ::;;;;; I ::;;;;; k a: Pf3E ( gf3z; s; a; e,(a) ) = � a/ y, ) exp(27T ijX ) where the coefficients in the expansion are given by aj (y, s) = 27TS I i Is - I / 2 f( s r\ly Ks - 1 / 2 (27T Ii I y ) 11/ S) , } i= 0, ao ( y , s) = �af3 e,(a) / + r;; f ( s - -!-) f(s r 1 i -Sl1o (S), Pf3(x*(y)e,(a) exp 2 7Tlj d(g;;lygf3 ) ; (3 . 1 .9) l1j (S) c ( g;; l ygf3 ) I c g;; lygf3 ) 1 2s by definition, a i= /3, a = /3, is the empty set , and} E lL. The series (3. 1 .9) converges absolutely in the norm of for Re s > 1 . THEOREM 3 . 1 .2.

j E lL

=

� �

except y :::: ( ra E r/l )oa# y E r,,\ r/r#

o

s

,

[

V

. .

1

III. FIRST REFINEMENT OF THE EXPANSION THEOREM

46

To complete the construction of the Fourier expansion for Eisenstein series it remains for us to consider the functions ( I v Pp )E( gp z; s; a; e l a » , where I v is the identity operator in the space V. The orthogonal projection onto the subspace V;- satisfies the relation -

Iv

h

-

Pp = }:; [ ek(f3) ® ek(f3) ] , k = kp+ 1

(3. 1 . lO)

h dim V, and ek( f3 ) is an element of a basis of V compatible with the decomposi­ tion V = Va EEl V;- . We consider the vector-valued function (Re s > 1) (3 . 1 . 1 1 ) g( z ) = [ ek(f3) ® ek(f3) ] E( gp z; s; a; e[( a » ) . We have =

g(Soo z ) =

}:;

y E fa\ f

� y E fa\ f

=

y S ( g� I ygp SOO z)[ ek(f3) ® ek(f3) ] x*(y)e[ ( a ) y S ( g� I ygp z) [ ek( f3 ) ® ek( f3 ) ] x ( Sp) x*(y)e[ ( a )

PkP g( z ) = g( z ) exp 27Ti8kP '

where 0 < 8kP < 1 . Consequently, the function [exp( -27Ti 8kp ( x ) ] g( z ) is periodic with respect to x Re z with period one. We set =

00

[ exp ( - 27Ti 8kP x )] g( z ) = }:;

j= - OO

X

bj(y , s ) exp (27T ij ) ,

We compute the coefficients b/ y, s) . We use the notation (3. 1 .3). We first suppose that a =1= {3. Just as in the case of (3. 1 .4) and (3. 1 .5), we get

bj ( y , s ) =

}:;

X

o

}:;

X

[ ekCB) ® ek(f3 ) ] X * (YSpm) etC a )

l 1y s ( g�l ySpmgp z ) exp [ -27Tix ( 8kp + j)] dx

y E f.,\ f /fp

f oo

�

y E fa\ f /fp m E 71.

[ eA f3 ) ® ek( f3 ) ] x*(y)e, ( a )

f-OO00ys ( g� l ygp z ) exp [-27Tix ( 8kp + j)] dx .

(3 . 1 . 12)

For the last integral in (3. 1 . 12) we obtain - 00

=

S y - - exp [ - 27Ti( OkP + j) x] dx ---=-I cz

+ d I2s

1 �2 [exp 27Ti( 8kp + j) d ] f oo ] c - 00 ( x 2 y 2 r exp [ - 2 7Tix( 8kp + j) dx Ici s I c 1- 2 s [ exp 27Ti( 8kP + j) � ] 27TS 1 8kP + j I s - I / 2 f ( sr I K _ 1 / 2 (27T I 0kP + j I y) vY . s -

=

Substituting this in (3. 1 . 1 2), we obtain the desired value of b/y , s) .

§3 . 1 . EISENSTEIN SERIES

a=

47

We now proceed to the case /3 . Here there is a clear analogy with (3. 1.8). However, in this case the term proportional to turns out always to equal zero, thanks to the relation [ ek( /3 ) ® ek( /3 ) ] e, ( /3 ) = ek( /3 ) ( ek( /3 ) , e,( /3 ) = 0 , since E �� E � . We have = � [ k ( /3 ) ® ek( /3 )] x * ( y ) e,( /3 )

ek (/3)

, e, (/3) b/y, s)

y E f"\ fjf,, y "", f" E f"

yS

e

(3 . 1 . 1 3) and the rest is analogous to (3. 1 . 12). We have proved the following theorem.

I

::;;;;;

3.1 .3.

a, /3 ::;;;;; n and I

THEOREM

The following Fourier expansion holds for the parameter values

::;;;;;

I ::;;;;; k a:

( I v - P/3 ) E ( g/3z ; s; a; e,(a) ) =

h

�

k = kf3+ 1

[ek( /3 )

h

�

00

�

k = kf3 + 1 j = - oo

,

'

® ek ( /3 ) ] E( g/3z ; s ; a; e, ( a ) )

bj (y, s ) exp [ 7T i ( j + Ok/3 )X] , 2

where the coefficients are given by b/ y, s) = 2 7T s 1 0k /3 + j IS- I /2 r (s r\/Y Ks - 1 /2 ( 2 7T 1 0k /3 + j 1 y ) �/ s) , �/ s) = y E f,,�\ fjff3 [ ek( /3 ) ® ek ( /3 ) ] x * ( y ) e,( a ) except y = ( f"Ef,,)oo:f3

d(( g�_ lygf3) . (3 . 1 . 14) ) c1 ga l yg/3 ) 1 2.1' ga l yg ) The series on the right in (3. 1 . 14) converges absolutely in the norm of V for Re s > 1 . The notation ( raEra) �a/3 is defined in Theorem 3 . 1 .2. ( _I

X

exp

[27Ti(

0k/3 + j

/3

C

l

Our Theorems 3. 1 .2 and 3. 1 .3 completely describe the Fourier expansion of any Eisenstein series (3. 1 . 1). As an application we now obtain the following facts.

The following asymptotic formula holds for the parameter values 1 a, /3 n and 1 I k a: E( g/3 z ; s; a; et( a) ) = �af3 e, ( a ) y S + 0( 1 ) , where Z = x + iy E IT a and Re s > 1 (s is fixed). LEMMA

::;;;;;

::;;;;;

3. 1 . 1 .

::;;;;;

::;;;;;

y ---> oo

The lemma follows from Theorems 3 . 1 .2 and 3. 1 .3 and the asymptotic behavior of KsCy) as 00 (see §2. 1). The proof is complete. PROOF.

y 3 . 1 .4. The following equalities hold for a , I and /3 as in the hypothesis of ......

Lemma 3. 1 . 1 1) ( ek ( /3 ) THEOREM

:

,

r;; r (s - ! ) r ( S rl l1o(S ; a ; I ; f3 )

) = @5 at,/3k(S ) ,

I I I . F I R ST REFI NEMENT OF THE EXPA N S I O N THEOREM

48

2) £ ( z ; s; IX; et( IX» = I/;( z; s; IX; I), where 3. 1 .2, and the functions f3 al./'Jk( s) and 1/;( z; s;

11 0( S ; IX; I; f3 ) = l1o( S ) is from Theorem IX; l) are defined in §2.3.

It is not hard to see that 2) implies 1 ), since equality of the functions implies equality of the coefficients in their asymptotic expansions. We prove 2). We note that both of the functions E( z; s; IX; et ( IX» and 1/;( z ; s; IX; I) are defined in the strip 1 < Re s < 2. From Lemma 3. 1 . 1 and Theorems 3. 1 . 1 and 2.3.2 it follows that the difference £( z; s; IX; etC IX» - 1/;( z; s; IX; I) is bounded in z and is an eigenfunc­ tion for the operator m ( f ; X ) with eigenvalue A = s( 1 - s) for any s, 1 < Re s < 2. Since m (f; X) is a selfadjoint operator, this is only possible if this difference is identically zero. The proof is complete. PROOF.

3. 1 .5 . 1 )

The functions E( z ; s; IX ; e{(IX» , l1/ S ), �/ s ), I/;( z; s; IX; I) and ® a ' , f3k( s) can be extended '!leromorphically in s onto all of C , 1 � a � n, 1 � I � k 1 � f3 � n, 1 � k � kf3,} E Z, z E H. 2) The kernel of the resolvent 91 (s; f; X) and the kernel of the operator iB (s; f; X) (see Chapter 2) can be extended meromorphically in s onto all of C . PROOF. 1 ) By Theorem 3 . 1 .4, it suffices to prove the assertion for E( z ; s ; IX; e,( IX» , l1)( S ) and �/s). The last two functions appear in the Fourier coefficients of the Eisenstein series; hence, it is sufficient to prove meromorphicity of E( z ; s; IX; e,(IX» . THEOREM

a'

By Theorems 2.3. 1 , 3. 1 . 1 , and 3. 1 .4, this function is meromorphic in the half-plane Re s > O. Consequently, the scattering matrix f3(s) is meromorphic in the half-plane Re s > O. Then its functional equation (2.3.2 1 ) implies that 8(s) is meromorphic on C , and the functional equation for the functions I/;( z ; s; IX; I) in Theorem 2.3.6 implies the desired meromorphicity of E(z; s; a ; et C a» . 2) This is a consequence of part 1 ) applied to the functions 1/;( z ; s; IX; I ), the functional equation (2.3. 1 4), and the formula (2.2.5). The proof is complete. §3.2.

The Maass-Selberg relation

In the scalar spectral theory of 2f(f; X ), i.e., in the special case of the theory when the dimension of the space V on which the representation X acts is equal to one, the Maass-Selberg relation is what one calls the formula expressing the inner product in :Je( f; X) of two Eisenstein series (modified in a special way) in terms of the constant terms in their Fourier expansion relative to parabolic subgroups fa C f (see [50], and also [28], Chapter II). This relation plays a vital rolejn one of Selberg's methods for proving meromorphic continuation of Eisenstein series onto C (see [52]) and in the method for proving the theorem : on expansion in eigenfunctions of m: (f; X ) which does not use the integral equation for the kernel of the resolvent 91 (s; f; X ) (see [46]; for references to other important works, see §8 of [66]). I n addition, the Maass-Selberg relation occupies an important place in the derivation of the Selberg trace formula (see [50]). In the theory developed in the present monograph, the Maass-Selberg relation (the multidimensional version) is essentially necessary only for the derivation of the Selberg trace formula (see §4.3). In the generality we need (dim V � 1 , X nontrivial) the relation was derived by Roelcke in [46] under the hypothesis of meromorphic continuation of Eisenstein series. Since we have now proved this meromorphic

§3.2. THE MAASS-SELBERG RELATION

. 49

continuation in Chapter 2 and §3. 1 , in §3.2 we shall only give the statement of the fundamental theorem of §§9- 1 1 in [46]; we shall limit ourselves to the special case of the relation which will be useful to us in §4.3. For the convenience of the reader, we mention that the derivation of the Maass-Selberg relation is based on Green's classical integral formula for the operator L (see § 1 . 1), the automorphic property of the Eisenstein series, and part 3) of Theorem 3. 1 . 1 . We introduce some notation. Recall the partition of the fundamental domain F into the sets Fj, j = 0, 1 , (see Theorem 1 .2.4). This partition depended on the number We now emphasize this dependence by letting Fj( denote the elements of the partition. For each Eisenstein s�ties E ( z; a; l a we define the function E( z; a; el a) ; by the following formula for z F:

s;

a.

. . . ,n

a)

E{z; s; a; e,( a ) ; a ) =

zE

a)

s; e )) E E { z; s; a; e,( a )) , z E Fo(a) , E{z; s; a; e,( a ) ) - y ( gplz rc5ape, ( a )

- .r;; r{ s - !)r- I ( s ) r] o( s )y ( gplz r-s,

, n,

z,

( 3 .2 . 1 )

where Fp , f3 = 1 , 2, . . . y( z ) = 1m and we extend the function by the automorphic property involving X to all of H (as was done for a; I ) In (2.3. 10)). In (3.2. 1 ) we have used the notation in Theorem 3. 1 .2.

l/t( z ; s ;

Suppose that 1 a ";;;;; n, 1 ,,;;;;; I ,,;;;;; ka and s E C, Re s 1 /2, is not a singular point for the operator �(s) on the interval ( 1 /2, 1) (see Theorem 2.3. 1). Then the following formula holds ( the Maass-Selberg relation): THEOREM

,,;;;;;

3.2. 1 .

>

... .

( 3 .2.2 )

where ® (s) is the scattering matrix (see §2.3), and the bar denotes complex conjuga­ tion. We note that the relation (3.2.2) can be extended by continuity onto the line Re = 1/2 as well. The Maass-Selberg relation enables us to fill in a small gap in the proof of part 3) of Theorem 2.3. 1 .

s

The point s = 1/2 is a regular point for each of the functions E( z ; s; a ; et( a)), I ,,;;;;; a ,,;;;;; n, 1 ,,;;;;; I ,,;;;;; ka• THEOREM

3.2.2.

The proof of the analogous fact in the scalar theory is given in §4.3 of [28]. If one supposes that the point = 1/2 is singular, one obtains a contradiction with (3.2.2) by means of a simple but rather lengthy argument. The proof of Theorem 3.2.2 is similar. We shall not give it here, instead referring the reader to Kubota's book.

s

50

III. FIRST REFINEMENT OF THE EXPANSION THEOREM

§3.3.

Incomplete theta-series cusp-vector-functions. The operators K( and

f; X )

The first part of this section is devoted to a certain intrinsic characterization of the subspace of the continuous spectrum-and to a lesser extent the subspace of the discrete spectrum-of the operator W (f; X ). The theory developed here uses ele­ ments from the spectral theory of Selberg Roelcke Godement Langlands and Kubota in which the main emphasis is on the properties of Eisenstein series and the Maass-Selberg relations, and especially the theorem on expansion in eigenfunctions of W(f; X) which we proved in Chapter 2 using Faddeev's method from the scalar theory of automorphic functions. Part 2) of Theorem is a resolvant version of a well-known theorem of Gel'fand and Pjateckii-Sapiro (see Chapter I, or Chapter I, §2). In the second part of the section we introduce a fundamental class of integral operators in the spectral theory of automorphic functions, and prove some proper­ ties of these operators. These operators were first introduced by Selberg (see [S I D. The idea of studying them using the resolvent f; X ) in the scalar theory is due to L. D. Faddeev, V. L. Kalinin and the author (see [72]). For every function E COOO([ 0, 00) ; C) we define the O-series

[33]

3.3.2

[52],

[28],

[13],

[46],

[14],

§6, [16],

9t(s;

l/; � l/;(y ( g� lz ) ) X* ( y )v O(z; l/;; a ; v) = y Ef,,\f

(3.1.1),

( 3.3.1)

(the incomplete theta-series), in analogy with the Eisenstein series where E Va ' = 1m and � a � Thanks to the finiteness of l/; and the discrete­ ness of the group r, for any fixed E H the series has only finitely many nonzero terms. We enumerate some simple properties of l/;; a ; which follow directly from the definition.

v

y( z )

1

z

n. z

(3.3.1) O(z;

v)

3.3.1. Every O-series has the following properties: l) O( yz; v) X( y )O( z; v)for any y r andz O( z; v) in the variable z.

THEOREM E E H. l/;; a'; l/; ; a ; = 2) l/;; a ; E COO( F; V; X) n :Je(f; X) We define e( f ; X) c :Je ( r ; X ) to be the closed subspace spanned by all of the O-series with E Co([ 0, 00); C ), E Va' � a � The set of O-series obviously forms a dense linear subset in e(f; X). We now introduce the orthogonal comple­ ment of e(f; X) in :Je(f; X):

v

l/;

1

:Je ( f ; X ) = :Jeo ( r ; X )

n.

EEl

( 3.3.2)

8(f ; X). It is not hard to describe the elements of :Jeo( f; X ). Let E :Je(r; X) be a continuous vector-valued function which is bounded on F in the norm of V. We have

f

the following equalities:

°=

{z

(f, O) F.x fF(f(z), O(z; l/; ; a ; v ) dp, (z) =

z f,

iy


( 3.3.3)

where IIo = E H, = x + I ° � x � 1, 0 oo } . We have used the auto­ morphic property of the unitary property of X, and Suppose that the orthogonality condition is fulfilled for any O-series. This obviously imposes, if

(3.3.3)

(3.3.1).

,

§3 .3. INCOMPLETE THETA-SERIES AND CUSP-VECTOR-FUNCTION

51

we use (3.3.3), the following necessary and sufficient conditions for an element E to belong to the subspace

f �K (f; X)

Xo(f; X): 1 10 (f(ga z ) , v ) dx = ° ( 3 .3 .4) identically in E (0, (0) for any v E Va' a = 1 , . . . ,no We shall call the elements of :Ko(f; X) cusp-vector-functions (cusp-functions), or parabolic forms of weight zero. We make the following observation. While the subspace 0(f; X) is in a certain sense constructed explicitly, and one can guarantee its non triviality and finite dimensionality for the f and X in this -chapter for which k(f; X) ° (see § 1 .2), the subspace Xo(f; X), on the other hand, is more mysterious. As we shall see later, it is by no means always possible (i.e., for all of the f and X under consideration) to y

-=1=

guarantee its nontriviality. However, in many important cases it is known that this space is nontrivial and finite dimensional (see Chapter 6). We now return to the subspace The theory of Eisenstein series (or the functions l{;( z; a ; I )) enables one to pick out a finite-dimensional subspace which is important from the spectral point of view. To do this, we consider the set of singular points of in the interval 1/2 < � 1 at which the functions £(z; a ; e,( a )), 1 � a ';;; 1 � I ,;;; ka' have simple poles. Recall that these func­ tions can have at most simple poles at the singular points of Q3 in ( l /2, 1 ], by virtue of Theorems 3. 1 .4 and 2.3. 1 . At each such point we take the residue of £(z; a; , a ) . It is now not hard to prove (see Theorems 3. 1 .2, 3. 1 .3 and 3.2. 1) that this residue is (now considered as a function of z ) an eigenfunction of X) lying in The vector space over C spanned by the residues of all vector-val­ ued functions E( z; a; a)), 1 ,;;; a � 1 ,;;; I � a , at all singular points 1 /2 < � 1, will be denoted by Because there are only finitely many singular points of in the interval 1 /2 < � 1 , and because V is finite dimensional, the space is also finite dimensional. We have the obvious inclusion c On the other hand, the relation of the form (3.3.3) implies that is orthogonal to the subspace Hence, we have the inclusion c We thereby have the following decomposition into a direct sum of orthogonal subspaces:

0(f; X).

s;

Q3(s)

s;

, -

s

n,

s; e ( ) X(f; X). n, s; et( s 01(f; X). Q3(s) 81(f; X) 01(f; X) X(f; X). 01(f; X) 0 1 (f; X) 0(f; X)·

01(f; X)

s

(s)

k

W(f;

s,

Xo(f; X).

( 3 .3 .5 )

8if; X) 0(f; X) 8 1 (f; X)· W(f; X(f; X) �if; X); Xo(f; X), 01(f; X) 0:lf; X) 3.3.2. Let s be a regular point for the resolvent 91 (s; f; X) of the operator W(f; X) ( ;\. = s( l s); ;\. tt [ 0, (0)). 1) The following operator identities hold in X(f; X): 91(s; f; X)�i f; X) = �/ f; X)91(s; f; X ) , j = 0, 1 , 2. 2) 91 (s; f; X) � o(f; X) is a Hilbert-Schmidt operator. = e where The decomposition (3.3.5) is compatible with the theorem on expansion in eigenfunctions of X ) (see Theorem 2.3.7). We now show this. We denote by 0, 1 , 2, the orthogonal projection of onto the subspace j= or corresponding to the index. THEOREM

-

52

I I I . FI RST REFINEMENT OF THE EXPANSION THEOREM

3) The following equality hold�': 82 ( f ; X ) = ( �f - 1{5 ) :JC ( f ; X ) , where the projection 1.13 was defined in Theorem 2.3.7, and (f is the identity uperator in :JC ( f; X ) . PROOF. 1) The definition of the subspace 8 1 ( f; X ) and the equality �f = I,(� () + I.t� 1 + 1{5 2 imply that it suffices to prove the claim ( 3 .3 .6) ffi ( s ; f ; X ) l{5 o ( f ; X ) = 1{5 0( f ; X ) ffi ( s ; f ; X ) . Suppose that f E :JCo( f; X ), s E C and Re s > 2. Then

�R ( s ; f ; x )f( z ) = j r ( z , z ' ; s )f( z ') dJ.L ( z ') F

=

jIIk ( z , z ' ; s )f( z ') dJ.L ( z ') .

( 3 .3 .7 )

We have used Theorem 1 .4.5 and ( 1 .4. 1). Next, because of the invariance of the kernel k( z , z ' ; s) (i.e., k( gz, gz ' ; s) = k ( z, z ' ; s), g E G , z E H ), from (3.3.7) we obtain for any a = I , . . . , n and v E Va

1o 1 ( ffi ( s )f( ga z ) , v ) dx = jIIk ( iy , Z ' ; s ) 110 ( f( ga( z ' + x ») , v > dxdJ.L( z ') ,

(3 .3 .8)

z = X + iy, z ' = x ' + iy ' . The right side of (3.3.8) is identically zero in y, because f( z ) is a parabolic form of weight zero. This proves part I ) for Re s > 2. I t is obvious how to extend this result to any regular point s. 2) We first suppose that s E C and Re s > 2. Let f E :Je(f; X ). We consider the expression for ffi ( s ) in Theorem 2. 1 . 1 . From the definition of the operator � ( s ) we have the following relations for the a-component of the function �(s )f( z): ( :� ( s )f ) o( z ) = 0, (3 .3.9 ) where a = I , . . . , n , z = x + iy and z ' = x ' + iy ' . If f E Xo(f; X ), then the right side of (3.3.9) is identically zero, since the kernel t(y, y '; s ) does not depend on x or x ' on ITa' Consequently, by Theorem 2. 1 . 1 we obtain 1

Since � o is a bounded operator in X(f ; X ) and, by Theorem 2. 1 .2, �(s), �(s) and �(s) are Hilbert-Schmidt operators, it follows that 91 (s)�o is a Hilbert-Schmidt operator. To prove the theorem for an arbitrary regular point s ' it suffices to make use of the Hilbert identity

91 ( s ') - 91 ( s ) = ( s '( 1 - s ') - s ( 1 - s » 91 ( s ') 91 ( s ) ,

( 3 .3 . 10 )

taking into account that s is chosen with Re s > 2. Multiplying (3.3. 10) on the right by � 0 and using the fact that 91 (s) � 0 is a Hilbert-Schmidt operator, we obtain the assertion in the theorem.

"

§3 .3. INCOMPLETE THETA-SERIES AND CUSP-VECTOR-FUNCTION

53

:Jeo(f;

X) is 3) From parts 1) and 2) proved above it follows that the subspace spanned by the eigenfunctions of the discrete spectrum of Hence X) X), X) � (3 . 3 . 1 1 ) and we would like to prove equality: X ) EB ( 3 . 3 . 1 2) X) . X) = It is not hard to verify that the functions which can be represented in the form

1,l3 :Je(f;

:Jeo(f; EE)(3 1 (f;

:Jeo(f;

8 1 (f;

W.

1,l3 :Je(f;

1 s ( 3 .3 . 1 3 ) ( 8(z; 0/ ; a ; v) = 2 JRe .V. =SoLvl ) E ( z; s; a ; v) ds, are a dense set in the subspace 8(f; X ) = 81(f; X) EB 8 (f; X); here the integral is taken along the line Re s = so, so > 1 , and E( z; s; a ; 2v) is the Eisenstein series (3. 1 . 1 ). In (3.3. l 3) Lis) is the Mellin transform 1 ( s=soL.is )yS ds, Lvl s) = 1000o/(y )y-S dy� , o/(y) 2 JRe 0/ E C:([ 0, (0); C), and the function 8( z; 0/; a; v) is the incomplete theta-series (3.3. 1 ). Hence, the subspace 8(f; X) does not contain any eigenfunctions of the discrete spectrum of W except for the elements of 8 1 (f; X ), and this, along with (3.3. 1 1), gives us (3.3. 12). The proof is complete. 'TTl

.

=

.

"

WI

.

We now proceed to the second part of this section, which is devoted to certain integral operators in :Je(f ; X ) which are functions of We introduce some notation. Let be a real eigenbasis for X ) in the subspace of the be the corresponding set of discrete spectrum X) EB X), and let eigenvalues .

{w( z; A)}j :Jeo(f;

W(f; { AJ

8 1 (f;

W.

1) If the function h: [ 0, (0 ) C is measurable and bounded, then the operator h(W(f; X)): :Je (f; X) :Je(f; X) is defined, and its kernel h(z, z' ; W) as an integral operator has the spectral decomposition h(z, z'; W) = �h (Aj) w (z; Aj) ® w (z' ; AJ 1 n k a 00 ( 1 + ) ( 1 r 2 E z; + ir; a ; e,( a ) ) � � f h THEOREM

3.3.3.

�

�

j

+ 4w 0' = 1

/= 1

- 00

4

2

( 3.3 14) .

The series and integral in (3.3.14) converge in the metric of :Je (f; X)· 2) Let the function h: C C have the following properties: a) h(s) h(s(1 - s)) is analytic in the strip -E < Re s < 1 + Efor some E > 0; and b) it satisfies the estimate h(s) = o ( ( 1 m s 1 + 1 ) -2 - 8), -E < Re s < 1 + E, for some 0 > O. Then the kernel of the operator heW) K(f; X) is given by the series h(z, z ' ; 2I ) = k(z, z ' ; f; X) � X {y )k { u { z, yz ' )) , ( 3.3.15 ) =

�

1

=

=

y E T'

III . FIRST REFINEMENT OF THE EXPANSION THEOREM

54

where u(z, z') is the fundamental invariant of a pair of points, and the function k: [ 0, 00) � C is connected with h by the following transformation (the inversion formula of Selberg and Harish-Chandra): foo Vtk-{ t)w dt Q { w ), k { t) -! f oo dVWQ {-wt) , (3 .3 . 1 6) Q{ exp u + exp { -u) - 2) = g { u), g{ u) 21'TT i: (exp - iru)h ( ! + r 2 ) dr. Here the series (3.3. 1 5) converges absolutely in the norm of V, converges uniformly in any compact subregion of H X H, and gives a continuous map H X H � =

=

w

'TT

t

=

V.

Part 1) is a direct consequence of Theorems 2.3.7 and 3.1 .4. We prove part 2). Let be a fixed number, -f < < 1 + f, where f is the number in the conditions of the theorem. A simple but fairly lengthy calculation, which we shall not carry out here, shows that, if the function h( satisfies conditions 2 a) and 2 b), then the inversion formula (3.3. 16) is equivalent to

So

So

PROOF.

k ( t) = - 2 �i i

Re s = So

k(t; s) k(u(z, z'); s). s so. k(t)

s)

(1 - 2

s ) h { s ( 1 - S )) k ( t; s ) ds,

(3.3 . l7)

where was defined in Theorem 1 . 1 . 1 and is the basic ingredient in the Green's function The integral in (3.3. 17) is taken over the vertical line Re This integral clearly does not depend on the choice of within the indicated region. We suppose that lies in the region 1 < < I + €. We show that is a continuous function on the semiaxis (0, 00), has a finite limit as 0, and satisfies the estimate (3.3 . 1 8) From the definition of it follows (see Theorem 1 . 1 . 1) that the following estimate holds for in any interval [ f l ' 00): =

t

So

So

So

t E [O, oo ).

k(t; s)

I k { t; s) I « ( 1 + t rRe "I

s,

t

�

(3 .3 . 1 9)

f l > 0. Because of this and assumption 2 b) of the theorem, the integral (3.3. 17) converges absolutely and uniformly for [ f l ' 00 ) and satisfies (3.3. 1 8) for � f l .

tE

k(t) t O.

t

This implies that is continuous on (0, 00). It remains for us to prove that there is a finite limit as � This is most ' easily done using (3.3. 16). The assumptions 2 a) and 2 b) concerning h(s) imply that behaves like u l +c2 as � 0, where f '::> ° is 2 some fixed number. From this it follows that Q( w ) w l /2+E2i2, W � 0, and this, in turn, implies that has a finite limit as 0. We now substitute the fundamental invariant in place of on the left and right in (3.3.17) and average the resulting kernels (multiplied by the representation over the discrete group. Using ( 1 .4. 1), we obtain

g(u)

k( t)

X)

2:

yET

X {y )k { u {z , yz')) -�'TT l 1 =

t

�

�

Re s = ,\'o

u( z, z')

(1 -

u

t

2s)h { s {1 - s))r { z, z ' ; s) ds.

(3 .3 .20)

§3 .4.

THE SELBERG-NEUNHOF FER INTEGRAL EQUATION

55

It is permissible to interchange the summation and integration in the right side of (3.3.20), because of the properties we proved for the integral in (3.3. 1 7) and Theorem 1 .2.5. Making the change of variables A = s( 1 s ) in the integral in (3.3.20), we find that the right side of this equality is the kernel of the operator 1 (3 .3.2 1 ) - - h ( A ) 9't ( A ) dA , 2m Q where �ft ( A ) = m ( s ) and Q is a contour which encloses the spectrum of 2! in the positive direction. Hence, by a well-known theorem in functional analysis, the integral (3.3.2 1) is the operator h( 2! ). Because of the properties of k( t ) which we proved and Theorem 1 .2.5, the series on the left in (3.3.20) determines a continuous map H X H V. The proof is complete. The class of functions h(s( 1 s )) for which the corresponding kernel k( z , z ' ; f; X) is given by an absolutely convergent series over the discrete group is not exhausted by the set defined in condition 2) of Theorem 3.3.3. To illustrate this, we shall be satisfied here with the following special fact, which will turn out to be useful in Chapter 6. -

.

->

j

-

THEOREM 3.3.4. Let the function h: C C have the following properties: a) h( s ) = h (s( 1 s )) is analytic in the region bounded by the rays Q l : s = -e or - iYJr, Q 2 : s = 1 + e + or + iYJr for r > 0, Q3 : s = -e or + irYJ , Q4 : s = 1 + e + or - i YJ r for r < 0, -

->

-

-

where e , 0 and YJ are fixed positive numbers. b) In this region it satisfies the estimate , .

Then the kernel of h ( 2! ) is given by the series (3.3. 1 5), which is absolutely convergent and uniformly convergent in {z, z ' } E H X H outside of an arbitrarily small neighbor­ hood of the surfaces z = z ' (mod f).

The proof of the theorem is similar to the preceding one, with the integral (3.3. 1 7) replaced by 1 t � el > O . (3.3 .22) (1 h s( s )) k ( t ; s ) ds , k(t) = -2 7T l !J4 U !J , - 2 s ) ( I PROOF .

.

j

Here one must use the easily verified estimate I k ( t ; s ) 1 « ( I + t ro r

for s E Q4 U Q 2 and t � e l > O. The proof is complete. We note that, in general, a weaker restriction on the growth of the function h in Theorem 3.3.4 than that in condition 2) of Theorem 3.3.3 will lead to singularity of the kernel k(z, z ' ; f; X) on the diagonal z = z ' (mod f ). §3.4. The integral equation which we shall discuss here was first proposed by Selberg in order to prove meromorphic continuation of Eisenstein series in the scalar theory (dim V = 1 ) (see [50]). In essence, it is a Fredholm equation of the third kind, whose kernel of the integral operator is the harmonic Green's function of the automorphic

The Selberg-Neunhoffer integral equation

56

I I I . FIRST REFINEMENT O F THE EXPANSION THEOREM

Laplacian modified in a special way. To prove the existence of this kernel, Selberg used the Dirichlet principle. Twenty years later, Neunhoffer [39] proposed a some­ what more general integral equation than the Selberg equation for proving meromor­ phicity of Eisenstein series (also in the scalar case). As the integral operator in this equation he used a modified resolvent ffi(s; f ; X), whose existence can possibly be proved by a simpler theory than the Dirichlet principle on a noncompact Riemann surface with elliptic singular points. We shall call the latter equation the Selberg­ Neunhoffer equation. The purpose of this section is to derive an even more general version of the Selberg-Neunhoffer equation, which is suitable for studying the vector-valued Eisen­ stein series (dim V � 1) which depend upon a representation X E 9C sC f), X: V ..... V. We derive the equation here not only for the sake of completeness of the exposition (we have actually already proved meromorphicity of Eisenstein series in Chapter 2), but because from the Selberg-Neunhoffer equation it is relatively simple to find an upper estimate for the order of meromorphicity of the Eisenstein series and the scattering matrix. Unlike in [50] and [39], we shall derive the Selberg-Neunhoffer equation from Faddeev's equation, using the information we now have concerning the resolvent ffi (s) of the operator m . We introduce the following kernel, defining it component by component (see § 1 .2):

ro:oo(z , z '; K; a ) = ro:o(z , z'; K ) , ro:°,B ( z , z' ; K; a ) = ro:,B (z, z ' ; K )

(3 .4. 1 )

a = 0 , 1 , . . . , n , /3 = 1 , . . . , n , where z ' = x ' + iy ' E ITa' r( z , Z ' ; K ) is the kernel of the resolvent of m , K is a fixed regular point, K > 3, and E( z ; K; a; et ( a» is the Eisenstein series (3. 1 . 1). From

(2.3. 1 1) and Theorem 3. 1 .4 we have

E{z; s ; a ; et( a )) = [ s ( I - s ) - K(I - K )] !. [ r ( z, z ' ; K) - r O (z , z' ; K ; a )] F

X E{z'; s ; a ; et( a )) dp. (z') + [s( I

-

s ) - K(l - K )]

(3.4.2) which can be regarded as a definmg equation for E( z ; s; a; et ( a » , 1 ,.;;;; a ,.;;;; n , 1 ,.;;;; I ,.;;;; k o: . From the definition (3.4. 1) we see that the free term, i.e., the first term on the right in (3.4.2), is equal to k s ( I - s ) - K(I - K ) n p E{z; K; /3; e k(/3 )) � � 2K - I /3= 1 k= l

§3 .4. THE SELBERG-NEUNHOFFER INTEGRAL EQUATION

57

We now use Theorem 3 . 1 .2 to compute the integral in (3.4.3) in terms of the constant term in the Fourier expansion of the Eisenstein series:

( 3 .4.4) Substituting this value in (3.4.3), we arrive at the following expression for the free term in (3.4.2):

s ( I - s ) - ,, ( 1 2 ,, - 1

- Ie)

1 2" - 1

X E{ z ;

kp

n

[

as -

al -

K.-S K. � � 8ap8kl ,,' ._ S + ® al ,Pk( s ) ,, + s _ I

fJ = 1 k = 1

1

kp

� � [ 8ap 8kl ( " + s - 1 ) a S-K. + ( " - S ) ® al,Pk( s ) a l - K. - s] n

fJ= 1 k = 1

Ie ;

( 3 .4.5 )

/3 ; e k( /3 ) ) .

In deriving (3.4.5) we used Theorem 3 . 1 .4. We introduce the notation

,

'

Val,Pk( S ) = 8ap 8kl( " + s - 1 ) a S-K. + ( " - S ) @5 al , Pk( s ) a l - K.-S , ( 3 .4.6) taking a and " to be fixed, and for the other indices taking 1 � a � n , 1 � I � k a ' 1 � /3 � n and 1 � k � kp . Just as in the definition of the matrix ® ( s ), we introduce new indices d and b according to (2.3.20). In this notation Val, Pk ( S ) = Vd,b( S ), 1 � d � kef ; X ), 1 � b � kef ; X). In order now to derive the desired Selberg-Neunhoffer equation from (3.4.2), we need only invert the matrix {Vd,b}��r�x? By Theorem 3. 1 .5, its determinant is a meromorphic function of s. We show that it is not identically zero. In fact, by Theorem 3. 1 .2, each function l1o( s), I � a, /3 � n and 1 � I � k a , can be represented by the Dirichlet series (3. 1 .9) in the region Re s > 1 .

From this and from Theorem 3. 1 .4 it is easy to see that each of the functions a -s ® d,b( s ) vanishes in the limit as Re s � 00 (we have the right to take the number a in Theorem 1 .2.4 to be large; in this connection, see also Theorem 3.5.2). Hence, if s has a sufficiently large real part, then the matrix { Vd,b( S )} is close to a multiple identity matrix, and so its determinant is nonzero for such s. We let Vi,b( S ) = Vai, Pk ( S ) denote the entries in the inverse matrix of {Vd,b(S)}. W e introduce the further notation:

U( z ; s ; /3 ; k ) =

n

k"

� � V;I,Pk ( s )E{ z ; s ; a ; e, ( a )),

a = 1 /= 1

( 3 .4.7 )

where /3 = 1 , . . . , n and k = 1 , . . , kp . From (3.4.2), (3.4.4) and (3.4.5) we obtain the desired generalization of the Selberg-Neunhoffer equation for the function .

U( z ; s ; /3 ; k ) =

2"

1

_ 1 E ( z ; ,, ; /3 ; e k ( /3 )) + [ s ( 1 - s )

f

-

X r O ( z ; z ' ; Ie ; a ) U( z' ; s; /3 ; k ) dp,( z' ) , F

where 1 � /3 � n and 1 � k � kp .

,,

( 1 - ,, ) ] ( 3 .4.8 )

III . FIRST REFINEMENT O F THE EXPANSION THEOREM

58

We proceed to study (3.4.8). An important difference between this equation and Faddeev's equation (2.2.6) is that the kernel of the integral operator in the former case depends more explicitly upon s. If the free term in (3.4.8) were an element of X(f; X) and the integral operator in the equation were a Hilbert-Schmidt operator, then this equation would be a Fredholm equation of the third kind, i.e., an equation of the sort well understood by classical mathematicians. However, it is not hard to see from Theorems 2. 1 . 1 , 2. 1 .2, 2.3.2, and 3 . 1 .4 that neither condition is fulfilled. Nevertheless, the question of solvability of (3.4.8) reduces to an investigation of a Fredholm equation of the third kind by means of a device similar to Selberg's " trick" in the scalar theory (see [50D. We make the following identity transforma­ tions in (3.4.8) :

[

at y( g�'z ) ] U( z ; k ) = � [ at y( g�'z ) ] E(z; K; 1,,[ t ( y( g� 'z ) - y( g�'z')) ] r O (z, z ' ; K; a ) X[ a� /( g;'Z')] U( z'; k ) z'),

exp -

2.

X

•

s; P;

\

exp - .

exp -

exIi -

•

P ; e k( p ) ) + [ s ( I - s ) - .(\ -

.)]

•

s ; P;

dl' (

( 3 .4.9 )

where y(z) = 1m z, z = x + iy , z' = x ' + iy', e is a parameter, e > 0, and z E F. We show that for fixed e (3.4.9) is a Fredholm equation for the vector-valued function

z E F, 1

� /3 � n , 1 � k �

kf3 .

In fact, from Theorems 2.3.2 and 3 . 1 .4 we have the inclusion

Then Theorems 2. 1 . 1 and 2. 1 .2 and the definition of :t( K ) in (2. 1 .27) imply that

fA

exp - 2 .

at ( y( g;'z ) - y( g';'z')) ] 1 r O t z, z' ;

. ; a ) I i- dl' ( z ) dl' ( z ' ) < 00 .

Thus, we can use the results of rredholm theory for (3.4.9) after passing to the limit as e -7 0, and also for (3.4.8). Here we shall limit ourselves to an outline of the steps. Equation (3.4.8) is uniquely solvable for all values of 'A = s ( l - s ) with the exception of a discrete set of singular points. In the resulting solution to (3.4.8) one can pass to the limit as e -7 ° for nonsingular A , and thereby obtain the desired solution U( z; s ; /3 ; k ) of (3.4.8). This solution can be represented in the form

U( z ; s ; /3 ; k) = W\(z ; 'A ; /3 ; k)/Wz( 'A ; /3 ; k ) , 'A = s ( 1 - s ) , ( 3 .4. 10) where W\(z; 'A; /3; k ) is a map F -7 V for fixed /3, k and 'A, and Wz('A; /3; k) is a complex-valued function for fixed /3 and k. As functions of 'A, W\(z; 'A; /3; k) and Wz('A; /3; k) are entire functions of order no greater than two. Now in order to prove

the exist�nce of a formula analogous to (3.4. 1 0) for Eisenstein series, one must

§3 .5 . THE DETERMINANT OF THE SCATTERING MATRIX

59

substitute on the right in (3.4.7) the asymptotic expansion as y -> 00 for the Eisenstein series E( g/3 z ; s ; f3; e r( f3 )), f3 = 1 , . . . , n , Z = x + iy -> E ITa (see Theo­ rems 2.3.2 and 3. 1 .4). Comparing the coefficients in the asymptotic formulas in the left and right in (3.4.7) leads to a proof of meromorphicity of the functions 'I',B-k ,ar ( s ), and hence of the functions 'I'/3 k , ar( S ) with order of meromorphicity in s no greater than four. This, in turn, implies meromorphicity of the Eisenstein series and the scattering matrix with order no greater than four, as well as formulas for them analogous to (3.4. 10). We have thereby proved a theorem which gives a refinement of Theorem 3 . 1 .5. 3.4. 1 . Each vJ the functions i,r- condition 1 ) of Theorem 3. 1 .5 has a representation as a ratio (3.4. 10); more precisely , THEOREM

E { z ; s ; a ; e r ( a ) ) = Wi z ; s ; a ; I )/ W4( s ; a ; I ) , �/ s ) = W7 ( s ; j ) / W8( s ; j ) ,

17/ S ) = Ws ( s ; j )/ ff6 ( s ; j ) ,

® ar , /3 k ( S ) = W9( s ; a ; I ; f3 ; k ) / WI O( s ; a ; I; f3 ; k ) ,

where 1 � a � n , 1 � I � k a , j E lL , 1 � f3 � n , 1 � k � k/3 and z for the indicated values of the parameters

E F. Furthermore,

W3( z ; s ; a ; I ) E V, W4( s ; a ; I ) E C , US ( s ; j ) E V, ff6 ( s ; j ) W7 ( s ; j ) E V, W8( s ; j ) E C , W9( s ; a ; I; f3 ; k ) E C ,

E C,

WI O ( s ; a ; I; f3 ; k ) E C . Each function ( or map ) Wm , m = 3, . . . , 1 0, is an entire function ( map ) of s of order no greater than four.

§3.5. In this section we explain several properties of the determinant of the scattering matrix { ® ar , /3k(S ) } (see (2.3. 1 7)); besides their independent interest, these properties will be useful in the derivation of the Selberg trace formula and in the study of the asymptotic behavior of a spectral function of the operator 21:(f; X) (see Chapters 4 and 5). Here we are generalizing results of the scalar theory which are due to Selberg (see [50], and also [68]). We recall that, with the change of indices (2.3.20), the scattering matrix is a square matrix of order k ef; X) = "1, � k a ' equal to the total degree of singularity of the representation X relative to the group f. We let �( s ) = �(s; f; X ) denote the determinant of the matrix ® ( s ) (see §2.3).

The determinant of the scattering matrix

3.5 . 1 . Let f and X be fixed. The function �(s ) has the following properties: 1) In the half-plane Re s > 1 it can be written in the form

THEOREM

k = k( f ; X ) , where I( s ) is an absolutely convergent Dirichlet series (Re s I( s ) =

00

� P; '

s

>

1 ),

Pm E C ' P ) =I= O ' 0 < Ql < Q2 < . .

·

·

m = 1 Qm 2) It is meromorphic on C with order of meromorphicity no greater than four. 3) It is regular in the half-plane Re s > 1 /2 except for a finite number of poles on the interval of the real axis s E 0 /2 , 1 ] ; each pole has multiplicity no greater than k e f; X ).

uv

I l l . t' l K :":>

1 K t r I N tMJ:: N T O r

THE EXPANSION THEOREM

4) For any r E IR

d ( i + ir ) 7"= O . 5) It satisfies the functional equations d ( s ) d ( 1 - s ) = 1 , des) = d ( s ) , where the bar denotes complex conjugation; in particular, 1 dU + ir ) 1 = 1 , r E IR . PROOF. All the assertions are direct consequences of theorems we have proved. In fact, 1) follows from Theorems 3. 1.2 and 3. 1.4; 2) is a consequence of Theorems 3. 1.5 and 3.4. 1 ; and 3) - 5) follow from Theorems 2.3.4 and 2.3.5. The proof is complete. The following theorems describe the subtler properties of the function d( s). THEOREM 3.5.2.

1) The function des) is bounded in the region l2 � Re s � l2 , 1 1m s I > 1 . 2) In the half-plane Re s � 3/2 the function q�Sd(s) is bounded, and, moreover, q�Sd ( s ) � O. Isl � oo

PROOF. In the Maass-Selberg relation (3.2.2) we set s = a + ir and sum the right and left sides .as I goes from 1 to k a and as a goes from 1 to n. The formula obtained in this way then gives us the inequality

[

1 20- 1 a k 2a - 1 +

/I

_

al - 20

±

�

±

a = 1 /= 1 {3 = 1

ka 1

� � 2ir [ ® al , al ( a + ir ) a2 i r - ® al , al ( a + ir ) a -2 ir] > O . a= 1 /= 1

From this we obtain

1 2 ka 4 .- 2 + 01 r a' ·- l ± a= 1

1

/I

� I ® al,.l o + ir ) I

/= 1

ka

/I

kp

> � � � � 1 ® al ,Pk ( a + ir ) 1 2 . a = 1 / = 1 {3 = 1 k = 1

( 3 .5 .1 )

From (3.5.1) it follows that each function ® al, Pk( a + ir) is bounded in the region 1 /2 � a � 3/2, 1 r I > 1 . Consequently, the determinant d( s) is bounded, and part 1) i s proved. Part 2) is a simple conseqllence of part 1) of Theorem 3.5.1. The proof is complete. Before stating the next theorem, we introduce some new notation. We let a1 � O � � o� denote all of the poles of Ll( s) in the interval s E (1/2; I ], 2 counting multiplicity (see part 3) of Theorem 3.5.1). By analogy with the scalar theory, we introduce the function •

•

•

'!)R,

d n;/ s) = q�S - I II

j= 1

s - a. � Ll(s ), · S - 1 aj

where the coefficient q l was defined in the conditions of Theorem

( 3 .5 .2 ) 3.5.1.

,

§3 .5 . THE DETERMINANT OF THE SCATTERING MATRIX

61

The function A reg( s) has the following properties: 1 ) It is regular in the half-plane Re s > 1/2 and continuous in Re s � 1/2. 2) A reg(s ) 0 as Re s 00 . + 3) It does not vanish for s = 1/2 ir, r E IR . 4) It satisfies the functional equations A reg( s ) A reg( I - s) = 1 , in particular, 1 A reg(l /2 + ir) 1 = 1 , r E IR . 5) It satisfies the boundedness condition I A reg ( S ) I .;;;;; 1' , . Re s � t . THEOREM

3.5.3. -i>

-i>

Parts 1 )-4) follow from the definition of A reg and Theorems 3.5.1 and 3.5.2. Part 5) is a consequence of parts 1 ) and 4) and the maximum modulus principle for analytic functions. The proof is complete. We now study the behavior of the function A(s ) in the half-plane Re s < 1 /2. Let p = f3 + iy be an arbitrary pole of A(s ) with f3 < 1 /2. Part 5) of Theorem 3.5 . 1 implies that p = f3 - iy is also a pole of A(s), and the points 1 - p and I - p are zeros. Obviously, p and p are poles of Areg(s), and I - p and 1 - P are zeros of this function. We consider the series PROOF.

_

�

.r:

f3 - 1/2 ' I p 12

(3 .5 .3)

where the summation is over all poles p of A(s) for which f3 = Re p < 1/2. THEOREM

3.5.4.

The series (3.5.3) converges.

PROOF. Since A reg(s) is analytic in the half-plane Re s > 1/2, and is continuous and bounded in the half-plane Re s � 1/2 (see Theorem 3.5.3), it follows by Carleman's theorem (see [56], §3.7 1) that �p f3/1 p 1 2 converges, and this implies the theorem. The proof is complete. From Theorems 3.5. 1 and 3.5.3 we already know that the functions A(s ) and A reg(s) are meromorphic with order no greater than four. We now construct special canonical products for these functions which are different from the usual Weierstrass product from the theory of entire and meromorphic functions. Using these products, we shall later refine our estimate for the order of meromorphicity of A(s ) and Areg(s) . THEOREM

3.5.5.

The logarithmic derivative A�eg I + - -lr 2 A reg -

(

.)

is nonnegative if r E IR . In addition, it can be represented by the series

where the summation is over all poles p = f3 + i Y of the function A( s) in the half-plane Re s < 1/2.

III . FIRST REt'INEMl::'N l " Ul' THE EXPAN SION THEOREM

£l reg( s), 3.5.4,

starting from the properties PROOF. We construct a canonical product for of this function described above. According to Theorem the following product is absolutely convergent:

II ( 1 (2s - 1) (s _2:)(s � p) ) +

(3.5 .4)

,

p

where the product is only over half of the poles p of in the half-plane Re s < (one pole is taken from the pair p, p ). The expression can also be written in the form

£l(s) (3.5 .4)

1/2

(3.5 .5) where the product is now over all p, but here we agree to take the factors for p and for together with one another. Next, we look for the function in the form

p

(s) £l r e g (3.5 .6) £l reg( s) = £l reg ( I ) II S -S _1 + p g ( - 21 ) ' ( s) £l 3.5 .3, g( s - �) = - !) - !f, 3.5.3, C2 £l reg(s) ± II s -s - p , £l(s) = ±ql- 2S II'JlL s -S -l oj II S -S -1 + - (3.5 .7) 1-2 _ £l'reg (l. ) � £l reg 2 - 1/2)2 (l/2) 4) E ± £l r e g 3. 5 . 3 ), q\, (3.5.2). ( s) £l r e g 3.5.5. 3.5 .6. , £l' ( 1 . ) = - £l£l'rreegg ( 1 . ) ( r12 ) , 2

P

p

exp

s

where, because of the estimate for the order of meromorphicity of result in Theorem we have

and the

c\ ( s

+ c2 ( s the constants c \ and C 2 must with c \ and constants. But, again by Theorem be zero. From this we obtain l + p

=

p

j= \

+ ir =

p

( f3

+

OJ

f3

+ (r

P

P

_

y)

2 ;;. 0

P

,

'

(see part where r �, the signs correspond to the possible values of of Theorem and the numbers OJ and M are defined in the context of · The proof is complete. The following fact is an immediate consequence of the definition of and Theorem THEOREM

+ r -T 2 z

where c3 is a constant.

2 + zr + c3 + 0

r � oo , r E IR

,

CHAPTER

4 THE SELBERG TRACE FORMULA :'.

In this chapter we shall suppose that f E 9J( and X E 9C( f ) (see and shall focus our attention primarily on the more difficult situation f E 9J( 2' X E 9C ( f). The modifications needed in the formulas in the alternative cases f E 9J( I ' X E 9C( f ) or f E 9J( 2 ' X E 9C rCf) are obvious, and involve a natural trivialization. As before, we shall suppose the group and the representation to be fixed, and shall not always indicate the dependence upon f and X in the notation.

§1.2),

s

§4.1. Nuclearity of the operator §4.1

x )Il3

0

(f; X)

In we shall prove a theorem which is in the spirit of the well-known theorem of Gel'fand and PjateckiI- Sapiro in the scalar theory of automorphic functions (see Chapter at the same time bringing in ideas from the theory of perturbations of continuous spectrum (see The proof will be based on our results from Chapter and Theorem We shall keep all of the notation used here. The proof of the theorem requires three lemmas. The first lemma was proved in Theorem For convenience, we shall state it here as a separate proposition.

[13],

1, §6),

) . [ 7 2] 3.3.2. 2.3.3.

2

, .

K( f ;

The operator O( s) satisfies the Hilbert identity; more precisely, the 4.1.1. following operator identity holds in :Je(f; X ) for any points s, s ' E C with s, LEMMA

Re s'

Re

> 1- :

O ( s ) - O ( s ,) = [ s { l - s ) - s'(l - s ')] O (s ) O(s ') . LEMMA The difference 9l (s) - O(s) is a Hilbert-Schmidt operator for any s E C satisfying the conditions Re s > 1- and s rt (1-,

4.1.2.

1].

PROOF. First suppose that Re s equality

>

2.

From Theorems

2.1.1 2.1.2 and

we have the

9l (s) - 0 (s) = � (s ) + :n (s) + (if (s ) + � ( s ) - 0 ( s ) ,

where �(s), :n(x) and

(if(s)

are Hilbert-Schmidt operators. From the definitions and it follows that the difference �(s) - O(s) is also a Hilbert-Schmidt operator, and this proves the lemma for s with Re s > Now let s be as in the lemma. We choose K E IR, K > and fix it. As in Chapter we let w(s) denote the function w(s) = - s) - K(1 - K), and let g denote the identity op�rator in :Je(f; X). We have

(2.1.27), (2.1.14), (2.2.13) (2.2.10) 2, s(1

2.

3,

( 63

4.1.1

)

1 v . 1 .t1� ��Ltl�KU I KAL� r U K M U LA

o ':t

In deriving (4.1.1) we have used the Hilbert identity for the resolvent 9T(s) and the operator D(s) (see Lemma 4. 1.1). From the Hilbert identity it also follows that

( 1 - W(S ) 9T ( K ))( 1 + w( s ) 9T (s )) = 1. (4. 1 .2 ) If we invert the operator ( 1 - W ( S )9T( K » on the left side of (4. 1 . 1 ) by means of (4. 1.2), we obtain an expression for 9T(s) - D(s) as a product of a bounded operator and the Hilbert-Schmidt operator 9T( K ) - D ( K ) (we have already proved the lemma for s = K ) ; this proves the lemma. Before stating the next lemma, we introduce some notation. We let II K II 62 denote the von Neumann-Schatten norm of an arbitrary Hilbert-Schmidt operator K (see [ 19]).

LEMMA 4. 1 .3. The following estimate holds in the region {s E e l Re s > 1 /2, s

(1/2, I ] ) :

tl

where n is the number of cusps in F, K is from Lemma 4. 1 .2, and w (s) = s( l - s ) K ( I - K ). PROOF. The formulas (4. 1 . 1) and (4.1 .2) and Lemma 4. 1.2 give us the estimate 11 9T ( s ) - D(s ) 11 62 � ( I + I w ( s ) 1 11 9T (s) 1I F.J ( 4. 1 .3 ) x ( 1 + I w( s ) I II D(s )II F,x ) II 9T ( K ) - D (K ) 1 1 6 2. Furthermore, by a general theorem in function analysis, the resolvent 9T(s) has norm bounded by

(4. 1 .4 ) Recall that the operator D(s) is associated to the kernel q(y, y ' ; s ) of the resolvent of a one-dimensional selfadjoint problem (see the situation in (2.2. 14» . Hence, its norm in :Je(f; X) satisfies an estimate similar to (4. 1.4):

II O ( s ) II F, x � n/I Im s ( l - s ) l . (4. 1 .5 ) If we substitute (4.1 .4) and (4. 1.5) in the right side of (4. 1.3), we obtain the lemma. The proof is complete. To formulate the basic theorem of this section we return to Theorem the definition of D(s) it is not hard to see that the operator

3.3.3. From

I

� {e s = so( I - 2s ) h ( s ( I - s )) O ( s ) ds

h( � ) = - 2 i

(4.-1 .6)

is correctly defined as a bounded operator in :Je(f; X ) if h(s(l - s» satisfies conditions 2a) and 2b) of Theorem 3.3.3 ; So was determined in the proof of that theorem. THEOREM

4. 1.1. I ) Suppose that the function Ii(s ) = h(s(1 - s» has the following

properties: a) it is analytic in the strip - e < Re s < 1 + e for some e > 0; and

, 65

§4.2. JUSTIFICATION OF THE SPECTRAL TRACE FORMULA

b) in this strip it satisfies the estimate

Ii(s ) =

o( ( 1 + I 1m s I) -4-8 )

for some 8 > O. Then h ( � ) - h( � ) and h ( � )�o(f; X) are Hilbert-Schmidt operators ( � o is the projection of :JC( f; X) onto :JCo(f; X » . 2) Suppose that the functions h(s), h1 (s) and 1i2(s ) satisfy conditions l a) and Ib) above, and also the equality Ii(s ) = h1 ( s )his ) ; then h( � ) - h(i) and h( � )�o( f; X) afe nuclear operators in :JC(f; X ). PROOF. Obviously, 2) is a consequence of I ), since a product of Hilbert-Schmidt operators is a nuclear operator. In addition, the definition of O(s) and the proof of part 2) of Theorem 3.3.2 imply the equality

h ( i) ) �o( f ; X ) = O. Thus, all of the assertions in the theorem reduce to the claim that the von Neumann-Schatten norm of the difference h( � ) - h(i) is finite if the function h(s(1 - s» is as in part I ) of the theorem. We prove this. We have

h( � ) - h(�)

=1

Re s = + e/2

1

( 1 - 2s )h(s ( 1 - s » [ ffi ( s ) - O (s )] ds .

( 4. 1 .7)

Lemmas 4. 1 .3 and 4. 1 .2 are now easily seen to imply the estimates

Il h ( � ) - h ( � ) 11 @52 « II ffi ( K ) - O ( K ) II @52 < ... . '

00 ,

where K is from Lemma 4. 1 .2. The proof is complete. THEOREM 4. 1 .2. Let the function Ii(s)

= h(s(1 - s» be as {n part 2) of Theorem

4. 1 . 1 . In the notation of Theorem 3.3.3, the integral operator with kernel p(z, z'; f; X)

= k(z, z'; f; X)

( � + ir; a ; e ( a ) ) dr , I

® E z' ;

(4 . 1 .8)

defined on F X F, is a nuclear operator in :JC(f; X ) , where k ( z , z ' ; f ; X ) = � X ( y )k ( u ( z , yz'» , yEr

and the functions k (t) and h(lj4 + r2) are related by the transformation (3.3. 1 6). PROOF. This theorem is a direct consequence of Theorems 3.3.2, 3.3.3, and 4. 1 . 1 . The proof is complete.

§4.2. Justification of the spectral trace formula In this section we prove that the trace of the integral operator with kernel (4. 1 .8) in :JC(f; X) can be computed as the integral of the value of the kernel on the diagonal. (According to Theorem 4. 1 .2, to do this it suffices to show that the kernel

... ..

.

... .L.L

...... ...... u

��.L...o.L, __

...

.L'-L 1.'-'

......

.I. '-'.I.'-.l.VLV L..tL"l..

x ) is defined and continuous on the diagonal.) Our method generalizes the method of [72] and the Selberg-Arthur method for justifying the trace formula for arithmetic groups on the rank one case (see [ID.

p(Z, Z'; r;

The next theorem will be stated and proved only in enough generality for our purposes in the proof of the Selberg trace formula. We let tr v denote the trace of an operator V � V. THEOREM 4.2.1 . Let the function h(s(l - s» be as in part 2) of Theorem 4.1 . 1 . In

addition, suppose that h l (s( I - s» = his(l - s» and that the function h I C A ) is real for values of the argument 0 � A � 00 . Then the following assertions are true: 1) The scalar kernel tr v p(z, z'; r ; X) (see (4.1 .8» is continuous in z for every z' E F and is continuous in z' for every z E F. 2) The function tr v p(z, z; r; X) is correctly defined, and is continuous on F. PROOF. 1) By Theorem 3.3 .3, it suffices to prove part 1) for the kernel

� t h ( ! + r' l E ( z ; ; + ir ; a ; e; { a ) l

T{z, z'; r ; X ) = L ±

a = i l= i

®

00

E (z' ; ! + ir ; a ; e, ( a )) dr

( 4.2 . 1 )

or for each term in (4.2. 1): T( z, z'; a ; I; r; X ) = } f_: h ( � + r 2 ) E ( z ; ± + ir ; a ; e, ( a) ) w ® E ( z'; � + ir ; a ; e t ( a) ) dr. ( 4.2.2 ) From part 1) of Theorem 3.3.3 we know that the kernel T(z, z'; a, I ) is defined for almost all z, z' E F. The scalar kernel tr vT(z, z'; a ; I ) is also defined almost everywhere. On the other hand, it is not hard to verify that the operator he m ) for h(s(l - s» as in the theorem is selfadjoint and nonnegative definite; and, by part 2) of Theorem 3.3.3, its kernel k( z, z ' ; r; X) is defined and continuous everywhere on F X F. Hence, tr v T( z, z; a ; I; r ; X) :;;;;; tr k ( z, z; r ; X ) , (4 .2.3 ) tr v T( z, z; r ; X ) :;;;;; tr k ( z, z ; r ; X ) for every z E F. By Holder's integral inequality we have I tr v T( z, z'; a ; l ) I 1 = 4 w I f_oo h ( ! + r 2 ) < E ( a ; � + ir ; a ; e, ( a ) ) , E ( z'; � + ir; a ; et ( a ) ) > drl v

v

oo / " 4� L C h ( ! + l lE ( z; ± + ir ; a ; e (a ) l [ dr ) 1 ' ( tooh ( i + r 2 l l E ( z'; ; + ir ; a ; e, { a) l l: dr ) - ' }

r'

,

I j'

X

trl/v 2 k ( z z · r ,· x ) tr l / 2 k ( z ' z,· r· X) (4.2.4) In deriving (4.2.4) we used the equality h(i + r 2 ) hrd + r 2 ) and the estimate (4.2.3). But we verified in the proof of part 2) of Theorem 3.3.3 that the integral �

"

v

·

'

"

.

=

§4. 3 . DERrvAnON OF THE FORMULA

67

0.3.20) for k(z, z ' ; f; X), regularized by means of (3.3.16), converges uniformly for z in a compact subregion of F. This implies uniform convergence of the integral on the left in (4.2.4) with respect to z in a compact region, for every fixed z' E F, and, in exactly the same way, its uniform convergence with respect to z' in a compact subregion of F, for any fixed z E F; this proves part 1). The proof of 2) is analogous to that of 1); it follows from (4.2.4) and part 2) of Theorem 3.3.3. The proof is complete. For convenience, we now combine the results in Theorems 4.1.1, 4.1.2, and 4.2. 1 into a single theorem. THEOREM 4.2.2. Suppose that the function h\(s) = h \(s(1 - s)) has the following properties: a) It is analytic in the strip - < Re s < 1 + for some O. b) In that strip it satisfies the estimate f

f

f >

for some 0 > O. c) h l "A) is real for 0 � "A < 00 . Then the function h (s( 1 - s)) = h �(s( I - s)) satisfies the trace formula � h ( AJ = j

,.

f tr p (z , z' ; f; x) d ( z ) , F

JL

v

(4.2.5)

where "Aj runs through the discrete spectrum of2:((f; X), and the kernel p( z, z'; f; X) is defined by (4.1.8).

§4.3 . Derivation of the Selberg trace formula

In this section we discuss transforming the spectral trace formula (4.2.5) to the Selberg trace formula. The following special cases of the Selberg trace formula for f E W1 and X E 9C(f) are well known and have been discussed several times in the literature: 1) f E W1 \ and X E 9C(f) (see [51], [13], [ 19] and others); 2) f E W1 2 and X E 9C /f), dim V = 1 (see [50], [28] and [72]); 3) f E W1 2 and X E 9C /f) (see [51)); and 4) f an arithmetic group in W1 2 and X E 9C(f) (see [23], [7], [1] and [18]) (for the notation, see 1.2). However, the general case of the Selberg trace formula for f E W1 and X E 9C /f) has not been considered before, as far as we know, either in the published literature or in Selberg' s lectures in Princeton (1952) and Gottingen (1954) (see [50)); hence, we shall focus our attention primarily on this case. As will be clear later, the Selberg trace formula we obtain for a general grollp f E W1 2 and a general representation X E 9C(f) in a certain sense includes all of the trace formulas I )-4) considered before, and its proof, modulo the results of Chapters 2 and 3, does not differ in any essential way from the proof of those formulas. We begin the transformation of the right side of (4.2.5) by observing that

§

f tr p ( z, z ; f ; X ) d ( z ) = lim f tr ( z ; f ; X ) d ( z ) lim f (tr k ( f; X ) - tr T( z, z ; f; X )) d ( z ) , F

JL

v

a -> 00

Fo( a )

a -> 00

v

z, z;

Fo( a )

vP

z,

v

JL

JL

(4.3.1)

68

IV. THE SELBERG TRACE FORMULA

where Fo(a) is a component in the partition of F (see §1.2 and (3.2.1» . We have to introduce an artificial passage to the limit in (4.3.1) because, in general, neither of the functions trv k(z, z; f; X ) or trv T(z, z; f; X) is integrable on F. The particular choice of passage to the limit is suggested by the Maass-Selberg relation (3.2.2). We now find asymptotic expansions for the integrals (4.3.2) fFo( a )trv k(z, z; f ; X) dJ.l. (z), (4.3.3)

as a � 00 . The divergent principal terms in these expansions will be the same, by Theorem 4.2.2, and so they will cancel in (4.3.l). After taking the limit as a � 00 , we will find that the other terms give the desired value for the matrix trace in (4.2.5). We now carry out this program. We first consider the integral (4.3.3). To construct its asymptotic expansion we shall need a certain consequence of the Maas-Selberg relation (3.2.2). THEOREM 4.3.1. Let r E IR . Then k f) E ( z ; s ; a ; e, ( a ) ; a ) l 2v d (z) = 2k ( f ; X)ln - 611' ( "21 + ir ) 11

�

a

a

1

_

I �I

a

J.l.

+

) ( + ir a 2 ir ( tr ® � 2�r

-

tr ® ( � + ir ) a- 2 ir )

( 4.3.4)

,

where 11'/11( s) is the logarithmic derivative of the determinant of the scattering matrix ® (s), tr ®(s) is the trace of the matrix ® (s), and k(f; X) is the total degree of singularity of the representation X relative to the group f.

The equality (4.3.4) is obtained from the Maass-Selberg relation (3.2.2) if we sum both sides over I and a, and then in the resulting relation pass to the limit as s � 1/2 + ir, r E IR. Here one must use Theorem 2.3.5 and (2.3.21). In fact, we consider the limit PROOF.

(

1

n

n

ka

a 2 a- 1 k ( f ; X ) - a l - 2 a � � � lim 17 --> 1 /2 211 - 1 a = I /= 1 {3 = 1

1 . hm (7 --> 1 /2 211 - 1

(

kp

� I ® a" ,8k ( J + ir)/

k= I

k ( r ; x ) k ( r; x )

k e f ; x )a 2 a- 1 . '--- a l - 2 a

�

d= 1

-

d 1

1 k ( r ; x) k ( r ; x)

2

d= I

b= 1

d= 1

b= 1

1 k ( r ; x ) k ( r ; x)

1

d ,b

1

b 1

® d ,b

® d ,b ,b

1

)

2 � I ® d,b ( + ir )1

2 / ) / = kef; x )ln a + (In a ) � � ( "2 + ir ( + ir ) ( + ir ) � � ® � ,b "2 "2 ) ) � ® ( 2 + ir ® � ( 2 + ir . - "2 � k ( r ; x) k(r ; x)

2

b= 1

11

)

1

1

(4.3.5)

§4.3. DERIVATION OF THE FORMULA

69

Next, we have

1 ® d,b ( 21 + ir 1 2 = tr [ ® ( 21 + ir ® ( 21 - ir = k ( r ; x ) . ) ) )] d�l b�l

k ( r ; x ) k ( r ; X)

(4 . 3 .6)

+ ir,

Differentiating the second equality in (2.3.21 ) with respect to we obtain

®'( 1 + ir ) ® (1 - ir )

,

. t5

s and setting s = 1/2

(1 + ir ) ®' (1 - ir ) .

(4 .3 .7 )

Finally, it is easy to verify the equality

tr[®'(1 + ir )®*(1 + ir )] = tr [ ®'(1 + ir )®(1 - ir )] = tr [ ®'(1 + ir )® - 1(1 + ir )] = det'( ®(1 + ir )) dec l ( ®(1 + ir )) = �' (1 + ir ) � (1 + ir ) . (4 . 3 . 8 ) -I

Substituting (4.3.6)-(4.3.8) in the right side of (4.3.5), we obtain the theorem. The proof is complete. We return to (4.3.3). We have

fFo(a)trv T(z, z; r ; X ) dp. ( z ) ,.

� � fFo( a)dp. ( z ) f_ dr 1 E ( z ; 2'1 + ir; a; e, ( a ) ) 1 2 h ( 41 + r 2 ) k" 1 1 4 = f dr h ( 4 + r 2 � � fF dp. ( z ) 1 E ( z ; 21 + ir; a; e, ( a ) 1 2 . 1 = 4'7T '7T

n

k"

a= 1

1= 1

00

00

)

- 00

00

n

V

/= 1

a= 1

)

o(a)

v

(4.3.9 )

We can interchange the order of integration in (4.3.9), because the integrand is nonnegative. We now prove a theorem which enables us to estimate the error which arises when we replace the sum under the integral sign on the right in (4.3.9) by the sum (4.3.4) as --') 00 .

a

Let h(s(l - s)) be as in Theorem 4.2.2. Then fFo( a )tr v T( z, z ; r ; X ) dp. ( z ) - -f- f drh ( ! + r 2 ) THEOREM 4.3.2.

'7T

x

�

�f

a = 1 1= 1

s; a; e,(a); a)

Fo( a )

00

- 00

i ( � + ir; a; e,(a); a ) i 2 = 0 ( 1 ) .

dp. ( z ) E z ;

V

a

-'>

00

(4.3 . 1 0 )

PROOF. According to the definition (3.2. 1) of the vector-valued function the difference on the left in (4.3. 10) is equal to E ( z;

70

IV. THE SELBERG TRACE FORMULA

. . ' 1 -\ + "S / / eJ a ) 2 2-,,;; a - / p + / 1' (ir )1' ( "2 + ir ) ( 21 + ir ) 2v] ' (4.3.11) �O

since integrating the Eisenstein series over IT a reduces to integration of the corre­ sponding constant term in the Fourier expansion (see Theorems 3.l.2 and 3.l.3). Now, because of (4.2 .3), which holds because the operator K(f ; X ) is . nonnegative definite, the right side of (4.3.11) is bounded above by the expression

[ k ( gp z, gp z ; 1' ; X ) - 4� t drh ( ! + r ' ) at /�1 �1 /I / I'(Z) oo P Xly l /2 +"Sape/ ( a ) + y ' /H';; r( ir )1' ( � + ir ) ( � + ir ) J - I �O

(4.3.12) as we see in our usual way, using (3.3.17),

But (4.3.12) is bounded by 0 (1) as a 00 , (3.3.20) and Theorem 2.1.1. We shall not dwell on the verification of this bound. The proof is complete. We now state a theorem which gives a final characterization of the asymptotic behavior of the integral (4.3 .3). �

.f

THEOREM 4.3.3. Let the function h(s(1 - s)) be as in Theorem 4.2.2. Then

Fo( a )

tr v T( z, z; f ; X ) dfJ. ( z ) =

1 k( f ; X)g(O)ln a - 47T

+ h(1{4) tr ® ( � )

1_0000 h ( 41 + r 2 ) Lf/1' ( "21 ir ) dr +

(4.3.13) 0(1), where the integral on the right in (4.3.13) is absolutely convergent; g( u) is the Fourier transform of the function h(1j4 + r 2 ) in (3.3.16). PROOF. By (4.3.9) and Theorems 4.3.1 and 4.3.2, we have f0)( a ) tr T( z, z; f; X ) d ( z ) 1 1_00 h ( ! + r 2 ) ( 2k( r; X ) In a - � ( � + ir ) + �. ( tr ® ( � + ir ) a 2 ir = 2r 4 7T 00 - tr ® ( � + ir ) a - 2 i r) ) dr + �� L v

J..t

+

. ' J

Q -). <X)

§4 . 3 . DERIVATION OF THE FORMULA

71

Next, from the Fourier integral formula we obtain 1 f_oo h ( ! + r2 ) � ( tr ® ( � + ir ) a2 ir - tr ® ( � + ir ) a -2 ir ) dr 4 w oo 2r =

h ( It4) tr ® ( � ) + 0 ( 1 ) . a -- 00

The integral involving the logarithmic derivative of the determinant of the scattering matrix is absolutely convergent because of the absolute convergence of (4.3.3) for each fixed a and because of Theorems 3.S.S and 3.S.6. The proof is complete. We now proceed to the asymptotic �xpansion for the integral (4.3.2). We make use of a device which is fundamental to the derivation of the analogous expansion in the scalar theory. The integral (4.3.2) is equal to the sum � � tr X ( Y ) i. k ( u (Z , y , - l yy'z ) ) d/-t (z), (4.3.14) F

'

v

{ Y } r y E r \r

y

o(a)

where { y h is the conjugacy class in f with representative y, fy is the centralizer of the element y E f in f, and the function k(u) is defined in Theorem 4.1.2. The summation in (4.3.14) is over all conjugacy classes { Y h and all cosets modulo the subgroups fy • A change in the variable of integration transforms (4.3.14) to the following form: (4.3.1S ) � tr x (y)j k(u(z, yz)) d/-t (z), {Y}r

v

B( a ; y )

where we have introduced the notation

B(a; y) =

U

Y ' E ry\ r

y'Fo (a).

It is easy to see that B(a; y ) becomes a fundamental domain for fy on the half-plane H as a 00 ; we shall denote this domain by Fy' i.e., B(oo; y ) = Fy• Just as in the scalar theory, we verify the following claim: ->

If y is the identity, is hyperbolic, or is elliptic, then the following integral is finite: lim j k(u(z, yz)) d/-t (z) = jFyk ( u(z, yz)) d/-t (z). (4.3.16) a -- 00 The structure of the centralizer fy is well known for such y, as is the procedure for computing the corresponding integrals (4.3.16) (see [SO], or [72], §3, or the Appendix to [28]). Hence we shall immediately give the result of the calculation of these integrals in terms of h (lj4 + r2) for the identity, for any hyperbolic element, and for any elliptic element. We introduce some notation. Every hyperbolic element P E G is conjugate in G to an element (transformation) z -> N(P )z , N(P) 1, z E H. Following Selberg, we shall call N(P) the norm of the hyperbolic element (or B(a; y)

>

the norm of the conjugacy class {Ph, since it is invariant under conjugation). THEOREM 4.3.4. 1) The following equality holds: t k(U (Z , z)) dl' (z) = 1;:1 f r(tanh wr) h ( ! + r ' ) dr,

oo E where E is the identity of the group f, I F I is the volume of F relative to the measure dp" and tanh is the hyperbolic tangent.

72

IV . THE SELBERG TRACE FORMULA

2) Suppose that k E 71... , k � 1 , and y = p k is a power of a hyperbolic element p E f. Then f k(u( z , yz )) d ( z ) In N(P) _ g(k ln N(P)), N(P) - N(P) 2 where g( u) is the Fourier transform of h(I/4 + r2) (see (3.3.16» , and In is the natural logarithm . 3) Let y = R k be a power of an elliptic element of order m, where k, m E k � 1, m � 2. Then f k ( u ( z , yZ )) d ( z f oo exp ( - 27Trk/m) h ( 1 + r ) dr. 1 ) - 2 m . (k 7T/m ) - f 1 - exp ( - 2 7T r ) 4 Thus, to construct the desired asymptotic expansion of the integral (4.3 .2), it remains for us to study the contribution to the sum (4.3.15) from all of the parabolic -

f.t

Fy

k /2

k/

71... ,

Fy

f.t

-

SIll

2

00

conjugacy classes in f; we now proceed to do this. We start with a definition. A parabolic element (or conjugacy class) will be called primitive if, first, it is not an integral power of any other element (conjugacy class) in f, and, second, in the group G = PSL(2, IR) it is equivalent (i.e., conjugate) to the transformation Soo: z -7 Z -+: J � Z E H. Any parabolic element in G is equivalent either to Soo or to S;;' 1 . The latter two elements are only equivalent relative to the reflection transformation. It is well known (see [SOD that the total number of primitive parabolic conjugacy classes in f is equal to n , the number of pairwise inequivalent cusps of F. For representatives of these classes we take the generators Sj ' 1 � j � n, of the groups If C f (see §1.2). The total contribution from all of the parabolic conjugacy classes to the sum (4.3.15) is equal to

tr ( Sf ) f a . k ( u ( z , SfZ )) df.t ( Z ) . (4.3.17) B( .S ) I We have used the equality B(a; 5;) = B(a; Sf), k E 71... , and the relation fSj = fj ' which follows from the discreteness of the group f (the group If was defined in §1.2),j = 1, . . . ,no We consider the integral 11

�

.j=

00

�

k = - 00 k of=. O

v

fB( a ,. S)k( u(z, 5;kZ ) ) dJL( z ) .

(4.3.18) Recall the equality t; ISfgj = S!. The. ,'change of variable of integration Z = gj Z ' reduces (4.3 .18) to the form (4.3.19) i -I B( a .,S)k{ u{z, z + k )) df.t ( z ) . We consider the domain of integration (tB(a; S). It is not hard to see that gjIB ( a; 5; ) = IT - IT a - D/ a ) , where, as in §1.2, IT a = {z E H I 0 � � 1 , a < y < oo}, ITo = IT a for a = 0, D;C a ) gj

X

is a measurable set lying strictly in ITo - ITc ' where the constant 0 depends only on the discrete group f, and, in addition, the Euclidean measure v(D;Ca)) of the set c >

§4.3 . DERlVATION OF THE FORMULA

73

approaches zero as a � 00 (see [72], §3). Now recall that the fundamental invariant of a pair of points is equal to Dia)

1 Z z ' 12 U ( Z , Z ') = -'--yy"--'-

and the function k(t) satisfies the estimate (3.3.l8): k(t) « ( 1 + t)- eo , where 1 1 + > o. From this we obtain <

€, €

<

So

!D/a)k( u(z, Z + k )) d/,(z) « k- 2 eop{Di a )) , l , _

n

�

OO

�

j = l k = - oo

bi= O

tr v ( S/ ) !Dj(a)k(u(z, z + k )) d/,(z) « 1 �(2so ) 1

n

� v { Di a ))

j= l

= 0 (1). a ---> oo

Consequently, the sum (4.3.17) is equal to

(4.3.20) We now represent each operator X(S/) in V as a sum

(4.3.21) where lj is the projection onto the subspace Vj C V (see § 1 .2), and I v is the identity operator in V. Corresponding to the partition (4.3.21), we pick out the following term from the sum (4.3.20): 00

�

1=

In

a

l O

k

( [ 2 ) dy

2 2 Y

Y

n

� j=

1

2 Re tr ( ljSj ) . 1

V

(4.3.22) .

Since tr v(Pj Sj) = trvlj = dim Vj = kj for any [ E lL, it follows that the sum (4.3.22) is equal to (

y

[ )d (4.3.23) 2k(f ; X ) � In k 22 2 ' Y l O Y The construction of an asymptotic expansion for (4.3.23) as a -> 00 is well known from the scalar theory (see [50], or [72], §3, or the Appendix to [28]). Here we shall only indicate the basic stages in the transformation of (4.3.23) which leads to the 00

/=

a

IV. THE SELBERG TRACE FORMULA

74

desired principal terms of the asymptotic formula. The sum (4.3.23) is equal to the expression (we are following [50]) 2 k(f; x) � 7I f oo k(u 2 ) du = 2k(f; x) r oo k(u 2 ) � 1 du 00

f= 1

fla

=

lr,

0

f";;;' au

2k( f; x)(1n a + C) 1 k( u 2 ) du o

00

7

2 k ( f ; X) Jrooo (In u ) k ( U 2 ) du + 0 ( �a ), V where C is Euler's constant. Taking (3.3.16) into account, we obtain a -> 00

+

-

(4.3.24)

1 f oo h ( 1 + r 2 ) f ' ( 1 + ir ) dr ; rOO exp In(1 -u)) dg(u) = -Cg(O) ( y Jo 2 7T 4 rOOf1 u2 du 21 g(O), (4.3.25) J k( ) - 00

=

o

where fe z ) is Euler' s gamma-function. From (4.3.24) and (4.3.2 5) we get the desired expression for the sum (4.3 .23):

kef; X)g(O)ln a - kef; x)(1n2)g(0) + kef; X) h(I{4) (4.3.26) - k( f27T;.il fOO h ( l.4 + r 2 ) ff' ( 1 + ir) dr + 0(a -->oo1). Now one has to consider the remaining part of the terms in (4.3.20), correspond­ ing to the projection (Iv - P) in (4.3.21): ak ( / 2 ) d 2 Re trv[ ( Iv - pJ x S (4.3.27 ) { f )] · y 2 ; j� l I�l 1a We use the notation in §2.1. The trace trv[(Iv - Po,)S.f] is equal to the sum of the corresponding eigenvalues (see (2.1.16) and (2.1.17» 4.3.28) tr A ( 1v - Pa)S:J = � exp 21Tip(}'a , ( I=k",+ where 0 < (}'a < 1 and h = dim We suppose that the eigenbasis of the operator X(Sa) is chosen in such a way that its first ka vectors correspond to the subspace Va' Thus, a typical term in (4.3.27) is an expression of the form (4.3.29) -- 00

n

00

h

V.

1

Similar sums were studied by Selberg in the scalar theory (see [50]). We shall give the computation of the sum (4.3.29) in terms of the function h(Ij4 + r 2 ), where we set () 21T(}/a for convenience, and we use (3.3.16). =

§4. 3 . DERIVA nON OF THE FORMULA

75

First of all, it is easy to verify that as a 00 the expression (4.3.29) has a finite limit, which is equal to 00 d = 2 1000 P00 (cos pO)k(p 2 U 2 ) duo (4.3.30) 2 1000 P� ( cos pO)k �22 ; �I ->

( )

I

We compute the right side of (4.3.30), introducing a small parameter and then passing to the limit as 00 : 00 (4.3.3 1 ) !�� 2 � 00 P� I (cos pO )k( p 2 U 2 ) duo EO

EO ->

Next,

f10 oo

:

oo � (cos pO)k(p 2 U 2 ) du = � (cos pO ) f k( p 2 U 2 ) du

p= I

p=I

10

(4.3.32)

Since 0 < Ola < 1, it follows that ei() =1= 1 . We have cos pO = In I I - exp i0 1 + 0 � I

-

P

C

V(eU

( ) Substituting (4.3.33) into (4.3.32), we obtain

� � u�

( 4.3.33 )

with a uniform constant in O. 00 00 f � (cos pO) k ( P 2 U 2 ) du = - "21 g( 0)In I I - exp i0 I + 0 ( (e) . 10

p= 1

Passing to the limit as left in (4.3.30):

EO ->

10 --> 0

0 in the last equality, we find a value for the sum on the -

g(O)ln l l - exp iO I .

Finally, the sum (4.3.27) is equal to n

-g(O) �

h

ln l l - exp 2 7T i0 a l + o(l).oo 1

�

(4.3.34)

a-+

If we now take (4.3.26) into account, we obtain the following result. THEOREM 4.3.5. The contribution to the sum (4.3.l5) from all of the parabolic

conjugacy classes is equal to k(f ; x) [ g(O) n a - g(O) In 2 + h( Y4) - 2� too h ( ! + r 2 ) i ( I + ir) dr] g (O) � � ln l l - exp 2 7T iO a l + 0( 1oo) . a-+ J

-

n

h

a = I I=k,,+ 1

l

Theorems 4.3.4 and 4.3.5 completely describe the asymptotic expansion of the integral (4.3.2). Along with Theorem 4.3.3, which gives an asymptotic expansion of the integral (4.3.3) as a 00 , they transform the spectral trace formula (4.2.5) into a Selberg trace formula. ->

r U l<. M U LA

1 V . 1 rIC :)CLtlCKI..J I KAI... �

To formulate the final result we need to introduce a little more notation and make some definitions. We first return to Theorem 4.3.4, which describes the contribution to the trace formula from the elliptic, hyperbolic, and identity conjugacy classes. Earlier we introduced the notion of a primitive parabolic element (conjugacy class). It is appropriate to introduce the analogous concepts for hyperbolic and elliptic elements (conjugacy classes) as well. Following Selberg, we call a hyperbolic element y E f primitive in f if it is not a power of any other element in f ; and similarly for a conjugacy class {y ) r . It is well known that every group f E 9)( has infinitely many primitive hyperbolic conjugacy classes. Later (§5.3) we shall find the asymptotic behavior of the distribution function for the values of their norms. We now say a few words concerning the elliptic elements in f. We consider the set of all elliptic elements in f having a common fixed point inside H. Among these elements we choose the one which gives a clockwise rotation through the smallest positive angle. Following Selberg, we call this element (and the corresponding conjugacy class in f) primitive. The number of primitive elliptic conjugacy classes is finite in every group f E 9)( .

(Selberg trace formula). Let the function h(s(1 - s)) have the

following properties: 1) h(s(1 - s)) = h f(s(1 - s)). 2) h\(s) = h\(s(1 - s)) is analytic in the strip - e < Re s < I + eforsome f > O. 3) In this strip h I es) satisfies the estimate THEOREM 4.3.6

for some � O. 4) h\( A ) is real for 0 � A < 00 . Then the following Selberg trace formula holds: >

�h }

( ! + 1/) = '2:' dim Vf_oooor (tanh 1Tr)h ( ! + r 2 ) dr +

�

{R}r

+2

�

d 1

k= 1

tdrsinv{xkk1T( R)/d) foo e +p exp-21Tr-21Tr k/d h .!. + r 2 ) dr (

� � � {P}r k = 1

)

(

x

- 00

I

(

(

) )

4

� trv{ xk ( P ) ) ln N(P ) g ( k ln N(P )) k/2 -k/2 - N( P )

N(P )

+ 211T f_:h ( ! + ·r � ) � ( � + ir ; f ; x ) dr - ke

f; X )

1T

fOO h ( .!. + r2 ) f'f ( 1 + ir) dr - 00

4

+ � ( k ( f ; X ) - tr ® ( � ; f ; X ) ) h ( ! )

- 2 ( k( r; X )ln 2 + at Ij: ln i l - eXP 2?TiOla l ) g(O). I

(4.3 .35)

§4.4. THE WEYL-SELBERG ASYMPTOTIC FORMULA

77

In the left side of (4.3.35) j runs through the set of all eigenvalues Aj = 1/4 + r/ of the discrete spectrum of the operator �(f; X), where we take both values of 1j which give the same Aj (i.e., the sum on the left in (4.3.35) gives twice the spectral trace). In the right side of (4.3.35) {R h runs through the set of all primitive elliptic conjugacy classes in f, d = d( R ) is the order of R, and {Ph runs through the set of all primitive hyperbolic conjugacy classes in f. PROOF. In the spectral trace formula (4.2.5) (more precisely, in the right side of it), we substitute the asymptotic expansions for the integrals (4.3.2) and (4.3.3) which are given by Theorems 4.3.3-4.3.5 and by (4.3.1 5). After passing to the limit as a � 00 and multiplying both sides of the resulting formula by two, we obtain the desired Selberg trace formula. The proof is complete. We now observe that, just as in the scalar theory, (4.3.35) remains true for a broader class of functions. We proceed to the definition of this class of functions. §4.4. The Weyl-Selberg asymptotic formula. Extension of the Selberg trace formula to a broader class of functions h

In this section we generalize the Weyl-Selberg asymptotic formula from the scalar theory (see [50]), refine our estimate of the order of meromorphicity of the determi­ nant �(s; f, X) of the scattering matrix, and, finally, describe a broader class of functions h (s( l s » than in the statement of Theorem 4.3.6 for which the Selberg trace formula still holds as an identity. We begin with an a priori estimate for the distribution function for the values of the norms of primitive hyperbolic conjugacy classes in f . We set -

'1T ( x ; f ) = { the number of primitive hyperbolic {P h I N( P} � x} . (4.4 . 1 ) The function '1T(x; f ) is obviously defined on the semiaxis 1 < x < 00 . Because the

series (4.3. 1 5) is absolutely convergent, we easily find the following estimate for this function (we shall later obtain an asymptotic formula). LEMMA 4.4. 1 .

'1T ( x ; f ) = O( x ) ,

x�

00 .

(4.4.2)

We now introduce the distribution function for the eigenvalues Aj of the discrete spectrum of �(f; X ) :

N( A ) = N( A ; f ; X ) = { the number of Aj for � ( f ; X ) I Aj � A} . The function N( A ) is obviously defined on the semiaxis 0 � A < 00 . By the Weyl-Selberg formula we mean the asymptotic formula, first obtained by Selberg [50] in the scalar theory (dim V = 1), for the sum of the function N(A; f; X ) and the value of the argument of the determinant of the scattering matrix at the point A of the continuous spectrum of �(f; X). In our opinion, this formula is d natural generalization of the well-known formula of Weyl in the spectral theory of the Laplace operator on a compact manifold (in particular, for � ( f; X), f E ITn I ) in the case when �(f; X) does have a continuous spectrum, i.e., f E 9)( 2 and X E 9( S < f). We note that in the theory of the operator �(f; X ) the Weyl-Selberg formula also degenerates into the usual Weyl formula on a noncompact fundamental domain F for f E ITn 2 if the representation X is regular.

IV . THE SELBERG TRACE FORMULA

78

4.4.1 (the Weyl-Selberg formula). The following asymptotic formula holds: . V · A , (4.4.3) N(A ; f; X ) - 41'IT f-TT T ( 21 + lr,. f; x ) dr A :OO I4'ITF l dlm where A = 1 /4 + T 2 , T > o. The second term on the left in (4.4.3) is absent if either f E 9)( 1 or f E 9)( 2 and X E 9C l f). PROOF. In the Selberg trace formula (4.3.35) we take for h(1 /4 + r2) the function h 0 + r 2 ; t ) = exp ( - (i + r 2 ) t ) , where the parameter t > 0 is fixed. It is easy to see that this function satisfies Theorem 4.3.6; in addition, its Fourier transform g(u; t) (see (3.3.16» can be THEOREM

!J. '

computed exactly:

g(u; t) = ( 2 F! ) l exp( t/4 - u 2/4t ) . We now successively find the asymptotic behavior as t � 0 of each term on the right in (4.3.35), except for the asymptotic behavior of the term with the logarithmic derivative !J.(1 /2 + ir; f; X). We have I F 1 dim V f oo r( tanh 'lT r)h ( 1 + r 2 ; t ) dr = I F 1 dim V 1. + 00 ), 2 'IT t ( ..... 0 4 2 'IT � tr� ( x k ( R» f oo exp( -2'ITrk/d) ( 1. � dsm(k'IT/d) 1 + exp(-2'ITr) h 4 + r 2 ; t ) dr = 0 ( 1), k (P»)ln N(P) 0 2 � � Ntrv(X (P) k/2 - N(P) _ k/2 g(k ln N(P); t) = 0 ), (4.4 .4) -

-

- 00

d

\

{Rh k= \

t

- 00

00

t

{Ph k= \

� ( k ( f ; X) - tr ® ( � ; f ; X ) ) h ( ! ; t ) = o( 1 ) ,

-2 kef; x)ln 2 +

(

n

�

dim

�

V

a = \ l= k", + \

t

)

O

.....

.....

O

O

( )

ln l l - exp 2 'ITi8la l g(O; t) = 0 r;1 .....

Vt

.

( ..... 0

The only relatively nontrivial of these formulas are the estimates for the integral with the logarithmic derivative of the gamma-function and the sum over the hyperbolic classes {Ph. We obtain the first estimate in §4 of [62], to which we refer the reader. The second estimate follows from Lemma 4.4.1 and the definition of g( u; t). Thus, we have 1 + 2 ,. t ) - 1 f oo h ( 1 + r 2 ,. t ) !J.' ( "21 + lr ) dr �h( £.J J

4

1

lj

2 'IT

_ 00

4

T

V 1. + 0 ( In t ) - I F I 2dim 'IT t vt _

r;

.

,

t

--'>

o.

(4.4 .5)

§4.4. THE WEYL-SELBERG ASYMPTOTIC FORMULA

79

The left side of (4.4.5) is up to O( 1/ Ii) equal to 2 foo ( exp t ;\ )d ( N( ;\ ; f ; X ) + Q ( ;\ ; f ; X )) (see Theorem 3.5.6), where, by definition o

Q(A ;

r;

(4.4.6)

-

X) =

! 41'71" fF� -

-

-

0,

VA - 1 /4

(1 l.r) dr, 2

/:).' re g

- +

/:). r e g

1

o � A � -4 ,

and the function �reg(s) defined in §3.5. According to Theorem 3.5.5, the function Q(;\; f; X ) is monotonically nondecreasing on [ 0, 00). The sum N( ;\; f; X ) + Q(;\; f; X) obviously has the same property. Hence, the theorem follows from (4.4.5), (4.4.6) and a well-known Tauberian theorem. The modifications needed in the proof of the theorem when the spectrum of 21(f; X ) is purely discrete, i.e., either f E W( I or f E 9Jl 2 and X E �n r(f) (see §1.2), are obvious and are connected with corresponding simplifications in the Selberg trace formula (4.3.35). The proof is complete. We now refine our information on the order of meromorphicity of �(s), s E C (see Theorem 3.5.1). We return to the notation of §3. THEOREM 4.4.2. � IYI"';; T 1 « T 2 , where the summation is over all poles p f3 + i y of the function �(s) in the half-plane Re s < 1/2. PROOF. We note that all poles p of �(s) with f3 < 1/2 lie in the strip - cj Re s < 1/2 for some sufficiently large Cj > 0 which depends only on f and x . This follows from parts 1) and 5) of Theorem 3.5.1, since the Dirichlet series l(s) does not vanish for Re s > Cj if cj > 0 is sufficiently large. Furthermore, from Theorems 3.5.5 and 4.4.1 we have =

",;;;;:

j

T

'" £.J

-T p

According to Theorem 3.5.4, �

I yl > 2 T

f

T

-

T (f3

-

1 - 2f3 ( f3 - 1/2)2 + ( r

_

y )2

dr « T 2 .

(4.4.7)

1 - 2f3 = 0 ( T ) . 1 - 2f3 T � dr « 2 2 2 1/2) + ( r - y ) I y I2 I yl > 2 T

T-> 00

Consequently, the following sum gives the basic contribution to the asymptotic behavior of the left side of (4.4.7 ): �

j

T

l y l ",;; 2 T - T

1 - 2f3 dr = 2 2 2 r + 1/2 ( y) (f3 - )

This leads to the estimate

�

l y l ",;; 2 T

j

( T -- Y )/( 1 /2 -. {3 )

- ( T + y )/( J /2 - {3 )

1 dr. 1 r2 +

1 ",;;;;: � f T ( f3 1/21 � +2f3( r - ) dr ",;;;;: C3 � 1 y - ) with constants 0 < C2 ",;;;;: c3 ; this implies the theorem. The proof is complete. The following fact follows from (3.5.6) and Theorems 3.5.1 and 4.4.2. THEOREM 4.4.3. �(s) is a meromorphic function on C of order no greater than two. c2

�

lyl ,;;; 2 T

lyl ",;; 2 T

--

T

2

l yl ,;;; 2 T

ov

I V . T H E SELBERG TRACE FORMULA

Using Theorem 4.4.1 and Lemma 4.4.1, we easily verify the following theorem, which extends the class of functions h( s(l - s)) for which the Selberg trace formula (4.3.35) is valid. THEOREM 4.4.4. The Selberg trace formula holds as an identity for any function h(s(l - s)) h(l/4 -+- r 2 ) which satisfies the following conditions: I) h(I/4 -+- r 2 ) is analytic as a function of r in the strip 1 1m r 1 < 1/2 + for some E > O. 2) h(l/4 + r 2 ) = 0((1 + 1 r 1 2 )- I -f) in this strip , and all of the series and integrals =

to

in (4.3.35) converge absolutely.

In conclusion, we note that the class of functions h(s(1 - s)) for which (4.3.35) remains valid can be extended further than in Theorem 4.4.4 ; however, the series over the hyperbolic conjugacy classes in the right side of (4.3.35) may stop converg­ ing absolutely (for example, for nonanalytic functions h ), and one must clarify what one means by convergence in such a case. We shall not dwell any more on this question here.

j

CHAPTER 5 ELEMENTS OF THE THEORY OF THE SELBERG ZETA-FUNCTION. SPECTRAL AND GEOMETRIC APPLICATIONS OF THE THEORY

In Chapter 5 all of the assumptions listed at the beginning of Chapter 4 are still in force. §5.1. The definition and basic properties of f; X) The material in this section is a generalization of analogous results in the scalar theory (dim V = 1) which we obtained in [68]. The brief remarks at the end of Selberg's lectures [50] served as a stimulus for this. By the f ; X) we mean the product

Z(s;

Selberg zeta-Junction Z(s; Z( s) = Z(s; f ; X ) = II

00

II

{Ph k=O

...

.

det v ( I v - x ( P ) N( P fS- k ) ,

(5 . 1 . 1)

where {Ph runs through the set of all primitive hyperbolic conjugacy classes in f, N( P ) is the norm of P f (see §4.3), I v is the identity operator in V, and det v is the determinant of an operator in V (see §1 .2). The definition (5. 1 . 1 ) is correct for Re > 1 , where the product converges absolutely, by Lemma 4.4. 1 . The zeta-function Z(s; f; X ) is connected with the Selberg trace formula in the same way as the Riemann zeta-function is connected with Weil's explicit formula in analytic number theory (for f 9)( 1 see [5 1], [ 1 8] and [68]). From (4.3.35) we shall derive an important representation for the logarithmic derivative of the Selberg zeta-function from which we shall later obtain all the basic properties of We now state and prove a theorem, using the notation from Theorems 3.5.3, 3.5.4 and 4.3.6.

E

s E. C, s

E

Z(s),

Z(s).

THEOREM 5. 1 . 1 . Suppose that s E C (Re s > 1), and let a E IR (a Then Z' (s '' f . x ) = _ ( s ! I F I dim V � ( _1_ 1 Z , 2) 7: s k a k ) �

+

'IT

_

d� l

k 0 trv { X k ( R ) ) d sin 'IT/ )

_

{Rh

k= I

be Jixed.

'IT

'IT

+

1)

+

(k d X [ ( 'lTexp ( - 2 ik ( s - � ) /d ) ) ( 1 - exp ( - 2 is ) ) - I _

>

i ( S - ; ) �1 ( exp ( -2 " ik ( I - ; ) /d ) ) [ ( S - ; ) ' - ( I - ; ) 'r]

I

81

V . Ttl!:': S!:':LH!:':KU Zb l A-l' U N CTlUN

Il' f; X) + ( s - "21 ) - "21 -X-(s; 1 1 X � [ ( s - 1/2)2 - ( p - 1/2)2 ( a - 1/2f - ( p - 1/2)2 1 f ' ( -3 - s ) - ( s - -21 ) � s 1/2f 1 + kef X) · ' f 2 /= ( - - «1/ - 1/2f 1 -2 k( f ; X) ( s - -21 ) k� = l ( s - 1/2) 2 - k 2 - ( k( f; X) - tr ® ( � ; f; X ) ) 2s � 1 1 1 ) + c4 + c5 ( s - -1 ) + � (s - 1/2)1 2 1/ s 2 2 ' (a - 1/2)2 + 1/ (5 1 .2) where c4 and c5 do not depend on s. PROOF. In the Selberg trace formula (4.3.35) we choose for h(1/4 + r2) the function h ( I /4 + r 2 ; s; a) which depends on two parameters s and a: h( t + r 2 ; s; a ) = [( s - 1) 2 + r 2r l - [( a - 1) 2 + r 2] - I. In order for this function to satisfy the conditions of Theorem 4.4.4, it is sufficient to require that 1 Re s a, where a is a fixed positive number, 1 < a. The corre­ sponding function g(u) = g(u; s; a) in (3.3.16) is easy to compute: g(u; s; a) = (2s - 1 )- l exp( - ( s - 1 ) I u I ) ( 5 . 1 . 3) - (2 a - 1 )- I exp ( - ( a - 1) I u I ) . The Selberg trace formula for the function h(1/4 + r 2 ; s; a) has the form � h ( ! + r/ ; s; a ) = I F I:: V f : r(tanh 7Tr )h ( ! + r2; s; a ) dr \ tr ( X k ( R )) OO .l . exP(-27Trk/d) d � + � \ d sin ( k7T/d ) f h ( 4 + r 2 ." s a ) 1 + exp(-27Tr) dr k= tr ( X k ( p )) In N(P) (5 . 1 .4) +2 � � N(P) k/2 - N(P)_k/2 g(kln N(P); s; a ) (Ph k=\ '. + 27T1 t h ( 1 + r 2; s; a ) -X-Il' ( 21 + ir ) dr oo k( f · X) i ( 1 - ; oo h 4 + r 2 ; s; a ) Tf ' ( 1 + ir) dr + l2 (k( f ·' X) - tr ® ( l2 ) ) ( ( s - 11/2) 2 ( a 1_) - 1 /2) 2 1 ) - ke f ; ) ln 2 + a� /=-� �+ \ ln l l - exp 2 7T i Ola l ) ( 1 \ s - 1/2 a - 1/2 ' p , /3 < 1 j2

�

1

00

j(

+

""[ j

.

<

<

-

J

v

(R}r

00

00

- 00

v

1

4

00

_

(

X

n

h

_

§5. 1 . DEFINITION AND BASIC PROPERTIES

83

h = dim We transform the terms on the right in (5.1.4). We begin by verifying the equality 1 Z' ( s) - I Z' ( a ) s - 1 12 Z a - 112 Z 00 tr v ( X k ( P ) ) In ( P ) 2 � � N( P ) k/2 - N( P ) _k/2 g ( k ln N( P ) ; s; a) , (5 .1.5) V.

=

{Ph

N

k= 1

where the summation on the right is only taken over the primitive hyperbolic conjugacy classes in f. In fact, from the definition (5.1.1) we have

� (s) = � =� ds

=� =

�

{Ph

� {Ph

�

00

{Ph

k=O

�

k =O

trv 1n( I v - x ( P )N( P fS- k )

� tr v ( - �

k=O 00

'= 1

X ( P )' I

N( P f( s +k) '

)

� � �0 (In N( P ))N( P f(s + k )' tr v ( X ( P ' ))

{Ph =

� In det v ( I v - x ( P )N( P fs - k ) t ,o _

k =O 1= 00

� � In N( P ) tr( x ( P ' ) ) N( P fS' ( l - N( P f' f l . {Ph

1= 1

(5.1.6)

Analogous equalities hold for s a as well. Combining these sums (5.1.6) for s and a with (5.1.3), we obtain the desired equality (5.1.5). To prove the theorem, we continue transforming the terms on the right in (5.104) to the form we need. The following integrals are computed using the theory of residues. We shall give the final results, since the computations are rather simple. We have 00 1 ) . ) OO ( 1 (1

=

i oo r(tanh 7Tr)h 4 + r 2 ; s ; a = 2 �0 s + k - a + k ' (5.1.7) k f OO h ( -41 + r 2 . s · a ) exp(-27Tkrld) dr " 1 + exp( - 27Tr) 7T ( exp ( -2 7Tik ( s - � ) /d ) ) (1 - exp(-27Tis ) r 1 = _ - 00

s 1/2 +i I exp ( -2 ?T ik ( m - � ) Id) ( ( s - ; ) r - ( m - ; rr + co ' (5.1.8) - kef'; X ) i h ( 1 + r 2 ; s; a ) f ' ( 1 + ir) dr y oo 4 = kef;- 112X) -ff' ( -23 - s ) + 2k( f; x ) k�= 1 (s - 1/21 )2 - k 2 + c7 · (5.1.9) In (5.1.8) and (5.1.9) the constants c6 and c7 do not depend on s. 00

-

S

00

84

V . THE SELBERG ZETA-FUNCTION

As our equalities (5.1.5) and (5.1.7)-(5. l .9) show, to derive the desired formula (5. l 2) it remains for us to compute the integral

1 h ( "41 + r 2 ; s ; a ) Lr/l' ( "21 + ir ) dr. 2 'Tf l oo

.

00

(5.1 .10)

This integral can also be computed using the residues of the integrand. We shall give the basic steps in this computation. We consider the integral

(5.1 .11)

where n T is the closed contour in the T-plane consisting of the segment of the line Re T 1/2 beginning at 1/2 - iT and ending at 1/2 + iT and the semicircle connecting { i + iT, i - T, i - iT} (the three points are given in the direction they are traversed by the contour); T is a positive parameter. By Cauchy's theorem, the integral (5.1.1 1) is equal to the sum of the residues of the integrand at the poles lying inside n T (we are supposing that T is taken so that n T does not pass through any singular points). On the other hand, the integral over the semicircle in (5.1.11) vanishes when T � 00 , because of the estimate we know for the order of meromor­ phicity of /l(s) (see Theorem 4.4.3) and because of our choice of the function h( T(l - T); s; a). The integral over the line segment [i - iT, i + iT] in (5.1.1 1) approaches the integral (5.1.10) as T � 00 . As a result, the integral (5.1.10) is equal to =

�

+ � [(s - i )2 _ ( aj - 1 )2r1 + Cg , (5.1 .12) j= 1

where p in the sum (5.1.12) runs through the set of all poles of /l(s) in the half-plane Re s < t f3 Re p, j runs through the set of all poles aj of /l(s) on the interval ( 1 , 1 ] (see §3.5), and cg does not depend on s. The proof is complete. The formula (5.1.2) is a fundamental tool for studying the properties of the zeta-function Z(s; f ; X). The first consequence of the formula and of Theorems 4.4.1 -4.4.3 is meromorphicity of the function Z(s) on C . Moreover, the following fact will be proved later, in Lemma 5.2 .3. =

THEOREM

5.1.2. Z(s; f; X) is a meromorphic function of s on C whose order of

meromorphicity is equal to two.

The second and third consequences of (5.1.2) are a fairly complete description of the zeros and poles, respectively, of Z( s). ,

5.1.3. The function Z(s; f; X) for s E C has zeros at the following points and nowhere else. " Nontrivial zeros ": 1) at points Sj on the line Re s 1 symmetric relative to the real axis and on the interval [0, 1 ] symmetric relative to the point s i ; each zero Sj has THEOREM

=

=

multiplicity equal to the multiplicity of the corresponding eigenvalue Aj of the discrete spectrum of the operator &(f ; X ) in the subspace of cusp-vector-functions 5(o(f ; X ) C 5((f ; X ), and Aj = s/ 1 - s) runs through the indicated part of the discrete spectrum of&(f ; X) on the semiaxis (0, 00 ) ;

§5. 1 . DEFINITION AND BASIC PROPERTIES

85

2) at points Sj on the interval 1- s 1 of the real axis; Sj has multiplicity no greater than kef; X), the total degree of singularity of the representation X relative to the group f; every such zero Sj is of the form Sj = (11 ' j = 1, . . . , �(see the definition (3.5. 2 » , and is connected with an eigenvalue Aj ( of the same multiplicity ) of the discrete spectrum of the operator �(f; X) in the subspace 8 1 (f; X) of incomplete theta-series by means of the formula s/ 1 - s) = Aj, and Aj runs through the indicated part of the discrete spectrum ofm(f; X) in the interval [ 0; �); 3) at the poles of the function �(s; f; X) which lie in the half-plane Re s 1 and have the same multiplicity ; and 4) at the points 1 - (1j , j = 1, . . . ,�; the 'multiplicity of the zero Sj 1 - (1j is equal to the multiplicity of the pole (1j of �(s; f; X) (see the remark following Theorems 5.1.3 and 5.1.4). The "trivial zeros" lie at the points s = -I, I = 0, 1, 2, . . . , and have, multiplicity k n [ = I F 1 dim V ( / + !2 ) {R�} r kd�= 1l dtrsm�( x(k('lTR/d)» ) in ( k 'lT (2d/ + 1 » ) . THEOREM 5.1. 4. The function Z(s; f ; X) has poles at the following points and nowhere else: 1) s = 1 with multiplicity l(k(f; X) - tr ® (t f; X» ; 2) s = -I + 1, I = 1, 2, 3, . . . , with multiplicity kef; X) ; and 3) s = 1 - (11 ' j = 1, 2, . . . ,�; the multiplicity of the pole at 1 - (1j is equal to the multiplicity of the pole (1j of �(s; f; X) (see the remark following Theorems 5.1.3 and 5.1.4). THE REM 5.1.3 5.1.4. The zeros and poles of Z(s; f; X) are <

:s;;;;

p

<

=

_

'IT

REMARK ON

O

S

S

AND

given in the theorems independently of one another. The final picture of the zeros and poles emerges after a comparison of both theorems. In particular, the zeros in part of Theorem cancel with the poles in part of Theorem The proof of Theorems and follows easily from a careful investigation of the residues of the logarithmic derivative of X) at all of the poles described in Finally, as a fourth consequence of we have a functional equation for the Selberg zeta-function f; X ).

4)

5.1.3

(5.1.2).

5.1.3

3) Z(s; f;

5.1.4

(5.1.2)

5.104.

Z(s; THEOREM 5.1.5. The function Z(s; f; X) satisfies the functional equation (5.1.13) Z(l - s; f; X ) = Z(s; f; X ) �( s ; f ; X ) 'l'r ( s ; X ) , where, by definition, k d - I tr ( ( R ) ) . 'l'r (s; X ) exp - I F l dlm VJ{o t tan 'lTt dt + { R�}r k�= d sm· (k /d ) 1 { (S- 1/2 [ exp ( - 2 'lT ikt/d) exp ( 2'lTikt/� ) 1 + 1 + exp 2'lTl t dt I + exp ( - 2'lTit) Jo ' + ( I - 2s) ( k( f ; x )ln 2 + at ��: ln l l - exp 2"iO'al ) -i arg a ( � f; X ) ' l k ) f(3/2 - s) ( r ;X) ( f(s + 1/2) =

x

[

s

1 /2

v

'IT

X

'IT

;

X

}1

86

V . THE SELBERG ZETA-FUNCTION

(5.1.2), we compute the sum of the logarithmic derivative of the Selberg zeta-function at s and at 1 - s. We have F dim V z:Z' (s; f ; X ) + z:Z' { l - s; f ; X ) = I I ( s - 21 ) k�O ( 1 - s1 + k - s +1 k ) l tr { X k ( R ) ) exP (- 2'lT ik(S - 1j2)d) + exp(2'lT ik(s - 1j2)jd) � d� 1 - exp 2'lT is 1 k= l dsin (k'lTjd ) [ 1 - exp( -2'lT is) f' 3 1 /1' ) 1 /1' - 2" � ( s ; f ; X ) - 2 � ( 1 - s ; f ; X ) + k(f; X ) r ( 2" - s (5.1.14) Integrating and exponentiating (5.1.14), we obtain the desired functional equation. PROOF. Starting from

00

'IT

_

{R}r

v

'IT

Here we use the formulas

�/1 ( s; f ; X ) = �/1 ( 1 - s; f ; X ) , 00 1 1 - 1 ) 1 ) tan 'lT ( s - 2" = ( � 1 s k O - +k s+k ' '

'

'IT

C4

= k(f; x )ln 2 +

n

dim V

a = I l= k ,, + I

�

�

In l 1 - exp(8/a 2'lTi) I .

The constant of integration is found from the obvious compatibility condition at the point = � . The proof is complete.

s

In

§S.2.

Remainder estimate for the Weyl-Selberg formula

§5.2 we shall construct an asymptotic expansion for the function /1 ' 1 f 1 f N( 'A ; ; X ) - 4 'lT � ( 2 + ir; f ; X ) dr, -

T

T

(5.2.1)

where 'A � 00 , 'A = t + T 2 , T > 0, with three principal terms and a remainder having order O(Tjln T) (see Theorem by means of the Selberg zeta-function (this will be our spectral application of the theory). Recall that the justification of the first principal term in the asymptotic formula for is the content of Theorem and is what leads to the ,Weyl-Selberg formula. The method of proof of the desired asymptotic expansion which we shall use generalizes a method well known in analytic number theory for constructing an asymptotic formula for the number of nontrivial zeros of the Riemann zeta-function in a " large" rectangle in the critical strip (see, for example, For a group f E W( ) and the trivial one-dimensional representation X, an analogous formula was obtained by Hejhal and Randol in and This was done for f E W( 2 and X E W(f), dim V = by the author in We note that the articles by Hejhal and Randol were helpful to us in The proof of the fundamental Theorem will be divided into eleven lemmas.

5.2.1)

4.4.1

[68].

[19] [44]. [68].

[57]).

5.2.1

(5.2.1)

1

§5.2. THE REMAINDER IN THE WEYL-SELBERG FORMULA

87

LEMMA 5.2. l .

2N

( -± + T 2 ; f ; X ) - 21TT i� � ( � + ir; f; X ) dr = ! arg 'l'r ( � + iT; X ) + � arg z ( � + iT; f ; X ) + ?� �

(see the notation in Theorems 4.4. 1 and 5.1 .5), where the functions arg 'l'r ( � + iT; X ) , argZ ( � + iT; f ; X ) are the values of the argument of 'l'r ( � + iT; X ) , Z ( � + iT; f ; X ) respectively, which are obtained by moving continuously from the point s = A , for some fixed A > 1, along a path consisting of the two line segments s = A A + iT 1- + iT. PROOF. We choose A > 1 large enough so that the half-plane Re s < 1 - A does not contain any nontrivial zeros of Z(s; f ; X). Such a choice of A is possible. In fact, all of the poles of �(s; f; X) in the half-plane Re s < � lie in the strip -C 1 � Re s < � (see the proof of Theorem 4.4.2). The other nontrivial zeros of Z(s; f; X) lie in the strip 0 � Re s � 1 (see Theorem 5.l.3). We integrate the logarithmic derivative of the Selberg zeta-function along a closed contour Qr A which is the boundary of the rectangle with vertices A iT, A + iT, 1 A + iT and 1 A iT, where T --->

-

-

--->

- -

and A are chosen so that the contour does not pass through the zeros or poles of Z(s; f; X). We fix A with these properties, and regard T as a large parameter. We have Z'Z ( s ) ds = N - Po, , 1 ( ( 5 .2.2) o,T,A T,A 2 TT l. JQr,A where No, r.A is the number of zeros of Z( s; f; X) inside Qr, A and Po, r,A is the number of poles there. By the functional equation for Z(s; f; X) (Theorem 5.1 .5), we have '1"

Z' Z' �' ( s ) ds = 2 i -Z ( s ) ds + i ""A ( s ) ds + i 'T,r ( s ) ds, iQT.A -Z o,r(n ) QT(n ) Qr(n ) ':I' r L.l

( 5 .2.3 )

where Qr(n) is the half of Qr,A to the right of the line Re s = t. Let N(Qr(n); �) be the number of zeros of �(s ; f; X) inside the region bounded by Qr( n ) and the line Re s = �. We let Qr(n ) denote the boundary of this rectangle. Thanks to Theorem 3.5.1 and (3.5.2), we have N( Qr ( n ) ; � )

=

�' s ds � , 1 ' f- ""A 2 TTl o,r(n) L.l ( ) +

( 5 .2.4)

where, as before, � is the number of poles of �(s; f; X ) in the interval ct , 1 ]. From (5.2.2)-(5.2.4) we obtain '1" Z' (s ) ds + 1 , ( ili 1 No, r,A = Po, r,A + ( 2 TTl JQr(n) ':I' ( s ) ds TTl JQr(n) Z ,

..

1 2 (S) ds - � . + N( Qr ( n ) ; � ) + 2 7Tl f1 /2 - ir A L.l 1 / + ir �'

( 5 .2.5 )

.

V . THE SELBERG ZETA- FUNCTION

88

5.1.3 gives us (5.2.6) No,TA - N( Q T (n); �) = 2N( T 2 + L f; X ) NTP' where NTP is the number of trivial zeros of Z(s; f; X) in the interval (1 - A, j :). But (5.2.5) and (5.2.6) give 2N ( T 2 + ! ; f; X ) - 217T i� � ( � + ir; f ; X ) dr I r (s; X ) ds + -1 . j -Z' (s; f ; X ) ds + Po,TA - NTP - �)TL , = -. j > T , 7TI Z 2 7T I r

Theorem

+

'1"

.

Q T( n )

Q T ( n ) 't"

from which the lemma easily follows. The proof is complete.

5.2.2. I F I dim V 2 2 k ( f ; X) 1 1 . - arg 'l'r ( - + IT,. X ) 2 7T T - 7T TIn T 2 7T . + -7T2 ( ke f; X ) - ke f; x ) ln 2 - � � Inl l - exp 27TiOtal T + 0(1) T�oo ) PROOF. The lemma follows from the definition of 'l'r (s; X) (see Theorem 5.1.5) and the following formulas, which are easy to verify; I F I dim V 2 1 arg exp ( I F I dim VJ(iT = t tan 7Tt dt a ) 27T T + ?� � , 7T iT[ exp(-27Tik T/d) + exp(27Tikt/d ) j dt = 0 (1) , arg exp la 1 + exp(-27Tit) I + exp(27Tit) T�oo r 2. arg ( f( 1 - iT ) k( ; X) 2 k (f ; X) ( -T l T + T) + 0 1) . ( n 7T f( 1 + iT) ) T�oo LEMMA

_

n

dim

V

a= 1 I=k + 1 "

-

=

The proof is complete.

7T

5.2 .3. For any numbers c9, c lO E IR , c9 � C I a , the function Z(s; f; X) satisfies the estimate Z( s; f; X) = O(exp I 1m s I ) in the region c9 ..:;; Re s ..:;; c l O , I 1m s I > 1 . PROOF. We shall use some facts from .�he theory of the Selberg zeta-function for a cocompact group f (see [19], Proposition 7.2). Because of the definition of the functions Z(s; f; X), �(s; f; X) and the func­ tional equation for Z(s; f; X), it suffices to prove the lemma in the region - 1 ":;; Re s ":;; 2, Im s > 1 . (5.2.7) We set s = (J + i T and take a generic T, i.e., not equal to the imaginary part of a zero of Z(s; f; X). We first prove an intermediate assertion: in the region (5.2.7) Z(s; f; X ) = exp 0(T2 ) . (5 .2.8) LEMMA

§5 .2. THE REMAINDER IN THE WEYL-SELBERG FORMULA

89

To do this it is sufficient to find a suitable estimate for the logarithmic derivative of Z{s; f ; X ) in the region (5.2.7) away from the nontrivial zeros of Z{s; f ; X ) or for " permissible" values of s. Then the resulting estimate for Z{s; f ; X ) will extend by continuity to all of (5.2.7). We make use of (5. 1.2), and find an estimate for each term on the right in that formula. The following estimate was proved in [19] (see p. 80):

( s � ) k�O ( s 1 k - 1 k ) = O( T a

In T )

,

for s in the region (5.2.7), s = a + iT. According to the Weyl-Selberg formula (see Theorem 4.4.1), we have N{ A; f; X ) � A. Hence, the sum

� j

[

j(

1 1 1 s 2 ( s - 1 /2) 2 + r./ ( a - 1 /2) 2 + r/

)

(5.2.9)

is bounded in the same way as the similar sum in Proposition 4.22 of [19], by a term of order 0(T 2 ) for an " admissible" value of T, in the region (5.2.7) where s varies. Next, it is not hard to verify that exp ( -27Tik ( a + iT - 1 /2)/d ) 1 - exp ( -27Ti ( a + iT ))

= 0( 1 ) ' T-> 00

( -27Tik(l ; 1 /2)/d ) ( a + iT _ I2 ) � ( a exp = 0( 1 ) , + iT - 1 /2 ) - ( 1 - 1 /2 ) 2 f' ( 3 a - iT = O 1n T , ( ) r 2 ) ( a + iT - 21 ) I�00 ( a + iT - 1 /2 - I )(1 a + iT - 1 /2 + I ) = O��� ) , I 1 ) ( a + iT _ I2 ) � 2 = o( ! 2 1= 1

T-" oo

T-" 00

-


j= 1

(a +

�

iT - 1 /2 ) - ( aj - 1 /2)

T-"

We now consider the sum

( s - -21 )

�

P . f3 < �

[

1 2 ( s - 1 /2 ) - ( p

-

1 /2 ) 2 ( a - 1 /2) 2

'

1 -

1(5 .2.. 10)

( p - 1 /2 ) 2

By Theorem 4.4.2, it is bounded, like the sum (5.2.9), by a term of order 0(T 2 ) for " admissible" values of T. We shall not give the details of this estimate, since they are standard in analytic number theory. Finally, Theorem 3.5.5 gives us the estimate

( d'/d )( s; f; X ) = 0(T 2 ) for an admissible value of T, by analogy with the estimate for (5.2 . 10). Combining

these results leads us to the final estimate

(Z'/Z)( s; f ; X ) = 0( T 2 )

( 5 .2. 1 1 )

v. THE SELBERG ZETA-FUNCTION

90 a

the region (5.2.7) and an " admissible" value of T. Integrating (5.2.1 1) with respect to s along the line segment from a + iT to 2 + iT, we obtain for s =

+

iT in

r; X ) 1 = 0( T 2 ) , T � 00 , which implies the desired estimate (5.2.8). We have also used the easily verified fact I ln Z( s ;

that I ln Z ( 2 +

iT; r; X ) 1 = 0(1 ) , T � 00 , as follows from the definition (5. 1 . 1). To derive the lemma from (5.2.8) we use the Phragmen-Lindelof principle. This principle will be applied in the sector

'IT/4 � arg( s + 1 - i ) � 'IT/2. ( 5 .2 . 1 2) We study the asymptotic behavior of Z(s; r; X) on the rays which bound this sector. In the first place, it is obvious that on the segment arg( s + 1 - i) = 'IT/ 4 we have Z(s; r; X) = 0(1), since this ray extends into the region where Z(s ; r; X) is given by the absolutely convergent product (5. 1 .1). Next, we set s = - 1 + iT, T > 1 . To estimate Z(- 1 + iT; r; X ) we make use of the functional equation for this zeta-func­ tion (see Theorem 5.1.5). In the right side of (5. 1 .1 3) we take s = 2 - iT, and we estimate the behavior of each factor for large values of T. The following estimates are easily verified:

Z(2 - iT; r; X ) = 0( 1 ) , � (2 - iT; r; X )

(

i

=

0( 1 ) ,

)

2 iT exp -I F I dim V 3/ - t tan 'ITt dt = exp o

( 3/2 - iT

J0

[ exp( - 2'ITikt/d ) 1 (-2 'IT it ) + exp

( � 1 F I dim V · T ) + 0( 1 ) ,

exp(2'ITikt/d )

1

+ 1 dt = 0( 1 ) ' + exp (2 'IT it ) T� 00

T� oo

l r ( r( ! + iT ) r( i iTf ( ; x ) = 0( 1 ) . -

T--> 00

-

We have thereby found that the function

r; x ) exp ( 1 1 F I dim Vis ) is analytic in the interior of the sector (5.2.12) and admits the estimate there, Frs) = exp 0 ( 1 s 1 2 ) , F(s ) = Z( s ;

while on the boundary of the sector it admits the estimate F(s ) =

( 5 .2.13 ) §5.61), the

0( 1 ) .

Consequently, by the Phragmen-Lindelof theorem (see [56], Theorem estimate (5.2. 13) also holds everywhere inside the sector (5.2. 1 2); and this proves the claim Z( s ;

r ; X)

=

O ( exp 1 1m s I ) ,

and hence the lemma. The proof is complete.

-1

� Re s � 2 , 1m s

> 1,

§S . 2 . THE REMAINDER IN THE WEYL-SELBERG FORMULA

LEMMA 5.2.4.

arg Z o - + iT; f ; X )

91

= 0( 1 ) ,

where the value of arg is defined in Lemma 5.2. 1 . PROOF. The lemma follows from Lemma 5 .2.3, and also from a version of Jensen' s theorem (see [57], Chapter 9, §4) and the following lower bound, which is easily derived from (5. 1 . 1 ): Re Z( a + it ; f; X) � 1-

for any a � a o = a o (f; X) > 2 for some fixed a o . (In this connection, see also [ 1 9], Theorem 7.3.) The proof is complete. It is clear from Lemmas 5.2. 1 , 5 .2.2 and 5 .2.4 that we have already refined the Weyl-Selberg asymptotic formula (see Theorem 4.4. 1 ), by taking out another prin­ cipal term proportional to TIn T. However, the remainder term in this refined formula still has order O(T), by Lemma 5 .2.4. In order to increase the precision of the formula, we must find a better estimate for the order of the function arg Z(-t + iT). We now show that

l arg Z o - + iT; f ; x ) l « Tjln T. Lemmas 5.2.5-5.2. 1 1 are devoted to proving this assertion. They are formal generali­ zations of corresponding facts from the scalar spectral theory for a cocompact group f (see [44]) or for f E WC2 (see [68]). Here we shall focus our attention primarily on the differences in the proof, and in the parts of the proof which are the same we shall refer the reader to these articles (mainly, to the detailed paper [44]). We introduce some standard notation. Let

Se T ) =

! arg z ( � + iT; f ; X ) ,

where the value of arg is defined in Lemma 5 .2. 1 . Next,

l

S I ( T ) = TS(t) dt, qJ ( t )

=

o

max

l ';;; u';;; t

l SI ( U ) I .

We shall now let p denote an arbitrary zero of the Selberg zeta-function Z(s; f; X ). We set s = a + it, t > 1 0, 1 < v < 5 j4 is fixed. LEMMA 5.2.5.

fV In I Z( a + it ; f; X ) I da = O( t ) . 1�

t - oo

PROOF. Lemma 5 .2.3 and Lemma 1 of [57], Chapter 3, §9, imply that Z' 1 Z

(s; f ; X )

=

-

� s - p + OCt ) , p

t- 00

(5 .2 . l 4)

which hold.s for I a - v I ,,;:;; 1 . The sum on the right in (5.2. 1 4) is taken over all zeros p lying in the disc I p - v - it I ,,;:;; 2. If we integrate the left and right sides of this

v.

THE SELBERG ZETA-FUNCTION

equality over the line segment from a + it to a In Z ( a + it ; f; X ) - In Z ( v + it ; f ; X )

+ it, we obtain

(5.2.15)

= � [In( a + it - p ) - In( v + it - p )] + O ( t ) .

4.4.2, 5.1.3 (5.2.15), V

t � oo

p

From Theorems and Lemmas of zeros p over which we are summing in Integrating we obtain

f

I�

5.2.1, 5.2.2, 5.2.4 it follows that the number (5.2.14) is a number of order OCt) ( t (0) . �

I ln Z ( a + it; f; X ) I da

= O( t) , t � oo

from which the lemma follows. The proof is complete. The proofs of the next three lemmas repeat almost verbatim the proofs of Lemmas in they are based on Lemma Here we shall limit ourselves to the statements of these lemmas.

5 - 7 [44] ;

5.2.6. I S I ( T ) 1 « T. LEMMA 5.2.7. Suppose that -t

5.2.5.

LEMMA

<

a � v . Then

1 ( 2t )1 f s - "2 - iy S( y ) dy + 0( 1) . In Z ( s ; f ; X ) = i

LEMMA

5.2.8: For a > -t and 0

In Z ( s ; f ; x) = i f

t+�

t-�

�

<

g < t/2

t � oo

( s - 1 - iy )- 1 S( y) dy + 2

0 ( g - I 9J(2t )) + 0( 1 ) . t � 00

t�

00

The proof of the next lemma is different from that of the analogous assertion in the spectral theory for a cocompact group f ; hence, we shall give the proof here. LEMMA

5.2.9.

PROOF. We set

. ��eg ( 1 1 . f ., X ) - 21 f T Xlr ., f ,. X ) dr + ' 4 -T reg 2 (see the notation in Theorem 4.4.1 and in (3. 5 .2» . According to Theorem 3. 5 .6 and Lemmas 5.2.1, 5.2.2 and 5.2.4 , we have N( T ) = c I I T 2 + c 2 Tln T + c I 3 T + R ( T ) , l T R ( ) = 12S(T ) + 0( 1 ) , where C I I ' C I 2 and c I 3 do not depend on T. Theorem 3.5.5 implies that the function (

N( T ) - 2 N T 2 _

+

'IT

N(T ) is monotonically nondecreasing ; hence, N(T + x E (0, T). We have

x) - N(T ) � 0 for x � O.

R ( T + x ) - R ( T ) � CI4Tx, where C I 4 > 0 does not depend on T or x. The inequality rT+x R ( y ) dy � xR ( T ) - C l 4 Tx 2/2, J T

Let

§5 .3. THE DISTRIBUTION FUNCTION '1T( x; r)

is easily verified. From it we obtain R (T ) � 2 X - I ( SI ( T + x ) - S I (T ) ) + 0( x - I'5(2T)) + O(Tx) . Setting x

=

93

0(1) + C l 4 Txj2

= (T-I'5(2T))1/2, we obtain

S (T) « ( T'5(2T ) ) 1/2 . The function -S( T ) has a similar estimate, as we see by integrating interval [T - x, T]. This gives us the lemma. The proof is complete.

-

LEMMA 5.2. 1 0.

jl

/2 + ln

1 /2

1

tln I Z ( (1 +

it ; f ; X ) I d(1

=

over the

O( t l /4 '5 3/4 (4t ) In- 1/2t ) .

The proof is analogous to that of Lemma 9 in LEMMA 5.2. 1 1 . S(T )

R( y )

= O(Tjln T).

[44],

so we shall not give it here.

The proof is based on Lemmas 5.2.8-5.2.10 and on the Hadamard three circle theorem. The proof is analogous to that of Theorem 1 in [44], so we shall refer the reader to that paper. The reader has probably already noticed that at the concluding stage of our estimate for the function S( T)-the normalized argument of the zeta-function Z(-!- + iT; f ; x )-in going from a term of order O(T) to a term of order O(Tjln T) we used the fact that the Selberg zeta-function Z(s; f; X) satisfies an analog of the Riemann hypothesis " to the right", i.e., in the region Re s > -!- (modulo a finite number of real zeros; see Theorem 5. 1.3). We have achieved the purpose of this section. The refinement of the Weyl-Selberg formula is given in the following theorem. THEOREM 5 . 2. 1 .

(

2N T2 +

The following asymptotic formula holds:

! ; f ; x ) - 2� i� � ( � + ir ; f ; x ) dr =

I F I dim V T2 2 'IT

+ +

( OU/n.T) . ....

_

2k( f ; X ) TIn T 'IT

n 2 k(f; X - k( f X In 2 ; ) ) - a� I 'IT

T

oo

t= E+ l In 1 1 - exp 2 'lTi01a I ) T

PROOF. The theorem is the combination of Lemmas

dim V

5.2. 1, 5.2.2,

and

5.2. 1 1 .

The

proof is complete.

§5.3. Asymptotic formula for the distribution function

'IT ( x ; f )

The purpose of this section i s to derive an asymptotic formula for the distribution function 'IT ( x; f) for the values of the norms of primitive hyperbolic conjugacy classes in f (see the definition (4.4. 1)) as x 00 ; this asymptotic formula will refine (4.4.2). The formula can be regarded as a geometric application of the theory of the ->

v. THE SELBERG ZETA-FUNCTION

94

Selberg zeta-function which we developed in §§5 . 1 and 5 .2, thanks to the well-known interpretation of the numbers In in the theory of Riemann surfaces (see (4.4. 1 )). We also note that our asymptotic formula for 7T( X; f) is analogous to a refined version of the· prime number theorem, and is connected with the Selberg zeta-function is the same way as that asymptotic law for prime numbers is connected with the Riemann zeta-function. The analogous formula in the theory of the Selberg zeta-function Z(s; f; X) for f E m l is well known, and apparently is due to Selberg and, independently, to Huber [2 1 ]. There are published proofs in the papers of Huber [2 1 ] and Hejhal [ 1 9]. The formula for groups f E m 2 was announced by A. I. Vinogradov and the author in the note [7 1 ] (see also [66]). Throughout §5.3 we shall assume that the representation X of the group f is chosen to be one-dimensional and trivial, i.e., X(y) 1 for any element y E f. We shall denote this representation X by 1 . We also introduce the following notation: {s) is the set of all zeros of the Selberg zeta-function Z(s; f; 1 ) in the interval ( - L 1 ], These zeros correspond to counting multiplicity (see Theorem 5 . 1 .3), j 1 , the eigenvalues Aj Sj( 1 Sj) of the operator � (f; 1 ) for eigenfunctions in the subspace 8 1 (f; 1 ) and, perhaps, in the subspace :JCo( f; 1) C :JC(f; 1 ). We shall assume that these zeros are ordered as follows: . . . ;;", Sj _ 1 ;;", Sj ;;", Sj+ 1 ;;", Following Roelcke, we now note that S I 1 and this is a simple zero (see [46], Theorem 5 . 1). We give the theorem whose proof is the subject of this section.

N(P)

=

=

=

-

. . . ,N.

=

THEOREM 5.3. 1 .

•

•

•

•

The following asymptotic formula holds:

7T{X; f )

�

= li x + j=�2 li{ x Sj ) + o{x7/8+S(1n xf l ) ,

where 8 > 0 is an arbitrary fixed number, and Ii x is the logarithm integral: Ii x

=

x

12 -1n1-t dt.

PROOF. We consider the Dirichlet series

GD { S ; f)

In N{ �: = {P}r � � ' k= 1 N{P)

( 5 .3 . 1

)

where the summation is over all primitive hyperbolic conjugacy classes {Ph in the group f. From Lemma 4.4. 1 it follows that this series converges absolutely in the region Re S > 1 . We use (5. 1 .6) to study the series. We have

Z' { s

z:

; f; 1)

1 00

- GD {s; f ) = � �

{Ph k= i

N{ p ) N{P)

In

ks

[

1

1

- N{P)

-k

1

-1 .

( 5 .3 .2)

It again follows from Lemma 4.4. 1 that the series on the right in (5.3.2) converges absolutely in the half-plane Re S > O. Because of this and Theorem 5 . 1 .2, (5.3.2) gives a meromorphic continuation of the series GD(s; f) in (5.3. 1) to the half-plane Re S > o. We keep the same designation GD(s; f) for this meromorphic function. The estimate (5.2. 1 1) for the logarithmic derivative of the Selberg zeta-function now implies that (5 .3 .3)

§S . 3 . THE DISTRIBUTION FUNCTION 1T ( X ; r )

everywhere in the region

1- +

E :O::::;; Re s � 2,

1 1m s I � 1 ,

95

(5 . 3 .4)

where E > 0 is an arbitrary fixed number. Since the estimate 6j)(s; f) = 00 ) holds for (ij) (S; f ) in the half-plane Re s � 2 (this follows from the definitiion (5.3. 1 » , it follows by the Phragmen-Lindelof principle that the estimate (5.3.3) in the region (5.3.4) can be improved as follows: (5 .3 .5 ) I Gj) ( s ; f) I « 1 1m s 1 4 rnax[ O, I + e - Re s j e

(see [ 1 9], Theorem 3 . 1 0). We introduce the following functions: 8( x ; f ) =

l . _

In N(P ) ,

� { Ph N( P ) � x

the summation is only taken over primitive conjugacy classes ( 8( x, f ) is the analog of the Tchebycheff psi-function). Using (5.3.5) and a standard device from analytic number theory (a discontinuous Dirichlet integral), we obtain estimate 8 1 (x; f ) as x -> 00 . Namely, 1 x s+ 1 ( 5 . 3 .6) 6V ( s ; f ) ds . 81( x ; f ) = -. 2 1T 1 Re = 2 S ( S + 1 )

f

s

Moving the path of integration in (5.3.6) to the left and using (5.3.5), ( 5. 1 . 2) and Cauchy' s theorem, we readily obtain 0L X l +SJ x2 + 0( X 7/4 H ) 81( x; f) = 2 + � l s + s J )=2 ) x --> oo

with arbitrary fixed 8 > O. This implies the asymptotic formula 0L. X Sj 8 ( x ; f ) = x + � - + 0( X 7/8 H ) . )= 2

Finally, if we use the equality 1T

( x ., f ) =

s}·

x --> oo

lX d8 ( t ; f ) 2

In t

'

in the same way as in Theorem 3. 1 4 of [ 1 9], we obtain the theorem. The proof is complete.

CHAPTER 6

SECOND REFINEMENT OF THE EXPANSION THEOREM FOR m ( f ; X ) . THE DISCRETE SPECTRUM §6.1 . Problems in the theory of the discrete spectrum. The spectrum of m ( f ; 1 ) for congruence-subgroups f

This section can be regarded as an introduction to Chapter 6, and to some extent Chapter 7. Vie begin by formulating the problems in the theory to which Chapter 6 and part of Chapter 7 are devoted; we shall base this on our survey article [66]. We then prove two theorems which illustrate a remark of Selberg in his lectures [50] concerning the discrete spectrum of the operator m(f; 1) for certain arithmetic groups f. We begin this section with a small refinement of the spectral decomposition (3.3.5). We recall (see §3.3 for the notation) that

:Je ( f ; X )

= :Jeo ( f ; X ) E9 8 1 ( f ; X ) E9 82 ( f ; X ) .

We consider the subspace %o(f; X ) of cusp-vector-functions. In this space we look at the subspace %ol(f; X ) c :Jeo(f; X) spanned by all the eigenfunctions of the discrete spectrum of m(f; X) which lie in :Jeo(f; X) and correspond to eigenvalues Aj with 0 � Aj < t . From Theorem 2.2.4 it follows that the subspace :Jeol(f; X) is finite dimensionaL Thus, all eigenfunctions of the discrete spectrum of m(f; X) whose eigenvalues lie outside the continuous spectrum, i.e., in the interval [ 0, t), and only such eigenfunctions, are in the subspace 8 1(f; X) EB :Jeo l (f; X). The subspace :Je02(f; X) = :Jeo(f; X) e %ol(f; X ) is spanned by all eigenfunctions of the discrete spectrum of m(f; X) whose eigenvalues lie in the continuous spectrum, i.e., on the serniaxis [ t , (0 ) . The refined spectral decomposition (3.3.5) has the form As indicated in the title of this chapter, the basic object of study here will be the discrete spectrum of m(f; X ). We suppose that f E we 2 and X E 9C sC f ) (see § 1 .2). It is under this assumption that the decompositions (3.3.5) and (6. 1 . 1 ) are conceivable, since otherwise, i.e., for r E we I or for f E we 2 but X E 9C ref), the operator m ( f; X) has a purely discrete spectrum, which is the spectrum neither of cusp-func­ tions nor of incomplete theta-series. For f E we I the spectrum of m( f ; X) has been thoroughly studied (see [66]). In this theory even simple ideas from the theory of selfadjoint operators may be useful. This is roughly the situation with the spectrum of m(f; X) if f E we 2 and X E 9C rC f). 97

98

VI . SECOND REFINEMENT O F THE EXPANSION THEOREM

In sharp contrast to this is the situation in the theory which studies the discrete spectrum of �(f; X ) for an arbitrary group f E m 2 and an arbitrary representation X E 91 sC f), for example, even if we are only interested in nontriviality of the subspaces in (6. 1 . 1). The general theorem on expansion in eigenfunctions of �(f; X ) (Theorem 2.3.7) has guaranteed us the existence of a continuous spectrum of multiplicity kef; X ) for arbitrary f E illC 2 and X E 91 sC f). Hence, the subspace 82(f; X ) is always nontrivial, i.e., does not consist only of zero. However, there are no general spectral theorems known at the present time which would guarantee, in general, that each of the other subspaces of the discrete spectrum in the decomposi­ tion (6. 1 . 1 ) is nontrivial. An exception is 8 1 (f; X) for a general group f E illC 2 and the trivial representation X = 1 (for the definition of the trivial representation, see §5.3); this subspace contains the normalized eigenfunction (v'fFT r l of the operator �(f; 1 ), which corresponds to the eigenvalue X = O. This function is also the residue of the corresponding Eisenstein series E( z ; s; a; 1 ; f; 1 ) at the point s = 1 (see [46], Theorem 5 . 1). For general f and X, the subspaces Xo I (f; X ) and Xolf; X ) are more mysterious. In this chapter, and a little in §7. 1 , we shall familiarize the reader with what we believe to be the basic methods currently known for investigating the discrete spectrum of �(f; X ), methods which at the same time enable one to prove infinite dimensionality of the entire subspace of the discrete spectrum

and thereby to guarantee the nontriviality and infinite dimensionality of the sub­ space Xo2(f; X). (The method developed in §7. 1 is local, and enables one to study the subspace XO I + 8 1 .) It should be mentioned that, as a rule, these methods require the use of additional information, and so are primarily suitable for special groups f and representations X. We shall illustrate some difficulties which stand in the way of proving nontriviality of the subspace of cusp-vector-functions for the general group f. These difficulties are similar in nature to those which obstruct the investigation of the zeros of the Riemann zeta-function in the "critical strip". It was Selberg and Roelcke who first called attention to the difficulties of studying the discrete spectrum of �(f; X ) in the scalar theory (dim V = 1 and X = 1 ). In his lectures [50], Selberg indicated that the formula of which a more general version is now what we call the Weyl-Selberg formula (see Theorem 4.4. 1 ) will not, in general, yield any information about the asymptotic behavior of the function N(X; f; 1 ) as X 00 , except in certain cases of special so-called arithmetic groups f, for which one can explicitly compute the function �(s; f; 1 ) in terms of the Riemann zeta­ function and other special functions of analytic number theory. Below we shall examine these cases in more detail. In [45], Roelcke considered other examples of groups (the family of Hecke groups fd (see §6.3» , and for each of them he proved that �

N( X ; fd ; 1 )

�

A ---+ oo

00 ,

( 6 . 1 .2)

which implies that the subspace X02(fd; 1) is infinite dimensional. Since it was Roelcke who conjectured that the assertion (6. 1 .2) holds in a more general situation

99

§6 . 1 . THEORY OF THE DISCRETE SPECTRUM

as well, we shall call the following assertion for f E W1 2 and X E 9( sC f) the Roelcke

conjecture:

N( "A ; f ; X )

(6.1 .3)

� 00 . ,\ � 00

Recall that N("A; f; X) is the distribution function for the eigenvalues of the discrete spectrum of m(f; X). We shall now attempt to reconstruct the proof of the above remark of Selberg concerning the asymptotic behavior of N("A ; f; 1) as "A � 00 for discrete arithmetic groups (Selberg's proof has not been published, and is not known to us). We give some standard definitions. By the modular group f we mean the discrete group f = PSL(2, Z) of transformations of i-I; where Z is, as usual, the ring of rational integers. It is well known that f E W1 2 ' and it contains only one primitive parabolic conjugacy class. Let fl and f2 be two subgroups of G = PSL(2, Ill ) . We shall call fl and f2 commensurable in the broad sense if there exists g E G such that the intersection z

z

z

(6.1.4)

has finite index in gf1 g- 1 and in f2 ' and we shall call them commensurable in the narrow sense if fl and f2 are commensurable in the broad sense and we can take g in

(6.1.4) to be the identity E of G. A group which is commensurable in the broad sense

with f will be called an arithmetic group f C G with noncompact fundamental domain F in H. It is easy to see that f E W1 2 • We let W1 2, a denote the subset of W1 2 consisting of all arithmetic groups f. For an arbitrary group f E W1 2 a' it is very problematic to compute the function �(s; f; 1) (see the definition in §3.5) in terms of the standard functions of analytic number theory. However, it is relatively simple to do this for f and for certain other arithmetic groups which are subgroups of f An example is the congruence­ subgroups fo(m), f\(m) and fim), which are defined by the following formulas: fo( m ) = { ( �: �) E f , c _ O (mod m ) } , 7L

7L .

z

z

f\ (m) = { ( � : � ) E fo(m), a l , d - l (mod m ) } , f2 ( m ) { ( � : �) E f\( m), b O (mod m ) } ,

(6.1.5)

=

where m is a natural number. The symbol denotes congruence of integers. (In (6.1.5) we allowed ourselves a minor and obvious notational inaccuracy, writing cosets in the form of matrices.) The group fi m) is called the principal congruence­ subgroup of level m. All of these groups play an important role in the arithmetic theory of automorphic forms. For f = f we have the following well-known formula for �(s; f; 1), given by Selberg in his lectures [50]: C f(s - 1/2)t(2s - 1 ) (6 . l . 6) � ( s; f ; 1) = ® ( s; f ; 1) = V 7T f(S)t(2s) , =

z

where t(s) is the Riemann zeta-function, and f(s) is Euler's gamma-function (this formula had been known before, but in the theory of the Epstein zeta-function). The

100

V I . SECOND REFINEMENT O F THE EXPANSION THEOREM

3.1.2

formula can be derived directly from Theorem by means of well-known formulas for summing arithmetic Dirichlet series (for a similar computation, see [28], (6. 1 .5) is odd and Chapter V). Next, we suppose that the natural number squarefree, � and that the representation X is trivial (dim V = 1 ). Formulas for Ll( ; f; 1) in the cases when f is one of the groups fo( ), fl ( ) or fi ) were found by Hejhal. Part of his results were published in § 10 of [ 1 8]. One has the formula

s

m 3,

m

Ll (S ; fo (m) ; 1 ) where

=

m

I/2)r(2s ) ( ;:;; f(s -f(s)r(2s)

- 1 ) n( m )

m

m

II(s, m),

( 6 . 1 .7)

2 ) p p I , P a (s) P - 1 ( P - P 1 - , p - I , ® denotes the tensor product of matrices, det is the determinant, m = P . . Pr is the prime factorization, and n(m) = 2 r is the number of primitive parabolic conjugacy classes of fo( m ). p

=

2s

1

s

s

I -- S

I

·

Using (6. 1 .6), (6. 1 .7) and Theorem 5 .2. 1 , we shall prove the following theorem.

Let f be any of the groups f or fo( m), where m � 3 is odd and squarefree. Then the distribution function for the eigenvalues of the discrete spectrum of � ( f; 1) satisfies the following refined Weyl-Selberg formula: N( T 2 ! ; f; 1 ) = 1;:'1 T 2 + c ( f )T ln T + c2 (f)T + o ( I:T ) . (6 . 1 .8) T-+ oo PROOF. It suffices to find the asymptotic behavior as T 00 of the following z

THEOREM 6. 1 . 1 .

+

l

�

integral in the refined Weyl-Selberg formula (see Theorem 5 .2. 1):

f-TT A U : + ir; f ; 1 ) dr. L1' Ll

Ll(s;

( 6 . l .9 )

If f is one of the specified groups, then f; 1 ) is given by the explicit formulas (6. 1 .6) and (6. 1 .7). From these formulas it is clear that finding the asymptotic behavior of (6. 1 .9) reduces to finding the asymptotic behavior of the argument of the and estimating the argument of the Riemann f-functions and fet + zeta-function r(1 + This information is well known to the degree of precision we need (see [57], Chapter 9). We hence find that the integral (6. 1 .9) is equal to

f(iT) c4

iT).

iT)

c3 ( f)T ln T + c4 (f)T + OT-+ On T) , oo

( 6 . l . 1O)

where c3 and depend only on the group f. Substituting (6. 1 . 1 0) in Theorem 5 .2. 1 and combining similar terms, we obtain the theorem. The proof is complete. The description of the determinant of the scattering matrix for the groups f l ) and f2 (m) is very complicated, and we shall not give it here even schematically (as Hejhal points out on p. 477 of [ 1 8], the detailed computation of the matrix f; 1) in terms of "elementary" functions of number theory would require about 250 pages of text for f = and f = f2( m ); hence, detailed proofs have not yet

® (s;

(m

f\(m)

§6. 1 . THEORY OF THE DISCRETE SPECTRUM

101

been published). Without hoping to obtain the best possible asymptotic formula for N(A; f; 1 ) in these cases, we shall derive the Weyl formula from our Theorem 4.4. 1 , making use of the equalities ( 10.5) and (10.7) in [ 1 8]. From these equalities it is not hard to obtain the estimate /1 ' ( 1 j oo h ( 1 + r 2 ., t ) 7l . ) + lr,. f ., 1 dr - 0 -00

_

4

2

( /i ) In t

t

.....

(6 . 1 . 1 1 )

'

O

2 where the function h(-l + r ; t) was defined in the proof of Theorem 4.4. 1 , and f is any of the groups f1(m) or f2(m ) (with certrun conditions on m; see below). THEOREM 6. 1 .2. Suppose that r is any of the groups

f1(m) or f2(m), and m ;;;:. 3 is

odd and squarefree. The distribution function for the eigenvalues of the discrete spectrum of � ( f; 1 ) satisfies the Weylformula N( A ; f; 1 ) -- I F 1 A/4'1T . >,

.....

00

We modify the proof of the general Theorem 4.4. 1 by adding (6. 1 . 1 1) to (4.4.4). From (4.4.5) and (4.4.6) we have /i oo (exp - tA ) dN( A ; f; 1 ) = (I F I/4 '1T )( 1 /t ) + O ln t/ , PROOF.

i

o

,

'

(

t

.....

O

)

and the theorem now follows from a Tauberian theorem. The proof is complete. The results in Theorems 6. 1 . 1 and 6. 1 .2 should be regarded as exceptionally good if viewed from the standpoint of the spectral theory of �(f; X) for general Fuchsian groups and from the standpoint of Roelcke's conjecture. They can be obtained thanks to the following properties of the function /1(s; f; 1), which hold for all of the groups f = fz , fo(m), f1(m) and f2(m), m ;;;:' 3 odd and squarefree: 1 ) In /1(s; f; 1 ) is defined for Re s > ! by an absolutely convergent Dirichlet senes; 2) /1(s; f; 1 ) is a meromorphic function of order one. We would now like to illustrate the difficulties which stand in the way of extending the above method for investigating the asymptotic behavior of N( A; f; 1) to more general groups f E WC 2• For an arbitrary arithmetic group f E WC 2, a, the verification of conditions 1 ) and 2) above for /1(s; f; 1 ) involves a difficult and as yet uncompleted computation, which does not seem to be justified by the final result, if we view the computation from the standpoint of proving Roelcke's conjecture. Using another method, in §6.4 we shall easily prove that Roelcke's conjecture holds for an arbitrary arithmetic group f E WC2, a , i.e., N( A; f; 1) � 00 as A � 00 (we are now concerned with the trivial representation X E 91 sCf)); however, that result will be weaker than the Weyl formula. In the case of an arbitrary nonarithmetic group f E WC 2 we do not have any hope whatsoever for verifying conditions 1) and 2) for the function /1( s ; f ; 1) by means of a direct computation, as was done for the congruence-subgroups. The reason is that, unlike in the arithmetic case, the only way known for defining f is by a system of generators and relations (see Fricke's Theorem 1 .2. 1). In practice, this way is unsuitable for computing the matrix entries for an arbitrary transformation y E f,

102

VI. SECOND REFINEMENT OF THE EXPANSION THEOREM

and It IS those matrix entries which occur in the definition of de s ; r; 1 ) by a Dirichlet series for Re s > 1 (see Theorem 3 . 1 .2). Even in the simplest case of the nonarithmetic Hecke group (see §6.3) the constructive description of this set of matrix entries is a well-known unsolved problem in the theory of Fuchsian groups. (Here the matrix entries are expressed in terms of certain continued fractions, but it is as yet unclear how useful such information is for the spectral theory.) We now proceed to other methods for studying the discrete spectrum of the operator � ( r; X).

El e ments of Arti n theory i n the spectral theory of automorphic functions, and Roelcke's conjecture

§6.2.

The following formula is well known in algebraic number theory:

m\ �K (S) = �k(S ) II l L(s ; X )di

(6 .2 . 1 ) X� it relates the zeta-function of an arbitrary algebraic number field k with that of a relative Galois extension K of finite degree. The formula plays an important role in class field theory. The formula (6.2. 1) was proved for an abelian Galois group by Takagi in [55], and for an arbitrary Galois group it was proved by Artin in [2]. In this section we derive a formula for the Selberg zeta-function of a compact Riemann surface which should be regarded as a transcendental analog of (6.2. 1 ). The ground field in oui situation corresponds to an arbitrary normal subgroup of finite index in the fundamental group of an arbitrary compact Riemann surface of genus no less than two. This formula is obtained as a consequence of a more general spectral theory for the resolvent of � ( r; X) which holds for an arbitrary group r E [)C and representation X E 9C ( r). Another consequence of the theory is the confirmation of Roelcke's conjecture (see §6. 1). Here we prove that every group r E [)C 2 has a subgroup of finite index rl C r such that N( A ; rl ; 1) � 00 as A � 00 ; X = 1 , as before, is the trivial representation of r. In addition, we give a lower bound for N( A; r; 1) in any finite interval [0, A ] of sufficient length. All of these results were first published in [67]. We shall use the earlier notation for the objects connected with the spectral decomposition of the operator � ( r; X), r E [)C , X E 9C ( r). Let r E [)C . We give a definition. A subgroup rl C r will be called regular if rl is a normal subgroup of finite index in r, i.e., in particular, [ r : rd < 00 . Let rl be a regular subgroup of r E [)C ; we let ( rl\ r)* denote the set of all finite-dimensional irreducible unitary representations X of the quotient group rl\ r. It is obvious that every element X E ( rl\ r)* can be regarded as a representation X E 9C ( r) whose kernel contains rl '

For an arbitrary group f E [)C and an arbitrary regular subgroup rl the following formula holds for the kernels of the resolvents of the operators m ( r; x ) and m ( rl ; 1 ) : l [ r : fl r 2: [ tr V( X ) r ( z , z ' ; s ; r ; x ) ] dim V( x ) = r ( z ; z ' ; s ; r \ ; I ) , (6 .2 .2) THEOREM 6.2. 1 .

x E [ rl\r]*

where VeX) is the space of the representation X E 9C ( r ), and tr V( x ) is the trace of an operator in VeX) .

dim VeX)

is its dimension,

§6 .2. ELEMENTS OF ARTIN THEORY

103

PROOF. It suffices to prove (6.2.2) in the half-plane Re s > 1 , since the equality will then extend onto all of C by meromorphic continuation. Thus, suppose that s is fixed, Re s > I . From § 1 .4 we know the following general formula:

r( z, z' ; s ; f ; X ) = � X( y)k (z, yz' ; s ) , yEf

which holds for any f E 9)( and X E 9C(f), and in which the senes converges absolutely for z , z' E H in general position. We have

r( z , z'; s; f ; X ) = �

� X( YIY2 )k ( z ' Y I Y2Z' ; S ) .

y E fl Y2 E I':i\ f

( 6 .2 .3 )

By the definition of X E

( fl\f)* , we have X( Y I Y2 ) X ( Y I ) X( Y2 ) = X ( Y2 ) , where YI E fl and Y2 E f2 . From (6.2.3) we obtain l [ f : fl r � [ tr r ( z , z' ; s; f ; x ) ] dim V(X ) =

x E ( fl\ f )*

= [ f : fl r l =

V( X )

'

� [ tr

�

x E ( fl \ f ) * Y2 E fl\ f

V( X )

X( Y2 )] dim V( x ) � k(z' Y I Y2Z' ; S ) y E fl

� k(z' Y lz' ; s ) = r(z, z' ; s; f l ; l)

yl E fl

as was to be proved. We have used the orthogonality and completeness formula for characters of irreducible representations, which is well known from the representa­ tion theory of finite groups (see [6], §3 1 ). In our context this formula has the form

The proof is complete. For f E 9)( and X E 9C(f) we let n/f; X) denote the multiplicity of the eigen­ value A)f; X) of the discrete spectrum of m:(f; X). THEOREM 6.2.2.

If the hypothesis of Theorem

6.2. 1

is fulfilled, then the following

assertions are true: 1 ) For any eigenvalue �\(f; X ) ofm:(f; X) the multiplicity satisfies the formula � n/f; x ) dim V( x ) = n/fl ; 1 ) . x E ( fl\f)*

2)

The equality

�

X E ( fl\ f)*

N( A ; f; x ) dim V(X ) = N( A ; fl ; 1 )

holds, where N(A; f; X) is the distribution function for the eigenvalues A/f; X ) of m:(f; x ), if f E 9)( 1 or f E 9)( 2 and X E 9Clf), and it is the distribution function for the eigenvalues A/f; X) of the discrete spectrum if f E 9)( 2 and X E 9C sef). PROOF. It. suffices to prove 1 ), since 2) is obtained from the formula in 1) by summing over all Aj(f; X) for which Aj � A.

104

V I . SECOND REFINEMENT OF THE EXPANSION THEOREM

We prove part 1 ). We make use of the general formula (2.2.26), which describes the behavior of the kernel of the resolvent in a neighborhood of the eigenvalue A/ f; X), and compare the residues of the left and right sides of (6.2.2). (\Ve derived (2.2.26) under the assumption that f E we 2 ; however, it also holds for f E We ) . ) We have

n/ f, :

I)

�

v ( z ; k ; j ; f , ; 1 ) v ( z ' ; k ; j ; f) ; 1 ) ,

k= 1

( 6.2.4 )

where { v ( z ; k; j; f; X)} , k = 1 , . . . , n)f; X), is a real orthonormal eigenbasis for W ( f; X) in the finite-dimensional subspace of X(f; X) corresponding to the eigen­ value A)f; X). If we set z = z'; integrate with respect to z in a fundamental domain F, for the group f) in H, use the equality

( 6.2.5) where, as before, F is a fundamental domain for f in H, and, finally, use the orthonormality of the eigenfunctions, then we obtain part 1 ) of the theorem. The proof is complete.

6.2.3 (analog of the Artin-Takagi formula). Let f be the fundamental group of an arbitrary compact Riemann surface of genus greater than or equal to two, and let fl be an arbitrary regular subgroup. Then the Selberg zeta-function satisfies the following formula: Z (s ', f " . 1 ) = II Z ( s ; f; X )dim V(X). ( 6 .2.6) THEOREM

PROOF. By hypothesis, f E we , is a cocompact Fuchsian group of the first kind without elliptic elements (a strictly hyperbolic group), and fj is a group with the same properties. The Selberg trace formula for the group f and for X E W(f) has the form (see Theorem 4.3.6)

(

� h 41 J

+

)

�/ ( f; X ) =

I F I dim V( X )

27T

f 00 r (tanh 7Tr ) h ( 1 + r 2 ) dr 4

_oo

k ( P )) In N(P) (X (x) V k g ( k ln N( P)) , k / / 2 2 N N P ) P) ( ( {Ph k = 1 ( 6 .2.7) where the function h(l + r2) satisfies the hypothesis of Theorem 4.4.4. For the other notation, see Theorem 4.3.6; 0 = 0(f; X) and Aj = 1 + 02• We multiply the left and right sides of (6.2.7) by dim VeX) and sum over X E ( f,\f)*. By part 1 ) of Theorem 6.2.2, we have � � h + 02(f ; x ) dim V( x ) = 2: h + 02( f, ; 1 ) . (6 .2 .8) +2

x E ( f, \ n* j

(!

� £.J

"" £.J

00 tr ,

)

j

(!

)

105

§6.2. ELEMENTS OF ARTIN THEORY

Next, by the well-known Burnside theorem from the representation theory of finite groups, we have

�

x E ( rl\ r ) *

2 v( X ) = [ f : f I ]

dim

( 6 .2 . 9 )

•

Finally, (6.2.5) implies that

( 6 .2 . 1 0)

Taking (6.2.8)-(6.2. 10) into account, we obtain the following formula from (6.2.7) after we sum in the way indicated above:

7 h ( ! + 1j2 ( r, ; 1 » ) - 1 ;� I .cr(tanh "r)h(! + r 2 ) dr �

x E ( rl\ f ) *

dim Ve X ) ' 2

00

� �

{ Ph k = i

tr V(x) (X k ( p ) ) In N(P ) k/2 -k/2 g ( k ln N( P)) .

N( P )

- N(P )

(6 .2 . 1 1 )

On the other hand, by the Selberg trace formula (6.2.7) for the group f = fl and the representation X = 1 , it follows that the left side of (6.2. 1 1) is equal to

( 6 .2 . 1 2) where {Pdr] runs through the set of all primitive hyperbolic conjugacy classes in f l ' If we now recall (5. 1 .5), we obtain the following equality from (6.2. l 1) and (6.2. l 2) :

Z' a f 1 ( ; l; ) -Z 1 /2 1 Z' . � dIm Ve X ) s 1 /2 -z (s; f ; X ) - a x E ( �\r y

Z' (s; f 1 l; ) - a -Z s - 1 /2 1

where, as in (5. 1 . 1 ),

1

_

[

_

00

Z (s; f ; X ) = II II

{Ph k= O

1 _

1

Z' ( a f X ) ; ; , -Z 1 /2 ( 6 .2. 1 3 )

det v ( I v - x ( P )N( P fs - k ) ,

Re s > 1 , a > 1 and a > Re s. If we let a approach infinity in (6.2. 1 3), integrate and then exponentiate the resulting equality, we obtain the desired formula (6.2.6). The proof is complete. We now consider the RoeIcke conjecture for f E we 2 ' First we recall some notation from § 1 .2. By Fricke' s theorem (Theorem 1 .2. 1 ), the group f E 9)( 2 is given by a system of generators. A I ' B I , · . . , A g , Bg are hyperbolic generators, R I ' . . . , R I are elliptic generators, S I ' . . . , Sr are parabolic generators,

106

VI . SECOND REFINEMENT O F THE EXPANSION THEOREM

and relations:

R'{'l = E, ( 6.2 . 14)

R[' = E,

E is the identity of f. Such a group has signature (g; m l , . . . , m{ ; r), f(g ; m l , . . . , m{ ; r). We have r =1= 0 for a group f E W1 2 .

where

f=

We prove a lemma.

LEMMA 6.2.1 (the Weyl formula). Suppose that f E W1 2 is an arbitrary group and X E 9C ref) is an arbitrary regular representation. Then the distribution function for the

eigenvalues of the spectrum of�(f ; X) satisfies thefollowingformula: N( A ; f ; X )

I F l dim V �

A ---> oo

4 'IT

A.

PROOF. The lemma is a special case of Theorem 4.4.1 . For f and X as in the lemma, the spectrum of �(f; X ) is purely discrete. In this case the integral of the logarithmic derivative of the scattering matrix is absent from the Weyl-Selberg formula (4.4.3). The proof is complete. Let fl be a regular subgroup of the group f E W1 2. We denote by ( fl\ f)ieg the set of all regular representations X in ( fl\ f)*. (Recall that there is an imbedding ( f l \ f)* C 9C (f).) THEOREM 6.2.4. Let fl

be any regular subgroup of any group f E W1 2. Then

N( A ; f I ;

1) �

for any sufficiently large A > O.

(

�

x E C rI \ r)*reg

2 dim V( X )

) 4'IT 1 fFI: Ifl ] A

PROOF. In part 2) of Theorem 6.2.2 (in the left side of the equality) we shall count only the regular representations. We have

N( A ; f ; x )dim Ve X ) .

( 6.2.15 )

For each distribution N( A; f; X ) with X E ( fl\ f)ieg we have Lemma 6.2. 1, i.e., the Weyl asymptotic formula; hence, for large values of A the inequality (6.2.1 5) can be continued as follows:

If we take (6.2. 10) into account, we obtain the theorem. The proof is complete. Thus, Theorem 6.2.4 gives a positive resolution of Roelcke' s conjecture for any regular subgroup f l of an arbitrary group f E W1 2 and the trivial representation X, as soon as we know that the set ( fl\ f)ieg is nonempty. The next theorem gives us information about the set ( fl\ f)ieg . Its proof is based upon the theorem of Fricke we cited before.

107

§6 .2. ELEMENTS OF ARTIN THEORY

6.2.5. 1) For any group r E 9)( 2 there exists a regular subgroup fl such that the set (fl\f)ieg is nonempty. 2) If f = f(g; m l , m,; r) is any group in W1 2 which is not of the form f(g; 0; 1 ), g ;:;. 1 , then there exists a regular subgroup fl C f with abelian quotient fl\f such that the set (fl \f)ieg is nonempty. THEOREM

•

•

•

,

PROOF. The plan of proof is as follows. First, for any group of the form f = f(g; 0; 1 ) E W1 2 , g ;:;. 1 , we construct a regular subgroup fl for which (fl\ f)ieg turns out to be nonempty. Then we prove part 2), thereby completing the proof of the theorem. ' Thus, suppose that f = f(g; 0; 1 ) E W1 2 , i ;:;. i . By assumption, f is generated by 2g hyperbolic generators A I ' B I , . . . ,Ag, Bg and one parabolic generator. It has one relation between the commutators:

(6.2. 16 ) The relation (6.2. 16) shows how f is distinguished from other groups in the set of all f(g; m l , ,m , ; r) E W1 2-namely, it does not have one-dimensional singular rep­ resentations X E 9C Cf). To prove the theorem for f(g; 0; 1 ) we construct the s following special two-dimensional representation X E 9C ( f). Let Z( g) denote the Gaussian ring of integers in O(g). We consider the following finite group, which is the intersection of a discrete group and a compact group : •

•

•

r

D = SL(2, Z({-l))

n

SU (2 ) .

Obviously, D is not abelian. We define the desired representation as follows on the generators of f:

Al X : BI

(� 0. ) , (� �), ( -6 -� ) ,

�

�

S�

-I

(0 �), (6 �),

A}. � 1 Bj �

(6.2. 17)

j = 2 , . . . ,g.

These matrices for the representation X clearly lie in of the representation is correct, since we have

X : [ A I ' B I ] = A I B I A ) IB ) I

�

D.

We note that the definition

( -6 _� ) .

From the definition (6.2. 17) it follows that the representation X is regular. We now consider the kernel fl of X in f. This is obviously a normal subgroup, and the quotient f l \f is finite and isomorphic to D ; this proves the theorem for the group f f(g; 0 ; 1), g ;:;. 1 . We now prove part 2) of the theorem. We divide the proof into three steps. First suppose that the group f = f(g; m l , ,m ,; r) E W1 2 has an even number of parabolic generators, i.e., r 2k, k E Z, k ;:;. 1 , and otherwise is arbitrary ; in other words, it may or may not have hyperbolic generators, and may or may not have elliptic generators. We shall denote the generators of f f(g ; m l , , m ,; 2k) in th� same way as in Fricke's theorem (see, in particular, (6.2.14)). We construct the

=

=

•

.

•

=

•

•

•

108

V I . SECOND REFINEMENT O F THE EXPANSION THEOREM

following special one-dimensional representation X E 9( ref), defining it on the generators of f:

{

j = l , . . . ,g , j = I , . . . , I, j = 1 , . . . , 2k

Aj -> 1 , Bj -> 1 , X : Rj -> l , 8; -> -1 ,

( 6 .2. 1 8 )

(if g = 0 or 1 = 0, then one must remove the corresponding row in (6.2. 1 8)). The correctness of the definition is easily verified using (6.2. 14), and the regularity of X is clear from the definition. Let f l be the kernel of X. The subgroup fl is regular, having index two, and the set (fl\f)ieg is nonempty. Now suppose that r is odd, r = 2k + 1 , k E lL , k � 0, and the group f contains at least one elliptic element R I , i.e., f = f( g; m l , . . . , m ,; 2k + 1 ), 1 � 1 . In this case the desired representation X has the form

Aj

->

1 , Bj

R}. -> 1 ' X : S.} -> -1 '

->

1,

exp ( 2'lTi/m l ) ' S2 k + I exp ( -2'lTijm I )

RI

->

j = 1 , . . . ,g, j = 2 , . . . , 1, j = 1 , . . . , 2k ,

( 6 .2 . 1 9 )

->

(one must remove the top row in (6.2. 1 9) if 1 = 0), where m l is the least order of the element R I • The subgroup f l = ker X has a finite abelian quotient f l \f , and the set (f l \f)�eg is nonempty. Finally, the last case for us to consider is when f is a group of the form f = f( g; 0 ; 2k + 1 ), k E lL , k � 1 . The representation X of f which gives the desired subgroup f l is constructed by associating the element one in lL to every hyperbolic generator and a primitive (2k + l )th root of unity to every parabolic generator. The subgroup fl C f for which 2) holds is then the kernel of the representation X. The proof is complete. Other choices of f I e f are possible in the proof. We conclude by noting that, in our view, the method described here for studying the resolvent of &(f; X ) has great promise in the spectral theory of automorphic functions, and is by no means exhausted by the theorems proved in this section. See §7.2 concerning one direction for developing this method. §6.3.

1)

The spectrum & (f; for a group f E WC nontrivial commensurable of,

with The basic purpose of this section is to extend the spectral theory of & (f; X ) in the case of special Fuchsian groups of the first kind f, namely, groups with nontrivial commensurable. Our fundamental results are 1 ) the construction of a compatible spectral decomposition for the operator &(f; X) and the Hecke operator T( g) (Theorems 6.3.3-6.3.5), and 2) the proof of Roelcke's conjecture for a group r E WC 2 with large commensurable (Theorem 6.3.6). These theorems were first published in [64]. Here we shall limit ourselves to the case of the trivial representa­ tion X , dim V = I (see § 1 .2).

§6.3. THE SPECTRUM OF W ( r ; 1 ) . FIRST CASE

109

Let f E IDe be arbitrary. Suppose that there exists an element g E G = PSL(2, IR) such that the groups f and g- l fg are commensurable in the narrow sense, i.e., the intersection

= f' ( g ) = g-lfg

n

(6 .3 . l ) has finite index in g-lrg and in f. Then we say that g lies in the commensurable f of the group r. It is not hard to verify that the commensurable f is a group containing f as a subgroup. We shall call f nontrivial if f =1= f; otherwise, we call it trivial. f'

r

Although the vast majority of groups f E IDe have trivial commensurable, a fair number of the groups f that arise in appijcations have the property that r =1= f. We shall give some examples of such groups at the end of the section. We let IDe n denote the set of all f E IDe for which f =1= f. We also introduce the sets IDej,n IDej n IDe n, j 1 , 2. All of the groups f in IDe n are of special interest from the standpoint of the spectral theory of the operator m (f; I ). We now explain this. Suppose that f E IDe n ' and the function f(z ) lies in the space C( F; C ; 1 ) (see § 1 .3). We decompose f into a union of cosets

=

=

e

f = U f'Yj ' j= l

( 6 .3 .2)

where f' is given in (6.3 . l ) and e E Z , e � 1 . We associate to fe z ) the following function: T( g )f( z ) =

e

� f( g Yjz ) .

j= l

( 6 .3 .3 )

This formula (6.3.3) defines a linear operator T( g) on C( F; C ; 1); it is an abstract shift operator which commutes with the differential operator L (see § 1 . 1 ), since its definition only involves the group operations. In addition, T( g) takes an automor­ phic function to an automorphic function. Namely, let Y E f be an arbitrary element. It is easy to verify the following equalities : T( g )f( yz ) =

e

�

j= l e

f( g Yj yz ) =

e

� f( g YYrp(j) z )

j= 1

e

= j=�1 f( ygYrp(j)z ) = j=�1 f( gYrp(j)z ) = T( g )f( z ) , where YjY = YYrp ( j ) ' Y E f ' , Y = g -I yg, Y E f, and qy( j ) is a permutation of the indices 1 , . . . , e. The operators T( g) play an extremely important role in the arithmetic theory of automorphic forms. In a special situation these operators were known as far back as Kronecker, and later Hurwitz. It was Hecke who studied them systematically in the theory of modular forms. They were introduced into the spectral theory of automor­ phic functions for cocompact groups by Selberg. We shall call the operators T( g) Hecke operators in the general situation of a group with nontrivial commensurable, since this is the customary name for them in the more special theory of automorphic forms.

llO

VI . SECOND REFINEMENT O F THE EXPANSION THEOREM

The formula (6.3.3) enables one to define the Hecke operator T(g) on a dense set in the space :Je(f; 1). We describe the properties of this operator in the next two theorems.

6.3. 1 . Let f E we Every nontrivial element g E f induces a nontrivial linear Hecke operator T(g) which is bounded as an operator in the space :Je(f; 1) and lies in the commutator algebra of W ( f; 1). PROOF. It suffices to show that T(g) is a bounded operator in :Je( f ; 1). For a function f E Ca( F; C ; 1) we have THEOREM

n"

(6 .3 .4) The set F' = �r Yj F is a fundamental domain for the group P, by (6.3.2), and the set over which we are integrating in the right side of (6.3.4) is a fundamental domain for the group gf 'g- I . However, gf 'g- I = f', and this, together with the f-automorphic­ ity of f, means that the integral on the right in (6.3.4) takes the form

IF' I f( Z ) 1 2 dp. ( z ) = [ f : f'l lFI f( Z ) 1 2 dp. ( z ) ,

( 6 .3 .5 )

where [ f : f'] is the index of f' in f. This implies that

( 6.3.6) But (6.3.6), which extends by continuity onto all of :Je( f; T( g) is bounded. The proof is complete.

1), shows that the operator

T(g)* = T(g- I ) for T(g) as in Theorem 6.3.1, where * denotes the adjoint operator in :Je ( f; 1). PROOF. For functions fl ' f2 E Co( F; C ; 1) we have THEOREM 6.3.2.

e

IF[T( g)fl ( z ) ] f2( z ) dp.( z ) = � IFfl ( gyjz ) f2 ( z ) dp. ( z ) = � /I (Z J!2 ( g- IZ ) dp. ( z ) = f /I ( z ) f2 ( g- IZ ) dp. ( z ) F F = I fl ( ) [T( g - I ) f2 ( Z ) ] dp. ( z ) . F j= J

z

Since T(g) is a bounded operator by Theorem 6.3. 1, this equality gives us the theorem. The proof is complete. From Theorems 6.3. 1 and 6.3.2 it follows that a group with a nontrivial com­ mensurable has at least one selfadjoint Hecke operator T( g) + T( g- I ). We introduce a new, slightly simpler notation (see §3. 1) for the function E( z ; s; a; a; v; f; X ) in accordance with our assumptions X = 1 and dim V = 1.

111

§6 . 3 . THE SPECTRU M OF � ( r : I ) . FIRST CASE

Thus,

E(z,, s ·, a) E( r, s ,· a ,' r ) where z E F, s E C and a = 1 , . . . , n. =

=

E(z , s ,· a ,' l ,' r ,· 1 ) .,

We prove the following important fact. THEOREM 6.3.3.

Suppose that r E WC 2, /1 and T(g) is a Hecke operator. Then /1

T(g )E( z ; s; a ) = � Hap( s; g )E( z; s; /3 ) ,

( 6 .3.7 )

where a = 1 , . . . , n , the/unction HaP(s ; g) is- defined by Hap( s; g ) = lim ( y- ST( g ; s )E( gp z ; s ; a ) ) , -+ 00

( 6 .3 .8)

P= I

)'

the transformations gp are defined in § 1 .2, z E IT a' y

=

1m z

and a, /3 = 1 , . . . , n .

PROOF. We first show that, for any a = 1, . . . , n , the function T(g) E ( z; s ; a ) has at most polynomial growth in a neighborhood of each cusp of the fundamental domain F of the group r. To do this, we make use of (6.3.6), in which we take as our fe z ) the function £(z; s; a; a) in (3.2. 1), which occurs in the Maass-Selberg relation (3.2.2). The Maass-Selberg relation shows that, for each a in the hypothesis of Theorem 1 .2.4, we have £ ( z; s ; a ; a) E X(r; 1). In particular, for any s E C, Re s > 1, the following estimate follows from (3.2.2):

( 6.3.9 ) By (6.3.6) and (6.3.9), we have

fF1 T( g) £ ( z ;

s;

a ; a ) 1 2 dJL ( z ) « a 2 Re S - I ,

= 1, . . . , n 1 T( g)E( gpz; s; a ) l « yRe s , 1m z and 1 .:;;;; a .:;;;; n.

and this shows that for every /3

=

(6 .3 . lO)

where z E ITa' y N ext, from the differential equation

-LT( g )E( z ; s ; a) = s ( l - s ) T( g )E(z ; s ; a ) and (6.3.l0), we find, just as in §3. 1 , that the following Fourier expansion holds for the values of the parameters 1 .:;;;; a, /3 .:;;;; n: T( g) E ( gp z; s ; a )

=

A ( lX ; /3; s ) y S + B ( a ; /3; s ) i - s + fY � Crn ( a ; /3; s ) Ks - 1 /2(27T 1 m 1 y ) exp 27Timx, m E Z, m*O

(6.3 . l 1 ) where z = x + iy E H, with various coefficients A( a; /3; s), B ( a ; /3; s) and Crn( a; /3; s) not depending on x or y . For a fixed value of a one can say that not all of the coefficients A( a; /3; s ) are identically zero. Otherwise, T(g)E( z; s ; a) would be a square integrable (on F ) eigenfunction for 2£ ( r; 1 ) with eigenvalue A = s(l - s ),

112

VI. SECOND REFINEMENT OF THE EXPANSION THEOREM

Re s > 1, and this would contradict the selfadjointness of difference

W(f ; 1). We consider the

n

T( g) E ( Z ; s ; a) - � A ( a ; fi ; s ) E ( z; s ; fi ) .

( 6.3 . 1 2)

{3 = 1

Since (see Lemma 3.1.1), it follows that the difference (6.3. 12) lies in the space X(f ; 1) and is an eigenfunction for the operator W(f ; 1) with eigenvalue s(l - s). Again using the selfadjointness of W(f ; 1), we find that the difference (6.3. 12) is identically zero; this proves (6.3.7). Here one should set

Ha{3( s ; g) = A( a ; fi ; s ) (see (6.3. 1 1» . Finally, (6.3.8) is an easy consequence (6.3.7). The proof is complete. We now describe some properties of the coefficients Hais ; g), where we assume that the group f satisfies the hypothesis of Theorem 6.3.3. The definitions (3. 1 . 1 ) and (6.3.3) imply that e

T( g)E(g{3 z ; s; a) = �

� y S ( g� lgYj g{3 Z ) ,

( 6 .3 . 1 3 )

j = 1 y E ra\ r

where the series converges absolutely for Re s > 1. From Theorem 6.3.3 it follows that Hais; g) is the coefficient of yS in (6.3. 13). This coefficient is already de­ termined by the terms of the series which correspond to transformations 0': H � H of the form ( 6 .3 . 1 4) O'Z = g� l ygYj g{3 z = p 2Z + q ,

where Z E H and p , q E R . We show that the set of such 0' in (6.3.13) is finite. In the first place, since (6.3. 13) converges absolutely, so does the series

( 6.3 . 1 5 )

a

for Re s > 1. By (6.3.2) and (6.3.3), the set of elements through f,j = 1, . . . , e, coincides with the double coset

g� l ygYj g{3'

g� l fgfg{3 .

where

y

runs

( 6.3 . 1 6)

Next, since r is a group, it follows that whenever g E r we also have g- l E r, i.e., there exists a (nontrivial) Hecke operator T(g- l ). If we consider the function which is defined by a series absolutely convergent series

T( g- I ) E ( gaz ; s ; fi ) , analogous to (6.3.1 3) for

where O" z = gp l yg- l Yj'gaz = p,2 + q ', Y E f{3\ f, j' that the set of elements gp l yg- l Yj'ga for y E f and with the double coset

Re s >

1,

we arrive at an

( 6 . 3 . 1 7) = I , . . . , e'. It is not hard to see Yj' E ( gfg - 1 n f)\f coincides ( 6 .3 . 1 8)

113

§6.3. THE SPECTRUM OF m: ( f ; I ) . FIRST CASE

However, the cosets (6.3. 1 6) and (6.3. 1 8) consist of mutually inverse elements; hence p = p '- I , and the series (6.3. l 5) and (6.3 . l 7) can only converge at the same time if one of them (and hence both) has only finitely many terms. We have proved the following theorem.

If r E ffiC 2, n and T(g) is a Hecke operator, then the coefficients HaP (s ; g) in the expansion (6.3.7) have the form Hap ( s ; g ) = � p 2 s ( a ; /3 ; I ; g) , ( 6 .3 . 1 9) THEOREM 6.3.4.

I

where p(l) = p( a; /3; I; g) E R and the . sum only has finitely many terms; a, /3 = 1, . . . ,no A s before, we shall let ® (s) denote the scattering matrix of the operator mer; 1 ), ® (s) = { ® ap(S)}:,P = 1 (see the context of (2.3.21» . Recall that in this section X = 1 and dim V = 1 . We introduce the notation for a matrix of order n: THEOREM 6.3.5. Suppose the conditions of Theorem 6.3.3 are fulfilled. H(s; g) = ® ( s )H( 1 - s; g)® (1 - s ) .

Then (6 .3 .20)

PROOF. The explicit formula (6.3. l9) implies that Hap (s; g) is an entire function of s for every a , /3 = 1, . . . , n. By Theorems 2.3.6 and 3 . 1 .4, we have the functional relation

n E ( z; s ; a) = � ® ap ( S ) E ( z ; s ; /3 ) , P= 1

a = 1, . . . ,no We have T( g )E(z ; s ; a) =

( 6 .3 .2 1 )

n � Hay (s; g )E( z ; s ; y )

y= 1

n n s; Ha g) � ® Y8(S )E(z; 1 - s; 8 ) . = � y( 8= 1 y= 1

(6 .3 .22)

The left side of (6.3.22) has the form

T( g)

[ Y� ® .y ( s )E(z ; 1 - s ; y) ] 1

=

® .y( s ) T( g)E(z; 1 - s ; y) t y

n n s ®a = � y ( ) � Hy8 ( I - s; g)E(z ; l - s; 8 ) . y= 1

8= 1

Because of the linear independence of the functions obtain

n � HaJ s ; g ) ® eP ( s )

f= 1

=

E(z; s; a), a =

n � ®a8 ( s )H8P ( 1 - s; g ) .

8= 1

1 , . . . , n, we (6 .3 .23)

From the relation ®(s)®(1 - s) In ' where In is the n X n identity matrix (see Theorem 2.3.5), and the equality (6.3.23), we obtain the desired relation (6.3.20). The proof is complete. =

1 14

VI . SECOND REFINEMENT OF THE EXPANSION THEOREM

We now prove a theorem which confirms Roe1cke's conjecture that there are infinitely many eigenvalues of the discrete spectrum of &(f ; X ) in the case of a group f E WC 2 n with a " large" commensurable and X = 1 (compare with Theorems 6.2.4, 6.2.5, 6. 1.1, and 6. 1 .2). For simplicity we shall not deal with the most general case. We introduce some notation. Let g E f for f E WC 2 n . We let 1) denote the j operator ,

,

1) =

T( gj )

+

T( gil ) ,

( 6.3.24)

and we let H ,a{J( s) denote the function

j

Hj ,a{J ( s ) = Ha{J ( s ; gJ + Ha{J(s ; gi l ) , where T(g) and T(gil) are Hecke operators, and Hap( s ; g) and Ha{J( s ; gil ) are the

corresponding coefficients in (6.3.7). We shall suppose that the group f has n 2 operators 1). Recall that n denotes the number of pairwise inequivalent cusps on the fundamental domain F for f. We make the change of indices

�1(S ) = Hj , a{J ( s ) ,

( 6.3 .25 )

where 1 = ( a l)n + /3. If a and /3 vary in the range 1 � a, /3 � n , then I runs through the set 1, . . . ,n 2 • We introduce the square n 2 X n 2 matrix made up of the functions (6.3.25), and its determinant -

� (s ) = det { �l ( s ) } ��l= . l THEOREM 6.3.6. Suppose that f E we 2 n is a group whose fundamental domain F has exactly n pairwise inequivalent cusps, and such that: 1) there exist n 2 of the operators 1) in (6.3.24), and 2) the determinant �(s) does not vanish in the region where the function h( s ) in Theorem 3.3.4 is analytic. Then the space of cusp-functions X02( f; 1) is infinite dimensional. PROOF. We consider the n 2 + 1 spectral decompositions (see (3.3.14)) 1) k/ z , z ' ; f ; 1 ) = � A ( j ; k ) h/ A k )w( z ; k ; j )w ( z ' ; k ; j ) 1 + -. 1 4 'TT l Re k n2+ I ( Z , Z ' ;

k

s=

hj (s( I - s ))

1 /2

n

n

� � � . a{J ( s ) E ( z ; I (3 = I

a=

j=

s; /3 ) E ( z ' ; s; a ) ds, 1 , . . . ,n 2 ;

( 6.3 .26 )

f ; 1 ) = � hn2+ I ( A k )w(z ; Ai Jw( z ' ; A k ) k

n 1 +4 'TT l. 1Re s = 1 /2hn2+ I ( s ( 1 - s )) L I E ( z ; s; a ) E ( z ' ; s; a ) ds. ( 6.3.27 ) {w( z ; k; j)} k is a compatible real eigenbasis for the operators the subspace Xo(f; 1) ffi 8 1 (f ; 1) of the discrete spectrum (the a=

In these formulas &(f; 1) and r;. in operators &(f ; 1) and 1) are selfadjoint and commute with one another; the operators 1) and 1), do not, in general, commute if j =1= j'). Furthermore, in (6.3.26)

§6 . 3 . THE SPECTRUM OF

9l ( r ; I ) . FIRST CASE

115

and (6.3.27) we have

l )w{ z ; k ; i ) = A k ( i ) w ( z ; k ; i ), 1jw( Z ; k ; i ) = Ak ( i ) w ( z ; k ; i ) , { W( Z; A k )} k is the standard real eigenbasis for m ( f; 1 ) in the subspace %o( f ; 1 ) E9 8 1 ( f; 1 ), and { A d is the corresponding set of eigenvalues. The functions h/s(l - s» satisfy the conditions of Theorem 3.3.4 for i = 1 , . . , n 2 • We are also supposing that the operator 1j acts on the variable z. The existence of the spectral decomposition m (f;

.

(6.3.26), (6.3.27) is a consequence of Theorems 3.3.4 and 6.3. 1 -6.3.5. The basis idea of the proof is as follows. We regard the set of spectral decomposi­ tions (6.3.26), (6.3.27) as a system of: linear equations for the integrals with the Eisenstein series, and we regard the set of functions hj as a set of free parameters. Starting from condition 2) of the theorem, we select these free parameters in such a way as to eliminate the unknown integrals from (6.3.27) and obtain a spectral decomposition containing only the eigenfunctions of the discrete spectrum of m ( f; 1); then our investigation of this discrete spectrum leads to the proof of the theorem. We introduce the notation

Xt(Z , z ' ; s ) = E(z; s ; 13) E( z'; s ; a ) ,

I= a

-

n

+ f3 n .

We consider the system of equations

n2 L Hjl ( s )Xt(z, z' ; s ) = I/ z, Z'; s),

J.

/= 1

- 1 , . . . ,n2 , -

( 6.3 .28 )

for the functions x t Cz, z '; s) with certain known kernels £ (z, z'; s). By assumption 2) of the theorem, the system (6.3.28) has a unique solution if s is not a zero of �(s). We have

n2 Xt( Z , Z'; S ) = L Qt/ s )£(Z, Z'; S ),

(6 .3 .29 )

j= 1

where

n2 L Hj / s ) Q r/ ( S ) = 13jt , r= 1

1

' ';:;;; J

,

I .;:;;;

n2,

and 13jt is the Kronecker symbol. From (6.3.27) we formally obtain

k n2 + I (Z, z' ; f ; 1 ) - L h n2 + I ( A k )W(Z; A k )W(Z' ; A k ) k n 1 ( h n 2 + I (s( 1 - s )) L xa n + a --n(z , z'; s ) ds = 4 WI J Re s = I /2 a= 1 .

n n L L Hj , yfl ( s )E(z ; s ; 13 ) E(z'; s ; y ) ds. X· fl = 1 y = 1

(6 .3 .30)

116

V I . SECOND REFINEMENT OF THE EXPANSION THEOREM

From Theorem 6.3.5 and the functional equation (6.3.2 1) we see that the function II

II

� � Hi, y/3 ( s ) E ( z; s; f3 ) E ( z' ; s; y ) /3= I y = I is invariant under the change of variable s 1 - s. Hence, the function --->

"

h n 2 + I ( S( 1 - s )) � Qan+a - n( s ) , ( 6.3.3 1 ) a= l 2 has the same property; we denote this function by hJ (s( l - s» , j 1 , . . , n • Using Theorem 6.3.4, we see that each of the functions hJ(s( l - s» satisfies the conditions of Theorem 3.3.4, provided that h,,2 + / S(l - s» satisfies these conditions, and we =

.

are assuming that this is the case. Thus, there exist kernels

kJ(z , z'; f ; 1 ) = � kJ ( u(z, yz')) , yEr

o

}

. -- 1 , . . . , n 2 ,

where the function kj (t) is determined from hJ(s(1 - s» , s 1- + ir, r E �, by (3.3. 16). Taking this into account, we obtain the desired spectral decomposition from (6.3.26) and (6.3.30): =

,,2 k,, 2 + I (Z , z' ; f; 1 ) - � Tj kJ(z , z' ; f; 1 ) j= l � h n 2 + 1 ( A k )W(Z ; A k )W(z' ; A k ) k =

� � A ( j; k ) hJ{A k ( j ))W(z; k; j)w(z'; k; j ) . (6 .3.32) = l k j We study (6.3.32) with the proof of nontriviality of Xoi f; 1) in mind. The proof

that this subspace is infinite dimensional will use proof by contradiction. Suppose that dim X02 (f; 1) < 00 , and hence the sum on the right in (6.3.32) is finite. All of the generalized eigenfunctions w(z; A k ) and w(z; k; j) may be assumed to be continuous on F, and thus the right side of (6.3.32) is a function on F X F which is continuous in z and z' for any functions hJ(s(1 - s » , j = 1, . . . , n 2 , satisfying the conditions of Theorem 3.3.4. For kn 2 + I ( Z, z'; f; 1) we now choose the kernel of the resolvent 91 ( K; f; I) (see §1 .4) of the operator 2f (f; 1). It is not hard to see that the corresponding function h n 2 + I (s(1 - s » has the form

h n 2 + I (S{1 - s ) ; K ) = 1/ (s{1 - s ) - K( 1 - X )) . ( 6.3.33 ) We suppose that K is a sufficiently large fixed positive number. From the definition (6.3.33) it is clear that the function h n2+ l (s(1 - s) ; K ), and hence also all of the functions hJ(s(1 - s) ; K ), which are connected with it by (6.3.3 1), satisfy the conditions of Theorem 3.3.4. Therefore, the right side of (6.3.32) is a continuous function of z and z ' for our h n 2 + ! and hJ. On the other hand, we now show that the kernel in the left side of (6.3.32), i.e., the �um n2 (6.3.34) r (z, z'; K ; f ; 1 ) - � Tj kJ(z, z'; f ; 1 ; K), ,

j= !

§6.3. THE SPECTRUM OF � (r ; 1 ) . FIRST CASE

117

is not a continuous function of z and z' in a neighborhood of the surface z = z '. In this manner we obtain a contradiction with the supposition that dim X02 ( f; 1) < 00 . I t suffices to prove that each of the kernels 1jkJ ( z , z'; f; 1 ; Ie) in (6.3.34) has a finite limit as z � z'. (The kernel r(z, z'; Ie; f; 1) obviously has a weak logarithmic singularity as z � z' (mod f).) This, in turn, follows from the definition (6.3.3), since

u( z , g/Ytz ) > 0, u ( z , g/Yfz ) > 0,

Yt E ( g;l fgj n f ) \ f, gj t£ f, yf E ( gjfg;l n f ) \ f, g/ t£ f,

where u(z, z') is the fundamental invariailt of a pair of points. The proof is complete. We now give a simple example to illustrate Theorem 6.3.6. Let f be any of the groups in Fricke's family. The Fricke groups play an important role in Markov's theory of the minima of quadratic forms and its generalizations (see [47]). By definition, f is a group in 9)1 2 with signature f = f( 1 ; 0; 1) (see § 1 .2). The set of all such groups up to conjugation in G forms a family which depends continuously on one complex parameter (the groups have a nontrivial deformation). As shown in [47], f is a subgroup of index two in a group fo having signature fo = f(O; 2, 2, 2; 1 ), and so it has a nontrivial commensurable. It is not hard to see that the commensura­ ble contains the parabolic transformation g; z � z + 1 , g t£ f, which induces a Hecke operator T( g). The determinant a(s ) is easily computed: a(s ) = 1 , since the group f only has one primitive parabolic conjugacy class. Consequently, Theorem 6.3.6 holds for f. We end this section by giving other examples of groups f E 9)1 n. We shall focus Ollr attention primarily on groups f E 9)1 2,n' although examples include cocompact groups as welL Later, in §6.4, we shall prove Roe1cke's conjecture for each group in this class, by carrying further the idea of the proof of Theorem 6.3.6. All of the examples are constructed from a single point of view: we work with groups which are commensurable with groups generated by reflections. In order to give a detailed description, we must first extend somewhat our definition of the commensurable of a group f. Before we assumed that f E G = PSL(2, IR). However, as we shall see later, it is useful to allow the commensurable to contain reflections relative to geodesics on H, i.e., motions of the second kind. [In §6.5 a) we shall give a group theoretic description of the motions of the second kind.] An example is reflection relative to the geodesic {z E H I Re z x = O } - this is the map ED : H � H, z � -z, where the bar denotes complex conjugation. Obviously, &; fl G. The condition for &; to belong to the commensurable is the same as before: the groups &; f&; and f must be commensurable in the narrow sense. We now describe the set of Fuchsian g£oups of the first kind which are connected with the groups generated by reflections relative to the sides of geodesic polygons in the Lobachevsky plane H. Let M be a regular polygon in H, i.e., it has the properties that 1) M is a closed polygon bounded by a finite number of geodesics in H, and 2) the interior angles of M are of the form 'TT/k, where k E 1, k > I or k = 00 , if the corresponding angle is zero. Let m = m(M) be the number of zero interior angles in M. If m =t= 0, then M is noncompact. However, in all cases we consider M has finite volume I M I relative to -

VI. SECOND REFINEMENT OF THE EXPANSION THEOREM

1 18

the measure dp.. Let ft be the group generated by reflections relative to all of the sides of M. We define fM to be the subgroup of index two in ft consisting of words of even length in the generators of ft (see [54]). We have fM E IDe . Its fundamental domain (in general, not canonical) can be chosen as follows: FM = M U SM, where S is reflection relative to some side of M. We note that the fM are classical groups. They were known to Klein in connection with the problem of existence of an analytic automorphic form for a given Fuchsian group (see [26b], and also [ 1 5]). For the groups fM the problem was solved in a particularly simple and elegant manner using Riemann's conformal mapping theo­ rem and the Schwartz symmetry principle. In our view, there is a curious analogy between this problem of Klein and Poincare and the problem of nontriviality of the subspace 8 \(f; 1 ) EB 5(o(f; I) which we stated in §6. 1 . As we shall show later, Roelcke's conjecture can also be proved in a relatively simple manner for the groups fM We now give a simple classical example of a family of groups fM ' This is the Hecke family. For every natural number k > 1 we define the group f(k ) to be the group of transformations H � H with the two generators z � -z - \ Z � Z + 2 cos 7T/k. It is well known that f(k) E IDe 2 ; in addition, the Hecke group f(k) can be obtained up to conjugation in G by the above procedure using a regular triangle with interior angles (0, 7T/2, 7T/k). In particular, the modular group fz is the Hecke group f(3). Hence, the set of all groups commensurable with the groups fM (in the case m(M) =1= 0) includes the set IDe 2 of all arithmetic groups, but it is much larger. Even the Hecke group f( k ) is arithmetic only in the cases when k = 3, 4, 6 or 00 (see [24]). In the next section we show that each of the groups fM ' and hence every group commensurable with it in G, has nontrivial commensurable. But from the standpoint of size of the com­ mensurable one must especially distinguish the arithmetic groups. It is well known that the modular group fz has GL(2, C) for its commensurable. Having such a gigantic commensurable is a characteristic property of an arithmetic group. ·

,

a

§6.4.

The spectrum of

�(f; I)

with

for a group commensurable the group f

fM

In this section we shall specialize the theory in §6.3 to the case of a group f which is commensurable with an arbitrary group fM (see §6.3 for the definition of fM ) ' The basic results are 1) establishment of a connection between the spectral theory of automorphic functions for fM and the Dirichlet and Neumann boundary value problem on M, and 2) proof of Roelcke's conjecture (and even a somewhat stronger conjecture) for the group f, with X = 1 . These results were first published in [6 1 ] and [70]. A special case of the conjecture which we are now calling Roelcke's conjecture was proved in Roelcke's dissertation [45] for the Hecke groups with X 1 . We begin by examining the theory for the groups rM (see §6.3). =

THEOREM 6.4. 1 .

Every group rM has a nontrivial commensurable.

PROOF. Suppose that the group rt is generated by the reflections R I , . . . ,Rk, with RJ = E, j = I , . . . , k, where E is the identity of the group. Any element 0 E rt is

represented by a word

§6 .4. THE SPECTRUM OF � ( f ;

I ). SECOND CASE

119

in the generators Rj' The group fM is identified in fZ. by the condition that the length of the words it contains must be even (the length of 0 is the number of letters in the word, i.e., the number of generators). If 0 E fM ' then RjoRj E fM' since the length of the word RjoRj is either the same, or two greater, or two less than the length of o. (We do not count the letters RjRj = E .) This gives us the inclusion RjfMRj C fM for any j = 1, . . . ,k, and we also have the reverse inclusion because of the equality R; = E; hence,

(6 . 4 . 1 ) Consequently, every reflection Rj is in,the commensurable fM' and since Rj fl fM' it follows that the commensurable is nontrivial. The proof is complete. In what follows we shall let &; = &;(M) denote the reflection relative to a side of the polygon. THEOREM 6.4.2.

Each reflection &; relative to a side of the regular polygon M induces a Hecke operator T(&;) connected with the group fM. For f E C( FM; C ; 1) T( &; )f( z ) = f( &; z ) . ( 6 .4.2 ) PROOF. The theorem is a direct consequence of the definition (6.3.3) of the Hecke operator and the equality (6.4. 1). 6.4.3. The operator T( E9) in Theorem 6.4.2 has the following properties: 1) It is bounded and selfadjoint in X ( fM; 1). 2) It commutes with m ( fM; 1). 3) The operators PI ( &; ) = 1/2( g - T(&;)) and P2 ( &; ) = 1/2( g + T(&; )), where g is the identity operator in X(fM; 1), are orthogonal projections of X(fM; 1) onto the mutually orthogonal subspaces VI ( fM ) and Vi fM) respectively; X ( fM; 1) = VI ( fM ) THEOREM

,

.

E9

V2 ( fM ) .

PROOF. Parts 1) and 2) are proved in a similar way to Theorems 6.3.1 and 6.3.2. The operator T( &; ) is selfadjoint because &; = E9 - I . To prove 3) it suffices to show that P I ( E9 ) and P2( E9 ) are selfadjoint and have the properties

P I ( E9 ) 2 = PI ( E9 ) , P2 ( E9 ) 2 = P2 ( E9 ) , ( 6.4.3 ) P I ( &; ) P2 ( E9 ) = P2 ( &; ) PI ( E9 ) = 0. The selfadjointness is obvious from part 1). Next, we have PI (&; ) 2 = 1/4 ( g - T( &; ) - T( &; ) + T( &; ) 2) = 1 /2 ( &; - T( E9 )) = P I ( E9 ) , P2 (E9 ) 2 = 1/4 ( g + T( &; ) + T( &; ) + T( E9 ) 2) = 1/2 ( g + T( &; )) = Pi &; ) , PI ( E9 ) P2 ( E9 ) = 1/4 ( g - T( &; ) 2) = 0, P2 ( E9 ) P I ( E9 ) = 0.

The proof is complete. As before, we let E (z; s; a) = E(z; s; a ; fM ) denote the Eisenstein series (more precisely, the corresponding meromorphic function of s E C ) for the group fM and the cusp z a of the fundamental domain EM ' a = 1 , . . . , n , n being the number of pairwise inequivalent cusps on FM (see §§1 .2 and 3.1). In Theorems 6.4.4 and 6.4.5 we shall assume that the regular polygon M has the property that m ( M) -=1= 0, where

12 0

V I . SECOND REFINEMENT OF THE EXPANSION THEOREM

m is the number of zero interior angles. This property is necessary and sufficient for

rM E 9J( 2 ·

Theorem 6.4.2 and/or every a = 1 , . . . , n , T( (9 ) E ( Z ; s; a ) E ( z ; s; a ) .

THEOREM 6.4.4. For every (9 as in

=

PROOF. We fix (9 as in the hypothesis. As we already pointed out in §6.3, the fundamental domain FM for can be chosen as the union FM = M U (9M. We fix such an F In general, it is not canonical, in the sense that it can have cusps which are equivalent under This is always the case if the number m(M) of zero angles is greater than one. We give a drawing to help visualize the situation. In it the arrows indicate the order in which the sides of FM are identified. We have chosen as our example a polygon having two zero angles and a line along the y-axis relative to which the reflection (9 is taken. The cusps Zil and Zi 2 with the stars in the diagram are equivalent. We now continue the proof of the theorem in the general situation. From the definition of the group it follows that the cusp Zi of the canonical fundamen­ tal domain corresponds to two equivalent cusps Zil and Zi2 of FM , which may coincide, if they border on the side relative to which the reflection (9 is taken. On the other hand, (9Zil Zi 2 • We draw two congruent oricycles W I and through the vertices Z i and Z /" respectively (see the figure). The reflection 0 takes the two oricycles into one another. We shall call this property a) of the transformation 0. We consider the Eisenstein series E ( z; s; a; Re s > 1. We define the function

rM

M.

rM .

rM

W2

=

I

2

rM )' tY I

M

By property a) of the reflection 0, the functions

E( giz ; s ; a ; rM ), E{ gi(9 Z ; s; a ; rM ) have the same asymptotic behavior with respect to y for Z E ITa' since 0 only acts on the variable x = Re z. In other words, E{gi0z ; s ; a ; rM ) �ia Y s + 0( 1 ) . =

y - oo

12 1

§6.4. THE SPECTRUM OF � ( r ; 1 ) . SECOND CASE

On the other hand, we know the asymptotic formula (see Lemma 3. 1 . 1 ) E ( giZ; s ; a ; fM ) = 8iay s + 0( 1 ) . y � oo

Because of this and the selfadjointness of � ( fM; 1), just as in the proof of Theorem 6.3.3 we obtain the desired equality E(z; s ; a ) = E ( f9z ; s ; a ) , a = 1 , . . . ,no The proof is complete.

The subspace Xoi fM ; 1) C X(fM; 1 ) is infinite dimensional. PROOF. The spectral decomposition for t4e kernel r(z, z ' ; K; fM; 1 ) of the re­

THEOREM 6.4.5.

solvent of � ( fM; 1 ) (see (3.3. 1 4» and Theorem 6.4.4 give us the decomposition

r(z, z' ; K ; fM ; 1 ) - r { z, f9z' ; fM ; 1 )

= 2� k

1

Ak - K( 1

- K

)

w ( z ; A k )W(Z ' ; A k ) , ( 6 .4.4)

where {w( z; A k ) h is the part of the real eigenbasis of the subspace of the discrete spectrum of � ( fM ; 1 ) in X(fM ; 1 ) whose elements satisfy the additional condition -w(z, A k ) = w { f9z; A k ) . ( 6 .4.5 )

Just as in the proof of Theorem 6.3.6, the supposition that dim Xoi fM ; 1) < 00 would imply continuity of the kernel on the right in (6.4.4) as a function of z, z ' E F. The left side of (6.4.4) can be represented as a series

� [ k( u(z, yz ') ; K ) - k( u(z, yf9z') ; K ) ] ,

yEr

( 6.4.6)

which is absolutely convergent for K , Re K > 1 and z =1= z ' (mod fM ). Hence, the kernel on the left in (6.4.4) has a singularity for Z = z ' (mod f), but this contradicts the assumption that the subspace X02 ( fM ; 1) is finite dimensional. The proof is complete. In parts 1) and 2) of the next theorem, the regular polygon M can be either compact (m(M) = 0) or noncompact (m(M) =1= 0). Part 3) only has content for noncompact M.

The restriction of the operator -� ( fM ; 1) to the subspace U1(fM ) is isomorphic to the operator of the Dirichlet boundary value problem on M for the Laplace differential operator L of the metric ds. 2) The restriction of -� ( fM; 1) to the subspace U2 (fM ) is isomorphic to the operator of the Neumann boundary value problem on M for the Laplace differential operator L of the metric ds. 3) The operator of the Dirichlet problem in part 1) has only a discrete spectrum. THEOREM 6.4.6. 1)

The subspaces U1(fM ) and U2( fM ) were defined in Theorem 6.4.3. PROOF. 1) We fix a fundamental domain FM for fM which is connected with a reflection f9, FM = M U f9M. It is sufficient to show that the elements of the intersection U1(fM ) n C( FM; C ; 1) vanish on the boundary of the polygon M. The proof that the indicated Dirichlet problem leads to the restriction of � (fM; 1 ) to the subspace U1(fM) is done by reversing the process, and we shall not carry this out here. Thus, suppose that f E U1( fM) n C( FM; C ; 1). The boundary of FM consists of

12 2

VI . SECOND REFINEMENT OF THE EXPANSION THEOREM

an even number of segments of geodesics on H. It can naturally be divided into pairs of segments. The elements of each pair are equivalent under a transformation in fM (i.e., one segment is mapped to the other by this transformation). If the point Zo is on the boundary of FM , then there is an element y E fM such that yZo is also on the boundary of FM . By the definition of f( z), we have

fe z ) = 1f(z ) - 1 T( ED )f( z ) , which gives us the equality

( 6.4.7 )

f( z ) = �f( EDz ) . The function f(z) satisfies the automorphicity condition; in particular,

f( z ) = f( yz ) . ( 6.4.8 ) The equalities (6.4.7) and (6.4.8) also hold for z = zoo Moreover, yZo = EDzo, and so f(yzo ) = f( EDzo ) . ( 6.4.9) Comparing (6.4.7)�(6.4.9), we arrive at the equality f(zo) = O. We have thereby proved that f( z) = 0 if z is on the part of the boundary of M which is also a part of

the boundary of FM . The rest of the boundary of M consists of the side of M relative to which the reflection ED is taken. On this side we have z = EDz, and so f( z ) = f( EDz); this, together with (6.4.7), leads to the equality fez ) = 0, thus completing the proof of part 1). 2) The proof is similar to that of 1), but for functions f E Ui fM ) n COO( F; C ; 1). The condition f E Ui fM ) leads to the equality

f( z ) = f( EDz ) .

( 6.4 . 10)

The automorphicity condition gives

af( z ) jan

=

( 6.4.1 1 )

-af( z )jan ,

where z and yz are equivalent boundary points on FM , and a jan is the exterior normal derivative to FM . For a boundary point z E FM we have yz EDz, and hence

f( yz ) = f( EDz ) .

=

( 6.4.12 )

Comparing (6.4. 10)�(6.4. 12), we arrive at the desired equality

af( z ) jan = 0,

(6.4. 1 3 )

if z is a common boundary point of FM and M. If z is on the other part of the boundary of M, we have z = EDz. Considering this equality in a neighborhood of z and taking (6.4.10) into account, we ll�rive at (6.4.1 3). This proves part 2). 3) The entire continuous spectrum of �(fM ; 1) is generated by the functions E(z; 1r; a ) , r E IR , a 1 , . . , n (see Theorems 2.3.7 and 3.1.4). According to Theo­ rems 6.4.3 and 6.4.4, each function E(z; s; a ) is annihilated by the projection PI( ED), regarded as a linear operator acting on the argument of the function; this proves part 3). The proof is complete. We now prove Roelcke's conjecture for an arbitrary group f which is com­ mensurable in G with one of the groups fM (m(M) 7'= 0; X = 1). Moreover, we prove that N( A ; f; 1) > c A starting from some A, where c > 0 is an effective constant. Recall that N( A ; f; 1) is the distribution function for the eigenvalues of the discrete spectrum of � (f; 1). =

.

§6 .4. THE SPECTRUM OF �l ( f ; I ) . SECOND CASE

123

Without loss of generality we may assume that the groups f and fM are commensurable in the narrow sense. Let f ' denote the intersection f ' f n fM ' It has finite index in f and in fM . Let {w(z; A k )h be a real eigenbasis for 2l( fM ; 1) in the subspace VI( fM ) (see Theorem 6.4.3), and let {A d be the set of eigenvalues. =

LEMMA 6.4.1.

Let the function h( s(1 - s)) satisfy condition 2) of Theorem 3.3.3, and let k(z, z'; fM ; 1) be the corresponding kernel of the operator h(2l"(fM ; 1)), given by the absolutely convergent series k(z, z'; fM ; 1 ) = � k( u(z, YZ')) . y ETM

Then one has the spectral decomposition 1 2 a E�fIf' al E�fIf' [k( O"z, 0" \ z ; fM ; I) - k( O"z, 0 0"\z; fM ; I )] =

� h ( Ak ) k

[

]

2 � w (O"z ; A k ) , ( 6.4.14)

a E fIf'

where &; is a fixed reflection relative to some side of the polygon M. PROOF. Theorems 3.3.3 and 6.4.4 imply the decomposition

1

"2 [ k (z,

z , ; fM ; 1 ) - k(z, 0Z'; fM ; 1 ) ]

=

� h ( A; ) W( Z ; A; ) w ( z'; Aj ) , ( 6.4. 15 ) .I

from which the desired equality (6.4. 14) follows. The proof is complete. We consider the equation O"Z Y0 0"\ Z in z E H, for fixed 0", 0" \ E f/f ' and y E fM' The set of its solutions obviously lies on a geodesic (in the Lobachevsky geometry of H) which depends on 0", 0" 1 and y. We call all such geodesics for 0", 0"\ and y as indicated " special". The following lemma follows immediately from the discreteness of the group fM and the finiteness of the set of cosets f/f ' . =

6.4.2. Any compact set on the Lobachevsky plane intersects only finitely many special geodesics. LEMMA

We now prove a theorem.

6.4.7. The inequality N(A; f; I ) > c r holds beginning with some A and c r , rM > 0 independent of A . ( The constant is computed in the course of the proof.) PROOF. In (6.4. 14) we choose h(s(1 - s)) to be the function THEOREM

h ( s (I - s ) ; t ) = exp [ -ts ( I - s ) ] , which depends on a parameter t > 0 (see the proof of Theorem 4.4.1). We use this function to construct k( u) = k(u ; t) froin the transformation (3.3. 16). If we take into account the explicit formula for the Fourier transform g(u; t) of the function h(t + r 2 ; t), s ! + ir, in (3.3. 16), namely =

124

VI . SECOND REFINEMENT OF THE EXPANSION THEOREM

we easily obtain the following two properties of k(u; t ) : l ) k(O; t )

� ;? � o > 0:

(4 17 t t l ; 2) for

=

where a and b are any positive numbers. We now integrate (6.4. 1 4) with the functions h and k fixed, over a compact set Q which does not intersect the special geodesics (see Lemma 6.4.2). As t 0 we obtain the equality �

p i Q I ( 8 17 t r

l

+ O{t b ) = � [ exp ( j

-

t AJ] Cj { Q ) ,

(6.4. 16)

where p is the number of triples a, a i ' Y such that yal = a, with y E fM and a, a I E fIf' (the definition of the groups f and fM implies that 1 � P < (0 ) , I Q I is the volume of Q relative to the measure dJL, and the function c/ Q) has the form

c/ Q ) =

1 dJL { Z ) g

[

�

o E r/r'

k

w ( az ; A j )

]

2

Let k be the minimum number of elements y, E fM such that a - I Q = U y, FM , ,= I

where FM is the fundamental domain for fM in H. We introduce the number q = max k . o E r/r'

We have c/ Q) � [ f : f,] 2q. We define a monotonic function Ng{ A ) =

However,

�

>...I ,,;;; >..

ci Q ) ·

(6.4 . 1 7) On the other hand, applying the Tauberian theorem to Nri A), from (6.4. 1 6) we obtain A � 00 ,

which, combined with (6.4. 1 7), gives the theorem with any constant cr r for which 1 cr.rM = c < p I Q I ( 817q [ f : f '] 2 ) - . .

The proof is complete. §6.5.

M

Selberg trace formula the Dirichlet boundary value problem on a regular polygon M for

This section is devoted to the derivation of a trace formula for a function of the operator in Dirichlet's problem in Theorem 6.4.6, a formula which seems to us to be a natural variant of the classical Selberg trace formula (see Chapter 4). The basic theorems proved here were published before in [65] (see also [62] and [66]). The plan of exposition is as follows. We first prove a theorem which applies to any regular polygon M. Then we separately consider the theory for compact M (§6.5 a» and for noncompact M (§6.5 b» .

12 5

§6 .S. SELBERG TRACE FORMULA FOR THE DIRICHLET PROBLEM

LEMMA 6.5.1. Let M be an arbitrary regular polygon, and let the function h(s(1

satisfy condition 2) of Theorem 3.3.3. Then 1

'2

� (k( u(z, yz' » - k( u (z, & yz'» )

y E rM

=

- s»

� h ( AJ w ( z ; AJW(Z'; AJ, (6.5 . 1 ) j

where Aj and w(z: A) respectively run through a complete set of eigenvalues and (real) eigenfunctions of the operator for Dirichlet 's problem in Theorem 6.4.6 ; k(t) is connected with h(s(l - s» by (3.3.16), and 0 is a fixed reflection relative to a side of M. PROOF. If M is noncompact, then the desired formula coincides with (6.4. 1 5). For a compact polygon M, (6.5.1) follows from the theorem on expansion in eigenfunc­ tions of 9f(fM ; 1) and Theorem 6.4.6. The proof is complete.

6.5.1 . Suppose that the conditions in Lemma 6.5.1 are fulfilled. In addition, let h(s(1 - s» = h i(s( l - s» , where the function h 1 (s(1 - s » satisfies the conditions of Theorem 4.2.2. Then the kernel on the left in (6.5. 1) is continuous on FM X FM , and determines a nuclear operator in the space :JC(fM ; 1). The spectral trace formula THEOREM

�f

� ( k( u (z, yz » - k { u(z, 0YZ » ) dp. { z ) = � h ( AJ j

FM y E rM

( 6.5.2 )

holds, where FM is the fundamental domain for fM . A ll of the series and the integral converge absolutely. PROOF. The continuity of the kernel follows from Theorem 3.3.3. The nuclearity of the corresponding operator is proved in a way analogous to Theorem 4. 1 .2. Finally, the spectral trace formula is obtained by integrating (6.5. 1). The proof is complete.

a) Trace formula for a compact polygon(6.5.2)M.

Here we shall be concerned with transforming the spectral trace formula to the Selberg trace formula for Dirichlet's problem on a compact regular polygon M. We begin the derivation of our formula by considering the integral

(6.5 .3 ) The integral (6.5.3) is obviously equal to the number

where Tr denotes the matrix trace of the operator, the function h(s(1 - s » is connected with k(t) by (3.3.16). Hence, the derivation of the classical Selberg trace formula consists of computing (6.5.3) in terms of the function h and the conj ugacy

126

VI. SECOND REFINEMENT OF THE EXPANSION THEOREM

classes in fM . We have (see §4.3)

�f

� k ( u( z , yz ) dJL( z))

FM -y E fM

=

=

� �

f k ( u(z, y'- l yy'Z )) dJL( z ) � f k( u (z, yz)) dJL( z ) = ' : ' f oo r ( tanh 7Tr)h ( ! + r 2 ) dr exp(-27Tkr/d ) h ( ! + r 2 ) dr + !4 � � d 1 d f oo 1 + exp( -27Tr ) 4 sin (k7T/ ) �

�

{ y hM Y ' E fM { y } \ fM FM

- 00

{y hM FM { y } d

\

- 00

{ R } rM k = \

1

00

+ "2 � �

(P hM k = \

In N( P )

N( P )

k/2

- N(P ) -k/2

g( k ln N( P )) .

(6.5 .4)

If we take into account Selberg's general remarks on computing the trace of the product T(g)&(f; X) for a cocompact group f with nontrivial commensurable (see [5 1 ], §2), where T(g) is an arbitrary Hecke operator, then we find an expression for the integral (6.5 .5) which is analogous to (6.5.4). However, in order to do this, we first make some preliminary transformations, as a result of which we shall be able to regard the reflection E9 as a group action. We translate the polygon M by an element in G = PSL(2, IR) in such a way that the side relative to which E9 is a reflection runs along the y-axis, y = 1m z, in the upper half-plane H. Here E9 takes the form E9 : Z � -z, (6 .5 .6) where the bar denotes complex conjugation. In what follows we shall consider M to be chosen in this way. (This choice of M in no way reduces the generality of the theory, and is only adopted for convenience.) We now construct an isomorphic model of H. This model is well known in the theory of symmetric spaces (see [5 1 D. We consider the set of positive definite symmetric matrices

Z ( x ,· y ) =

(

y

+ X 2Y - I -I .x'y ,

,

xy - 1

Y

-

I

where x E IR and y > o. If g E G, we denote g by

g=

( � �)

(mod ± 1 )

)

'

(6 .5 .7)

,

thereby expressing the fact that g E PSL(2, IR). We now define an action of G on the set { z ( x; y ) } . By definition, for g E G we have

(6.5 .8) gz( x ; y ) g [ z( x ; y )] g , where gt is the transpose of g, and the product on the right in (6.5.8) is the usual =

t

product of matrices. It is not hard to verify that the set fl = { z ( x; y)} with this

§6 . S . SELBERG TRACE FORMULA FOR THE DIRICHLET PROBLEM

12 7

action has the structure of a symmetric space and is isomorphic to the upper half-plane H. The isomorphism is given by the map z(x; y ) -) Z x + iy. This model ii of the upper half-plane has the following useful property which is not so obvious in the case of H. The reflection ED (6.5.6) for the model ii is given by the " matrix" =

( mod

± 1),

(6 .5 .9)

i.e., it is an element in GL(2, IR )/( ±E), where E is the identity matrix. It therefore makes sense to consider the products ED g and gED for g E G, regarding them as the corresponding cosets in GL(2, IR ). In the sequel we shall not distinguish between H and ii; we shall keep in mind the possibility of giving the reflection ED (6.5.6) by formula (6.5.9). We now return to the integral (6.5.5). We have

- � {0,,}�r y ' E TM �{ 0y} M fFMk( u(Z, y ,-I EDyy'z )) dll { z ) M � { 0�Y }r fFM( f9 y )k { u{ z , EDyz )) dll { Z ) . \r

(6 .5 . 1 0)

M

We have denoted by {EDy hM the relative conjugacy class in ED fM with representative EDy by fM(EDy) the relative centralizer in fM of the element EDy E ED fM and FM(EDy) the fundamental domain for fM(EDy) in H. Recall that EDy and EDy' are in the class {EDyhM if and only if there exists an element Yo E fM such that YoEDyyol = EDy'. Furthermore, fM(EDy) = {y' E fM I y'EDy = EDyy'}. The relative centralizer fM(EDy) is a group. The sum on the right in (6.5. 10) is taken over all relative classes {EDy hM ' In order to make the next transformation of the integral (6.5.5), we study the set of classes {EDyhM in some detaiL Let y E fM. We have det EDy = - 1 and tr EDy = a , with the usual notation for the determinant and trace of a matrix. We let II I and II I denote the eigenvalues of EDy . A simple computation shows that the eigenvalues can be chosen as follows: II I =

al2 - va ll4 +

1.

From this it is clear that the eigenvalues of EDy are always real and unequal. Hence, there is a transformation g E G which takes EDy to diagonal form:

gEDyg - 1

( Ill

-Ill ( Ill I )

=

0

_\

o

)

(mod

-+

1).

We shall denote the element EDy as follows:

EDy =

0

0

-Ill

( mod

-+

1).

(6 .5 . 1 1 )

We now find the relative centralizer of EDy in fM for y E fM' Here one should distinguish two cases. We first suppose that tr EDy = a =1= O. In this case the relative

VI. SECOND REFINEMENT OF THE EXPANSION THEOREM

centralizer in G of the diagonal matrix in (6.5. 1 1 ) (under the assumption (mod ± 1» consists only of diagonal matrices:

( �: _Z-I )

( mod

IL l =1=

1

± 1) ;

and this implies that the relative centralizer fM ( E9 y) c fM consists only of the hyperbolic elements and the identity. From the discreteness of fM one can conclude that the group fM ( E9y) is cyclic hyperbolic, i.e., it is generated by a single hyperbolic generator P( E9y). We now consider the alternative case tr E9y O. It is not hard to verify that the relative centralizer in G of the diagonal matrix (6.5. 1 1) with IL l = 1 (mod ± 1) consists of elements of the form =

( 6 .5 . 12 )

Obviously, P2 is an elliptic element of order two. Thus, in this case fM ( E9y) generally consists of hyperbolic, elliptic, and the identity elements. However, if we consider all possible products of the form P�P2 ' n E lL, we discover that fM ( E9y ) contains infinitely many elliptic conjugacy classes, and this contradicts the relation fM E we. Since fM is discrete, it follows that the centralizer fM ( E9y) for E9y with tr E9-y = 0 either consists of hyperbolic elements and the identity, in which case the group fM ( E9y) is cyclic hyperbolic and is generated by P( E9y) E fM ' or else it is generated by a single elliptic generator of order two. Later on, using analytic considerations, we shall show that the second case is impossible for cocompact fM ' We now proceed to compute the sum on the right in (6.5. 10). In it we consider the terms corresponding to classes {E9yhM with tr(E9y) =F O. For any y E fM we have E9yE9y E fM ' For any y under consideration, tr( E9y) =1= 0, the element E9yf9y is hyperbolic. By definition, by the norm of the class {E9yhM (or of the element E9-y) we shall mean the number N( f9y ) = 1 N( E9yE9y ) 1 1 / 2 , where the norm of a hyperbolic element was defined in §4.3, tr( E9y) =F O. In addition, we shall call a relative conjugacy class {E9yhM primitive if it is not an odd power of any other relative class {f9y ' hM • Part of the terms in the sum on the right in (6.5.10) we transform as follows: -

�

f

�

{ 6:i Y } rM FM( 6:i y ) tr 6:i y # O

� = -� = -

1

2

k{ u { z , (f9yz ))) dIL ( z )

�'

�

f

�'

�

f

�

f

{ 6:i Y } rM k = ! tr ( 6:i y ) 7"' 0

FM( 6:i y )

k k ( u ( z , ( E9 y ) 2 - I Z ) ) dIL ( z ) '

{ 6:i y hM k = ! g( 6:i y ) FM( 6:i y ) tr ( 6:i y ) #O �, £.J

£.J

{ 6:i Y } rM k = ! tr ( t9 y ) # O

!

N( P( t9 y »

k k( u(z, g( E9y )( E9 Y f - l g - l ( E9y )z)) dp, ( z )

(

I z + N( E9y )2 k - l jz I dr (" dcp . 2 k ) 2k- ! r J,0 sm cp y 2N ( E9y

)

'

(6.5 . 1 3 )

§6.S. SELBERG TRACE FORMULA FOR THE DIRICHLET PROBLEM

129

where the summation in �' is only taken over primitive relative classes, and P( f9y) is the generator of fM (f9y). The element g(f9y) E G is chosen so that g( f9y)f9yg- I (f9y) is diagonal. The region g( f9y) FM ( f9y) is a fundamental domain for the group of " diagonal matrices" g(f9y)fM (f9y)g- I (f9y). In polar coordinates x = r cos cp, Y r sin cp on H, this region is chosen as follows:

=

g(f9y) FM (f9y)

=

{z E H I I � r < N( P(f9y)) , O < cp < '1T} .

Recall that rM ( f9y) is generated by a single hyperbolic generator P( f9y). Next, the first part of (6.5. 1 3) is equal to

+

(N( f9 y ) k - I /2 - N( f9y fk + I /2) 2 ) dt 1

2

k k Q ( N( f9 y f - l + N( f9 y ) -2 + 1 � In N( P( f9y)) k N(f9y) - I/2 + N( f9y fk + 1/2 k= 1 00

�'

{ t9 y hM tr( t9 y ) 'i"' O

2)

, 00 1 In N( P(f9y)) - "2 � � k - I/2 + N( f9y) -k + I/2 g « 2 k - 1 )ln N( f9y)) ; { t9 Y } rM k = 1 N( f9y)

tr( t9 y ) 'i"' O

(6.5 . 14) the sum with the prime has the same meaning as in (6.5. 1 3), and the functions Q (w) and g(u) are determined from k(t) by (3.3.1 6). We now make a small digression from the immediate question of studying the integral (6.5.5). We shall later have need of more detailed information concerning P( f9y), the generator of rM (f9y) for f9y a primitive element, in our case tr(f9y) =1= O. For this purpose we consider the full discrete group rz. ::J fM (see the proof of Theorem 6.4.1 ) and the usual centralizer fZ.( f9y). It consists only of the powers of f9y, and perhaps the element f9. It contains f9 if and only if f9y = yf9. Correspond­ ingly, the relative centralizer fM (f9y) = rM n rz.(f9y) is generated by:

( 6.5 . 1 5 ) (6 .5 . 16)

1 ) either p ( f9y) = f9yf9y, or p ( f9y) y.

2)

=

We have the following alternatives for the norms:

N( P(f9y))

=

{

N 2 ( f9y ) N( f9y )

for 1 ) , for

2).

( 6 .5 . 1 7)

In order to finish the computation of the integral (6.5.5), it remains to consider in the sum on the right in (6.5. 1 0) the terms which correspond to relative classes {f9y h satisfying the condition tr( f9y) = O. Recall that an f9y for which tr( f9y) = 0 can have a centralizer fM ( f9y) of either of two types. The group fM (f9y) is M

..

V .1 . j,:)J: \...... V l�.LJ �..cr .11"11 r..1V.1.cl"11 1 v.C' I nn CArl"\.l"'f ':>.1Vl� J. n.c.v�..c.1VJ.

... ..,v

generated either by a single hyperbolic generator P ( f9y) or else by a single elliptic generator R(f9y) of order two. In the first case, just as in (6.5. l3) and (6.5.14), we have

f

FM( f9 y )

k( u (z, f9yz )) dJt ( z ) = j I

=

d q; k (cot 2 ) l q; . r sm2 q;

N( P ( f9Y » dr

1

'TT

0

2 In N(P( f9y )) g (0) .

In the second case we have

( )

4x 2 dxdy = 1 k(u(z, f9yz » dp. ( z ) = 2 J( k H , Y2 Y2 FM( f9 y )

f

(6.5 . 1 8)

00 .

(6.5.19)

The last integral diverges for all functions k(t) for which g(O) =ft 0 (see (3.3. 16)). Now, from the well-known absolute convergence of the integral (6.5.5) and the series (6.5.14), we conclude from (6.5.10) that the series and integral

�

f

{ f9 Y } rM FM ( f9 y ) tr( f9 y ) = O

k( u(z, f9yz» dp. ( z )

must converge absolutely. Taking into account (6.5.18) and (6.5. 1 9), we hence find that rM (for compact M) does not contain classes {f9y hM with tr( f9y) = 0 for which the relative centralizer fM ( f9y) consists of two elements. In addition, the series over the remaining {f9yhM with hyperbolic centralizer fM (f9y)

� In N( p ( f9y »

{ f9 Y } rM tr( f9 y ) = 0

( 6.5 .20)

is absolutely convergent. The discreteness of rM implies that the values of the norms N( P(f9y)) are isolated away from one; hence, the series (6.5.20) consists of only finitely many terms. We have completed the construction of the Selberg trace formula for Dirichlet's problem on a compact polygon M. We gather together our results in the form of a theorem, where we use all of the notation us�d in the context of (6.5.4), (6.5. 14) and 1

(6.5.20).

THEOREM 6.5.2. Suppose that M is an arbitrary regular compact polygon,

h (s( 1 - s »

=

h f { s { l - s»

and the function h )(s(1 - s)) satisfies the conditions of Theorem 4.2.2. Then the following Selberg trace formula holds for the operator of Dirichlet 's problem in Theorem 6.4.6:

§6 . S . SELBERG TRACE FORMULA FOR THE DIRICHLET PROBLEM

� h ( Aj ) .J

=

131

� I i: r (tanh wr ) h ( ! + r 2 ) dr d� 1 1 j oo exp+ (- 2 w rk/d ) h ( ! + r 2 ) dr +! I

4 ( R�}r k = I d sin kw/d 1 M In N( P ) 1

+ '2 �

- 00

00

exp (- 2 wr )

4

� -k g ( k In N( p )) k (P} rM k = 1 N( P ) /2 - N( P ) /2

1 l: � : . . In N( P ( E9 y ) ) 2 ( �� Y}r k = N( E9 y ) k - I/2 + N( E9 y rk+ I/2 M 1 tr( ��y)*o

1 X g ( ( 2k - I )In N( E9 y ) ) - 4

�

{ ��Y } rM tr( &y)=o

In N( P( E9 y ) ) · g(O) , (6 .5 .21 )

where Aj on the left runs through the set of all eigenvalues of this Dirichlet problem, and the summation on the right is taken over all primitive classes {R hM ' { P hM and { E9 y hM , tr( E9 y ) =1= O. All the series and integrals in (6.5.21) are absolutely convergent, and the sum over {E9yhM , tr( E9 y ) 0, contains only finitely many terms. Now, just as in §4.4, we extend the class of functions h for which (6.5.21) remains true. We give two definitions (here M is any regular compact polygon): w ( x ; E9 fM ) =

{number of all primitive relative conjugacy classes {E9yhM I N( E9y) ..;;; x} , NM ( A ) {number of all eigenvalues Aj of Dirichlet's problem for the operator -L on M I Aj ";;; A}. The following lemma follows from Theorem 6.5. 1 . LEMMA 6.5.2. w ( x ; E9 fM ) = O( x ) as x -> 00 . LEMMA 6.5.3 (Weyl' s formula). NM ( A ) I M I A/rw as A -> 00 . PROOF. The method of proof is analogous to the method of proving Theorem 4.4. 1 ; it is based on the trace formula (6.5.21 ) which we derived. We have = =

�

(6.5 .22) ( -> 0 , (>0

The lemma follows from the Tauberian theorem, because of the monotonicity of the function NM ( A). The proof is complete. From Lemmas 6.5.2 and 6.5.3 we obtain a theorem analogous to Theorem 4.4.4. THEOREM 6.5.3. The Selberg trace formula for the Dirichlet problem (6.5.21) is true as an identity for any function h( A) satisfying the conditions of Theorem 4.4.4.

Trace formul a for a noncompact pol y gon M. Here we shall keep the basic b) notation of the preceding subsection. Recall that the classical Selberg trace formula for a Fuchsian group of the first kind f with noncompact fundamental domain differs from the cocompact version of the trace formula because of the presence of additional terms coming from the parabolic conjugacy classes in f and the terms which are connected with the continuous spectrum of the operator m(f; X) (see

132

VI. SECOND REFINEMENT OF THE EXPANSION THEOREM

§4.3). The Selberg trace formula for Dirichlet' s problem, which we shall now derive from the spectral trace formula (6.5.2) for a noncompact regular polygon M, also differs from (6.5.21) because of the presence of new terms. In the preceding subsection we gave a classification of the relative conjugacy classes {51' hM for a cocompact group fM . If fM is constructed from a noncompact regular polygon M, then the classification of the {5yhM is analogous, except that there are new classes {5yhM for which tr(5y) = ° and the centralizer fM (5y) is generated by a single elliptic generator of order two. In other words, for each such I' E fM there exists an element g E G having the properties

{ ( � _°1 ) (mod 1 ) } , g fM ( 5y ) g I { ( � � ) ' ( _�_ � ) (mod ± I ) } , I g5yg- 1 -

=

5=

+

(6.5 .23)

=

where a ;;:. 1. These classes play the role of the parabolic classes in the classical Selberg trace formula. We proceed now to derive the desired formula. By analogy with (4.3. 14) - (4.3. 16), (6.5.4) and (6.5 . 1 0), the left side of (6.5.2) is equal to

� fFMk( u(z, z » dJL ( z ) + � ( R�}r fFM( R )k ( u( z , Rz » dJL ( z ) M

.

-� 1 2

�

fF ( �)Y)k ( u ( z , 5yz » _

{ [, Y } ru tr( �; y ) "", 0

M

dJL( z )

�

{ fD y hu tr( fD y ) = O rM ( [, y ) hyperb.

�

�

{ fD y hu y' E rM( fD y )\rM tr( fD y ) = O rM( fD y)

k ( u ( Z , y , 1 5yy z )) dJL( z ) , (6.5 .24) -

'

ellip .

where the sum is taken over all elliptic conjugacy classes {R hM ' hyperbolic con­ jugacy classes {Ph , and parabolic conjugacy classes {Sh in fM with representa­ M M tives R, P and S, respectively, over all relative nondegenerate classes {51' hM (i.e. {5yhM , tr( & y ) =F 0), and over all relative degenerate classes {5yh , tr(5y) = O. M Here we consider separately the sum over all {5yhM for which the centralizer

§6.5. SELBERG TRACE FORMULA FOR THE DIRICHLET PROBLEM

133

is a cyclic hyperbolic group, and the sum over {0 Y h with an elliptic M centralizer fM(0y). Some of the other notation in (6.5.24) is as follows: FJ = Fo = Fo( a) is from Theorem 1.2.4, where a = Y; f = fM; FJ is the compact part of the fundamental domain FM, which we choose depending on a large parameter Y > 0; and FM( R ), FM( P ) and FM( 0 y ) are the fundamental domains in H for the centralizer fM( R), the centralizer fM( P ), and the relative centralizer fM( 0 Y ), respectively, of the elements R, P and 0 Y (all of these centralizers are taken in the group fM). If the function h(s(1 - s» satisfies the condition in Theorem 6.5.1 , then the terms in the sum (6.5.24) corresponding to the identity, the elliptic classes, the hyperbolic classes, the nondegenerate {0yh , and the degenerate {0 Y h with hyperbolic M M centralizer, are defined by absolutely convergent series and .integrals, and are computed by analogy with the corresponding contributions to the classical Selberg trace formula (see §4.3) and the trace formula in the last subsection. We shall not carry out these computations here, but shall immediately formulate the result in Theorem 6.5.4 ; first we focus our attention on the last term in the sum (6.5.24) with the limit. By Theorem 4.3.5, for f = fM' X = 1 and dim V = 1 we have fM( 0 Y )

n

- 47T

f oo h ( "41 + r 2 ) r f' ( 1 + ir) dr + 0 ( 1 ) , - 00

y- oo

( 6.5.25 )

where n is the number of pairwise inequivalent cusps on the fundamental domain FM' or the number of primitive parabolic conjugacy classes in fM' and g( u) is the function in (3.3.16). From the definition of fM it is not hard to show that n is equal to m(M), the number of zero interior angles in the polygon M. We now find the asymptotic behavior as Y 00 of the remainder term in the integral under the limit in (6.5.24):

-

�

y' E rM( f9 y )\ rM

k ( u ( z , y , - 1 0 YY'Z)) dfL( Z ) .

(6.5.26 )

It can be shown that in the class of functions k(t) considered in Theorem 6.5.1, the asymptotic behavior of (6.5.26) does not depend on the order of the summation and integration. In addition,

134

VI. SECOND REFINEMENT OF THE EXPANSION THEOREM

where I fM ( E9y ) I is the order of the group fM ( E9y ). Hence, the integral (6.5.26) is equal to the sum

1

4

( 0 Y } rM tr( 0 y ) = O rM( 0y) ellip.

f

k ( u( z , E9yz )) dp.( z ) .

U y'Fj; , y ' E rM

( 6.5 .27 )

We now find the asymptotic behavior as Y � 00 of each term in (6.5.27). The technicalities of the derivation of this asymptotic behavior are reminiscent of the computation of the contribution of the degenerate elliptic elements to the Selberg trace formula for three-dimensional Lobachevsky space (see [58], and also [59], §5.4). Let E9y be as in (6.5.23). For simplicity, we first suppose that g = E, the identity of G, in (6.5.23). In addition, let E9 be a reflection relative to a side of M which is adjacent to a zero interior angle of M; let E9 be given by (6.5.6). We must find the asymptotic behavior of the integral

f

k(u(z , E9 z )) dp.( z ) .

U y Fj; y E rM

( 6.5 .28 )

We have the obvious equality ( 6 .5 .29 )

where z = x + iy. The domain of integration in (6.5.28) is all of the half-plane H except for a certain set concentrated in neighborhoods of the fM'parabolic points of the absolute of H whose Lebesgue measure vanishes in the limit as Y � 00 . On the other hand, if we study the behavior of the function k (4x2Iy2) for z = x + iy on H, we see that the set of all points at which (6.5.28) may diverge is concentrated in the region x � Y and in a neighborhood of the point x = 0, y = 0. By our assumption, the centralizer fM ( (9 ) contains an elliptic element of order two (mod ± 1 ) .

We let B( Y ) denote the set obtained from H by removing from it the two regions B \ ( Y ) = { z E H I x � Y} and B2 ( Y ) = yoB\(Y), i.e., B ( Y ) = H B\ ( Y ) - B2 ( Y ) ' ( 6.5 . 30 ) We fix a sufficiently large Yo > 0, and we consider the intersection B ( Yo ) : n U yFi; . ( 6 5 . 31 ) -

.

From the definition of B( Yo ) it follows that the difference

vanishes in the limit as Y �

f

-

This gives us

k( U ( Z , 0Z )) dp.( z ) =

U y Fit

y E TM

00 .

f

( )

4x 2 dxdy + 0( 1 ) . k - B( Y ) y2 y2 Y � oo

§6 . S . SELBERG TRACE FORMULA FOR THE DIRICHLET PROBLEM

) dxdy

We now find an asymptotic expansion for the integral 4X 2 y2 y2 ·

o

00

k(t) x

B(Y), the integral (6.5.32). is equal to the sum

dx fa Y2/ y ydy2 k ( 4yX22 ) la2!r ydy2 fyvaoo 2!I'Y- dxk ( 4yx22 ) · 1

By the definition of 1

f8(Y)k ( +

0

135

(6.5 .32)

(6.5 .33)

Recall that was chosen in accordance with Theorem 6.5. 1 , and it is connected with h(s( l - s » by the transformation (3.3. 1 6). We make a change of the variable of integration in (6.5.33): 4

Y

dx dt/ {i . =

The sum (6.5.33) is equal to dy 1 00 4 y 0

!fY a2/ y

+

k(t) dt ! la 2/ Y dy f oo k(t) dt . 4 Y 4(a2/yY-I) {i {i 0

The first term in (6.5.34) is equal to I 1 2" g(O)ln Y - 2" g(O)ln

( 6 .5 .34)

a,

a 2(yYtl T.

where g( u ) is from (3.3. 1 6). In the second term in (6.5.34) we make a change of the variable of integration y : = This term is equal to

! f oo dT f oo k( t) dt = ! f ood(In T ) f oo k({it) dt 4 I T 4(T-1) {i 4 I 4(T- I) k(t) dT d f oo --dt - 1 f oo (In T)dT 4(T-I) {i k( t) In 2 1 [ 00 - -g(O) T + In( t + 4) {i dt. 4

=

-

I

4

_

Thus, the integral (6.3.32) is equal to 1 1 In 2 1 2" g ( O ln - 2" g(O)ln a T g(O) + 4

) Y

-

Jo

l OO In(t + 4) k{i( t ) dt . 0

( 6 .5 .35)

We now transform the integral

lo°O ln(t + 4) ky�t ) dt.

( 6 .5 .36)

The integral (6.3.36) equals (see (3.3. 1 6» - � 1 00 'TT

0

dQ ( w )lw In(Vwt + 4)t dt

=

=

0

{i

-

1 1 1 {i {l=t dt � l °O Q (w ) l l (w 4/dTdw T)Fb - T 1 00 dT 2 1n 2g (O ) - 1 Q (w ) d w 1 1 (w + 4/T ) yT y� l -T

� g(O ) 2 1n 2 'TT

+

0

'TT

0

+

'TT

0

0

0

c

+

.

V I . SECON D REFIN EMENT OF THE EXPANSION THEOREM

136

00 1 1 dT 1- 1 Q(w) dw 1 1 7T (w + 4/T ) F f1=-; = -7T Q(W)7T W + 4 + 2/w 4 dw 00 x 1 1 - exp( -u) 1 fX 1 = I g( u )tanh ( -4U ) du = -7T - x h ( 4 + r 2 ) exp (-2iru) 1 + exp ( -u ) dudr ( l + 2ir)) 2 1 x g(2u) du !7T f Xx h ( �4 + r 2 ) l OO exp(I +-uexp( dudr u) 2:I h ( "4I ) 7T2 fX_ x h ( 4I + r 2 ) { 2:I ( rf' (I + ir) - rf' ( 21 + ir ) ) } dr I x f' = 2:I h ( "4I ) + 7T i h ( 41 + r 2 ) r ( 2:I + ir ) dr x - � i: h ( 1 + r 2 ) � ( 1 + ir) dr.

We continue the computation as follows : 00 0

0

o

=

+

0

0

o

0

_

=

-

Consequently, the integral (6.5.36) is equal to

2g(O)ln 2 + � h ( ! ) + � i:h ( ! + r 2 ) � ( � + ir ) dr oo ( f ! 7T - 00 h �4 + r 2 ) .cf ( I + ir) dr,

1 h ( -1 ) + -1 foo00 h ( -I + r ) -f' ( -1 + ir ) dr 1 1 O)ln Y - -g(O)In + a -g( 4 f 2 2 8 4 4 7T 2 - 417T i:h ( ! + r 2 ) � ( I + ir) dr. (6.5 .37)

and the integral (6.5.32) is equal to

2

Finally, the integral (6.5.28) differs from (6.5.37) by 0(1) as Y 00 . We now consider an arbitrary term in the sum (6.5.27) which corresponds to a having the property (6.5.23). The integral (6.5.28) is replaced by class �

{S y hM

f

k(u(z, Sz)) dp,(z).

U gy Fi; y E rM

We easily observe that the asymptotic behavior of this integral as Y 00 is the same as that of (6.5.28), except that it is computed with respect to the variable vY, where v is a fixed positive number which only depends on the transformation in (6.5.23), and hence on We set v = We sh,;tJ� also let denote the number a in (6.5.23). Taking (6.5.37) into account, we ob tain the following value for the sum (6.5.27) as Y 00 : �

S y.

v(Sy).

�

-

1 ( M )g(O)ln Y + "81

"8 q

�

{ & Y } rM tr( & y ) = O rM ( & y ) eHip.

a(Sy)

g

) ) g(O) Y In( a(S v( S y)

1 f oo h ( ! + r ) f ' ( ! + ir ) dr - �4 q ( M ) [�8 h ( �4 ) + _ 4 f 2 47T 1 h ( ! + r 2 ) � ( 1 + ir) dr] + 0(1 ), . 4 7T i: 2

- 00

Y � 00 ,

(6.5 .38)

137

§6 .5 . SELBERG TRACE FORMULA FOR THE DIRICHLET PROBLEM

where q(M ) is the number of classes {£yhM having the properties that tr( £ y ) = 0 and fM ( £ y ) is elliptic. Since the limit in the sum (6.5.24) is finite as Y � 00 , from (6.5.25) we conclude that the number q(M) is finite, and q(M ) = 4m(M ), where m( M ) is the number of zero interior angles in M. We finally obtain the following expression for the limit in (6.5.24):

�

lim

f

(

�

�

Y--+ oo FJ, {S hM y ' E rM(S ) \ rM

k( u ( z , y ,- ISy 'z ))

�

�

y ' E rM ( & y ) \ rM { & Y} rM tr( & y ) = O rM( & y ) ellip .

�

{ & Y } rM tr( & y ) = o rM( & y ) ellip.

In

k(�(Z, y'-I£YY'Z))

)

dp. ( z )

a ( £y ) - m f oo h I r 2 f' 1 r "2 + lr dr. 4'17' 00 4 + p( £y) -

(

) (

.

)

( 6.5 .39)

We note that finiteness of the number of classes {£yhM for which tr( £ y ) = 0 but the centralizer fM ( £y) is a hyperbolic group can be proved directly from the finiteness of (6.5.24), just as was done for the analogous assertion in the preceding subsection. We have proved the following theorem, which is analogous to Theorem 6.5.2. We shall state it using the notation adopted in this section. THEOREM

6.5.4. Suppose that M is an arbitrary noncompact regular polygon,

h(s(1 - s» = h�(s(1 - s» , and the function hl(s(1 - s» satisfies the conditions of Theorem 4.2.2. Then the following Selberg trace formula holds for the operator of the Dirichlet problem in Theorem 6.4.6: � h ( AJ £ r (tanh 'lT r ) h + r 2 dr )

= 1 ::;1 : 1 +4

{

�

d� )

rM k = )

00 1 + "2 � � { P }r k = )

M

1

2 +

)

(!

1

d sin( k 'lTjd )

f oo exp ( -2 'lTrkjd ) h ( 1 + r 2 ) dr - 00

-

1 + exp( 2 'lT r )

4

In N( P ) g( k ln N( P » k/2 - N( P ) -k/2 N( P )

�

In N( P ( £ y » g (( 2 k - I )ln N( £y » k 1/2 /2 N( £ y ) k - I + N( £ y r + { & Y } rM k = l

�

tr( & y ) * O

g(O) 1 2

4

�

{ & Y hM tr( & y ) = O I'M( & Y ) ellip .

a( 0Y )

In p

( £y )

- m In 2

- "21

m f OO h 1 r -4'17' 4+ -. 00

�

{ & Y}rM tr( & y ) = O I'M ( 0 Y ) hyperb.

In N( P( £y »

( - 2 ) -[[' ( -21 + lr. ) dr,

( 6 .5 .40 )

138

VI. SECOND REFINEMENT OF THE EXPANSION THEOREM

where 'Aj on the left runs through the set of all eigenvalues of this Dirichlet problem, and the summation on the right is taken over all primitive classes {RhM , {PhM and {t9yh , tr(t9y) =1= O. All the series and integrals in (6.5.40) converge absolutely, and both ofMthe sums over the classes {t9yhM , tr(t9y) = 0, contain only finitely many terms. We extend the definitions of the distribution functions 7T (X; t9 fM) ' NM('A) (see the previous subsection) word for word to the case of a noncompact regular polygon M. LEMMA 6.5.4. l) 7T ( x ; t9fM) = O( x ), x 2) NM('A) � I M I 'A/4 7T , 'A � 00 .

� 00 ;

The proof of these facts is similar to the proof of Lemmas 6.5.2 and 6.5.3. In part 2), instead of (6.5.22), from (6.5.40) we have the estimate

t ---> O,t>O

which, by the Tauberian theorem, also leads to the desired result. We conclude the section with a theorem analogous to Theorem 6.5.3. THEOREM 6.5.5. The Selberg trace formula for the Dirichlet problem (6.5.40) is true as an identity for any function h('A) satisfying the conditions of Theorem 4.4.4. It remains for us to note here that the Selberg trace formula for the Neumann boundary value problem on a regular polygon M now follows trivially by Theorem 6.4.6, the classical Selberg trace formula (4.3.35), and the Selberg trace formulas (6.5.21) and (6.5.40) for the Dirichlet problem. §6.6.

Elements of the theory of the Seibert zeta-function for the Dirichlet boundary value problem on a regular polygon

In this section we shall define a zeta-function Z M (s), which we shall call the Selberg zeta-function for the Dirichlet boundary value problem on the regular polygon M C H; then we shall prove several of its basic properties. The basic results of the section were first published in [65] (see also [66]). We shall carry out the proofs in the context of an arbitrary noncompact regular polygon. However, since all of the results we obtain will be consequences of the Selberg trace formula for the Dirichlet problem, and since (6.5.2 1 ) is formally a special case of (6.5.40) (in (6.5.40) it sUffices to take m = 0, and to take the set of classes {t9yh for which tr(t9y) = 0 and fM (t9y) is elliptic to be the empty set), it M follows that, in particular, we shall obtain the results for an arbitrary compact regular polygon as well. We proceed to the definition of the zeta-function. In accordance with (6.5. 1 5)­ (6.5. l 7), we introduce the following numerical function on the classes {t9Y}r and M tr( t9y) =1= 0: a( t9y)

=

{

2 p(t9y) ' 1 , p( t9y )

( t9 y ) 2 , = y. =

(6.6 . 1 )

§6.6. THEORY OF THE SELBERG ZETA-FUNCTION

139

We introduce the following product over the primitive classes {PhM and { 0Y hM , tr( &� y ) =1=

0:

Z M (S) = II II (I _ N(PfS- k ) 2 00

( P } rM k = O

S 1, Z M (s)' is

4.4.1, 6.5.2

6.5.4.

(6.6.2)

which converges absolutely for Re > by Lemmas and From the definition oLa(0Y) it follows that an analytic function of in this half-plane. We now derive a representation for the logarithmic derivative of which is analogous to the formula for the classical Selberg zeta-function; all of the basic properties of and the spectral applications will follow from this formula.

Z M (S)

(5.1.2) Z M (S)

s

6.6.1. Suppose that s E C, b E IR , Re s > 1 and b > 1 is fixed. Then ZM (s) 4 1 M 1 ( s _ 1. ) � ( _1_ _ 1 ) S+k b+k 2 ZM 7T - 2 { R�} rM kd�'= 1 d Sin( !,,/d l [ 7T ( exP ( -21Tik ( S - � ) /d ) ) (I - exp(- 21TiS)) - 1 THEOREM =

_

k =O

+4 ( s - -21 ) � [ ( s - 1/2)I 2 + r/ - (b 1/2)l 2 + r/ l + + 1 6 ( s - -21 ) ' j ( 6.6.3) where C l 5 and C I6 do not depend on s, and Aj = 1/4 + r/- (The rest of the notation is from Theorem 6.5.4.) PROOF. In the trace formula (6.5.40) we choose for h(l/4 + r2) a function h(l/4 + r 2 ; s; b) which depends on two parameters: h ( I/4 + r 2 ; s; b ) = (s - 1/2)1 2 + r 2 ( b - 1/21 f + r 2 . c1 5

_

5.1.1,

Just as in Theorem we first suppose that in has the form

1

<

Re

c

s b. The function g( u) = <

g(u; s; b) (3.3.16) g( u) g( u; s; b) 2s � 1 exp ( - ( s - � ) I u I ) - 2b � 1 exp ( - ( b - � ) I u I ) . =

=

140

V I . SECOND REFINEMENT OF THE EXPANSION THEOREM

We have the formula 4� h J

( ± + r/; s; b ) 2 I � I i: r( tanh 7Tr ) h ( ! + r 2 ; s; b ) dr d�l 1 f e p ( - 27Trk/d ) h(� 2 . . +2 � 1 + exp(-27Tr) 4 + r , S, b ) dr d sin k 7T/d =

oo

k= l

{ R } rM

x

- 00

Z� 1 Z� 1 -(b) + 2c -(s) M (s - 1/2) (s - 1/2) ZM (b - 1/2) Z M 2m f h ( 1 + r 2 ; s; b ) rr' ( 1 + ir ) dr, - --;;;(6 .6.4 ) 2 4 1

+

_

oo

oo

1/2 'j

1/2 - i'j

where both values + and leading to the same Aj taken in the sum over j on the left. In addition, 1 cM = 4

_�

{ b ' Y } rM tr( b� y ) rM( &� y ) el1ip.

=0

In ( a(( &;&;yy )) ) - m In 2 - 2"1 v

�

{ &� y hM tr( &� y ) rM( b� y ) hyperh .

=0

=

1/4 + r/ are

In N(P(&;y)) . ( 6.6.5)

To prove (6.6.4) it suffices to verify the equality

ZM 1 � In N( P) g(kln N(P); s ; b) (s) (b) 4 � s - 1/2 Z M b - 1/2 Z M k = N( p ) k/2 N( p ) -k/2 ( ) I N(&; y ) g ( (2 k - 1) ln N(&; y ) ; s; b) (6.6.6) -4 � a &;y n k - I /2 + N( &;y fk+l /2 ) N(&; k= y 0 tr(t;; y)* where a( &;y ) was defined in (6.6. 1), and to use the trace formula (6.5.40). We now

I

ZM

=

00

{ b� Y } rM

I

{ P } rM

_

I

prove (6.6.6). From (5. 1 .5) we have

Z' r 1 ) - 1 Z' b ; 1 ) -Z s - 1/2 (s ; M ; b 1/2 -Z ( ; rM 2 � k=� l N(P) kIn/2 N-( NP)( P)_k/2 g(kln N(P) ; s ; b) , 1

_

=

00

(6.6.7)

{ P } rM

where Z( ; rM; X ) is the classical Selberg zeta-function for the group rM and the trivial representation X, dim V = 1 (see (5.5. l )). Hence, it suffices to consider the sum

s

-4

'" �

{ 0 Y } rM

tr( 0 y ) * O

�

a (&; y ) ln N(&;yk)-g(l (22 k - I ) In Nk +1(&;y/2 ) ; s ; b) . N( &; y ) / N( &; y f k= l �

+

(6.6.8)

§6.6. THEORY OF THE SELBERG ZETA-FUNCTION

This sum is equal to -

4 ,�

{ t9 Y } rM tr( fD y ) �o

(

�

a ( t9 y ) In N( t9y ) I N ( t9y rs(2k - l) -2k + l 2s - 1 k= I I + N( <9"" y ) = -4 X

�

00

{ fD Y } rM tr( fD y) � O

_

141

I

N rb(2k - l » 2b - 1 ( t9y

)

00

� � a ( t9y ) In N( t9y )

k= l / =O

{ 2s � 1 N( t9y r(s + /)(�k- l)

-

2b

� 1 N( t9y r( b+ /)(2 k - l) } ( _ 1 ) / .

(6.6.9)

Here we have expanded in a geometric series. Furthermore, the antiderivative with respect to s of the function

-4 �

{ fDyl rM tr( fD y ) � O

� �

k= I /=0

is ��( t9y ) N( t9y r(s+ /)(2k- l)(_ 1 ) 1

a ( t9 Y l

is equal to

� � (- 1 )

1

s - 1 /2

{ fD y l rM tr( fD y ) � O

k.J

k.J

k = I / =0 2k

_

1

1

N( <9"" y ) -( s+ /)(2k - l ) +

e\7 '

( 6 .6 . 1 0)

We note that the desired formula (6.6.3) and (6.6.6) determine Z M (S) up to a multiplicative constant; hence, at this point we can ignore the constant of integration e \ 7 ' Using the series 00 1 +x x 2k- 1 In 1 x = 2 � ! 2k - 1 ' k= -

I :� 1 < 1 ,

we rewrite (6.6. 1 0) (except for e \ 7 ) in the form 1

S - 1/2

After we multiply by s

-

(6.6. 1 1 ) t the sum (6.6. 1 1) is the natural logarithm of the product

We make analogous transformations with the other sum in (6.6.9), which depends on b. Comparing the resulting formulas with (6.6.2) and taking (5. 1 . 1 ) and (6.6.7) into account, we obtain the desired equality (6.6.6). We now return to (6.6.4), from which the theorem will easily follow. In fact, one need only recall the proof of Theorem 5. 1 . 1, namely, (5. 1 .7) and (5. 1.8). In addition,

142

VI . SECOND REFINEMENT OF THE EXPANSION THEOREM

one must use the easily verified equality

2m

-;-

f oo h ( 41 + r 2., s,. b ) rf ' ( 2"1 + I.r ) dr -_ - 2m rf ' ( 1. - s ) s 1/2 -

00

_

00

1 2 2 k = O ( s - 1 /2) - (k + 1/2 )

+ 4m �

+

Cl8

( C 1 8 does not depend on s), which replaces (5. 1.9). The proof is complete.

It is not hard to prove the following theorem from Lemmas 6.5.3 and 6 . 5.4 and Theorem 6.6.1 . THEOREM 6.6.2. ZM(S) is a meromorphic function of s and C of order equal to two. (The exact estimate for the order of meromorphicity will be a consequence of Lemma 6.7.3.) The next two theorems are analogous to Theorems 5.1.3 and 5.1.4, and are proved by studying the residues on the right side of (6.6.3). THEOREM 6.6.3. The zeros of the function ZM(S), s E C, are at the following points

and only at those points (see the remark following Theorems 6.6.3 and 6.6.4). I. Nontrivial zeros: on the line Re s = t symmetric relative to the point t and on the interval [0, 1], symmetric relative to the point ±. Call those zeros Sj . Each Sj has multiplicity equal to four times the multiplicity of the corresponding eigenvalue Aj of the operator of the Dirichlet boundary value problem (see Theorem 6.4.6), Aj = sil - Sj ) = ± + 'j2 , and Aj runs through the entire spectrum of that operator. II. Trivial zeros: 1) at the points s = -I, 1 = 0, 1, 2, . . . , with multiplicity n , equal to

( for the notation, see §6.5); 2) at the points s = 1 = 1 , 2, . . . with multiplicity 2m. THEOREM 6.6.4. The poles of the function Z M(S ) are at the points s = I, I E 7L (with

multiplicity 2m), and only at those points (see the remark following Theorems 6.6.3 and 6.6.4). REMARK ON THEOREMS 6.6.3. AND 6.6.4. I) The zeros and poles of Z M ( S ) are given

independently of one another in the theorems. The final picture of the zeros and poles is obtained by comparing the assertions in both theorems. 2) There is reason to expect that there are actually no nontrivial zeros of Z M ( s) on the interval s E [0, 1], s =1= 1, just as there are no corresponding eigenvalues of the operator of the Dirichlet problem. THEOREM 6.6.5. Z M( s) satisfies the functional equation

( 6.6 . 12 )

where , by definition,

§6 . 7 . THE REMAINDER IN WEYL 'S FORMULA

'l'M ( S ) = ( eXP { - 4 I M l j

's - 1 / 2

(

[

o

t tan '7Tt dt + 2'7T �

{ R hM

143

d-I 1 k�= I d sm. ( k '7Tjd )

l)

( 2'7Tiktj�) dt exp ( -2'7Tiktjd ) + exp 1 + exp ( -2'7Tit ) 1 + exp 2'7Tlt }o f 2 (1 - s) I ( 6.6 . l 3 ) + 4 cM s - -2 f2 m ( S ) ' with the constant C M defined by (6.6.5). x

r s - 1 /2

(

)}} m

t , _

PROOF. Starting with (6.6.3) , we write out the sum of the logarithmic derivatives of

the zeta-function at s and 1 - s. We have 4 1 MI Z� Z� I OO 1 s s ) ( ) ( + = "2 Z '7T Z

M

(S ) ( I - 1 ) k�O I - s + k s + k

M

- 2'7T X

d- I � �

{ R } rM

k= I

1 d sin (k '7Tjd )

[ exP(-2'7Ti k(S - Ij2) jd ) exp(2'7Tik(s - I j2) jd ) 1 + 1 - exp(2'7Tis) 1 - exp( -2'7Tis )

f' f' + 2m r ( 1 - s) + 2m r (s) - 4 cM

·

If we integrate and exponentiate this formula, we obtain the theorem. The proof is complete.

§6.7.

Estimate of the remainder in Weyl's formula for the Dirichlet boundary value problem on M

The purpose of this section is to prove the following asymptotic formula for the distribution function NM ( A. ) for the eigenvalues of the operator P I ( E9 ) m ( fM ; 1) for the Dirichlet boundary value problem (see Theorems 6.4.3 and 6.4.6) on an arbitrary regular polygon M: T ---> oo

where I M I is the d/L-volume of m is the number of zero interior angles in and the constant cM is defined by (6.6.5). A similar formula, but with a somewhat worse remainder of order O( T ), was first published in [65] (see also [66]). The method for deriving (6.7.1), like the method for refining the Weyl-Selberg formula for N( A.; f; X) (see §5.2), generalizes a well-known method in analytic number theory for construct­ ing an asymptotic formula for the number of nontrivial zeros of the Riemann zeta-function in a large rectangle in the critical strip (see [57]) ; but here it is based on a study of the Selberg zeta-function for Dirichlet's problem (see §6.6). In deriving (6.7. 1), we shall use results from the theory of the classical Selberg zeta-function Z( s ; f; 1) for cocompact groups f (see [19] and [44]) and for f E 9)( 2 (see §5.1 ).

M,

M,

144

VI . SECON D REFINEMENT OF THE EXPANSION THEOREM

LEMMA 6.7.1 .

8NM

( � + T 2 ) = ! arg i'M ( � + iT ) + ! arg ZM ( � + iT )

-t

��� ,

where the function i'M ( S ) is defined by (6.6. 1 3). The values of arg i'M d + iT) and arg Z M (t + iT) are chosen in the same way as in Lemma 5.2. 1 for the functions i'r(s; x) and Z(s; f; X), respectively. PROOF. We take positive numbers A > 1 and T, and we consider the contour QT = Q(T; A ) which is the boundary of the rectangle with vertices A - iT, A + iT, 1 - A + iT and 1 - A - iT; here A and T are chosen so that the contour does not pass through the zeros or poles of Z M ( s). We fix A with this property, and we regard T as a large parameter. We have Z'M 1 (6.7.2) s ds = NnT - PnT ' 2 7Tl. j.OT Z M ( ) where Nn T is the number of zeros of Z M (S) inside the contour Q T ' and Pn T is the r.

number of poles there. From Theorem 6.6.5 we obtain

'1"

1QT Z'Z MM ( s ) ds = 2 jQT( n ) Zz'MM (s ) ds + 1QT( n ) i'MM ( s ) ds (6.7 .3)

where Q T( n ) is the half of the contour QT to the right of the line Re s = 1 /2. The values of the arguments of the functions are obtained by continously moving from the point s = A along the broken line consisting of the two segments s A A + iT 1/2 + iT. N ow Theorems 6.6.3 and 6.6.4 imply that =

--+

Nn T - PQT = 8NM

--+

{ � + T 2 ) + 0( 1 ) . T .... oo

Comparing this with (6.7.2) and (6.7.3), we obtain the lemma. The proof is complete. LEMMA 6.7.2.

{

)

! arg 'I'M 1 + iT = 2 / M / T 2 - TIn T . 4m + i ( cM + m ) T + 0( 1 ) 2 7T 7T 7T 7T � "" oo (see the notation in (6.6.4» . PROOF. We make use of (6.6. 1 3), setting s = 1 /2 + iT. We have already consid­ ered the contributions from the identity and the elliptic classes in the course of proving Lemma 5.2.2; hence, it suffices to estimate the contribution from the factor with the gamma-function. Stirling's formula gives us

(

)

2m 4m 4m arg f 1 - iT = -1 arg f ( 1/2 - iT) = (-Tln T + T) + O( l ) . 2 7T 7T 7T T .... 00 f 2 m ( 1/2 + iT ) The proof is complete.

145

§6.7. THE REMAINDER IN WEYL'S FORMULA

LEMMA 6.7.3.

estimate

For any

C1 9 '

c20 E IR,

ZM( S )

C 1 9 :S;;;;

c20, the function ZM(S) satisfies the

= O(exp 1 1m s I)

in the region C l 9 :s;;;; Re s :s;;;; c20 ' 1 1m s I > 1 .

The proof of this lemma is close to the proof of Lemma 5.2.3, so we shall limit ourselves to an outline of the basic steps. It suffices to prove the lemma in the region - 1 :s;;;; Re s :S;;;; 2, (6.7.4) Im s > 1 . We set s =

a

+ i T and first prove the estimate

(6.7.5) We do this by estimating the logarithmic derivative of ZM( S ) in the region (6 . 7 .4) for " admissible" values of T. We use (6.6.3), estimating each term on the right in that formula. As shown in the proof of Lemma 5.2.3, the contributions from the identity and elliptic classes are bounded, respectively, by O(Tln T) and 0(1 ), T � 00 . From the Weyl formula for NM( "A ) (see Lemmas 6.5.3 and 6.5.4) we derive the estimate

[

(a + iT - 1.)2 � (a + iT - 11/2)2 + r/ - ( b - 1 /12) 2 + r/ j

In addition, it is not hard to prove that ( r'/r )( 1

- - iT ) a

=

1

=

O( T 2 ) . T-> oo

O(ln T ) .

This leads us to the formula T� Integrating over the line segment from a +

00 .

(6.7.6)

iT to 2 + iT, we obtain T�

00 ,

which proves (6.7.5). To prove the lemma we now use the Phragmen-Lindelof principle for the sector ?T/4 :S;;;;

arg( s + 1

- i)

:s;;;;

?T/2.

(6.7.7)

On part of the boundary of the sector, i.e., on the segment arg(s + 1 i ) = ?T/4 , we have Z M (S) = 0(1) by the definition of ZM(S) in (6.6.2). Next, we set s = - 1 + iT, T > 1 . To estimate Z M ( - 1 + i T) for large T we use the functional equation for Z M (S). On the right in (6.6. 1 2) we take s = 2 - iT and estimate the behavior of each factor. It follows from the proof of Lemma 5.2.3 that

-

Z M (- l +

iT ) = exp(6 1 M I T ) + 0( 1 ) . T-> 00

Thus, the function

F( s ) = Z M ( S )exp(6 1 M 1 is ) is analytic inside the sector (6.7.7) and satisfies the following estimate there:

F(s ) = exp 0(1 s 2 1) ,

.L<J:O

VI. SECOND REFINEMENT OF THE EXPANSION THEOREM

and the following estimate on its boundary:

F( s )

=

(6 .7 .8)

0(1 ) .

Consequently, (6.7.8) holds everywhere inside the sector (6.7.7). This gives us the lemma. The proof is complete. The proof of the next lemma is analogous to that of Lemma 5.2.4. LEMMA 6.7.4.

T � 00 , where the choice of the argument is described in Lemmas 6.7. 1 and 5.2. 1. Lemmas 6.7.1 - 6.7.4 lead us to a formula of the type (6.7.1), but with a worse order for the remainder term, namely OCT). THEOREM 6.7.1 . The formula (6.7.1) holds. PROOF. For the reasons mentioned at the beginning of the section, we shall limit ourselves to an outline of the basic steps in the proof. It suffices to prove the estimate

T�

(6.7.9)

00 .

By analogy with §5.2, we introduce the notation

)

(

SeT ) = 'TTl arg z M 21 + iT ,

S (X) I , 'J(t ) = max I x tI I We let p denote an arbitrary zero of Z M (S). We set s = a + it, t > 10, and 1 < a < 5/4, a fixed. ";;'

,,;;,

We successively verify the estimates

fa In I Z M ( a + it ) I d a = 0 ( t ) , 1 /2

For a value of the variable a, In Z M( s ) =

i

S I ( T ) = OCT ) . t < a � a, we have

!,�; ( s - ; - iy ( S( y ) dy + 0( 1 ) ,

t � 00 .

For values of the variables a and � , a > 1. 0 < � < t/2, we have In Z M( s ) =

i

f

t

+ t�

�

(s

-

1/2 - jy r I S( y ) d Y + 0 ( �- I 'J(2t )) 0( 1 ) . 1

Next, setting N(T) = 8NM (T 2 + t), we have

t� 00

N( T ) = c2 l T2 + cnTln T + c 23 T + R ( T ), R ( T ) = 2S(T ) + 0 ( 1) ,

t ---> 00

where c2l ' cn. and cn do not depend on s. By definition, N(T) is a monotonic nondecreasing function, i.e., N(T + x) - N(T) ;;;. 0 for x ;;;. O. Let x E (0, T). We have

R ( T + x ) - R ( T ) ;;;. c24 Tx,

§6 .7. THE REMAINDER IN WEYL'S FORMULA

where

C24 does not depend on T or

x.

.

147

Next, we obtain

S( T ) = ( ( T§, 2 T ) 1/ ) , which implies another estimate: +

O

( ) 2

jl/2+ln -llln I Z M ( o it) I do O {t1/4'?f3/4 (4t)ln-1/2 t ) . IP

=

1 - 00

Finally, from the last estimate and the well-known Hadamard three circle theorem we obtain the desired estimate (6.7.9) and the theorem. The proof is complete.

CHAPTER 7

THE SPECTRAL THEORY OF PERTURBATIONS OF THE SPECTRUM OF 1lJE OPERATOR � ( f ; X ) . SOME PERSPECTIVES ON THE DEVELOPMENT OF THE SPECTRAL THEORY OF AUTOMORPIDC FUNCTIONS §7.1. Deformations of the group f, the spectrum of � ( f ; X ) ,

and singular points of the resolvent 9t ( s ; f ; X )

In this section we shall prove a theorem on continuity of the singular points of !R (s; f; X ) as a function of a regular deformation of the discrete group f E WC 2 . For groups f E WC I such a result is well known (see [ 1 1 ]) and lies within the framework of the classical theory of perturbations of a selfadjoint operator with purely discrete spectrum. This is by no means the case for perturbations of � (f; X ) for f E WC 2 and X E 91 if). We begin the section with a short introduction. The perturbation theory for the spectrum of an abstract selfadjoint operator in Hilbert space lies at the base of the proof of the theorem on expansion in eigenfunctions of the automorphic Laplacian � ( f; X) (see Chapter 2). The principle involved can be stated as follows. Operators which are close to one another in some sense must, in general, have spectral properties which are not too sharply different. The quasiresolvent D(s; f; X ) with which one compares the resolvent !R (s; f; X) of �(f; X ) is induced by a simple operator with the same continuous spectrum as �(f; X ). Although D(s; f; X) and !R (s; f; X) are not near to one another in the usual norm in X(f; X ), their proximity in this situation means that the difference 9t (s; f; X) - D(s; f; X ) is a compact operator. Once the theorem on expansion in eigenfunctions of �(f; X ) has been proved, one can ask what are the spectral properties of a selfadjoint operator � e in :Je(f; X) if its resolvent !R eCs) is close to !R (s; f; X ) in the norm. (We are comparing the resolvents rather than the operators themselves, because the former are defined on the entire space :Je(f; X).) Specialists in operator theory will say that there is no clear-cut answer to this question. However, it becomes a well-posed question if it is formulated in a more specialized manner, within the framework of the spectral theory of automorphic functions. We consider a family of groups fe E WC which depends continuously on a parameter E E [0, 1 ] and has the following properties: 1 ) All the fE ' E E [0, 1 ], are algebraically isomorphic to one another, i.e., they have the same signature fE = f( g; m l , . . . , m , ; n ) (see § 1 .2). 2) Let 8E : fo fs be the isomorphism in condition 1). Then for every E E [0, 1 ] there exists a continuous differentiable one-to-one map E E : H H, such that for any y E fo we have EeC yz ) = 8eC y ) EeC z ) for any z E H. �

�

1A Q

150

VII. SPECTRAL THEORY OF AUTOMORPHIC FUNCTIONS

3) The restriction of Ee to Fo, the fundamental domain for fo in H, induces a shift transformation which is a bounded linear operator X( fo ; fe) : X(fo; X) X(fe; X ), which depends continuously on e and is strongly convergent to the identity operator § in X(fo; X) as e 0. The map X( fo; fe) defined above will be called a regular deformation of the space X ( fo; X). The regular deformation X(fo; fe) is said to be trivial if it is associated with the trivial deformation of the group fo, i.e., fE = g(e)fog( etl , where g(e) belongs to G and depends continuously on e. It can be verified that nontrivial regular deformations exist. It is especially simple to verify this for groups fo with signature fo = f( g; 0; 1), but we shall not dwell on this here. We now prove the basic theorem of this section. -,>

-,>

THEOREM 7. 1 . 1 . Suppose that there exists a regular deformation X(fo; fe). Further,

suppose that So is an arbitrary singular point with multiplicity ko of the resolvent 9l (s ; fo; X) as an integral operator, and the disc of radius ro around So has no other singular points of 9l(s ; fo; X) (see §2.2). Then there is an eo > ° such that, for every e os:;; eo and all r os:;; ro , the disc of radius r around So has exactly ko singular points of the resolvent 9l(s ; fe ; X). PROOF. We first prove that the following assertion holds under the conditions of the theorem: For every fixed s with Re s > 1 , the resolvent 9l(s ; fe ; X) depends continuously on e E [0, 1]. In fact, for such s the kernel of 9l(s ; fe; X) is represented by the absolutely convergent series ( 1 04. 1)

r ( z, z'; s ; fe; K ) = � X ( y)k ( z, yz' ; s ),

(7. 1 . 1 )

y E r,

where z *" z' (mod fE). We show that (7.1 .1) converges uniformly for e E [0, 1 ]. It is not hard to verify (see [59], Lemma 4. 1) that the kernel k(z, z', s) is majorized by a function k\(z, z') with the properties a) fHk l (Z, z') dp, (z') < 00 and b) k 1 (z, z') has regular growth, in Selberg's terminology (see [5 1]), i.e., there exist positive constants a, o and c such that for all z, z' E H with u(z, z') ;;>- a we have

where u(z, z') is the fundamental invariant of a pair of points. From the discreteness of fE it follows that the set {z E H I u( yz', z) < o} for fixed y E fE and z' E H can only intersect with finitely many sets of the same form but with y' instead of y, y' E fE. We let N(z'; y) denote this nurp�er of intersections. It is not hard to see that N(z' ; y ) does not depend on z' or y, 'but only on the group fE' and it depends on the latter continuously in e E [0, 1]. We set Nmax = max N. f E [O, I]

We have the estimate �

y ET,

I k ( u ( z, yz') ; s ) 1 os:;; cNmaxf I k \ ( z, z ") I dp,( z") , H

where u(z, z') ;;>- a (mod fJ, and this implies that the series (7. 1 . 1) is uniformly convergent in e E [0, 1] and that 9l(s; fE; X) is continuous as desired.

§7 . 1 . DEFORMAnONS OF THE GROUP r

151

We now proceed directly to the proof of the theorem. We introduce the operator � i s ): �X( fo ; X ) X( fo; X ) by the formula � Js) = X-I( fo ; fe) � (s; fe; K ) X( fo ; fe) . ( 7.1 . 2) The operator X( fo; fJ has the inverse X -I(fo; fe ) , by condition 2), which is a bounded operator for any e, at least in a neighborhood of the point e = 0, by condition 3) in the definition of a regular deformation. Furthermore, since the resolvent has been proved to be continuous, it follows that the difference �( s; fo; X ) - � e< s ) is an operator which is bounded on X( fo; X ), depends continuously on e, and vanishes in the limit as e O. Recallthat s is fixed, Re s > l . We now consider the Faddeev equation (2.2.6) for the operator ?B ( s; fo; X ) and the analogous equa­ tion for ?B ( s; fe; X ), where in both equations we take s in the half-plane Re s > O. We have ( 7. 1 .3 ) ?B ( s ; fo ; X ) = m ( K ; fo ; X ) + w &J ( s ; fo ; X ) ?B (s; fo ; X ) , ( 7. 1 .4) 58 (s; fe ; X ) = m ( K ; fe; X ) + w&J(s; fe ; X ) ?B (s; fe; X ) , where K is fixed, K > 3 (see §2.2). We transform (7.l .4) as follows: ( 7. 1 .5 ) ?B e(s) = m J X ) + w &J e(s ) m e(s ) , where, by definition, ?Be(s) = X-I( fo ; fe) ?B ( s ; fe; X ) X( fo ; fJ , m e( x ) = X-I( fo ; fe) m ( K ; fe; X ) X( fo ; fJ , &)e(s ) = X - I ( fo ; fJ&J (s; fe ; X ) X( fo ; fJ . We note that the operators in (7. l .5) have already been defined as operators in X( fo; X) for Re s > 1/2, s f1. (1/2 , 1] , and in 0[,_ ( fo; X ) for s with Re s > O. To I for the difference ?B ( s; fo; X ) prove the theorem we compose a new integral equation - ?B i s ) from (7. l .3) and (7. l .4). We have ?B (s; fo ; X ) - ?B e( s ) = m ( K ; fo ; X ) - mee K ) + w [ &J ( s ; fo ; X ) - &J e( s ) ] ?B e( s ) + w&J(s; fo ; x ) [ ?B (s ; fo ; X ) - ?B e(s ) ] , ( 7.1 .6 ) and so this difference is determined by the formula ?B ( s ; fo ; X ) - ?B e( x ) = ( 10 - w&J (s; fo ; X ))- I �

�

X

[ m ( K ; fo ; X ) - m e( K ) + w ( &J ( s; fo ; X ) - &J e( s ) ) ?B e( s ) ] ,

( 7 . 1 .7 )

where 10 is the identity operator in X( fo; X ). Recall that (see §2.2) &J (s; fo ; X ) = m ( K ; fo ; x ) ( 10 + w O es ; fo ; X ) ) , 91 (s; fo ; X ) = � ( K; fo ; X ) + m ( x ; fo ; X ) (and analogous formulas hold for fe ) , where � ( K; fo; X ) and O( s; fo; X ) are defined by (2. 1 .27) and (2.2.13), respectively. One can prove that the differences �(K ; fo ; X ) - X - I ( fo ; fJ � ( K ; fe; X ) X( fo ; fe) , O (s; fo ; X ) - X - I ( fo ; fe) O (s; fe; X ) X( fo ; fe)

152

VII. SPECTRAL THEORY OF AUTOMORPHIC FUNCTIONS

are bounded operators in :K (fo ; X) for Re s > 1/2 and in Cj� _ I (fo; X ) for Re s > 0, which depend continuously on f E [0, 0], 0 ';;;: 1 , and vanish in the limit as f O. By this, together with the continuity property above for m ( K; fF ; X) , it follows that the difference --->

have the same properties. We proceed to the consideration of (7. 1.7). We rewrite this formula in terms of the kernels of the integral operators which appear in it, and we take the free variables z, z' E Fo in general position; Fo is a fundamental domain for the group fo. Let So be the singular point in the theorem; it is a pole of multiplicity ko for the operator 'R(s; fo ; X) and ( 10 - w&j(s; fo; X » - I (see Theorems 2.2.3, 2.2.5 and 2.2.6). Let {Sj }7�:i e) be the set of all poles (counting multiplicity) of the operator 'R is) in the disc of radius r around so. If we compare the coefficients in the Laurent expansions at all singular points in the disc of radius ro around So on the left and right sides of (7. 1 .7) (written in terms of kernels, as indicated above), we arrive at the following relation : a s

( ) k ( s - so )

I)

-

l1 ( r ; E ) o j=

�

C/f)

I S - Sj ( f )

1 .

[

n ( ro ; e) ci f) b(s ) . ( f) f, s + 12 ( ) j�= l s - Sj ( f + h ( f , s ) (7.1 .8) II k ) ( s - so ) where a ( s ) , b( s ) , 12 ( f; s ) , h( f; s) are analytic functions of S in the disc of radius ro around so ' for all f E [0, 0 ]. The functions /1(f), /:z(f; s ) vanish in the limit as f 0, c.I.( f) =1= 0, f E [0, 0]. Now suppose that the theorem is false. Then there exist infinite sequences fm and rm, m = 1, 2, 3, . . . , such that fm 0 and rm 0 as m ---> 00 and the numbers n(rm ; fm) fail to be equal to ko for all m . But (7.1.8) easily leads to a -

-

()

--->

--->

__

--->

contradiction with this supposition. To see this, it is enough to pass to the limit zero over the sequence fm in (7. 1.8). The proof is complete. This theorem essentially says that the singular points of the resolvent m(s; f ; X ) depend continuously on a regular deformation of the discrete group. Recall (see §2.2) that the set of singular points in question is situated on the interval s E (1/2 , 1] and the half-plane Re s � 1 /2. Precisely the ones which lie in the interval s E (1/2 , 1] and on the line Re s = 1/2 correspond to eigenvalues X of the discrete spectrum of W(f; X), X = s(l - s) (Re s ;;::' 1/2). In principle, Theorem 7. 1 . 1 even gives new information concerning the discrete spectrum of W (fe ; X) for any group fe which regularly deforms to fo (and lies in a neighborhood of the latter) if we already have information concerning the discrete spectrum of W(fo; X). It is first of all natural to take for fo some arithmetic group fo E 9JC 2, a which has a nontrivial regular deformation; by Theorems 6.1 . 1 , 6. 1.2 and 6.4.7, for such a group the spectrum of W(fo ; X) is very rich. (Here we are restricting ourselves to the case of the trivial representation X, dim V = 1 .) As an example of such a group fo we can take the commutator [fz ' f�], of the modular group fz , i.e., the group generated by all the commutators of fz . It is well known that [fz , fz ] is contained in a one-parameter family of Fricke groups (see §6.3) and has a nontrivial regular deformation.

§7 .2. ZEROS OF ZETA- AND L-FUNCTIONS

153

In [53] for number-theoretic purposes (more precisely, in order to construct counterexamples to the general Petersson conjecture estimating the Fourier coeffi­ cients of cusp form), Selberg constructed a special subgroup of finite index fq C [ fz , fz ] depending on a parameter q E lL. He proved that there is a nontrivial eigenvalue A = s(l - s) of � ( fq; 1) lying between ° and 1/4 (this corresponds to 1 /2 < s � 1 ) and arbitrarily close to 0, depending on q E 71.. Theorem 7. 1 . 1 allows one to infer that there exist nonarithmetic groups f which can be deformed to fq, also having eigenvalues of � ( f ; 1 ) close to A = 0. We shall not dwell on this here, except to note that our " method of contin:q.ity" is especially suitable for studying precisely the eigenvalues Aj (e) = sj (e)(l - si e)) of � ( fe' X ) for which sin) lies in a neighborhood of s = 1 . In fact, the selfadjointness of all the operators � ( fe ; X ) "only permits" the deformation X( fo; fe) to move si e) as a function of e along the interval Sj E ( l /2 , 1 ], and a small deformation cannot throw si e) into the half-plane Re s < 1 /2. On the other hand, Theorem 7. 1 . 1 does not prevent an arbitrarily large discrete spectrum { AiO) = siO)(l - sin))} of � ( fo ; X } for which the corresponding sin) lie on the line Re s = 1/2 from vanishing completely for an arbitrarily small nontrivial regular deformation, because of the displacement of all the si e) into the half-plane Re s < 1/2. This circumstance also shows how difficult Roelcke's conjecture is (see §6. I). It seems to us that it is an interesting (if difficult) problem to study the directions of displacement of the singular points of 9t(s; fe; X ) as a function of a nontrivial regular deformation X( fo; fe)' A predominantly vertical displacement of the singular points (parallel to the 1m s axis) would support the Roelcke conjecture. A good model of deformation in analytic number theory is the behavior of the zeros of the Hurwitz zeta-function ns; a) as it degenerates into the Riemann zeta-func­ tion, i.e., for small values of the parameter a. Recall that the Hurwitz zeta-function is the meromorphic function on C which is given in the half-plane Re s > 1 by the Dirichlet series res; a ) §7.2.

=

CXJ

�

n= ]

1

(n + af '

a E �.

Zeros of zeta-and L-functions of imaginary quadratic fields and the eigenValues of ( 1 ) � fz ;

As we mentioned in the survey [66] (see also Chapter 6), the basic stimuli for the modern development of the spectral theory of automorphic forms for Fuchsian groups f E me 2 are the open problems in the theory of the discrete spectrum of the operator�(f; X ), X E � sef), in particular, the Roelcke problem. If one speaks of analogies and historical roots, then these problems have above all a function-theo­ retic meaning. For example, as already mentioned in §6.3, the problem of the existence of cusp-functions (i.e., the problem of nontriviality of the space Xo( f; X)) is related to the classical problem of Klein and Poincare in function theory at the end of the nineteenth century on the existence of a meromorphic automorphic function for a given Fuchsian group. However, there are other important directions for the development of the modern spectral theory of automorphic functions which arise because of the needs of number-theoretic applications. We now know some number-theoretic directions in spectral theory, such as the study of the connection

154

VII. SPECTRAL THEORY OF AUTOMORPHIC FUNCTIONS

and analogy between the Selberg trace formula and the Voronoi-Hardy summation formulas for the circle problem (see [ 1 8] and [66]), the connection between the spectral theory and estimates for Kloosterman sums in number theory (see [3]), and also its connection with the refined Kummer conjecture for cubic characters (see [41 ], and also [ 1 7]), and so on. Here we shall only discuss one aspect, which we did not discuss in our earlier survey [66]. This is the interrelation between the spectral theory and the theory of the Riemann zeta-function from the point of view of the prospects for the development of the former theory. We indicate some facts which by themselves stimulate the development of the spectral theory. 1 . The analogy between Weirs explicit formulas in number theory and the Selberg trace formula led Selberg to the definition of his zeta-function. 2. The explicit Selberg trace formula for the modular group rz made it possible to derive a new formula for the Tchebycheff psi-function, expressing l{;( x) in terms of the distribution function 71'( x ; rz ) for the norms of primitive hyperbolic conjugacy classes in rz and in terms of the eigenvalues of m-(rz ; 1 ) (see [63]). Formulas of this type can be obtained from the Selberg trace formula with Hecke operators for rz. Although at the present time these formulas do not have practical value for number theory, in the first place they point to the existence of an indirect connection between the spectral theory of automorphic forms for rz and the theory of the Riemann zeta-function; in the second place, they stimulate the study of the function 71' ( x; rz ) and the distribution of eigenvalues of m-(rz; 1). 3. The asymptotic formulas obtained by Hejhal by considering the kernel of the resolvent of 2l(r; 1) for special quaternionic discrete arithmetic groups r E WC \,a (see [20]), for example,

� N{n )N{5n + 1 ) = � ( 1n c) 2 X + � a k x sk + -.0 (X 2 / 3 ) , (7 .2 . 1 ) x oo 71' k n �x where N(n ) is the number of ideals in the field 0(/2) with norm n. The numbers Sk

on the right in (7.2. 1) have a spectral origin. Analogous formulas can be obtained by studying the kernels of the resolvents at special points for other arithmetic groups r E WC 2 • All of these formulas will then have practical value for number theory, because of the extraordinary good (from the point of view of analytic number theory) remainder term. 4. The proof that the Eisenstein series E( z ; s; rz ) has no poles on the line Re S = 1/2 by methods of operator theory (Theorem 2.3. 1 , X = 1 , dim V = 1) led us to an independent proof of the asymptotic prime number theorem (n2s) =1= 0, Re s = 1/2, �(s) the Riemann zeta-function). Thus, the prime number theorem is a consequence of the selfadjointness of the operator U(rz; 1). Finally, we cannot avoid mentioning the connection between sums of Klooster­ man sums and the so-called density theorems in number theory. Progress in estimating these sums based on spectral theory (see [3]) is extremely important for the theory of the distribution of prime numbers and the theory of the Riemann zeta-function. The number of facts of this sort can be multiplied, but this would carry us too far afield. The basic purpose of this section is to prove a theorem which enables one to compare the spectral singularities of the resolvants of automorphic Laplacians

§7 . 2 . ZEROS OF ZETA- AND L-FUNCTIONS

155

defined in two-dimensional and three-dimensional Lobachevsky space. Based on this theorem, we state a conjecture on a possible intersection of the set of zeros of the Selberg zeta-function Z( s; fz ; 1 ) for Re s = 1 /2 and the set of zeros of the Dede­ kind zeta-functions of imaginary quadratic fields. At the end of the section we indicate a new direction in the spectral theory of automorphic functions, whose development is naturally stimulated by this conjecture. We published the basic results before in [69]. We introduce some notation and definitions (compatible with Chapter 1) from the spectral theory of automorphic functions for m-dimensional Lobachevsky space. In what follows it is dimensions m 2 and m, . 3 which will be important to us. Let Hm be m-dimensional Lobachevsky space; Hm = Gm/Km' where Gm is the corre­ sponding Lie group and Km is its maximal compact subgroup. Next, let dS m be the Riemannian metric on Hm which is invariant relative to Gm; let Lm be the Laplace differential operator for the metric dsm; and let dP,m be the Riemannian measure on Hm determined by the metric dsm. We introduce the notation :JCm = Lz{Hm; C ; dP,m ). In addition, m m is the selfadjoint operator on :JCm induced by the operator -Lm on a suitable dense set. We have the resolvent 1Rm(s) = ( m m - sCm - s - l)t i , and the integral operator has kernel =

( 7.2.2 )

where um(z, z') is the fundamental invariant of a pair of points. The following fact follows from Lemma l .2 of [59]. LEMMA 7.2. 1 . The function k m( u; s) in (7.2.2) has the integral representation

where Re s > m/2 - 1 , f( s) is Euler 's gamma-function, and c( m) is a constant depending only on m. Let fm C Gm be a discrete subgroup acting on Hm, and let Fm be a fundamental domain for fm in Hm. We introduce the Hilbert space of automorphic functions on Hm: :JCm(fm ) = Lz{Fm; C ; dp, ; 1 ) (see § 1 .3) and the nonnegative selfadjoint operator m m(fm): :JCm(fm) � :JCm(fm), induced by -Lm. We denote the kernel of its resolvent by rm(zm' z:n; s; fm). The following lemma is proved by analogy with Lemma 1 .4 of [59]. LEMMA 7.2.2. The following expansion in an absolutely convergent series holds in the region Re s > m - 1 : rm ( zm , z:n ; s ; fm) = � km ( um ( z m , yz:n) ; s ) , y Efm

where zm ' z:n E Hm, zm =i= z :n (mod fm), and the series converges uniformly with respect to Zm and z:n in any compact subregion of Hm X Hm which does not intersect the surfaces Zm = z:n (mod fm)·

V l l . Sl'l:.C I KAL I Hl:.U K Y U.t" A U I UMUKl'H I L t· U N L. I I V N :>

As mentioned above, we shall only be interested in the dimensions m = 2 and m = 3. We set where d is an arbitrary squarefree natural number and 7L(�J) is the ring of integers in the imaginary quadratic field OCr-d). LEMMA 7.2.3. 1) Let z 2 ' z 2 ' E H2 be fixed points in general position. The function ri z 2 ' z ;; s; f2 ) has the following poles in the region Re s > 0 which will be of special importance to us: a) a simple pole at the point s = 1 ; b) simple poles Sj on the line Re s = 1 /2 , s =1= 1 /2 , where each pole is connected with an eigenvalue Aj ,2 of the discrete spectrum of m i f2 ) by the formula s/1 - s) = Aj .2 ; and c) poles p at the zeros of the Riemann zetajunction �(2s) having the same multiplicity Re s < 1/2. 2) Let d correspond to a field O(N) of class number one, and let Z3' z � E H3 be

fixed points in general position. The function ri z 3' z�; s; fid » has the following poles in the region Re s > 0 which will be of special importance to us: a) a simple pole at the point s = 2; b) simple poles Sj on the line Re s = 1 , s =1= 1 , where each pole Sj is connected with an eigenvalue Aj,3 ofm l fid» by the formula sj (2 - Sj ) = Aj,3 ; and c) poles p at the zeros of the Dedekind zetajunction �-A s) of O(N) having the same multiplicity, Re s < 1 . PROOF. The simple poles in la) and 2a) correspond to constant eigenfunctions of the operators m i f2 ) and m 3( f3 )' The simple poles in 1 b) and 2b) are guaranteed by our Theorem 2.2.6 and by Theorem 3. 1 of [59]. We proceed to consider the poles in c). From our Theorem 2.3.4 and from Theorem 3 . 1 and Lemma 3.5 of [59] it follows that at least part of the poles of the functions ri z 2 ' z ;; s; f2 ) and ri z 3 ' z�; s; f3 ) are induced by the poles of the determinants of the scattering matrices �is) and �is), respectively. The functions �is) and �is) can be computed explicitly: �

2( s ) =

r;; f ( s - 1/2) �( 2s - 1 ) f ( s ) �( 2s )

(see (6. 1 .6» , and

s 1) � 3 ( s ) = S -7T_1 LA r_d ( S ) _

-

(see [29]); and this implies l c) and 2c). The proof i s complete. In the sum in Lemma 7.2.2 for m . 3 we look at the terms corresponding to summation over f2 C f3( d). In our notation we have

r3 ( z3 ' z� ; s ; f2 ) = � k3 ( U 3 ( Z3 ' 'Yz� ) ; s ) .

(7.2.3)

y E f2

The kernel (7.2.3) is the kernel of the resolvent of m i r2 ) in the space Xif2 )' (Here f2 is considered as a group of motions of the three-dimensional space H3, PSL(2, 7L) C PSL(2, 7L(N» ). Consequently, one would expect the functions ri z 3, z�; s; fid » and ri z 3' z�; s ; f2 ) to have some poles in common i n the region Re s > O. On the other hand, we have the following theorem.

§7.2. ZEROS OF ZETA- AND L-FUNCTIONS

157

There exists a map p: H2 � H3 such that the difference f - I (2s ) r3 ( p ( z2 ) ' P ( Z� ) ; s ; fJ , r2 ( z 2 ' z � ; s; f2 ) - cf( s )f(2s - l ) f- l s

THEOREM 7.2. 1 .

( - �)

where c is a constant and z 2 and z� are fixed points in general position, is a regular function in the region Re s > 0, except at the point s = 1 /2, where it has a pole of order no greater than two. PROOF. We realize the space H3 as the quotient SL(2, C)/SU(2). We have local coordinates {y, x, v}, y > 0; x, v E IR, of H3• In these coordinates the action of SL(2, C) is given as follows (see [59], §5,.�):

z' = gz, z' = {y', x', v'} , i ' = x' + iv', i = x + iv E C , E SL(2, C ) , g=

( � �)

The fundamental invariants of a pair of points are

z -z ' u 2 ( Z 2 ' z 2' ) - I 2 � f Y2 Y2 y - y(,» ) 2 + ( x - X (,» ) 2 + ( v - V (,» ) 2 ( U3 ( Z3 ' z�'» ) = yy ( ,) If the upper half-plane H2 is given by the coordinates x, y, z = x + iy (see § 1 . 1) , then the map p : H2 � H3 is defined as follows: p : x + iy � { y, x, O } . Let Re s > 2. From Lemma 7.2.2 we have r2 ( z2 , z� ; s ; f2 ) = � k 2 ( u i z 2 ' YZ� ) ; s ) , (7.2.4) _

"

r3 ( p ( z 2 ) ' p ( z� ) ; s ; f2 ) = � k3 ( U 2 ( Z 2 ' yz� ) ; s ) . y E r2

We consider the difference

r2 ( z 2 ' z � ; s; f2 ) - g(s)r3 ( p (z2 ) P (Z� ) ; s; f2 ) ,

(7.2.5) where g(s) is specially chosen so that (7.2.5) is given by an absolutely convergent series of the type (7.2.4) in the region Re s > 0 as well. The proof of the theorem amounts essentially to proving the existence of such a function g(s). For Re s > 2, by Lemma 7.2. 1 , the difference (7.2.5) is equal to �

y E r2

{

I I [t ( 1 - t )] S - I ( t + ! Ui Z2 ' yz� ) rS dt 3 g ( s ) ( 3 ) f ( s ) 11 [ t ( 1 t ) ] S 2 ( t + 1 u 2 ( Z 2 ' YZ 2, ) ) -S dt} . ( 7.2.6)

C(2) _

0

c

f( s

_

1 /2)

0

-

--

/

4

•

... _

. ...... ... ....... ........ __ .&

_

.&

_ ... ..... ...

_

.... ...... ... ,I.

"" ... .. ... "".:"... ... ... L '-' J." U J."I '-- J. J.Ul"'�

N ext, integrating by parts, we obtain l 1 1 [t ( 1 - t ) ] S - I ( t + u rs dt ( 1 + u rS l [t ( I - t ) ] S- 1 dt o t + u ) -S - I l t [ T ( 1 - T ) ] S - I d T dt , +s o ( 7.2.7) =

11(

Ia

0

0

Ia

+s \ t + u rS - 1 t [ T( 1 - T ) ] s-3 /2 d T dt .

Using (7.2.7), we choose g(s) in (7.2.6) from the condition l c (2)( 1 + u rSl [t ( 1 - t ) ] S - 1 dt o

= g( s ) c (3 )

f(S ) ( 1 + u rs l [t ( 1 - t ) ] s-3/2 dt , f ( s - 1 /2 )

01

from which it follows that g( s )

=

c (2) f ( s ) f ( s ) f (2s - I ) f ( s - 1 /2) - c (2) f ( s ) f(2s - 1 ) . c ( 3 ) f (2s ) f ( s - I /2)f ( s - I/2)f ( s ) c ( 3 ) f(s - I /2) f ( 2s )

After this choice of g(s), the difference (7.2.5) takes the form

-

X

c

f 2 ( s ) f (2s - I ) ( 2) s 2 f ( S I /2) f( 2s ) _

f.' ( t + ! u , ( z" yz; ) r-' f.' [ T( I - T ) ] , - 3/' dT dt} .

(7.2.8)

We now prove that for u 2 ;;;;. e > 0 the general term in (7.2.8) is bounded from above by O( UiZ2 ' yz� ) -Re s - I ), where the constant in the 0 in general depends on s. By Theorem 1 .2.5, this estimate ensures the absolute convergence of (7.2.8) for Re s > 0 and its meromorphicity in this half-plane. We consider the integral

11( 1 + u ) -S - I lt [ T( I -. T )] S - I d T dt . o

0

(7.2.9)

If u > 0, then obviously (7.2.9) is a regular function in the half-plane Re s > 0; hence, for fixed Z 2 and z� with z 2 =1= z� (mod f2 ) the first series in (7.2.8) with an integral of the type (7.2.9) is a regular function in the half-plane Re s > o. We now consider the second integral ( 7.2. 1 0)

§7.:?. ZEROS OF ZETA· AND L·FUNCTIONS

where we suppose that u > O. If we represent (7. 2 . 1 0) in the form

l \ t + ufs- I l t { ( [ ,. ( 1 - ,. )] S -3/2 - ,. s - 3/ 2 o

0

-

(1

159

- ,.r - 3/2 ) ,. s- 3/ 2 + ( 1 - ,.r - 3 /2 } d T dt , +

it is not hard to see that this integral is equal to a sum of the form

I I ( s; u ) + 12 ( s; u ) / (s - 1/2) + 13 ( s; u) / ( s - 1/2 ) , where for each fixed u > 0 the function 1/ s; u ) is regular in the half-plane Re s > 0, } = 1 , 2, 3, and 1z{1/2 ; u) =F 0, /3 ( 1/2 ; u) =F O. In addition, for each } = 1 , 2, 3 we have

I l'J.(s :

I

and hence the series

Z2

'- - ,

") « - /

u-Res - I .-

,

(mod f2 ), converges absolutely in the half-plane Re s > 0, where it repre­ sents a regular function, except at the point z = 1/2, where it has pole of order no greater than one. To complete the proof of the theorem, it remains for us to observe that the factor =F

z;

f 2 (S) f (2s - 1 ) f 2 (s - 1/2)f(2s) in the second integral in (7.2.8) contributes only one more pole to the function (7.2.5) (with our choice of g(s)) in the half-plane Re s > 0, namely, a simple pole at the point s = 1/2. The proof is complete. Roughly speaking, Theorem 7.2.1 says that the functions have the same spectral poles in the half-plane Re s > O. It seems to us that the combination of all these results gives support for the following conjecture. CONJECTURE I. The set of numbers Sj in part 1 b) of Lemma 7.2.3 has nonempty

intersection with the set of zeros in the Dedekind zeta-function �_As) of any class one field Q(V-d). Recall that all of these numbers Sj lie on the line Re S = 1/2, and they are connected with eigenvalues Aj 2 of the discrete spectrum of � z{f2 ) by the formula

Aj,2 = s/1 - s).

,

It seems that an analogous conjecture can be formulated for a field Q(../-d ) of class number greater than one, but we shall not discuss this here. A less general conjecture than Conjecture I (for the Rieman zeta-function) was stated eariler by L. D. Faddeev, based on other heuristic considerations. Faddeev's conjecture �timulated P. Cartier to undertake computer calculations of approximate values for the eigenvalues of the operator � 2(f2 ) (see [4] and [5]); however, it is still too soon to claim any confirmation of the conjectures, because of the high degree of

.J. U V

V 11.

::IrCL

I lV"\.L I nevI'\. 1 v r t'l. U 1

V1VIVKrniL r- U N L I IUN ::'

impreclSlon in these computations. This work is primarily concerned with the eigenvalues which are connected with the Neumann boundary value problem (see Theorem 6.4.6, rM = r z). These approximate computations undoubtedly are of theoretical interest, and may turn out to be useful for supporting (or, to some extent, refuting) Faddeev's conjecture and Conjecture I. However, they cannot furnish a proof, nor even shed much light on a more general conjecture concerning the numbers Sj and the zeros of all the Dedekind zeta-functions t_is) for O(N), since the set of zeros here is too large, and one needs purely theoretical investigations on this question. What type of theory would be useful here? In §6.2, when discussing the Artin theory for the Selberg zeta-function, we derived the formula (6.2.2), which connects the kernels of the resolvents of � ( r ; X) and � ( r l ; 1), where rl is a normal subgroup of finite index in an arbitrary Fuchsian group r of the first kind. In our view, a generalization of this formula to the case of a nonnormal subgroup of infinite index in a discrete group acting in H3, in particular, to the case rl = PSL(2, Z) c r = PSL(2, Z(N» , combined with Theorem 7.2. 1, might lead to an explanation of Conjecture I. In the present state of the art, it is too difficult to derive such a formula. As a first step in this direction one might generalize the spectral theory of automorphic functions for the group r E Wl considered in this paper to infinite-di­ mensional representations of the groups r, and carry over the entire theory to discrete subgroups acting in Lobachevsky space.

BIBLIOGRAPHY

1 . James Arthur, The Selberg trace formula for groups of F-rank one, Ann. of Math. (2) 100 ( 1 974), 326-385. 2. Emil Artin, Ober eine neue A rt von L-R,e.ihen , Abh. Math. Sem. Univ. Hamburg 3 (1 923/24), 89- 1 08; reprinted in his Collect�dpapers, Addison-Wesley, 1 965. 3. R. W. Bruggeman, Fourier coefficients of cusp forms, Invent. Math. 45 ( 1 978), 1 - 1 8. 4. P. Cartier, Some numerical computations relating to automorphic functions, Computers in Number Theory (Proc. Sci. Res. Council Atlas Sympos. No. 2, Oxford, 1 969), Academic Press, 1 97 1 , pp. 37-48. 5. , A nalyse numerique d ' un probleme des valeurs propres a haute precision applications. aux fonctions automorphes, Preprint, Inst. Hautes Etudes Sci., Bures-sur-Yvette, 1 978. 6. Charles W. Curtis and Irving Reiner, Representation theory offinite groups and associative algebras, Interscience, 1 962. 7. Michel Duflo and Jean-Pierre Labesse, Sur la formule des traces de Selberg, Ann. Sci. Ecole Norm. Sup. (4) 4 ( 1 97 1), 1 93-284. 8. Jiirgen Elstrodt, Die Resolvente zum Eigenwertproblem der automorphen Formen i� der hyperbo/ischen Ebene. I, II, III, Math. Ann. 203 ( 1 973 ) , 295-330; Math. Z. 132 ( 1 973), 99- 1 34; Math. Ann. 208 ( 1 974), 99- 1 32. 9. L. D. Faddeey, Expansion in eigenfunctions of the Laplace operator on the fundamental domain of a discrete group on the Lobachevsky plane, Trudy Moskov. Mat. Obsc. 17 ( 1 967), 323-350; English transl. in Trans. Moscow Math. Soc. 17 ( 1 967). 1 0. John D. Fay, Fourier coefficients of the resolvent for a Fuchsian group, J. Reine Angew. Math. 293/294 ( 1 977), 1 43 -203. . 1 1 . ___ , Perturbation of the spectrum of a compact Riemann surface, Preprint, Bowdoin College, Brunswick, Maine, 1 975. 1 2. , Ana�vtic torsion and Prym differentials, Riemann Surfaces and Related Topics (Proc. 1 978 Stony Brook Conf.), Ann. of Math. Studies, Vol. 97, Princeton Uniy. Press, Princeton, N.J., 1 98 1 , pp. 1 07- 1 22. 1 3 . I. M. Gel/fand, M. I. Graey and I. I. Pjatecki'i-Sapiro, Generalizedfunctions. Vol. 6: Representation theory and automorphic functions, " Nauka", Moscow, 1 966; English transl., Saunders, Philadelphia, Pa., 1 969. 14. R. Godement, The decomposition of L2( G/f) for f = SL(2, Z), Algebraic Groups and Discontinu­ ous Subgroups, Proc. Sympos. Pure Math., Vol. 9, Amer. Math. Soc., Providence, R.I., 1 966, pp. 2 1 1 -224. 1 5. Z. Adamar [Jacques Hadamard], Non-Euclidean geometry in the theory of automorphic functions, GITTL, Moscow, 1 95 1 . (Russian) 1 6. Harish-Chandra, A utomorphic forms on semisimple Lie groups, Lecture Notes in Math., Vol. 62, Springer-Verlag, 1 968. 1 7. D. R. Heath-Brown and S. J. Patterson, The distribution of Kummer sums at prime arguments, J. Reine Angew. Math. 310 ( 1 979), 1 1 1 - 1 30. 1 8. Dennis A. Hejhal, The Selberg trace formula and the Riemann zeta function , Duke Math. J. 43 ( 1 976), 44 1 -482. 1 9. , The Selberg trace formula for PSL(2, R), Lecture Notes in Math., Vol. 548, Springer-Verlag, 1 976. 20. , Sur certaines series de Dirichlet dont les poles sont sur les /ignes critiques, C. R. Acad. Sci. Paris Ser. A-B 287 ( 1 978), A383-A385. 2 1 . Heinz Huber, Zur ana(vtischen Theorie hyperbolischer Raumformen und Bewegungsgruppen , Math. Ann. 142 ( 1 960j6 1 ), 385-398; addendum, ibid. 143 ( 1 96 1 ), 463-464. 22. ___ , Ueber die Eigenwerte des Laplace-Operators auf kompakten Riemannschen Fliichen . I, II, Comment. Math. HelY. 51 ( 1 976), 2 1 5-23 1 ; 53 ( 1 958), 458-469. 23. H. Jacquet and R. P. Langlands, A utomorphic forms on GL(2), Lecture Notes in Math. , Vol. 1 1 4, Springer-Verlag, 1 970. ___

___

___

___

161

24. D. A. Kaidan, Construction of r-rational groups for certain discrete subgroups r of the group SL(2, R ), Funkcional. Anal. i Prilozen. 2 ( 1 968), no. 1 , 36-39; English transl. in Functional Anal. Appl. 2 ( 1 968). 25. Felix Klein, Gesammelte mathematische A bhandlungen. Band III, Springer-Verlag, 1 923. 26. a) Felix Klein and Robert Fricke, Vorlesungen uber die Theorie der elliptischen Modulfunctionen. I, II, Teubner, Leipzig, 1 890, 1 892. b) Robert Fricke and Felix Klein, Vorlesungen uber die Theorie der automorphen Functionen . I, II, Teubner, Leipzig, 1 897, 1 9 1 2. 27. M. G. Krein, Some new studies in the theory of perturbations of selfadjoint operators, First Math. Summer School (Kanev, 1 963), Part I, " N aukova Dumka", Kiev, 1 964, pp. 1 03- 1 87. (Russian) 28. Tomio Kubota, Elementary theory of Eisenstein series, Kodansha, Ltd., Tokyo, and Wiley, New York, 1 973. 29. , Ober diskontinuierliche Gruppen Picardschen Typus und zugehorige Eisensteinsche Peihen , Nagoya Math. J. 32 ( 1 968), 259-27 1 . 30. N. V. Kuznecov, Petersson 's conjecture for cusp forms of weight zero and Linnik 's cotijecture. Sums of Kloosterman sums, Mat. Sb. 1 11(153) ( 1 980), 334-383; English transl. in Math. USSR Sb. 39 ( 1 98 1 ). 3 1 . Gilles Lachaud, Spectral analysis of automorphic forms on rank one groups by perturbation methods, Harmonic Analysis on Homogeneous Spaces, Proc. Sympos. Pure Math., Vol. 26, Amer. Math. Soc., Providence, R.I., 1 973, pp. 44 1 -450. 32. Serge Lang, SL 2(R), Addison-Wesley, 1 975. 33. Robert P. Langlands, On the functional equations satisfied by Eisenstein series, Lecture Notes in Math., Vol. 544, Springer-Verlag, 1 976. 34. Peter D. Lax and Ralph S. Phillips, Scattering theory for automorphic functions, Princeton Univ. Press, Princeton, N.J., 1 976. 35. Joseph Lehner, Discontinuous groups and automorphic functions, Amer. Math. Soc., Providence, R.I., 1 964. 36. Hans Maass, Ober eine neue A rt von nichtanalytischen automorphen Funktionen und die Bestimmung Dirichletscher Reihen durch Funktionalgleichungen, Math. Ann. 121 ( 1 949), 1 4 1 - 1 83. 37. , Lectures on modular functions of one complex variable, Tata Inst. Fund. Res., Bombay, 1 964. 38. H. P. McKean, Selberg 's trace formula as applied to a compact Riemann surface, Comm. Pure Appl. Math. 25 ( 1 972), 225-246. 39. H. Neunhoffer, Ober die analytische Fortsetzung von Poincarereihen , S.-B. Heidelberger Akad. Wiss. Math.-Nat. Kl. 1973 , 33-90. 40. S. J. Patterson, Spectral theory and Fuchsian groups, Math. Proc. Cambridge Philos. Soc. 81 ( 1 977), 59-75. 41. , On Dirichlet series associated with cubic Gauss sums, J. Reine Angew. Math. 303/304 ( 1 978), 102 - 1 25 . 42. B . S . Pavlov and L. D . Faddeev, Scattering theory and automorphic functions, Zap. Nauen. Sem. Leningrad. Otdel. Mat. Inst. Steklov. (LOMI) 27 ( 1 972), 1 6 1 - 1 93 ; English transl. in 1. Soviet Math. 3 ( 1 975), no. 4. 43. Burton Randol, Small eigenvalues of the Laplace operator on compact Riemann surfaces, Bull. Amer. Math. Soc. 80 ( 1 974), 996 - 1 000. 44. , The Riemann hypothesis for Selberg 's zeta-function and the asymptotic behavior of eigenvalues of the Laplace operator, Trans. Amer. Math. Soc. 236 ( 1 978), 209-223. 45. Walter Rbelcke, Ober die Wellengleichung bei Grenzkreisgruppen erster Art, S.-B. Heidelberger Akad. Wiss. Math.-Nat. Kl. 1953/55, ( 1 956) 1 59-267. 46. , Das Eigenwertproblem der automorphen .Frrmen in der hyperbolischen Ebene. I, II, Math. Ann. 167 ( 1 966), 292-337; ibid. 168 ( 1 967), 26 1 -324. 47. Asmus L. Schmidt, Minimum of quadratic forms with respect to Fuchsian groups. I, 1. Reine Angew. Math. 286/287 ( 1 976), 341 -368. 48. Atle Selberg, On the remainder in the formula for N( T), the number of zeros of r< s ) in the strip 0 < t < T, Avh. Norske Vid. Akad. Oslo I 1944, no. I . 49. , Contributions to the theory of the Riemann zeta-function , Arch. Math. Naturvid. 48 ( 1 946), 89- 1 55. 50. , Harmonic ana�vsis. Part 2, Lecture Notes, G6ttingen, 1 954. 51. , Harmonic ana�vsis and discontinuous groups in weak�v symmetric Riemannian spaces with applications to Dirichlet series, J. Indian Math. Soc. (N.S.) 20 ( 1 956), 47-87. 52. , Discontinuous groups and harmonic ana�vsis, Proc. Internat. Congr. Math. (Stockholm, 1 962), Inst. Mittag-Leffler, Djursholm, 1 963, pp. 1 77- 1 89. ___

___

___

___

___

___

___

___

___

BIBLIOGRAPHY

163

53. On the estimation of Fourier coefficients of modular forms, Theory of Numbers, Proc. Sympos. Pure Math., Vol. 8, Amer. Math. Soc., Providence, R.I., 1 965 , pp. 1 - 15. , Recent developments in the theory of discontinuous groups of motions of symmetric spaces, 54. Proc. Fifteenth Scandinavian Congr. (Oslo, 1 968), Lecture Notes in Math. , Vol. 1 1 8, Springer-Verlag, 1 969, pp. 99- 1 20. 55. Teiji Takaga, Uber eine Theorie des relative A bel 'schen Zahlkorpers , J. ColI. Sci. Imp. Univ. Tokyo 41 ( 1 9 1 7/2 1), art. 9 ( 1 920). 56. E. C. Titchmarsh, The theory offunctions, 2nd ed., Oxford Univ. Press, 1 939. 57. , The theory of the Riemann zeta-junction, 2nd. ed., Clarendon Press, Oxford, 1 95 1 . 58. A. B . Venkov, Expansion in automorphic eigenfunctions of the Laplace operator and the Selberg trace formula in the space SO( n , 1)/SO ( n ), Dokl. Akad. Nauk SSSR 200 ( 1 97 1), 266-268; English transl. in Soviet Math. Dokl. 12 ( 1 97 1 ) . 59. , Expansion in automorphic eigenfunctions of the Laplace-Beltrami operator in classical symmetric spaces of rank one, and the Selberg trace formula, Trudy Mat. Inst. Steklov. 125 ( 1 973), 6-5 5 ; English transl. i n Proc. Steklov. Inst. Math. 125 ( 1 973). 60. , On an asymptotic formula connected with the number of eigenvalues corresponding to odd eigenfunctions of the Laplace-Beltrami operator on a fundamental region of the modular group PSL(2, Z), Dokl. Akad. Nauk SSSR 233 ( 1 977), no. 6, 1 02 1 - 1 023 ; English transl. in Soviet Math. Dokl. 18 ( 1 977). 61. , On the space of cusp forms for certain Fuchsian groups generated by reflections, Dokl. Akad. N auk SSSR 236 ( 1 977), 525-527; English transl. in Soviet. Math. Dokl. 18 ( 1 977). 62. , Selberg 's trace formula for the Heeke operator generated by an involution , and the eigenvalues of the Laplace-Beltrami operator on the fundamental domain of the modular group PSL(2, Z), Izv. Akad. Nauk SSSR Ser. Mat. 42 ( 1 978), 484-499; English transl. in Math. USSR Izv. 42 ( 1 978). 63. , A formula for the Tchebycheff psi-function , Mat. Zametki 23 ( 1 978), 497-503 ; English transl. in Math. Notes 23 ( 1 978). 64. On the space of cusp functions for a Fuchsian group of the first kind with nontrivial commensurator, Dokl. Akad. Nauk SSSR 239 ( 1 978), 5 1 1 -5 14; English transl. in Soviet Math. Dokl. 19 ( 1 978). 65. , Selberg 's trace formula and non-Euclidean vibrations of an infinite membrane, Dokl. Akad. Nauk SSSR 240 ( 1 978), 1 02 1 - 1024; English transl. in Soviet Math. Dokl. 19 ( 1 978). 66. , Spectral theory of automorphic functions, the Selberg zeta-function, and some problems of analytic number theory and mathematical physics , Uspehi Mat. Nauk 34 ( 1 979), no. 3 (207), 69- 1 35 ; English transl. in Russian Math. Surveys 34 ( 1 979). 67. , The A rtin-Takagi formula for Selberg 'S zeta-function and the Roc/cke conjecture, Dokl. Akad. Nauk SSSR 247 ( 1 979), 540-543 ; English transl. in Soviet Math. Dokl. 20 ( 1 979). , On the remainder term in the Weyl-Selberg asymptotic formula, Zap Nauen. Sem. Leningrad. 68. Otdel. Mat. Inst. Steklov. (LOMI) 91 ( 1 979), 5-24; English transl. in J. Soviet Math. 17 ( 1 98 1), no. 5 . 69. Zeros of r- and L-functions of imaginary quadratic fields and the eigenvalues of the PSL(2Z)-automorphic Laplacian , Dokl. Akad. Nauk SSSR 250 ( 1 980), 528-53 1 ; English Transl. in Soviet Math. Dokl. 21 ( 1 980). 70. A. B. Venkov and M. M. Skriganov, On the question of H. Weyl 's formula in the spectral theory of automorphic functions, Funkcional. Anal. i Prilozen 13 ( 1 979), no. 1 , 67-68; English transl. in Functional Anal. Appl. 13 ( 1 979). 7 1 . A. B. Venkov and A. I. Vinogradov, The asymptotic distribution of the norms of hyperbolic classes and spectral characteristics of cusp forms of weight zero for a Fuchsian group, Dokl. Akad. Nauk SSSR 243 ( 1 978), 1 373- 1 376 ; English transl. in Soviet Math. Dokl. 19 ( 1 978). 72. A. B. Venkov, V. L. Kalinin and L. D. Faddeev, A nonarithmetic derivation of the Selberg trace formula, Zap. Nauen. Sem. Leningrad. Otdel. Mat. Inst. Steklov. (LOM!) 37 ( 1 973), 5-42; English transl. in J. Soviet Math. 8 ( 1 977), no. 2. 73 . Marie-France Vigneras, Examples de sous-groupes discrete non conjugues de PSL(2, R) qui ont meme fonction zeta de Selberg, C. R. Acad. Sci. Paris Ser. A-B 1J!,7 ( 1 978), A47-A49. 74. Andre Weil, a) Sur les "formules explicites " de la theorie des nombres premiers, Comm. Sem. Math. Univ. Lund Medd. Lunds Univ. Mat. Sem. Tome Suppl. ( 1 952), 252-265. b) Sur les formules explicites de la theorie des nombres, Izv. Akad. N auk SSSR Ser. Mat. 36 ( 1 972), 3- 1 8; reprinted in Math. USSR Izv. 6 ( 1 972). ___ ,

___

___

___

___

___

___

__

___ ,

___

___

___

___

___ ,

=

ABCDEFGHU -AMS -89876S 4 3