Monte Carlo and Quasi-Monte Carlo Methods

Monte Carlo and Quasi-Monte Carlo Methods 2004

Harald Niederreiter Denis Talay Editors

Monte Carlo and Quasi-Monte Carlo Methods 2004 With 73 Figures and 29 Tables

ABC

Editors Harald Niederreiter

Denis Talay

Department of Mathematics National University of Singapore 2 Science Drive 2 Singapore 117543 Republic of Singapore email: [email protected]

INRIA Sophia Antipolis route des lucioles 2004 06902 Sophia Antipolis Cedex France email: [email protected]

Library of Congress Control Number: 2005930449 Primary: 11K45, 65-06, 65C05, 65C10, 65C30 Secondary: 11K38, 65D18, 65D30, 65D32, 65R20, 91B28 ISBN-10 3-540-25541-9 Springer Berlin Heidelberg New York ISBN-13 978-3-540-25541-3 Springer Berlin Heidelberg New York This work is subject to copyright. All rights are reserved, whether the whole or part of the material is concerned, specifically the rights of translation, reprinting, reuse of illustrations, recitation, broadcasting, reproduction on microfilm or in any other way, and storage in data banks. Duplication of this publication or parts thereof is permitted only under the provisions of the German Copyright Law of September 9, 1965, in its current version, and permission for use must always be obtained from Springer. Violations are liable for prosecution under the German Copyright Law. Springer is a part of Springer Science+Business Media springeronline.com c Springer-Verlag Berlin Heidelberg 2006
Printed in The Netherlands The use of general descriptive names, registered names, trademarks, etc. in this publication does not imply, even in the absence of a specific statement, that such names are exempt from the relevant protective laws and regulations and therefore free for general use. Typesetting: by the authors and TechBooks using a Springer LATEX macro package Cover design: design & production GmbH, Heidelberg Printed on acid-free paper

SPIN: 11366959

46/TechBooks

543210

Preface

This volume represents the refereed proceedings of the Sixth International Conference on Monte Carlo and Quasi-Monte Carlo Methods in Scientific Computing which was held in conjunction with the Second International Conference on Monte Carlo and Probabilistic Methods for Partial Differential Equations at Juan-les-Pins, France, from 7–10 June 2004. The programme of this conference was arranged by a committee consisting of Henri Faure (Universit´e de Marseille), Paul Glasserman (Columbia University), Stefan Heinrich (Universit¨ at Kaiserslautern), Fred J. Hickernell (Hong Kong Baptist University), Damien Lamberton (Universit´e de Marne la Vall´ee), Bernard Lapeyre (ENPC-CERMICS), Pierre L’Ecuyer (Universit´e de Montr´eal), Pierre-Louis Lions (Coll`ege de France), Harald Niederreiter (National University of Singapore, co-chair), Erich Novak (Universit¨ at Jena), Art B. Owen (Stanford University), Gilles Pag`es (Universit´e Paris 6), Philip Protter (Cornell University), Ian H. Sloan (University of New South Wales), Denis Talay (INRIA Sophia Antipolis, co-chair), Simon Tavar´e (University of Southern California) and Henryk Wo´zniakowski (Columbia University and University of Warsaw). The organization of the conference was arranged by a committee consisting of Mireille Bossy and Etienne Tanr´e (INRIA Sophia Antipolis), and Madalina Deaconu (INRIA Lorraine). Local arrangements were in the hands of Monique Simonetti and Marie–Line Ramfos (INRIA Sophia Antipolis). This conference continued the tradition of biennial MCQMC conferences which was begun at the University of Nevada in Las Vegas, Nevada, USA, in June 1994 and followed by conferences at the University of Salzburg, Austria, in July 1996, the Claremont Colleges in Claremont, California, USA, in June 1998, Hong Kong Baptist University in Hong Kong, China, in November 2000 and the National University of Singapore, Republic of Singapore, in November 2002. The proceedings of these previous conferences were all published by Springer-Verlag, under the titles Monte Carlo and Quasi-Monte Carlo Methods in Scientific Computing (H. Niederreiter and P.J.-S. Shiue, eds.), Monte Carlo and Quasi-Monte Carlo Methods 1996 (H. Niederreiter, P. Hellekalek, G. Larcher and P. Zinterhof, eds.), Monte Carlo and Quasi-Monte Carlo

VI

Preface

Methods 1998 (H. Niederreiter and J. Spanier, eds.), Monte Carlo and QuasiMonte Carlo Methods 2000 (K.-T. Fang, F.J. Hickernell and H. Niederreiter, eds.) and Monte Carlo and Quasi-Monte Carlo Methods 2002 (H. Niederreiter, ed.). The next MCQMC conference will be held in Ulm, Germany, in August 2006. The programme of the conference was rich and varied, with over 150 talks being presented. Highlights were the invited plenary talks given by Mark Broadie (Columbia University), Benjamin Jourdain (Ecole Nationale des Ponts et Chauss´ees), Alexander Keller (Universit¨ at Ulm), Wilfrid S. Kendall (University of Warwick), Roland Keunings (Universit´e Catholique de Louvain), Pierre-Louis Lions (Coll`ege de France), Art B. Owen (Stanford University) and Henryk Wo´zniakowski (Columbia University and University of Warsaw) as well as the special sessions that were organized by designated chairpersons. The papers in this volume were carefully screened and cover both the theory and the applications of Monte Carlo and quasi-Monte Carlo methods. Several papers are also devoted to stochastic methods for partial differential equations. We gratefully acknowledge generous financial support of the conference by the PACA Regional Council and the General Council of Cˆ ote d’Azur. We also thank the members of the Programme Committee and many others who contributed enormously to the excellent quality of the conference presentations and to the high standards for publication in these proceedings by careful review of the abstracts and manuscripts that were submitted. Finally, we want to express our gratitude to Springer-Verlag, and especially to Dr. Martin Peters, for publishing this volume and for the very helpful support and kind advice we have received from his staff.

May 2005

Harald Niederreiter Denis Talay

Contents

Invariance Principles with Logarithmic Averaging for Ergodic Simulations Olivier Bardou . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

1

Technical Analysis Techniques versus Mathematical Models: Boundaries of Their Validity Domains Christophette Blanchet-Scalliet, Awa Diop, Rajna Gibson, Denis Talay, Etienne Tanr´e . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 15 Weak Approximation of Stopped Dffusions F.M. Buchmann, W.P. Petersen . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 31 Approximation of Stochastic Programming Problems Christine Choirat, Christian Hess, Raffaello Seri . . . . . . . . . . . . . . . . . . . . . 45 The Asymptotic Distribution of Quadratic Discrepancies Christine Choirat, Raffaello Seri . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 61 Weighted Star Discrepancy of Digital Nets in Prime Bases Josef Dick, Harald Niederreiter, Friedrich Pillichshammer . . . . . . . . . . . . . 77 Explaining Effective Low-Dimensionality Andrew Dickinson . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 97 Selection Criteria for (Random) Generation of Digital (0,s)-Sequences Henri Faure . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 113 Imaging of a Dissipative Layer in a Random Medium Using a Time Reversal Method Jean-Pierre Fouque, Josselin Garnier, Andr´e Nachbin, Knut Sølna . . . . 127

VIII

Contents

A Stochastic Numerical Method for Diffusion Equations and Applications to Spatially Inhomogeneous Coagulation Processes Flavius Guia¸s . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 147 Non-Uniform Low-Discrepancy Sequence Generation and Integration of Singular Integrands J¨ urgen Hartinger, Reinhold Kainhofer . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 163 Construction of Good Rank-1 Lattice Rules Based on the Weighted Star Discrepancy Stephen Joe . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 181 Probabilistic Approximation via Spatial Derivation of Some Nonlinear Parabolic Evolution Equations B. Jourdain . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 197 Myths of Computer Graphics Alexander Keller . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 217 Illumination in the Presence of Weak Singularities Thomas Kollig, Alexander Keller . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 245 Irradiance Filtering for Monte Carlo Ray Tracing Janne Kontkanen, Jussi R¨ as¨ anen, Alexander Keller . . . . . . . . . . . . . . . . . . 259 On the Star Discrepancy of Digital Nets and Sequences in Three Dimensions Peter Kritzer . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 273 Lattice Rules for Multivariate Approximation in the Worst Case Setting Frances Y. Kuo, Ian H. Sloan, Henryk Wo´zniakowski . . . . . . . . . . . . . . . . . 289 Randomized Quasi-Monte Carlo Simulation of Markov Chains with an Ordered State Space Pierre L’Ecuyer, Christian L´ecot, Bruno Tuffin . . . . . . . . . . . . . . . . . . . . . 331 Experimental Designs Using Digital Nets with Small Numbers of Points Kwong-Ip Liu, Fred J. Hickernell . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 343 Concentration Inequalities for Euler Schemes Florent Malrieu, Denis Talay . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 355 Fast Component-by-Component Construction, a Reprise for Different Kernels Dirk Nuyens, Ronald Cools . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 373

Contents

IX

A Reversible Jump MCMC Sampler for Object Detection in Image Processing Mathias Ortner, Xavier Descombes, Josiane Zerubia . . . . . . . . . . . . . . . . . . 389 Quasi-Monte Carlo for Integrands with Point Singularities at Unknown Locations Art B. Owen . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 403 Infinite-Dimensional Highly-Uniform Point Sets Defined via Linear Recurrences in F2w Fran¸cois Panneton, Pierre L’Ecuyer . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 419 Monte Carlo Studies of Effective Diffusivities for Inertial Particles G.A. Pavliotis, A.M. Stuart, L. Band . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 431 An Adaptive Importance Sampling Technique Teemu Pennanen, Matti Koivu . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 443 MinT: A Database for Optimal Net Parameters Rudolf Sch¨ urer, Wolfgang Ch. Schmid . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 457 On Ergodic Measures for McKean–Vlasov Stochastic Equations A. Yu. Veretennikov . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 471 On the Distribution of Some New Explicit Inversive Pseudorandom Numbers and Vectors Arne Winterhof . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 487 Error Analysis of Splines for Periodic Problems Using Lattice Designs Xiaoyan Zeng, King-Tai Leung, Fred J. Hickernell . . . . . . . . . . . . . . . . . . . 501

Invariance Principles with Logarithmic Averaging for Ergodic Simulations Olivier Bardou INRIA, 2004 routes des Lucioles, 06902 Sophia Antipolis, France [email protected]

Summary. In this contribution, we consider the problem of variance estimation in the computation of the invariant measure of a random dynamical system via ergodic simulations. An adaptive estimator of the variance for such simulations is deduced from a general result stating an almost sure central limit theorem for empirical means. We also provide a speed of convergence for this estimator.

1 Ergodic Simulations We consider the Stochastic Differential Equation in Rn , dXt = b(Xt ) dt + σ(Xt ) dWt ,

(1)

driven by a d-dimensional Brownian motion W . We shall work under hypotheses ensuring that the solution to the system (1) has a unique invariant measure µ. Under additional assumptions, this measure has a density p which solves the stationary Fokker-Planck equation L∗ p = 0

(2)

where L is the infinitesimal generator of the process X. The Fokker-Planck equation occurs in numerous domains, from random mechanics (see e.g. Soize [16]) to magnetoencephalograhy (see e.g. Martinez [13]). The numerical resolution of this equation can be extremely difficult or even impossible
when the dimension n is large. Therefore, to estimate expectations such as f dµ for a given function f , ergodic simulations are often required in practice. These simulations belong to the family of Monte-Carlo methods and it is well known that the estimation of their variance plays an essential role in their numerical implementation (see e.g. Talay [17] or Lamberton and Pag`es [11] for discretization issues and Bardou [1] for a first glance at variance reduction technics in this context). Unfortunately, as far as we know, there does not exist any tractable way to estimate the variance of ergodic simulations. So, the aim of this contribution is to introduce an adaptive estimator of this quantity.

2

O. Bardou

In a first part, we recall some well known results but, to our knowledge, never stated under our hypotheses. The second part of this paper is devoted to the proof of an almost sure central limit theorem for empirical means and in the third part we provide a speed of convergence for this invariance principle. Then, we deduce from these results an adaptive, strongly consistent and asymptotically normal estimator for the variance of ergodic simulations. Finally, we extend all these results to the local time estimator of the one dimensional invariant density. Unless otherwise stated, we shall now suppose that (H1) The function a := σσ∗ : Rn → Rn is uniformly continuous, bounded and there exists a positive constant λ such that: ξ · (a(x)ξ) ≥ λ|ξ|2 , ∀x ∈ Rn , ∀ξ ∈ Rn . (H2) The function b : Rn → Rn is locally bounded and there exists α > −1, r > 0 and M > 0 such that x ≤ −r|x|α , |x| ≥ M . b(x) · |x| (H3) The function f : Rn → R is not a constant and there exists β ≥ 0, C1 > 0 and C2 > 0 such that |f (x)| ≤ C1 + C2 |x|β . We call Px , the probability under which X0 = x. Under these hypotheses, it is proved in Pardoux and Veretennikov [15] that the process X has a unique invariant measure µ and that every function f satisfying (H3) belongs to L1 (µ). Therefore, the ergodic theorem applies: Theorem 1. Under the hypotheses (H1)–(H3),   1 T Px -a.s. f (Xs ) ds −−−−−→ f dµ, ∀x ∈ Rn . T →+∞ T 0 This result is at the root of ergodic simulations since it provides a consis
tent estimator for the unknown expectation f dµ. We refer to Has’minskiˇı [7] for a proof. We are now interested in the speed of convergence in this theorem. For convenience, we set  ˜ f := f − f dµ . The starting point of this work is the following proposition from Pardoux and Veretennikov [15] concerning the solution Gf to the Poisson equation LGf = −f˜ .

(3)

2 In the following, Wp, loc stands for the space of functions which belong locally to the Sobolev class Wp2 (Rn ); the space of functions with weak derivatives up to order 2 in Lp (Rn ).

Invariance Principles with Logarithmic Averaging for Ergodic Simulations

Proposition 1. Under the hypotheses (H1)–(H3), the function  +∞ Gf (x) := Ex [f˜(Xt )] dt, x ∈ Rn ,

3

(4)

0

 2 is continuous, belongs to p>1 Wp, loc , is solution to the problem (3) and satisfies the three following properties. For all m > β ∨ 2, there exists some  > 0 such that constants Cm > 0 and Cm |Gf (x)| ≤ Cm (1 + |x|m ), ∀x ∈ Rn ,

(5)

and

 |∇Gf (x)| ≤ Cm (1 + |x|m ), ∀x ∈ Rn .  Thus, the functions Gf and ∇Gf belong to p≥1 Lp (µ). Moreover,  Gf dµ = 0

and Gf is the only solution to the problem (3) belonging to satisfying properties (5) and (7).

 p>1

(6)

(7)

2 Wp, loc and

This proposition has two important consequences. The first one is the existence of our main object of investigation. Lemma 1. Let us define

 σf2 :=

|σ∇Gf |2 dµ .

(8)

Then, under the hypotheses (H1)–(H3), 0 < σf2 < +∞ . Proof. This is a consequence of the hypotheses (H1) and (H3), and of property (6) from proposition 1.  Remark 1. Using an integration by parts and the Fokker-Planck equation (2), we get the alternative expression  σf2 = −2 f Gf dµ . The second lemma is a key result in the study of the speed of convergence for ergodic simulations. Lemma 2. Under the hypotheses (H1)–(H3), we have the Poisson representation  T  T (σ∇Gf ) (Xs ) · dWs + Gf (x) − Gf (XT ) , f˜(Xs ) ds = 0

0

Px -almost surely, for all x ∈ Rn .

4

O. Bardou

Proof. As the functions b and σ are locally bounded and σσ ∗ is nondegenerate, this results from the application of the Itˆ o-Krylov formula to the function  2 . See Krylov [8] for a statement in bounded domains and Gf ∈ p>1 Wp, loc Pardoux and Veretennikov [15] for the extension to the whole space.  From these two lemmas, we get the following central limit theorem. Proposition 2. Under the hypotheses (H1)–(H3),  T 1 Law √ f˜(Xs ) ds −−−−−→ N (0, σf2 ) . T →+∞ T 0

(9)

Proof. The functional form of this theorem is proved in Bhattacharya [3]. In our setting, the proof is straightforward. From lemma 2, we have  T  T 1 Gf (x) − Gf (XT ) 1 √ √ (σ∇Gf ) (Xs ) · dWs + f˜(Xs ) ds = √ . T 0 T T 0 By the ergodic theorem and lemma 1, we check that  1 T P -a.s. |σ∇Gf |2 (Xs ) ds −−x−−−→ σf2 < +∞, ∀x ∈ Rn . T →+∞ T 0 Therefore, the central limit theorem for continuous local martingales applies (see Kutoyants [9]) and then  T 1 Law √ (σ∇Gf ) (Xs ) · dWs −−−−−→ N (0, σf2 ) . T →+∞ T 0 Moreover, as XT converges in law and Gf is continuous, Gf (XT ) also converges in law and then Gf (x) − Gf (XT ) Px √ −−−− −→ 0 . T →+∞ T Slutsky’s lemma (see Montfort [14]) leads to the conclusion.



We refer to Bardou [1] for an Edgeworth expansion refining this proposition. The remainder of this paper is devoted to the construction of an efficient estimator of the asymptotic variance σf2 and to the study of its speed of convergence.

2 Almost Sure Central Limit Theorem Let us define, for 0 ≤ τ ≤ 1 and K > 0,   Gτ := ϕ ∈ L2 (N (0, 1))/|ϕ(v) − ϕ(u)| ≤ K|v − u|(|v|τ + |u|τ + 1), (u, v) ∈ R2 .

Our main result is the following theorem.

Invariance Principles with Logarithmic Averaging for Ergodic Simulations

5

Theorem 2. Under the hypotheses (H1)–(H3), for any 0 ≤ τ ≤ 1 and any function ϕ ∈ Gτ , we have   T    t 1 dt Px -a.s. 1 ϕ √ −−−−−→ ϕ dN (0, σf2 ), ∀x ∈ Rn . f˜(Xs ) ds ln T 1 t T →+∞ t 0 Proof. Remember the Poisson representation from lemma 2,  t  t (σ∇Gf ) (Xs ) · dWs − (Gf (Xt ) − Gf (x)) . f˜(Xs ) ds = 

 0 0 

 =:Gt

(10)

=:Mt

First, note that under our hypotheses, for all m ∈ N, there exists a constant Cm , independent of T , such that Ex [|XT |m ] ≤ Cm (see Pardoux and Veretennikov [15]) . Therefore, we can check that MT is a real martingale. Moreover, from the ergodic theorem 1 and lemma 1, we know that M T Px -a.s. −−−−−→ σf2 with 0 < σf2 < +∞, ∀x ∈ Rn . T →+∞ T Under this assumption, it is proved in Maaouia [12] and Chaabane [5] the following almost sure central limit theorem for continuous martingales,   T   Mt dt Px -a.s. 1 ϕ √ −−−−−→ ϕ dN (0, σf2 ), ∀x ∈ Rn (11) ln T 1 t T →+∞ t for any ϕ ∈ Gτ . Let us now consider the difference    T  1  T  1  t dt Mt dt 1 ˜ − ϕ √ ϕ √ DT := f (Xs ) ds ln T 1 t ln T 1 t t 0 t     t  T  1 t dt ˜(Xs ) ds − ϕ M ϕ √1 √ ≤ f ln T 1 t 0 t t     T  M dt 1 G ϕ M √ t − √t − ϕ √ t ≤ using (10) ln T 1 t t t t τ   T  Gt Mt Mt τ dt K Gt √ √ − √ + √ +1 ≤ as ϕ ∈ Gτ ln T 1 t t t t t  T  T  T |Gt | |Mt − Gt |τ dt |Gt | |Mt |τ dt |Gt | dt K K K √ √ √τ √ ≤ + + . √τ ln T 1 t ln T 1 t ln T 1 t t t t t t

 

 

  1 =:DT

2 =:DT

3 =:DT

We first consider the term DT3 . After an integration by parts, we get

6

O. Bardou

DT3

K = ln T





1

T

t

3 + 2



T

1



t



|Gt | dt |Gs | ds dt 5/2 t3/2 1 1 t 1 1  T  T  t 1 K 1 3K dt = |G | dt + |Gs | ds . t ln T T 3/2 1 2 ln T 1 t3/2 1 t

As the process X is ergodic and Gf ∈ L1 (µ), we check the convergence of the first term in DT3 , 1



t

P -a.s.

x |Gs | ds −− −−→ 0, ∀x ∈ Rn .

t3/2

t→+∞

1

The second term of DT3 is the Cesaro mean of a bounded function converging towards 0 and thus converges also towards 0. Therefore, P -a.s.

DT3 −−x−−−→ 0, ∀x ∈ Rn . T →+∞

To deal with DT2 , we apply the Cauchy-Schwarz inequality,  DT2

≤K

1 ln T



T

1

|Gt |2 dt t t

 12 

1 ln T



T

1

1 Mt 2τ dt 2 √ . t t

With the same arguments as above, we show that the first term converges towards 0. For the second term, note that, for any 0 ≤ τ ≤ 1, the map y → |y|2τ belongs to Gτ . Thus, we can apply (11) and deduce that P -a.s

x −−→ 0, ∀x ∈ Rn . DT2 −−−

T →+∞

Finally, to deal with DT1 , we write DT1 =

K ln T

K ≤ ln T K ≤ ln T



T

|Gt | |Mt − Gt |τ dt √ √τ t t t

T

|Gt | |Mt | + |Gt | + 1 dt √ as 0 ≤ τ ≤ 1 √τ t t t  T  T |Gt | |Mt | dt |Gt |2 dt |Gt | dt K K + + . √τ √ √ τ +1 √ τ +1 ln T 1 t ln T 1 t t t t t t

1



1



1

T

Then, with the same arguments, we check that P -a.s.

DT1 −−x−−−→ 0, ∀x ∈ Rn T →+∞

and the proof is complete.



Invariance Principles with Logarithmic Averaging for Ergodic Simulations

7

3 Speed of Convergence in the Almost Sure Central Limit Theorem The speed of convergence in theorem 2 is made precise by the following. Theorem 3. Under the hypotheses (H1)–(H3), for any 0 ≤ τ ≤ 1 and any function ϕ ∈ Gτ , we define,  ϕ¯ := ϕ − ϕ dN (0, σf2 ) . Then, we have 1 √ ln T





T

ϕ¯ 1

1 √ t





t

f˜(Xs ) ds 0

dt Law −−−−−→ N (0, σ ¯ϕ2 ) t T →+∞



with σ ¯ϕ2 := −2

ϕG ¯ ϕ dN (0, σf2 )

and Gϕ is a solution to the Poisson equation ¨ ϕ (y) = −2ϕ¯ . −y G˙ ϕ (y) + σf2 G To prove this result, we need a preliminary lemma specifying the speed of convergence in the ergodic theorem 1. Lemma 3. Under the hypotheses (H1)–(H3) and with the previous notations,   M T P -a.s. − σf2 −−x−−−→ 0, ∀x ∈ Rn . (ln T )2 T →+∞ T Proof. In Maaouia [12], it is proved that this result is a consequence of the following condition on the resolvant associated to the semi-group of the process X;
(R–R) There exists a non negative function f0 ∈ L1 (µ) such that f0 dµ > 0 and, for all x ∈ Rn , the map    +∞  1 −λt λ → e Ex [f0 (Xt )] dt − f0 dµ (12) λ λ 0 has a finite limit when λ → 0. 1
It is clear that there always exists a function f0 ∈ L (µ) such that for exemple the characteristic function of a Borel set f0 dµ > 0. Consider
B such that B dµ > 12 . Such a set exists as µ is of total mass one. And moreover, the characteristic function satisfies (H3). The function defined by (12) can also be written  +∞   λ → e−λt Ex f˜0 (Xt ) dt . 0

8

O. Bardou

Therefore, as λ → 0, this function converges towards the solution Gf0 (x) to the Poisson equation LGf0 = −f˜0 and we know that this function is finite for  any f0 satisfying (H3) and any x ∈ Rn by proposition 1. We can now proceed to the proof of theorem 3. Proof (of theorem 3). We keep the notations from the proof of theorem 2. From Chaabane [5], we know that under the assumption   M T P -a.s. 2 (ln T ) − σf2 −−x−−−→ 0, ∀x ∈ Rn , (13) T →+∞ T we have the following speed of convergence in the almost sure central limit theorem for continuous martingales,   T  1 Mt dt Law √ ϕ¯ √ −−−−−→ N (0, σ ¯ϕ2 ) . (14) t t T →+∞ ln T 1 As we have checked in lemma 3 that (13) is verified under our hypotheses, we get (14). Now, we want to prove that the difference     T   t dt Mt 1 1 ˜ ϕ¯ √ DT := √ f (Xs ) ds − ϕ¯ √ t t 0 t ln T 1 converges almost surely towards 0. From the proof of theorem 2, we have the inequality  T |Gt | |Mt − Gt |τ dt K √ DT ≤ √ √τ t t ln T 1 t

  K + √ ln T 

 1

1 =:DT

T

|Gt | |Mt |τ dt K √ √τ +√ t t ln T t

 

2 =:DT



|Gt | dt √ . t t 1

 T

3 =:DT

With the arguments from the proof of theorem 2, we also have  T  T  t 1 1 K 3K dt √ DT3 = √ |G | dt + |Gs | ds . t t 2 ln T 1 t3/2 1 ln T T 3/2 1 The first term clearly converges √ towards 0. To deal with the second term we note that the derivative of t → ln t is t → 2t√1ln t and we make appear the Cesaro mean  T  T √  t  1 1 ln t t dt dt 1 √ . |Gs | ds = √ |Gs | ds √ 3/2 3/2 t ln T 1 t ln T 1 t t ln t 1 1 As

Invariance Principles with Logarithmic Averaging for Ergodic Simulations

9

√  ln t t P |Gs | ds −−-a.s. −−→ 0, ∀x ∈ Rn , 3/2 t→+∞ t 1 we get 1 √ ln T



T

1

√



ln t

t3/2

t

1

and then

dt Px -a.s. |Gs | ds √ −− −−→ 0, ∀x ∈ Rn t ln t t→+∞

P -a.s.

x −−→ 0, ∀x ∈ Rn . DT3 −−

t→+∞

In the same way, following the proof of theorem 2 and enlightening the presence of well choosen Cesaro means, it is a simple matter to prove that DT1 and DT2 also converge towards 0 and we obtain the claimed result. 

4 Application: Dynamical Estimation of the Asymptotic Variance Note that the function y → y 2 belongs to G1 . Therefore, as a straightforward consequence of theorems 2 and 3 we get the following proposition, Proposition 3. Under the hypotheses (H1)–(H3), the random variable 

1 ln T

T

1



1 √ t



2

t

f˜(Xs ) ds 0

dt t

is a strongly consistent and asymptotically normal estimator of the asymptotic variance σf2 . That is, on the one hand, 1 ln T

 1

T



1 √ t



2

t

f˜(Xs ) ds 0

dt Px -a.s. 2 −−−−−→ σ , ∀x ∈ Rn , t T →+∞ f

and, on the other hand,  2  T   t 1 1 dt Law 2 ˜ √ √ −−−−−→ N (0, σf2 ) . f (Xs ) ds − σf t T →+∞ t 0 ln T 1 The speed of convergence of this estimator is logarithmic so that, in practice, we have no hope to get a precise estimate of the variance. Nevertheless, for a cost equivalent to the computation of the empirical mean, we can estimate the order of magnitude of the variance and thus, the time requested for
the estimation of f dµ with a desired precision. Moreover, notice that the asymptotic variance of this estimator is precisely σf2 , the asymptotic variance of the simulation. Thus, the smaller is this variance, the best it will be estimated. We refer to Bertail and Cl´emen¸con [2] for another estimator of the asymptotic variance in the context of atomic Markov chains.

10

O. Bardou

5 Invariance Principles for the Local Time of a Diffusion In a one dimensional setting, it is possible to generalize the preceding results thanks to the Itˆ o-Tanaka formula. From now on, we set the dimension of the process X to be n = 1. It is known (see G¯ıhman and Skorohod [6]) that, in dimension one, if the process X admits an invariant measure µ with density p, this one is given by
y exp(2 0 σb(z) 2 (z) dz) , ∀y ∈ R . (15) p(y) = σ 2 (y) Therefore, in order to ensure the existence and differentiabilty of this density p, we shall now suppose that (H4)

 exp(2
y 0 R

b(z) σ 2 (z)

σ 2 (y)

dz) dy < +∞ .

(H5) The function σ is bounded away from 0 and differentiable. The one dimensional empirical measure of the process X is defined by  1 T µT (x) := 1 (Xs ≤x) ds . T 0 The occupation time formula gives the alternative expression,  1 (y≤x) LyT 1 µT (x) = dy T R σ 2 (y)  x LyT 1 dy = T −∞ σ 2 (y) where (Lyt )t≥0 is the local time of the process X at point y. Therefore, it is natural to define the empirical density by pT (x) :=

LxT . T σ 2 (x)

(16)

It is known (see e.g. Bosq and Davydov [4]) that this is a strongly consistent estimator of the density p(x). Moreover, from Kutoyants [10], we have the following Poisson representation, Px -almost surely for all x ∈ R,  1 T 1 pT (x) − p(x) = σ(Xt )G˙ x (Xt ) dWt − (Gx (XT ) − Gx (x)) (17) T 0 T deduced from the application of the Itˆ o-Tanaka formula to the function  y 1 (u>x) − µ(u) du, (18) Gx (y) := −2p(x) σ 2 (u)p(u) y0

Invariance Principles with Logarithmic Averaging for Ergodic Simulations

11




where the constant y0 is choosen to ensure that Gx dµ = 0. With these two elements, we can extend the previous invariance principles to the local time estimator (16). As a first step, we prove that the function Gx defined in (18) is a weak solution to a Poisson equation. Thus, for any C ∞ function ψ with compact support,  

 σ2 ¨ Gx (y)ψ(y) dy by the definition of L bG˙ x + 2 R    σ2 ˙ ˙ Gx (y) (b − σ σ) ψ (y) dy after integration ˙ ψ− = 2 R by parts     2 σ ˙ p˙ b − σ σ˙ p˙ = Gx ψ − ψ˙ (y) dy because = 2 due 2 p p σ2 R to (15)    µ(y) − 1 (y>x) p˙ = p(x) ψ − ψ˙ (y) dy using (18) p(y) p R = p(x) ψ(y) dy − ψ(x) after integration by parts.

 R

LGx (y)ψ(y) dy =

R

Therefore, we have proved that, formally, LGx = −(δx − p(x)) .

(19)

Now, as a consequence of (18), we get the central limit theorem, √

Law

T (pT (x) − p(x)) −−−−−→ N (0, σx2 ) T →+∞

with the asymptotic variance   σx2

2

2 1 (y>x) − µ(y) dy σ 2 (y)p(y)

:= 4p(x) R  = (σ G˙ x )2 dµ .

After an integration by parts,
using (19) and the fact that the function Gx is defined in such a way that Gx dµ = 0, we get the alternative expression: σx2 = −2p(x)Gx (x) .

(20)

Finally, for any 0 ≤ τ ≤ 1 and any function ϕ ∈ Gτ , let us define the solution Gϕ to the Poisson equation ¨ ϕ (y) = −2ϕ¯ , −y G˙ ϕ (y) + σx2 G and the variance

12

O. Bardou

 σ ¯ϕ2 := −2

ϕG ¯ ϕ dN (0, σx2 ) 

with ϕ¯ := ϕ −

ϕ dN (0, σx2 ) .

Then, following the proofs of theorems 2 and 3 with (18) as a starting point, we can prove the following almost sure central limit theorem    T √  dt Px -a.s. Lxt 1 − p(x) ϕ t − − − − − → ϕ dN (0, σx2 ), ∀x ∈ R , ln T 1 tσ 2 (x) t T →+∞ whose speed of convergence is given by   T √  dt Law Lxt 1 √ − p(x) ϕ¯ t −−−−−→ N (0, σ ¯ϕ2 ) . tσ 2 (x) t T →+∞ ln T 1

Acknowledgement I wish to thank Professor Denis Talay for suggesting the use of invariance principles in the estimation of the asymptotic variance.

References 1. O. Bardou, Contrˆ ole dynamique des erreurs de simulation et d’estimation de processus de diffusion, Ph.D. thesis, Universit´e de Nice–Sophia Antipolis, 2005. 2. P. Bertail and S. Cl´emen¸con, Edgeworth expansions of suitably normalized sample mean statistics for atomic Markov chains, Probab. Theory Related Fields (2004). 3. R.N. Bhattacharya, On the functional central limit theorem and the law of the iterared logarithm for Markov processes, Z. Wahrscheinlichkeitstheorie verw. Gebiete 60 (1982), 185–201. 4. D. Bosq and Yu. Davydov, Local time and density estimation in continuous time, Math. Methods Stat. 8 (1999), no. 1, 22–45. 5. F. Chaabane, Invariance principles with logarithmic averaging for continuous local martingales, Statist. Probab. Lett. 59 (2002), no. 2, 209–217. 6. ˘I.¯I. G¯ıhman and A.V. Skorohod, Stochastic Differential Equations, SpringerVerlag, New York, 1972, Translated from the Russian by Kenneth Wickwire, Ergebnisse der Mathematik und ihrer Grenzgebiete, Band 72. 7. R.Z. Has’minskiˇı, Stochastic stability of differential equations, Monographs and Textbooks on Mechanics of Solids and Fluids: Mechanics and Analysis, vol. 7, Sijthoff & Noordhoff, Alphen aan den Rijn, 1980, Translated from the Russian by D. Louvish. 8. N.V. Krylov, Controlled diffusion processes, Applications of Mathematics, vol. 14, Springer-Verlag, New York, 1980, Translated from the Russian by A. B. Aries.

Invariance Principles with Logarithmic Averaging for Ergodic Simulations

13

9. Yu.A. Kutoyants, On a hypotheses testing problem and asymptotic normality of stochastic integral, Theory of probability and its applications (1975). , Statistical inference for ergodic diffusion processes., Springer Series in 10. Statistics, Springer-Verlag, 2003. 11. D. Lamberton and G. Pag`es, Recursive computation of the invariant distribution of a diffusion, Bernoulli 8 (2002), no. 3. 12. F. Maaouia, Principes d’invariance par moyennisation logarithmique pour les processus de Markov, Ann. Probab. 29 (2001), no. 4, 1859–1902. 13. M. Martinez, Interpr´etations probabilistes d’op´ erateurs sous forme divergence et analyse de m´ ethodes num´ eriques probabilistes associ´ees, Ph.D. thesis, Universit´e de Marseilles, 2004. 14. M. Montfort, Cours de statistique math´ ematique, Economica, 1982. 15. E. Pardoux and A.Yu. Veretennikov, On the Poisson equation and diffusion approximation. I, Ann. Probab. 29 (2001), no. 3. 16. C. Soize, The Fokker-Planck equation for stochastic dynamical systems and its explicit steady state solutions, Series on Advances in Mathematics for Applied Sciences, vol. 17, World Scientific Publishing Co. Inc., River Edge, NJ, 1994. 17. D. Talay, Second-order discretization schemes of stochastic differential systems for the computation of the invariant law., Stochastics Stochastics Rep. 29 (1990), no. 1, 13–36.

Technical Analysis Techniques versus Mathematical Models: Boundaries of Their Validity Domains Christophette Blanchet-Scalliet1 , Awa Diop2 , Rajna Gibson3 , Denis Talay2 , and Etienne Tanr´e2 1

2

3

Laboratoire Dieudonn´e, universit´e Nice Sophia-Antipolis Parc Valrose 06108 Nice Cedex 2, France [email protected] INRIA, Projet OMEGA, 2004 route des Lucioles, BP93, 06902 Sophia-Antipolis, France {Awa.Diop,Denis.Talay,Etienne.Tanre}@sophia.inria.fr NCCR FINRISK, Swiss Banking Institute, University of Z¨ urich, Plattenstrasse 14, Z¨ urich 8032, Switzerland [email protected]

Abstract We aim to compare financial technical analysis techniques to strategies which depend on a mathematical model. In this paper, we consider the moving average indicator and an investor using a risky asset whose instantaneous rate of return changes at an unknown random time. We construct mathematical strategies. We compare their performances to technical analysis techniques when the model is misspecified. The comparisons are based on Monte Carlo simulations.

1 Introduction In the financial industry, there are three main approaches to investment: the fundamental approach, where strategies are based on fundamental economic principles, the technical analysis approach, where strategies are based on past prices behavior, and the mathematical approach where strategies are based on mathematical models and studies. The main advantage of technical analysis is that it avoids model specification, and thus calibration problems, misspecification risks, etc. On the other hand, technical analysis techniques have limited theoretical justifications, and therefore no one can assert that they are riskless, or even efficient (see [LMW00]). Consider a nonstationary financial economy. It is impossible to specify and calibrate models which can capture all the sources of instability during a long time interval. Thus it might be useful to compare the performances obtained

16

C. Blanchet-Scalliet et al.

by using erroneously calibrated mathematical models and the performances obtained by technical analysis techniques. To our knowledge, this question has not yet been investigated in the literature. The purpose of this paper is to present its mathematical complexity and preliminary results. Here we consider the case of an asset whose instantaneous expected rate of return changes at an unknown random time. We compare the performances of traders who respectively use: • a strategy which is optimal when the model is perfectly specified and calibrated, • mathematical strategies for misspecified situations, • a technical analysis technique. In all this paper, we limit ourselves to the case in which the trader’s utility function is logarithmic. Of course, it is a severe limitation from a financial point of view. This choice is also questionable from a numerical point of view because logarithmic utilities tend to smoothen the effects of the different strategies. However, we will see that, even in this case and within a simplified model, the analytical formulae are rather cumbersome and that our analysis requires nonelementary mathematical and numerical tools. See also the Remark 1 below. Our study is divided into two parts: a mathematical part which, when possible, provides analytical formulae for portfolios managed by means of mathematical and technical analysis strategies; a numerical part which provides quantitative comparisons between all these various strategies.

2 Description of the Setting The financial market consists of two assets which are traded continuously. The first one is an asset without systematic risk, typically a bond (or a bank account), whose price at time t evolves according to  0 dSt = St0 rdt, (1) S00 = 1. The second asset is a stock subject to systematic risk. We model the evolution of its price at time t by the linear stochastic differential equation    dSt = St µ2 + (µ1 − µ2 )1 (t≤τ ) dt + σSt dBt , (2) S0 = S 0 , where (Bt )0≤t≤T is a one-dimensional Brownian motion on a given probability space (Ω, F, P). At the random time τ , which is neither known, nor directly observable, the instantaneous return rate changes from µ1 to µ2 . A simple computation shows that

Technical Analysis Techniques versus Mathematical Models

17

   t σ2 0 )t + (µ2 − µ1 ) St = S exp σBt + (µ1 − 1 (τ ≤s) ds =: S 0 exp(Rt ), 2 0 where the process (Rt )t≥0 is defined as 

σ2 Rt = σBt + µ1 − 2





t

t + (µ2 − µ1 )

1 (τ ≤s) ds.

(3)

0

This model was considered by Shiryaev [Shi63] who studied the problem of detecting the change time τ as early and reliably as possible when one only observes the process (St )t≥0 . Assumptions and Notation • The σ algebra generated by the observations at time t is denoted by FtS := σ (Su , 0 ≤ u ≤ t) , t ∈ [0, T ]. Note that the Brownian motion (Bt )0≤t≤T is not adapted to the filtration (FtS )t≥0 . • The Brownian motion (Bt )t≥0 and the random variable τ are independent. • The change time τ follows an exponential law 1 of parameter λ: P (τ > t) = e−λt ,

t ≥ 0.

(4)

• The value of the portfolio at time t is denoted by Wt . • We denote by Ft the conditional a posteriori probability (constructed by means of the observation of the process S) that the change time has occurred within the interval [0, t]:   (5) Ft := P τ ≤ t/FtS . • We denote by (Lt )t≥0 the following exponential likelihood-ratio process :

1 Lt = exp (µ2 − µ1 )Rt σ2 (6)    2 1 σ 2 ) t . − 2 (µ2 − µ1 ) + 2(µ2 − µ1 )(µ1 − 2σ 2 • Finally, the parameters µ1 , µ2 , σ > 0 and r ≥ 0 are such that µ1 − 1

σ2 σ2 < r < µ2 − . 2 2

Any other law is allowed to derive our main results.

18

C. Blanchet-Scalliet et al.

3 A Technical Analysis Detection Strategy 3.1 Introduction Technical analysis is an approach which is based on the prediction of the future evolution of a financial instrument price using only its price history. Thus, technical analysts compute indicators which result from the past history of transaction prices and volumes. These indicators are used as signals to anticipate future changes in prices (see, e.g., the book by Steve Achelis [Ach00]). Here, we limit ourselves to the moving average indicator because it is simple and often used to detect changes in return rates. To obtain its value, one averages the closing prices of the stock during the δ most recent time periods. 3.2 Moving Average Based on the Prices Our trader takes decisions at discrete times. We thus consider a regular parT : tition of the interval [0, T ] with step ∆t = N 0 = t0 < t1 < . . . < tN = T,

tn = n∆t.

We denote by πt ∈ {0, 1} the proportion of the agent’s wealth invested in the risky asset at time t, and by Mtδ the moving average of the prices. Therefore, Mtδ

1 = δ



t

Su du.

(7)

t−δ

We suppose that, at time 0, the agent knows the history of the risky asset prices before time 0 and has enough data to compute M0δ . At each tn , n ∈ [1 · · · N ], the agent invests all his/her wealth into the risky asset if the price Stn is larger than the moving average Mtδn . Otherwise, he/she invests all the wealth into the riskless asset. Consequently, πtn = 1 (St ≥M δ ) . n tn

(8)

Denote by x the initial wealth of the trader. The wealth at time tn+1 is   St0n+1 Stn+1 Wtn+1 = Wtn πtn + 0 (1 − πtn ) , Stn Stn and therefore, since St0n+1 /St0n = exp(r∆t), WT = x

N −1  n=0

    πtn exp(Rtn+1 − Rtn ) − exp(r∆t) + exp(r∆t) .

(9)

Technical Analysis Techniques versus Mathematical Models

19

3.3 The Particular Case of the Logarithmic Utility Function One of our key results is Proposition 1. The expectation of the logarithmic utility function of the agent’s wealth is   σ2 (1) E log(WT ) = log(x) + rT + µ2 − − r T pδ 2    1 − e−λT  (2) σ2 (1) λδ (3) (p −r +∆t µ2 − − p )e + p δ δ δ 2 1 − e−λ∆t 1 − e−λT (3) −∆t(µ2 − µ1 )(e−λ∆t − λ∆t) p , 1 − e−λ∆t δ where we have set    ∞ ∞ µ2 −3/2 (µ2 /σ−σ/2)2 δ (1+z2 ) z z − 2σ 2 y − (1) 2 e iσ2 δ/2 pδ = dzdy, (10) 2y σ2 y 0 y  δ µ −3/2 (µ2 /σ−σ/2)2 (δ−v) (1+z22 ) z 2 − 2σ 2 y − (2)  2 2 2 1 e pδ = z1 2y2 δy2 ≥ +z2 0 R4 y1   µ1 −3/2 (µ1 /σ−σ/2)2 v (1+z2 )   z2 z1 z1 − − 2σ 2 y1 2 1 iσ2 (δ−v)/2 e iσ2 v/2 σ 2 y2 2y1 σ 2 y1 e−λv dy1 dz1 dy2 dz2 dv,    ∞ ∞ µ1 −3/2 (µ1 /σ−σ/2)2 δ (1+z2 ) z z − − 2σ 2 y (3) 2 e iσ2 δ/2 pδ = dzdy, 2y σ2 y 0 y  2 zeπ /4y ∞ −z cosh(u)−u2 /4y e sinh(u) sin(πu/2y)du. iy (z) = √ π πy 0

(11) (12) (13)

Proof. The tedious calculation
t involves an explicit formula, due to Yor [Yor01], for the density of ( 0 exp(2Bs )ds, Bt ). See [BSDG+ 05] for details. 3.4 Empirical Determination of a Good Windowing One can optimize the choice of δ by using Proposition 1 and deterministic numerical optimization procedures, or by means of Monte Carlo simulations. In this subsection we present results obtained from Monte Carlo simulations, which show that bad choices of δ may weaken the performance of the technical analyst strategy. For each value of δ we have simulated 500,000 trajectories of the asset price and computed the expectation E log(WT ) by a Monte Carlo method. The parameters used to obtain Figure 1(a) and Figure 1(b) are all equal but the volatility. It is clear from the figures that the optimal choice of δ varies. When the volatility is 5 percent, the optimal choice of δ is around 0.3

20

C. Blanchet-Scalliet et al. 4.8 4.78

E(log(W_T))

E(log(W_T))

4.79 4.77 4.76 4.75 4.74 4.73 4.72 0

0.2

0.4

0.6

0.8

1

1.2

1.4

4.88 4.87 4.86 4.85 4.84 4.83 4.82 4.81 4.8 4.79 4.78 4.77

1.6

0

0.1

order of moving average = delta

0.2

0.3

0.4

0.5

0.6

0.7

order of moving average = delta

(a)

(b) 5.07

E(log(W_T))

5.06 5.05 5.04 5.03 5.02 5.01 0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

order of moving average = delta

(c)

Fig. 1. E(log(WT )) as a function of δ.

whereas, when the volatility is 15 percent, the optimal choice of δ is around 0.8. The parameters used to obtain Figure 1(b) and Figure 1(c) are all identical but the maturity. The optimal choice of δ is around 0.3 when the maturity is 2 years, and is around 0.4 when the maturity is 3 years. The empirical variance of log(WT ) is around 0.04. Thus, the Monte Carlo error on E log(WT ) is of order 5.10−4 with probability 0.99. The number of trajectories used for these simulations seems to be too large; however, considered as a function of δ, the quantity E log(WT ) varies very slowly, so that we really need a large number of simulations to obtain the smooth curves (Figure 1). Parameter Figure 1(a) Figure 1(b) Figure 1(c)

µ1 −0.2 −0.2 −0.2

µ2 0.2 0.2 0.2

λ 2 2 2

σ 0.15 0.05 0.05

r 0.0 0.0 0.0

T 2.0 2.0 3.0

Figure 2 below illustrates the impact of the parameters µ1 , µ2 and σ on the optimal choice of δ.

Technical Analysis Techniques versus Mathematical Models µ1=−0.2µ2=0.2

µ1=−0.1µ2=0.2 4.86

4.82

4.84

E[log(WT)]

4.84

4.8

4.82

4.78

4.8

4.76

4.78

4.74

0

0.5

1

21

1.5

2

4.76

0

0.5

1

1.5

2

1.5

2

µ1=−0.1µ2=0.1

µ1=−0.2µ2=0.1 4.7

4.7 4.69 E[log(WT)]

4.69

4.68 4.68

4.67 4.67

4.66 4.65

0

0.5

1 delta (years)

1.5

2

4.66

0

0.5

1 delta (years)

Fig. 2. Volatility and Optimal Moving Average Window Size: Plot of Expected Value of the Log of Terminal Wealth vs. Window size with T = 2, λ = 2, - σ = 0.1, - - σ = 0.15, and -. σ = 0.2.

Remark 1. One can observe that the empirical optimal choices of δ are close to the classical values used by the technical analysts, that is, around 200 days or 50 days. One can also observe from Monte Carlo simulations that these optimal values also hold when the trader’s utility function belongs to the HARA family: see [BSDG+ 05].

4 The Optimal Portfolio Allocation Strategy 4.1 A General Formula In this section our aim is to make explicit the optimal wealth and strategy of a trader who perfectly knows all the parameters µ1 , µ2 , λ and σ. Of course, this situation is unrealistic. However it is worth computing the best financial performances that one can expect within our model. To be able to compare this optimal strategy to a technical analyst strategy, we impose constraints on the portfolio. Indeed, a technical analyst is only allowed to invest all his/her

22

C. Blanchet-Scalliet et al.

wealth in the stock or the bond. Therefore the proportions of the trader’s wealth invested in the stock are constrained to lie within the interval [0, 1]. To compute the constrained optimal wealth we use the martingale approach to stochastic control problems as developed by Karatzas, Shreve, Cvitani´c, etc. More precisely, we follow and carefully adapt the martingale approach to the celebrated Merton problem [Mer71]. We emphasize that our situation differs from the Merton problem by two aspects: • The drift coefficient of the dynamics of the risky asset is not constant over time (since it changes at the random time τ ). • Here we must face some subtle measurability issues since the trader’s strategy needs to be adapted with respect to the filtration generated by (St ): as already noticed, the drift change at the random time τ makes this filtration different from the filtration generated by the Brownian motion (Bt ). Let πt be the proportion of the trader’s wealth invested in the stock at time t; the remaining proportion 1 − πt is invested in the bond. For a given nonrandom initial capital x > 0, let W·x,π denote the wealth process corresponding to the portfolio (π· ). Let A(x) denote the set of admissible portfolios, that is, A(x) := {π· − FtS − progressively measurable process s.t. W0x,π = x, Wtx,π > 0 for all t > 0, π· ∈ [0, 1]}. The investor’s objective is to maximize his/her expected utility U of wealth at the terminal time T . The value function thus is V (x) :=

sup E U (WTπ ).

π· ∈A(x)

As in Karatzas-Shreve [KS98], we introduce an auxiliary unconstrained market defined as follows. We first decompose the process R in its own filtration as   σ2 dRt = (µ1 − ) + (µ2 − µ1 )Ft dt + σdB t , 2 where B · is the innovation process, i.e., the FtS - Brownian motion defined as    t 1 σ2 )t − (µ2 − µ1 ) Bt = Fs ds , t ≥ 0, Rt − (µ1 − σ 2 0 where F is the conditional a posteriori probability (5). Let D the subset of the {FtS }− progressively measurable processes ν : [0, T ] × Ω → R such that 

T

E

ν − (t)dt < ∞ , where ν − (t) := − inf(0, ν(t)).

0

The bond price process S 0 (ν) and the stock price S(ν) satisfy

Technical Analysis Techniques versus Mathematical Models

23


t St0 (ν) = 1 + 0 Su0 (ν)(r + ν − (u))du,  
t St (ν) = S0 + 0 Su (ν) (µ1 + (µ2 − µ1 )Fu + ν(u)− + ν(u))du + σdB u . For each auxiliary unconstrained market driven by a process ν, the value function is V (ν, x) := sup Ex U (WTπ (ν)), π· ∈A(ν,x)

where    dWtπ (ν) = Wtπ (ν) (r+ν − (t))dt+πt ν(t)dt+(µ2 −µ1 )Ft dt+(µ1 −r)dt+σdB t . Proposition 2. If there exists ν such that V ( ν , x) = inf V (ν, x)

(14)

ν∈D

then there exists an optimal portfolio π ∗ for which the optimal wealth is ∗

ν ). Wt∗ = Wtπ (

(15)

An optimal portfolio allocation strategy is   φt µ1 − r + (µ2 − µ1 )Ft + ν(t) ∗ −1
t πt := σ + , (16) − σ Htν Wt∗ e−rt− 0 ν (s)ds where Ft defined in (5) satisfies Ft =


t

e−λs L−1 s ds ,
t λt −λs 1 + λe Lt 0 e L−1 s ds λeλt Lt

0

and Htν is the exponential process defined by   
t µ1 − r + ν(s) (µ2 − µ1 )Fs ν  Ht = exp − 0 + dB s σ σ   2 1
t µ1 − r + ν(s) (µ2 − µ1 )Fs + − 0 ds , 2 σ σ and φ is a FtS adapted process which satisfies  t  
T −
T − φs dB s . E HTν e−rT − 0 ν (t)dt (U  )−1 (υHTν e−rT − 0 ν (t)dt ) / FtS = x + 0

Here, v is the Lagrange multiplier which makes the expectation of the left hand side equal to x for all x. Proof. See Karatzas-Shreve [KS98, p. 275] to prove (15). We obtain (16) by solving the classical unconstrained problem for ν.

24

C. Blanchet-Scalliet et al.

4.2 The Particular Case of the Logarithmic Utility Function Proposition 3. If U (·) = log(·) and the initial endowment is x, then the optimal wealth process and strategy are
T

−

xer(T −t)+ t ν (t)dt = , Htν   µ1 − r + (µ2 − µ1 )Ft + ν(t) ∗ πt = , σ2 Wt∗,x

where

(17)

⎧ ⎪ ⎪ ⎪ ⎨ − (µ1 − r + (µ2 − µ1 )Ft )

µ1 − r + (µ2 − µ1 )Ft if < 0, σ2 µ1 − r + (µ2 − µ1 )Ft (18) ν(t) = 0 if ∈ [0, 1], ⎪ ⎪ 2 ⎪ σ ⎩ 2 σ − (µ1 − r + (µ2 − µ1 )Ft ) otherwise, and, as above,

ν− (t) = − inf (0, ν(t)) .

Remark 2. The optimal strategies for the constrained problem are the projections on [0, 1] of the optimal strategies for the unconstrained problem. Remark 3. In the case of the logarithmic utility function, when t is small and thus before the change time τ with high probability, one has Ft close to 0; as, ≤ 0, the optimal strategy is close to 0 ; after by hypothesis, one also has µ1σ−r 2 the change time τ , one has Ft close to 1, and the optimal strategy is close to min(1, µ2σ−r 2 ). In both cases, we approximately recover the optimal strategies of the constrained Merton problem with drift parameters equal to µ1 or µ2 respectively. Using (18) one can obtain an explicit formula for the value function corresponding to the optimal strategy: E log(WT ) = log(x) + rT   T  ∞ " σ2 a ⎫ − µ1 − r + (µ2 − µ1 ) + 1⎧ ⎨ σ 2 − µ1 + r ⎬ 1+a 2 0 0 a> ⎩ µ2 − σ 2 + r ⎭ &  2 1 a ⎧ ⎫ 1⎨ µ − r µ1 − r + (µ2 − µ1 ) σ 2 − µ1 + r ⎬ 1 σ2 1+a
Technical Analysis Techniques versus Mathematical Models





t

exp 0

25



µ2 − µ1 ˜ (µ2 − µ1 )2 Bs − s + λs ds, σ 2σ 2

˜ is a Brownian Motion. This density admits a rather complex explicit where B expression which involves the function iy (z) defined as in (13): see Yor [Yor01] and Borodin and Salminen [BS02].

5 A Model and Detect Strategy The optimal strategy of the preceding section assumes that the trader has chosen a mathematical model and controls the investment policy to optimize the expected utility function of his/her wealth, and that the investment policy is time continuous, whereas the technical analyst does not control the policy and invests at discrete times according to a rupture detection rule. We now consider the case of a trader who chooses a mathematical model and wants to reinvest the portfolio only once, namely at the time where the change time τ is optimally detected owing to the price history. In this section we describe the wealth of such a trader, supposing that the reinvestment rule is the same as the technical analyst’s one: at the detected change time from µ1 to µ2 , all the portfolio is reinvested in the risky asset. We also continue to suppose for a while that the trader perfectly knows all the parameters of the model. In the full report [BSDG+ 05], we consider two detection methods proposed by Karatzas [Kar03] and by Shiryaev [Shi02]. Here, we limit ourselves to consider Karatzas’ optimal stopping rule ΘK which minimizes the expected miss R(Θ) := E|Θ − τ | (20) over all stopping rules Θ, where τ is a positive random variable. In view of the results in [Kar03] one has here Proposition 4. The stopping rule ΘK that minimizes the expected miss E|Θ− τ | over all the stopping rules Θ with E(Θ) < ∞ is  '  t p∗ ΘK = inf t ≥ 0 λeλt Lt e−λs L−1 ds ≥ , s 1 − p∗ 0 where Lt is defined as in (6), and p∗ is the unique solution in ( 12 , 1) of the equation  0

1/2

(1 − 2s)e−β/s 2−β s ds = (1 − s)2+β

with β = 2λσ 2 /(µ2 − µ1 )2 .



p∗

1/2

(2s − 1)e−β/s 2−β s ds (1 − s)2+β

26

C. Blanchet-Scalliet et al.

We thus are in a position to compute the wealth of the trader who uses the strategy consisting in investing all of his/her money in the bond until ΘK and in the stock after ΘK . The value of the portfolio at maturity T is WT =

xSθ0K ST 1 (θK ≤T ) + xST0 1 (θK >T ) . Sθ K

In the particular case of the logarithmic utility function, one can exhibit an exact formula for E(log(WT )). Unfortunately this new formula has a complexity similar to (19): see [BSDG+ 05]. However we can numerically compare the performances of the two change time detection strategies (one based on technical analysis, the other one based on a mathematical model), and the optimal portfolio allocation strategy. Figure 3 illustrates, based on the typical results that we have obtained so far, that the methods using mathematical models have better performances than the technical analyst method. 4.85

Optimal allocation Model and detect Technical Analyst

E[log(Wt )]

4.8 4.75 4.7 4.65 4.6

0

0.2

0.4

0.6

0.8

1

1.2

1.4

1.6

1.8

2

Time Fig. 3. Comparison.

6 The Performances of the Strategies Based on Misspecified Models 6.1 Introduction In practice, it is extremely difficult to know parameters exactly. If one may hope to calibrate µ1 and σ relatively well owing to historical data, the value of µ2 cannot be determined a priori (i.e. before the occurrence of the drift change), and the law of τ cannot be calibrated accurately because of the lack of data concerning τ . Consider a trader who believes that the stock price is   dSt = St µ2 + (µ1 − µ2 )1 (t≤τ ) dt + σSt dBt , (21)

Technical Analysis Techniques versus Mathematical Models

27

where the law of τ is exponential with parameter λ. We suppose that the true stock price is given by (2). Our aim is to study the misspecified optimal allocation strategy and the misspecified model and detect strategy. Notation. As above we set Rt = log(St ), where St is the actual price. We define Lt and F t as follows:  '   1 σ2 1 2 ) t , Lt = exp (µ − µ1 )Rt − 2 (µ2 − µ1 ) + 2(µ2 − µ1 )(µ1 − 2 σ2 2 2σ
−1 t λeλt Lt 0 e−λs Ls ds . Ft =
t −1 1 + λeλt Lt 0 e−λs Ls ds 6.2 On the Misspecified Optimal Allocation Strategy Observing the stock price St , the trader computes a pseudo optimal allocation by using the erroneous parameters µ1 , µ2 , σ and τ . Thus the value of his/her misspecified optimal allocation strategy is π ∗t = proj[0,1]

(µ1 − r + (µ2 − µ1 )F t ) , σ2

and the corresponding wealth is ∗ Wt

 rt

= e exp

t



π ∗u d(e−ru Su )

.

0

Numerical Example In this section, we compare numerically the performance of two traders who respectively use a misspecified model and the true model. We fix the value of µ1 = −0.2, σ = 0.15, r = 0.0 and λ = 2.0, and we assume that they are perfectly known by the trader. A contrario µ2 is misspecified. Its true value is µ2 = 0.2. Figure 4 shows the functions t → E(log(Wt )) for three values of µ2 . It suggests that it is better to overestimate µ2 (µ2 > µ2 ) than to underestimate it (µ2 < µ2 ). 6.3 On Misspecified Model and Detect Strategies The erroneous stopping rule is   t K −1 λt Θ = inf t ≥ 0, λe Lt e−λs Ls ds ≥ 0

p∗ 1 − p∗

'

1 where p∗ is the unique solution in ( , 1) of the equation 2  1/2  p∗ (1 − 2s)e−β/s 2−β (2s − 1)e−β/s 2−β s ds = s ds (1 − s)2+β (1 − s)2+β 0 1/2

28

C. Blanchet-Scalliet et al.

µ2 = 0.2 µ2 = 0.3 µ2 = 0.1

4.85

E[log(Wt )]

4.8 4.75 4.7 4.65 4.6

0

0.2

0.4

0.6

0.8

1

1.2

1.4

1.6

1.8

2

Time Fig. 4. Error on µ2 for the optimal trader.

with β = 2λσ 2 /(µ2 − µ1 )2 . The value of the corresponding portfolio is 0 W T = xSΘ K

ST 1 K + xST0 1 (ΘK >T ) . SΘK (Θ ≤T )

6.4 A Comparison Between Misspecified Strategies and the Technical Analysis Technique Our main question is: Is it better to invest according to a mathematical strategy based on a misspecified model, or according to a strategy which does not depend on any mathematical model? Because of the analytical complexity of all the explicit formulae that we have obtained for the various expected utilities of wealth at maturity, we have not yet succeeded to find a mathematical 1 is large, e.g.). answer to this question, even in asymptotic cases (when µ2σ−µ 2 As this part of our work is still in progress, we present here a few numerical results obtained from Monte Carlo simulations. Consider the following study case. Parameters of the model True values

µ1 µ2 λ σ r −0.2 0.2 2 0.15 0.0

Parameters used by the trader µ1 µ2 λ σ r Misspecified values (case I) −0.3 0.1 1.0 0.25 0.0 Misspecified values (case II) −0.3 0.1 3.0 0.25 0.0 Figure 5 shows that the technical analyst overperforms misspecified optimal allocation strategies when the parameter λ is underestimated. We have looked for other cases where the technical analyst is able to overperform the misspecified optimal allocation strategies. Consider the case where the true values of the parameters are in Table 1. Table 2 summarizes our results. It must be read as follows. For the misspecified values µ2 = 0.1, σ = 0.25, λ = 1, if the trader chooses µ1 in the interval (−0.5, −0.05) then

Technical Analysis Techniques versus Mathematical Models

29

4.85

MSP Case II Technical Analyst

E[log(Wt )]

4.8 4.75

MSP Case I 4.7

4.65 4.6

0

0.2

0.4

0.6

0.8

1

Time

1.2

1.4

1.6

1.8

2

Fig. 5. A technical analyst may overperform misspecified optimal allocation strategies. Table 1. True values of the parameters Parameter True Value µ1 -0.2 µ2 0.2 σ 0.15 λ 2

Table 2. Misspecified values and range of the parameters µ1 (-0.5,-0.05) µ2 0.1 σ 0.25 λ 1

µ1 -0.3 µ2 (0,0.13) σ 0.25 λ 1

µ1 -0.3 µ2 0.1 σ (0.2,→) λ 1

µ1 -0.3 µ2 0.1 σ 0.25 λ (0,1.5)

the misspecified optimal strategy is worse than the technical analyst’s one. In fact, other numerical studies show that a single misspecified parameter is not sufficient to allow the technical analyst to overperform the Model and Detect traders. Astonishingly, other simulations show that the technical analyst may overperform the misspecified optimal allocation strategy but not the misspecified model and detect strategy. One can also observe that, when µ2 /µ1 decreases, the performances of well specified and misspecified model and detect strategies decrease. See [BSDG+ 05].

7 Conclusions and Remarks We have compared strategies designed from possibly misspecified mathematical models and strategies designed from technical analysis techniques. We have made explicit the trader’s expected logarithmic utility of wealth in all the cases under study. Unfortunately, the explicit formulae are not propitious to mathematical comparisons. Therefore we have used Monte Carlo numeri-

30

C. Blanchet-Scalliet et al.

cal experiments, and observed from these experiments that technical analysis techniques may overperform mathematical techniques in the case of severe misspecifications. Our study also brings some information on the range of misspecifications for which this observation holds true. Jointly with M. Martinez (INRIA) and S. Rubenthaler (University of Nice Sophia Antipolis) we are now considering the infinite time case where the instantaneous expected rate of return of the stock changes at the jump times of a Poisson process and the values after each change time are unknown. We also plan to consider technical analysis techniques different from the moving average considered here.

Acknowledgment This research is part of the Swiss national science foundation research program NCCR Finrisk which has funded A. Diop during her postdoc studies at INRIA and University of Z¨ urich.

References [Ach00] [BS02]

S. Achelis. Technical Analysis from A to Z. McGraw Hill, 2000. A. N. Borodin and P. Salminen. Handbook of Brownian Motion—Facts and Formulae. Probability and its Applications. Birkh¨ auser Verlag, Basel, second edition, 2002. [BSDG+ 05] C. Blanchet-Scalliet, A. Diop, R. Gibson, D. Talay, E. Tanr´e, and K. Kaminski. Technical Analysis Compared to Mathematical Models Based Methods Under Misspecification. NCCR-FINRISK Working Paper Series 253, 2005. http://www.nccr-finrisk.unizh.ch/wp/index.php [Kar03] I. Karatzas. A note on Bayesian detection of change-points with an expected miss criterion. Statist. Decisions, 21(1):3–13, 2003. [KS98] I. Karatzas and S. E. Shreve. Methods of Mathematical Finance, volume 39 of Applications of Mathematics. Springer-Verlag, New York, 1998. [LMW00] A. W. Lo, H. Mamaysky, and J. Wang. Foundations of technical analysis: Computational algorithms, statistical inference, and empirical implementation. Journal of Finance, LV(4):1705–1770, 2000. [Mer71] R. C. Merton. Optimum consumption and portfolio rules in a continuous-time model. J. Econom. Theory, 3(4):373–413, 1971. [Shi63] A. N. Shiryaev. On optimum methods in quickest detection problems. Theory Probab. Applications, 8:22–46, 1963. [Shi02] A. N. Shiryaev. Quickest detection problems in the technical analysis of the financial data. In Mathematical Finance—Bachelier Congress, 2000 (Paris), Springer Finance, pages 487–521. Springer, Berlin, 2002. [Yor01] M. Yor. Exponential Functionals of Brownian Motion and Related Processes. Springer Finance. Springer, Berlin, 2001.

Weak Approximation of Stopped Dffusions F.M. Buchmann and W.P. Petersen Seminar for Applied Mathematics, Swiss Federal Institute of Technology CH-8092 Z¨ urich, Switzerland [email protected] [email protected] Summary. In this work we study standard Euler updates for simulating stopped diffusions. We make extensive use of the fact that in many applications approximations have to be good only in the weak sense. This means that for good convergence properties only an accurate approximation of the distribution is essential whereas path wise convergence is not needed. Consequently, we sample needed random variables from their analytical distributions (or suitable approximations). As an immediate application we discuss the computation of first exit times of diffusions from a domain. We focus on one dimensional situations but illustrate extensions and applications in higher dimensional settings. We include a series of numerical experiments confirming the conjectured accuracy of our methods (they are of weak order one).

1 Introduction The simulation of a stopped diffusion is of significant interest in many applications. Often, an approximation of the first exit time of a stochastic process from a domain is needed to get good convergence in numerical simulation. Usually, this approximation has to be accurate only in a weak sense: only the resulting distributions are required whereas path wise convergence is not. A typical application is the probabilistic solution of Dirichlet problems in bounded domains. With this in mind, we introduce in this paper a method to weakly approximate stopped diffusions. Our main goal is to find good approximations to both first exit times and corresponding first exit points in a weak sense. We will apply the simplest method to numerically integrate a stochastic differential equation, the Euler method, and apply some a posteriori corrections to improve estimates for processes exiting a domain D with boundary ∂D. We start with the derivation of our method in one dimension where D is a bounded interval. Subsequently, we extend our method to the weak approximation of diffusions in a bounded domain in n-space, n ≥ 2, by reducing the problem

32

F.M. Buchmann and W.P. Petersen

to 1d. To this end, we approximate the domain D close to its boundary ∂D locally by a half-space.

2 Problem Formulation Consider the diffusion in the sense of Itˆo  t  t µ(X(s)) ds + σ(X(s)) dW (s) X(t) = x + 0

(1)

0

with values in R. The numerical approximation of the solution to (1), X(t), over a fixed (deterministic) time interval [0, T ] is nowadays standard [KP92, Mil95]. The simplest simulation method is the Euler scheme. Applied to (1) with a fixed step size h > 0, it takes the following form: One initializes X0 = x and computes for k = 0, . . . , K − 1 the successive updates (where T = Kh for some K ∈ N) √ (2) Xk+1 = Xk + µ(Xk )h + σ(Xk ) hξk+1 . The random variables ξk+1 in (2) are i.i.d. standard normal with zero mean and unit variance. We denote such random variables as ξ ∼ N (0, 1). It is well known, that under some continuity and growth conditions on the coefficients σ(·) and µ(·), method (2) is then accurate of first order in the weak sense. That is, we have the estimate for the approximation (2) to (1) ∀f ∈ F

∃ Cf

s.t.

|E[f (X(T ))] − E[f (XK )]| ≤ Cf h,

h→0,

(3)

where F is a suitable class of test functions and the constant Cf depends on the function f but is independent of h. See the references [KP92,Mil95] above. The situation becomes much more difficult if one considers X(t) in a bounded domain, in particular, if one stops the integration in (1) as soon as the trajectory reaches the boundary of this domain. Let D ⊂ R be such a domain (a bounded interval in one dimension) and assume x ∈ D. The first exit time of diffusion X from D is τ = τ (x) = inf {X(t) ∈ D} = inf {X(t) ∈ ∂D} , t>0

t>0

X(0) = x .

(4)

As the increment Xk+1 − Xk is unbounded in (2) due to the infinitely long tails of ξ ∼ N (0, 1), the approximation of τ (and of X(τ )) becomes difficult in simulation. It is well known that stopping only if Xk+1 ∈ D and approximating the corresponding first exit time by kh results in a loss of accuracy: Xk approximates X(τ ) no longer with first order of accuracy in the weak sense (and the same holds for Xk+1 or any fixed value in between). Thus, our objective is simulate an approximation of X(t) for 0 ≤ t ≤ τ and, in particular, of both X(τ ) and τ in the weak sense (see (3)).

Weak Approximation of Stopped Dffusions

33

Our motivation for this task is as follows: Consider in addition to (1) for g : R → R the integral  t g(X(s)) ds (5) f (t) = 0

with Euler approximation for a fixed T = Kh (in addition to (2)) f (T ) ≈ h

K−1 (

g(Xk ) .

(6)

k=0

Again imposing continuity and growth conditions on g(·), one has an equivalent estimate as the one given above (see (3)) for the approximation (6) to (5). With τ from (4) we might define in addition to X(τ ) the stopped integral f (τ ). Recalling x = X(0) is the starting point for diffusion (1) we have u(x) = E [ψ(X(τ )) + f (τ )] ,

(7)

which solves the following boundary value problem in the domain D: d σ(x)2 d2 u + g(x) = 0, x ∈ D; u(x) = ψ(x), x ∈ ∂D . (8) u + µ(x) 2 2 dx dx The exact conditions on the functions appearing in (8) for the representation (7) to exist can be found in [Fre85]. We always assume that (8) is well posed and that its representation (7) for X from (1), f from (5), and τ defined in (4) exists so that we can concentrate on the weak approximation of both X(τ ) and f (τ ). These approximation are, again, much harder due to the difficulty in finding τ . Note that setting g ≡ 1 in (5) we have f (τ ) = τ . Let us remark that because τ in (4) is different for every realization of (1), a more rigorous notation of (1) and (5) is in the equations dX = µ(X)1{t<τ } dt + σ(X)1{t<τ } dW and

df = g(X)1{t<τ } dt

with X(0) = x , with f (0) = 0.

(9) (10)

For simplicity of notation we prefer nevertheless the notation without the indicator function 1A of the set A but always assume implicitly that we stop integration for each individual sample at the corresponding first exit time τ . It is important to note that for the error in (3) and likewise in the approximation of the expectation in (7) to be small, path wise convergence is not needed: E[|X(T ) − XK |] does not need to be small. A good approximation of the distributions, however, is essential. We will make extensive use of this observation when constructing our method in the sequel.

3 The Method in Dimension One Our method extends a method for the weak approximation of killed diffusions in a natural way. This method was introduced and analyzed by various authors, see [Bal95,Gob00] and references therein. There, a trajectory is killed if

34

F.M. Buchmann and W.P. Petersen

τ ≤ T for some predefined (deterministic) T < ∞, i.e. it does not contribute to an expectation of the form (7). After an Euler step (2) which results in Xk+1 ∈ D the probability of a possible excursion within tk and tk + h is estimated with an a posteriori test involving an additional Bernoulli random variable (if Xk+1 ∈ D obviously the path is killed). This test is constructed by analyzing the pinned process (i.e. the bridge satisfying X(tk ) = Xk and X(tk+1 ) = Xk+1 )) and estimating a possible intermediate excursion across the closest boundary. 3.1 Overview When considering stopped (instead of killed) diffusions, the situation is more complicated. Looking at (7) it is immediately clear that knowing that τ is not bigger than T is not enough. An estimate of τ is needed. In other words, it is not enough to detect a first exit from D but rather an approximation for its first exit time is needed. To fit our needs, this approximation, however, has to be good only in a weak sense, i.e. its distribution should be as close to the exact one as possible. We make extensive use of this fact in our approach. It can be summarized as follows: After a (preliminary) Euler step tk → tk+1 we generate a random variable τ from a bridge distribution between tk and tk+1 . We then decide if (and when) the domain was left for the first time and set τ ≈ tk + τ . This allows us to add a last Euler step of length τ yielding the approximation (compare with (6)) f (τ ) ≈ h

k−1 (

g(Xi ) + τ g(Xk ) .

(11)

i=0

The construction of the distribution for sampling τ will be different depending on the position of Xk+1 . Namely, if Xk+1 ∈ D we estimate an excursion, whereas in the case that Xk+1 ∈ D we estimate the clear case τ ≤ tk+1 . 3.2 Construction We always approximate D locally by a half space. To derive our method in 1d we assume that the closest boundary point is b and hence consider D = (−∞, b) with x < b. The Euler approximation (2) has the continuous-time approximation (frozen coefficient approximation) X(t) = Xk + µ(Xk )(t − tk ) + σ(Xk )(W (t) − W (tk )), t ∈ [tk , tk+1 ) , √ where W (tk+1 ) − W (tk ) = hξk+1 . To derive our formulae we therefore consider the constant coefficient diffusion X µ,σ (t) = X µ,σ (0) + µt + σW (t),

t>0.

Weak Approximation of Stopped Dffusions

35

The corresponding measure is denoted by Pµ,σ X µ,σ (0) . We further set y = Xk and (from (2)) √ z = Xk+1 = y + µ(y)h + σ(y) hξk+1 with ξk+1 ∼ N (0, 1) is our preliminary Euler update (in analogy to the method for killed diffuµ,σ sions). Finally, we write Xy,h,z for the bridge process obtained by pinning µ,σ X (t) at y and z. When applying the scheme (2) with step size h, consequently, it has length h. We denote its law by Pµ,σ y,h,z . In the case of the rare event z = b (it has probability zero in theory but might occur in finite number arithmetic in simulation) we set τ = tk+1 . Otherwise, we proceed as follows. Inside (z < b): Test for an Excursion We first examine the situation where both y < b and z < b. A test for a possible excursion is needed and a (weak) approximation of the corresponding first exit time must be found. Consider the function f (u) = u/σ and define F (t) = f (X µ,σ (t)). By Itˆo’s formula [BS02,RW00], F satisfies dF = µ/σ dt+ dW with F (0) = X µ,σ (0)/σ, i.e. (F (t))t≥0 is a Brownian motion with drift ν = µ/σ (starting at X µ,σ (0)/σ), see [BS02, I.IV.5. and II.2.]. Let τ be the first hitting time of level b > y. Obviously,    µ,σ [τ ≤ h] = P sup X (t) ≥ b Pµ,σ y,h,z y,h,z 0≤t≤h

 =P

  sup Ff (y),h,f (z) (t) ≥ f (b) ,

0≤t≤h

where Ff (y),h,f (z) denotes the version of F which is pinned at f (y) and f (z) and has length h. To simplify notation we set v = f (y), w = f (z) and c = f (b) and find     Pv sup0≤t≤h F (t) ≥ c, F (h) ∈ dw , P sup Fv,h,w (t) ≥ c = Pv [F (h) ∈ dw] 0≤t≤h where Pv is the law corresponding to the diffusion with F (0) = v. Inserting [BS02, formulae (2.1.0.6) and (2.1.1.8)] yields − tσ2 (b−z)(b−y) Pµ,σ , y,h,z [τ ≤ t] = e 2

t>0,

(12)

which is equivalent to [Gob00, p.169]. We now want to generate a random number, T1 , with distribution (12). Recall that we have frozen σ = σ(y). The simplicity of (12) allows to apply the inverse transform method [KP92, p.12]. Inverting (12) we get

36

F.M. Buchmann and W.P. Petersen

T1 = −

2(b − y)(b − z) , σ 2 (y) log u

(13)

where u ∼ U(0, 1) is distributed uniformly in (0, 1). The path hit b between tk and tk + h if T1 ≤ h (in a statistical sense). Furthermore, b was hit for the first time at t = tk +T1 . Otherwise, we estimate there was no excursion within this time step. Outside (z > b): Compute First Exit Time We now consider the situation y < b < z. We first construct the needed density and then show how to sample from it. Constructing the density of τ : Using absolute continuity of the measures Py and Py,h,z , we have for 0 ≤ t < h [Szn89, (3.7), p.371] Pµ,σ y,h,z [τ ∈ dt] =

pµ,σ (h − t; b, z) µ,σ P [τ ∈ dt] , pµ,σ (h; y, z) y

(14)

  2 exp − (y−µt−x) 2 2tσ √ pµ,σ (t; x, y) = 2πtσ 2 denotes the transition density of the solution to dX = µ dt+σ dW , To find the density of the first hitting time one solves for α > 0 the differential equation   2 2 σ d d − α u(x) = 0 . + µ 2 dx2 dx where

and combines the increasing and decreasing solutions (which we denote by −ατ u↑ and u↓ respectively) to get the Laplace transform Eµ,σ ] (see [BS02, y [e p.18]), namely,  ↑ ' √ µ2 +2ασ 2  −ατ  µ u (y)/u↑ (b), y ≤ b (b−y)− |b−y| σ2 σ2 e = Eµ,σ . = e ↓ ↓ y u (y)/u (b), y ≥ b Inverting the Laplace transform we find 2 b − y − (b−µt−y) 2tσ 2 e dt , Pµ,σ y [τ ∈ dt] = √ 2πt3 σ 2

and inserting everything into (14) yields the density for 0 < t < h, ) h b−y µ,σ (15) Py,h,z [τ ∈ dt] = √ 2πσ 2 t3 (h − t)    (z − b)2 1 (z − y)2 (b − y)2 × exp − 2 − + dt . 2σ (h − t) h t

Weak Approximation of Stopped Dffusions

37

Sampling from (15): We next show how to generate a random variable with density (15). After some algebra in the exponent we find from (15) by the substitution x = t/(1 − t) ≥ 0 )   2 (h(b − y) − t(z − y)) h b − y exp − Pµ,σ dt y,h,z [τ ∈ dt] = √ 2σ 2 ht(h − t) 2πσ 2 t3 (h − t) ⎛  2 ⎞ x (z − y) (b − y) − 1+x b−y ⎟ ⎜ exp ⎝− =√ ⎠ dx x 2 2 3 2σ h (1+x)2 2πhσ x 

1 (b − y)2 (z − b)2 exp − =√ 2x hσ 2 (b − y)2 2πhσ 2 x3 b−y



b−y x− z−b

 dx .

A random variable ζ follows the inverse Gaussian distribution with parameters γ > 0, δ > 0 (and we write ζ ∼ IG(γ, δ)) if it has the density [FC78] 0   γ γ(x − δ)2 exp − dx, x > 0 . P[ζ ∈ dx] = 2πx3 2δ 2 x From above calculations it is immediate that if ζ ∼ IG((b − y)2 /hσ 2 , (b − y)/(z − b)) then T2 = hζ/(1 + ζ) is a random variable with density (15). Michael et al. presented an algorithm to generate random variables following the inverse Gaussian distribution [MSH76, p.89]. 3.3 Algorithm To summarize how to approximate f (τ ) (see 5) in a weak sense, recall that τ is the first exit time of diffusion X(t) from bounded D (see (4)), and apply the usual Euler integration to numerically integrate (1) and (5). Suppose that Xk < b (with Xk ≈ X(tk )). Then proceed as follows: 1. Set y√ = Xk , generate ξk+1 ∼ N (0, 1) and compute z = y + µ(y)h + σ(y) hξk+1 . 2. Depending on z there are two cases: a) z < b: Generate T1 according to (13). Depending on T1 there are again two cases: i. T1 ≤ h: add a last Euler step and stop: f (τ ) ≈ fk + T1 g(y). ii. T1 > h: continue integration: fk+1 = fk + hg(y) and Xk+1 = z (and go to 1.). b) z > b: Generate ζ ∼ IG((b − y)2 /hσ 2 , (b − y)/(z − b)), set T2 = hζ/(1 + ζ), add a last Euler step and stop: f (τ ) ≈ fk + T2 g(y). This yields the following approximation for the first exit time τ :

38

F.M. Buchmann and W.P. Petersen

a) τ = tk + T1 (T1 by the inverse transform method (13)), b) τ = tk + T2 (T2 = hζ/(1 + ζ) with ζ ∼ IG(γ, δ), see Sec. 3.2), if Xk < b and a) Xk+1 < b and T1 ≤ h (excursion), b) Xk+1 > b. and out of this we approximate f (τ ) as described in (11).

4 Higher Dimensions: the Brownian Motion Case Let D ⊂ Rn be a bounded domain with boundary ∂D. Consider 1  u(x) + g(x) = 0, 2

x ∈ D,

u(x) = ψ(x),

x ∈ ∂D .

Then u(x) = E[ψ(X(τ ))+f (τ )] where dX = dW , X(0) = x, df = g(X(t)) dt, f (0) = 0 and τ = inf t>0 {X(t) ∈ D}. Here X has values in Rn , f is scalar, g : Rn → R and ψ : Rn → R. We assume that D is such that we can always approximate ∂D locally by a half space. To derive our method, we assume that the local approximation is given by H = {x ∈ Rn : x1 < b} with boundary ∂H = {x ∈ Rn : x1 = b} (assuming here that every Xb ∈ ∂D can be mapped to (b, 0, . . . , 0)T ). Then   τ = inf {X(t) ∈ H} = inf X 1 (t) ≥ b , X(0) = x ∈ D , t>0

t>0

and we can proceed as in Sect. 3 to find τ . To evaluate the boundary condition, we need additionally an approximation for the first exit point X(τ ). We compute it after an approximation for τ is found (because the domain was left or an excursion was detected). In this case, by construction, we approximate X 1 (τ ) ≈ b ∈ ∂H and we sample the remaining n − 1 components on the hyperplane ∂H (the correct distribution can be found easily from [LS89, Lemma 7]) )   τ i i i i τ X (τ ) ≈ y + (z − y ) + τ 1 − i = 2, . . . , n . · ξ i , ξ i ∼ N (0, 1), h h Finally, we project back to ∂D. It is clear that these approximations make only sense if h is small enough: That is, h is much smaller than the radius of surface curvature.

5 Numerical Experiments We show results of extensive tests performed with the algorithm derived previously. We first show results from a statistical test in one dimension where we

Weak Approximation of Stopped Dffusions

39

compare the numerically obtained density of a simple first hitting time (i.e. a histogram) with the known analytical density. We compare our algorithm with a variety of other approaches. At a later stage, we show the performance of our algorithm when applied to the numerical solution of some Dirichlet problems in various dimensions via the stochastic representation of the solution. 5.1 A Statistical Comparison in Dimension One We compute numerically the first hitting time of level b = 1 of a Brownian motion. Its density has its maximum at t = 13 where it forms a non-symmetrically shaped peak, and it has a very long tail. We show results obtained when trying to approximate the peak (peak test) and the tail (tail test) respectively. To measure the quality of the approximations we performed a χ2 -test over equidistant time intervals (the bins of the histogram). As a measure of 1Nb −1 2 (Ni − N pi ) /(N pi ) where approximation we computed [KS73] χ2 = i=0 Nb denotes the number of bins, N the sample size, Ni the number of trials that fell in bin i and pi the relative expected frequency of bin i. Asymptotically, χ2 has a χ2 -distribution with Nb − 1 degrees of freedom (DOFs). For the peak (tail) test we have chosen the bins 0.05, 0.06, . . . , 1.0 (0, 1, . . . , 250) and the sample size N = 1e5 (1e6). The tails of the histogram were not included for computing χ2 . We compare our method (S) with T: The trivial attempt which stops only if a discrete sample satisfies Xk ≥ 1, K: A method applying a killing test and stopping at tk (Kb), at tk + h/2 (Km) and at tk + h (Ke) E: The exponential time stepping method with a killing test [JL00, JL03] and stopping at k/λ (Eb), at (k + 1)/λ (Ee) and at (k + u)/λ where u ∼ U (Eu) and λ is the parameter for this method. The procedure for exponential time stepping is straightforward: after sampling a positive time step h from an exponential distribution with parameter λ, one chooses an increment ∆W from a two-sided exponential distribution with parameter √ 2λ. This increment will be normally distributed with variance equal to the previously chosen h. See, for example, equation 1.1.0.5 on page 153 of [BS02]. We obtained the results shown in Tables 1 (for the peak test) and 2 (tail test). Table 1 shows that when comparing peaks (at t = 1/3) of hitting time distributions, our statistical procedure (S, from Sect. 3) obviously works much better than the other tested procedures (T, Kb, Km, Ke, Eb, Ee, Eu) for all step sizes. The resulting χ2 /DOF values for the other methods when measuring the peak are clearly not acceptable. Table 2 shows that the tails of other procedures’ distributions can be acceptable for small step sizes, particularly the killing tests (Kb, Km, Ke). Considered together, that is both

40

F.M. Buchmann and W.P. Petersen

Table 1. χ2 per DOF obtained with various algorithms for various step sizes h = 1/2k , λ = 2k for the peak test step sizes h = 2−k and λ = 2k method k = 0 k = 1 k = 2 k = 3 k = 4 k = 5 k = 6 S T Kb Km Ke Eb Ee Eu

6.38e1 7.62e3 1.15e4 2.52e4 4.35e4 1.28e4 2.54e4 2.79e2

9.38e1 5.89e3 1.05e4 1.45e4 1.72e4 9.20e3 1.26e4 3.21e2

1.99e1 4.15e3 7.75e3 8.22e3 8.25e3 6.12e3 6.70e3 2.83e2

2.16 2.51e3 4.51e3 3.99e3 3.86e3 3.78e3 3.44e3 9.94e1

1.58 1.29e3 2.09e3 1.78e3 1.77e3 2.53e3 1.74e3 3.55e1

1.34 5.57e2 7.60e2 7.24e2 7.18e2 7.89e2 7.03e2 1.37e1

1.14 1.60e2 1.94e2 1.91e2 1.89e2 1.98e2 1.86e2 7.75

Table 2. χ2 per DOF obtained with various algorithms for various step sizes h = 1/2k , λ = 2k for the tail test step sizes h = 2−k and λ = 2k method k = 0 k = 1 k = 2 k = 3 k = 4 k = 5 S T Kb Km Ke Eb Ee Eu

1.02 6.29e2 1.04 1.04 2.29e3 1.24e2 2.20e3 1.24e2

1.03 3.85e2 1.10 1.10 6.10e2 4.64e1 7.17e2 4.64e1

0.973 2.31e2 9.45e-1 9.45e-1 1.11e2 1.55e1 1.97e2 1.55e1

1.05 1.35e2 1.08 1.08 2.43e1 5.55 5.02e1 5.55

1.02 7.56e1 9.35e-1 9.35e-1 6.70 2.04 1.26e1 2.04

9.20e-1 4.18e1 1.12 1.12 2.51 1.34 3.92 1.34

k=6 9.64e-1 2.22e1 1.02 1.02 1.49 1.16 1.90 1.16

Table 1 and 2, method (S) works definitely better. Some remarks concerning these methods’ computational speeds are noteworthy. For acceptable accuracy, method (T) is slow compared to the others because many more steps are required. Although single steps are faster, no intermediate excursions are detected. Methods (Kb, Km, Ke) and (S) cost roughly the same because their exit tests have similar computational requirements although the formulations are different. Only the last step of (S) is more costly because of the inverse Gaussian distribution. For example, comparing (Kb, Km, Ke) and (S) on the 1-D problem (17) shown in Figure 2, using 105 sample paths and initial timestep h = 1/8: timings taken on an Intel Pentium III workstation gave 23.3, 23.5, 23.1, and 24.0 seconds respectively. Method (T) only took 16.8 seconds, but the resulting relative error was unacceptably 20 times worse than (Kb, Km, Ke) and (S). When using exponential time stepping as in (Eb, Ee, Eu), sampling the Brownian increments is cheaper because the only function calls are logarithms - not square roots, logarithms, and circular functions as in the Box-Muller procedure for normals. There, however, for the

Weak Approximation of Stopped Dffusions

41

same accuracy, one is forced to take more steps because of smearing. That is, the exit time is given by the number of steps times the expected value of the time increment per step. Exponential tails are not long, so there are many small steps for the same final time. Jansons and Lythe [JL00] discuss this in some detail. 5.2 Application to the Feynman-Kac Representation We show results from the following two examples in unit balls D = {x ∈ Rn : |x| < 1}: 1  u(x) + 1 = 0, 2

x ∈ D;

u(x) = 0,

x ∈ ∂D,

(16)

and n  1  u(x) + cos(2πk i xi ) = 0, 2 i=1 2n cos(2πk i xi ) u(x) = i=1 2 2 , 2π |k|

x ∈ D;

(17)

x ∈ ∂D.

Results for problem (16) are collected in Figure 1 while those obtained for problem (17) are shown in Figure 2. From the test results on the exit-time problem (16) in Figure 1, we see that for X(0) = x near the center in the left plot, we get that for any modest step size, statistical errors dominate and the finite step size truncation errors are insignificant. Closer to the edge of the interval with length 2 in the left figure we evidently get a slope one (order h) truncation error. In higher dimensions we see from the results shown in the right figure that asymptotically (i.e. when the step size h is small enough) weak order one convergence is always observed. The same holds for the more general test problem (17), see Figure 2. 100 10–1

101

x=0.99, N=1.6e6 x=0.99, N=6.4e7 x=0.9, N=1.6e6 x=0.9, N=6.4e7 x=0, N=1.6e6 x=0, N=6.4e7

n=128 n=64

100

10–2

10–1

–3

10

10–2 10–4 10–3

10–5 10–2

10–1

10–4

10–3

10–2

10–1

100

Fig. 1. Relative errors vs. step size h for test problem (16). Parameters are on the left n = 1 (dimension), N ∈ {1.6e6, 6.4e7} (sample size) and x ∈ {0, 0.9, 0.99} (where solution is evaluated) whilst on the right we show results for n ∈ {64, 128}, N = 4e6, x = 0.

42

F.M. Buchmann and W.P. Petersen 2

10

i

x =0 i x =0.05 i x =0.1

2

10

1

i

x =0 i x =0.05 i x =0.1

1

10

10

100

100

10–1

10–1

10–4

10–3

10–2

10–1

100

10–4

10–3

10–2

10–1

100

Fig. 2. Relative errors vs. step size h for test problem (17). Parameters are n = 32 (dimension), N = 1e6 (sample size) and x = (x1 , . . . , xn ) with xi ∈ {0, 0.05, 0.1} for i = 1, . . . , n (where solution is evaluated). The chosen frequencies are k = (1, . . . , 1) on the left whilst on the right we show results for k = (3, 2, 1, . . . , 1).

6 Conclusion and Summary In Sect. 2 we stressed the notion that only weak approximations are needed for probabilistic representations of solutions for Dirichlet problems which motivated this stopped diffusion study. To this end, we approached the problems of finding the boundary and process sampling thereon with the intent of accurately representing the first exit time τ and X(τ ) distributions accurately. More extensive studies [Buc05, Buc04], show our approach more successful than step size control [BP03,Mil97], or other cruder inter-step sampling methods. An earlier paper [Buc05], described the 1-D situation in some detail. A much expanded version of the current work was submitted to SIAM J. on Scientific and Statistical Computing. In that pending paper, more emphasis is given to high dimensional problems and the mechanisms for transformations near boundaries. These transformations allow the one dimensional procedure given here and in [Buc05] to be used because a slowly varying smooth surface looks flat when an interior point is close enough to it. Obviously, when a boundary is cusped, detecting an exit depends upon being able to accurately distinquish the inside of D from the outside. Near an interior pointing sharp cusp, a step from inside to inside may have actually gone outside and returned on the other side of the cusp. In that case, which of Sec. 3.2 or Sec. 3.2 can be used is likely to be difficult to determine. We have presented an algorithm which leads itself to an efficient implementation for the simulation of stopped diffusions. Our approach used standard Euler updates and it was based on a method for the simulation of killed diffusions. Instead of simply checking if a path has reached a certain level within or at the end of a time step, we constructed a true stopping time to stop the integration. To achieve this goal, we sampled random numbers having approximatively the right distributions. In the case of diffusions with constant coefficients, these distributions are by construction exact. This allowed

Weak Approximation of Stopped Dffusions

43

us to add a final Euler step of corresponding length to the simulated path and connected integrals. Our numerical tests showed evidence that the resulting distributions and thereof constructed weak approximations are of very high quality.

References [Bal95]

Paolo Baldi. Exact asymptotics for the probability of exit from a domain and applications to simulation. Ann. Probab., 23(4):1644–1670, 1995. [BP03] F.M. Buchmann and W.P. Petersen. Solving Dirichlet problems numerically using the Feynman-Kac representation. BIT, 43(3):519–540, 2003. [BS02] Andrei N. Borodin and Paavo Salminen. Handbook of Brownian motion— facts and formulae. Probability and its Applications. Birkh¨ auser Verlag, Basel, second edition, 2002. [Buc04] F.M. Buchmann. Solving high dimensional Dirichlet problems numerically using the Feynman-Kac representation. PhD thesis, Swiss Federal Institute of Technology Zurich, 2004. [Buc05] F. M. Buchmann. Simulation of stopped diffusions. J. Comp. Phys., 202(2):446–462, 2005. [FC78] J. L. Folks and R. S. Chhikara. The inverse Gaussian distribution and its statistical application—a review. J. Roy. Statist. Soc. Ser. B, 40(3):263– 289, 1978. With discussion. [Fre85] Mark Freidlin. Functional integration and partial differential equations, volume 109 of Annals of Mathematics Studies. Princeton University Press, Princeton, NJ, 1985. [Gob00] Emmanuel Gobet. Weak approximation of killed diffusion using Euler schemes. Stochastic Process. Appl., 87(2):167–197, 2000. [JL00] Kalvis M. Jansons and G. D. Lythe. Efficient numerical solution of stochastic differential equations using exponential timestepping. J. Statist. Phys., 100(5-6):1097–1109, 2000. [JL03] Kalvis M. Jansons and G. D. Lythe. Exponential timestepping with boundary test for stochastic differential equations. SIAM J. Sci. Comput., 24(5):1809–1822 (electronic), 2003. [KP92] Peter E. Kloeden and Eckhard Platen. Numerical solution of stochastic differential equations, volume 23 of Applications of Mathematics (New York). Springer-Verlag, Berlin, 1992. [KS73] Maurice G. Kendall and Alan Stuart. The advanced theory of statistics. Vol. 2. Hafner Publishing Co., New York, third edition, 1973. Inference and relationship. [LS89] H. R. Lerche and D. Siegmund. Approximate exit probabilities for a Brownian bridge on a short time interval, and applications. Adv. in Appl. Probab., 21(1):1–19, 1989. [Mil95] G. N. Milstein. Numerical integration of stochastic differential equations, volume 313 of Mathematics and its Applications. Kluwer Academic Publishers Group, Dordrecht, 1995. Translated and revised from the 1988 Russian original. [Mil97] G. N. Milstein. Weak approximation of a diffusion process in a bounded domain. Stochastics Stochastics Rep., 62(1-2):147–200, 1997.

44

F.M. Buchmann and W.P. Petersen

[MSH76] John R. Michael, William R. Schucany, and Roy W. Haas. Generating random variates using transformations with multiple roots. Am. Stat., 30:88–90, 1976. [RW00] L. C. G. Rogers and David Williams. Diffusions, Markov processes, and martingales. Vol. 2. Cambridge Mathematical Library. Cambridge University Press, Cambridge, 2000. Itˆ o calculus, Reprint of the second (1994) edition. [Szn89] Alain-Sol Sznitman. A limiting result for the structure of collisions between many independent diffusions. Probab. Theory Related Fields, 81(3):353– 381, 1989.

Approximation of Stochastic Programming Problems Christine Choirat1 , Christian Hess2 , and Raffaello Seri3 1

2

3

Dipartimento di Economia, Universit` a degli Studi dell’Insubria, Via Ravasi 2, 21100 Varese, Italy [email protected], Centre de Recherche Viabilit´e, Jeux, Contrˆ ole, Universit´e Paris 9 Dauphine, 75775 Paris CEDEX, France [email protected] Dipartimento di Economia, Universit` a degli Studi dell’Insubria, Via Ravasi 2, 21100 Varese, Italy [email protected]

Summary. In Stochastic Programming, the aim is often the optimization of a criterion function that can be written as an integral or mean functional with respect to a probability measure P. When this functional cannot be computed in closed form, it is customary to approximate it through an empirical mean functional based on a random Monte Carlo sample. Several improved methods have been proposed, using quasi-Monte Carlo samples, quadrature rules, etc. In this paper, we propose a result on the epigraphical approximation of an integral functional through an approximate one. This result allows us to deal with Monte Carlo, quasi-Monte Carlo and quadrature methods. We propose an application to the epi-convergence of stochastic programs approximated through the empirical measure based on an asymptotically mean stationary (ams) sequence. Because of the large scope of applications of ams measures in Applied Probability, this result turns out to be relevant for approximation of stochastic programs through real data.

1 Introduction In Stochastic Programming, one has often to solve an optimization problem of the form (1) (see [4], p. 332): inf Eg (Y, x)

x∈X

(1)

where X ⊂ Rp and Y is a Rq −valued random variable. Most of the time the integral functional (also called the mean functional ) Eg (Y, x) cannot be explicitly calculated, but can be approximated through sampling methods

46

C. Choirat et al.

(see [4], Chap. 10):4 suppose that a sample of realizations of the random variable Y , say (Yi )i=1,...,n , are available. We would like to find conditions under which the solution of the approximated problem 1( g (Yi , x) x∈X n i=1 n

inf

converges almost surely to the solution of the original problem (1). It is common to assume that (Yi )i=1,...,n is a sample of independent and identically distributed realizations of the random variable Y and that the approximated objective function converges uniformly on X almost surely to the original one: n 1 ( as; g (Yi , x) − Eg (Y, x) → 0 sup x∈X n i=1

this can be cast in the framework of the Glivenko-Cantelli problem and implies the convergence of the minimizers. The previous structure can be extended to encompass also the so-called M −estimation used in Statistics and Econometrics, that is, estimation obtained by optimizing a function with respect to some parameters. In the present paper, we shall focus our attention on epigraphical convergence for sequences of stochastic functions defined on a metric space. Epigraphical convergence (epi-convergence, for short) is weaker than uniform convergence, but it is well-suited to approximate minimization problems. Indeed, under suitable compactness assumptions, it entails the convergence of infima and minimizers (see e.g. [3] or [9]). A symmetric notion, called hypographical convergence, enjoys similar properties with respect to maximization problems. Further, as it is known, this type of convergence is closely related to the Painlev´e-Kuratowski convergence for sequences of subsets, so that it has an interesting geometric interpretation allowing for connections with the theory of random sets. Similar results have been already derived in [2], when (Yi )i=1,...,n is a sample of independent and identically distributed Monte Carlo realizations, in [14,15], when (Yi )i=1,...,n is a sample of pairwise independent and identically distributed Monte Carlo realizations, and in [7, 19, 32, 33], when (Yi )i=1,...,n is a stationary ergodic sequence. A particular mention should be made to [27], where a general result similar in scope to the present one is proposed. The next Section contains some preliminaries on epi-convergence (other results on epi-convergence are gathered in Appendix A). Then, Sect. 3, and Theorem 1 in particular, provides a result on the epi-convergence of the objective functions to be minimized. This result is motivated by examples. Section 4 studies in more details its application to asymptotically mean stationary processes. The proofs are deferred to Sect. 5. 4

Much more general stochastic programming problems can be dealt with in this way, such as multistage stochastic programs with recourse and stochastic programs with chance constraints.

Approximation of Stochastic Programming Problems

47

2 Preliminaries on Epi-convergence In this Section, we define the concept of epi-convergence. Its main properties are briefly recalled in Appendix A. For a more complete treatment of the subject, we refer the reader to the monographs [3] or [9]. Let (E, d) be a metric space and φ : E → R = [−∞, +∞] be a function from X into the extended reals. Its epigraph (or upper graph) is defined by: Epi (φ) = {(x, λ) ∈ E × R : φ (x) ≤ λ} . Its hypograph (or lower graph) Hypo (φ) is defined by reversing the inequality. Further, given a sequence (φn )n≥1 of functions from E into R, the functions epi − lim infφn and epi − lim supφn are defined on E by epi−lim infφn (x) = sup lim inf

inf

φn (y) ,

(2)

epi − lim supφn (x) = sup lim sup

inf

φn (y) ,

(3)

k≥1 n→∞ y∈B(x,1/k)

k≥1 n→∞ y∈B(x,1/k)

where B (x, 1/k) denotes the open ball of radius 1/k centered at x. These functions are called respectively the lower epi-limit and the upper epi-limit of (φn )n≥1 . Moreover, both functions are lower semi-continuous (lsc for short) and one has for all x ∈ E epi − lim infφn (x) ≤ epi − lim supφn (x) .

(4)

When the equality holds in (4) at some point x ∈ E, the sequence (φn ) is said to be epi-convergent at x. If this holds for all x ∈ E, the common value defines a function φ that is called the epi-limit of the sequence. This is denoted by φ = epi− lim φn . Equalities (2) and (3) have a geometric counterpart involving the Painlev´eKuratowski convergence of epigraphs on the space of closed sets of E × R. Let us give a general definition. Given a sequence of sets (Cn )n≥1 in the metric space E, we define PK− lim inf Cn
{x ∈ E : x = lim xn , xn ∈ Cn , ∀n ≥ 1} n→∞   PK− lim supCn
x ∈ E : x = lim xi , xi ∈ Cn(i) , ∀i ≥ 1 n→∞





where Cn(i) i≥1 is a subsequence of (Cn )n≥1 . The subsets PK− lim inf Cn and PK− lim supCn are the lower limit and the upper limit of the sequence (Cn )n≥1 . They are both closed and satisfy PK− lim infCn ⊂ PK− lim supCn . A sequence (Cn )n≥1 is said to converge to C in the sense of Painlev´eKuratowski, if C = PK− lim inf Cn = PK− lim supCn . This is denoted by C = PK− limn→∞ Cn . As mentioned above, this notion is strongly connected with epi-convergence: the sequence of functions φn epi-converges to φ on E if and only if the sequence (Epi (φn ))n≥1 PK−converges to Epi (φ), in E × R.

48

C. Choirat et al.

3 Main Results In this Section, we present the mathematical framework of our main result. Consider a metric space (X, d) (usually a subset of an Euclidean space Rq ) endowed with its Borel σ−algebra B (X) and a metric space (Y, ρ) endowed with its Borel σ−algebra B (Y) and with a probability measure P defined on (Y, B (Y)). We want to approximate the stochastic programming problem: inf Eg (Y, x)

(5)

x∈X

where E is the mean with respect to P, through a simpler problem given by inf En g (Y, x)

(6)

x∈X

where En stands for the expectation under Pn . Pn will often be a transition probability measure, i.e. a probability measure depending on a random parameter ω; this point will be made clearer in the following. We look for some conditions under which the solution of the approximated problem (6) converges to the solution of (5). Some remarks and definitions about g, Y and Pn are necessary. We say that an extended function g : Y × X → R is an integrand if it is B (Y) ⊗ B (X) −measurable.5 Furthermore, g is called a normal integrand if g (y, ·) is a lower semi-continuous (lsc) function for P−almost all y ∈ Y. The function g is said to be k−Lipschitz on X if for P−almost all y ∈ Y and for all x, x ∈ X, |g (y, x) − g (y, x )| ≤ k · d (x, x ) . An integrand is said to be positive if, for P−almost every y ∈ Y, g (y, ·) takes on its values in [0, +∞]. Given a normal integrand g and an integer k ≥ 1, the Lipschitz approximation of order k of g is defined by: g k (y, x)
inf {g (y, x ) + k · d (x, x )} ,  x ∈X

k≥1.

We will resort to Lipschitz approximations to derive a characterization of epi-convergence (see Proposition 5) used in our main result.
approximate the mean Eg (Y, x) =
As concerns Y, since our aim is to g (y, x) P (dy) through En g (Y, x) = Y g (y, x) Pn (dy), the only constraint Y is the kind of space on which the approximate probability measure Pn can be q defined: as an example, if the space Y is the unit hypercube [0, 1] , we can take Pn to be the empirical measure defined by a quasi-Monte Carlo sequence (see below). On the other hand, Monte Carlo can be used also in more abstract spaces. On the other hand, also the nature of the probability measure Pn has to be better specified. 5

For real-valued functions, this corresponds exactly to the definition of a measurable random real function in [22] (Definition III-4-4, p. 86) and [12] (Definition 1, p. 157).

Approximation of Stochastic Programming Problems

49

Example 1. Consider the case of Monte Carlo integration through a sample of independent and identically distributed (iid) random variables (Yi )i=1,...,n . Pn can be specified by the empirical distribution: 1( Pn (ω, A)
δY (ω) (A) n i=1 i n

for a set A, where δy (·) is a Dirac delta. Remark that the distribution Pn is random (and depends on ω) since the sequence of iid random variables (Yi )i=1,...,n defining Pn is random. This situation is quite general. Therefore, we suppose that Pn takes the following form: n ( wi · δYi (ω) (A) (7) Pn (ω, A) = i=1

  where wi i=1,...,n is a sequence of weights and the points (Yi (ω))i=1,...,n can be chosen in a stochastic or deterministic way. In a rigorous probabilistic setting, Pn is defined as a transition probability (see e.g. [22], Chapter III-2), i.e. as the mapping: Pn : Ω × B (Y) → [0, 1] (ω, A) → Pn (ω, A) such that: • for any ω ∈ Ω, Pn (ω, ·) is a probability on (Y, B (Y)); • for any A ∈ B (Y), Pn (·, A) is A−measurable. In order to stress the characteristics of P and Pn , we
will use the notations
Eg (Y, x)
Y g (y, x) P (dy) and [En h (Y, x)] (ω)
Y h (y, x) Pn (ω, dy) to indicate respectively the mean with respect to P and Pn . Example 2. (Monte Carlo) The kind of approximation introduced in Example 1 is considered e.g. in [2]. Hess [14,15] covers the case in which (Yi )i=1,...,n is a sample of identically distributed and pairwise independent random variables. The situation in which (Yi )i=1,...,n is a stationary ergodic sequence is considered in [7, 19, 32, 33]. Example 3. (Quasi-Monte Carlo) A different method is to use quasi-Monte Carlo integration methods: in this case, the random sample is substituted by a low discrepancy point set (yi )i=1,...,n and Pn is given by Pn (A)
1n 1 i=1 δyi (A) for A ∈ B (Y); remark that in this case the empirical distribn ution is not random. Quasi-Monte Carlo integration methods are in general more efficient than Monte Carlo ones. However, most of quasi-Monte Carlo methods are limited to sequences of points uniformly scattered in the unit hyd percube [0, 1] (see [10], Sect. 1.5, for some algorithms yielding uniform points

50

C. Choirat et al.

on sets different from the unit hypercube). Using an argument based on the probability integral transform (see e.g. [1], for the result and [27], for applications to Stochastic Programming) and on its multivariate generalization ( [29]; see [20], p. 354, for an application to Numerical Analysis), it is possible to create random vectors with arbitrary distributions: however, this method often requires the evaluation of complex functions (quantile functions of conditional distributions) and can be quite time-demanding in practice. Moreover, quasiMonte Carlo methods require more stringent hypotheses on the behavior of the function g: indeed, while any Lebesgue integrable function can be integrated using Monte Carlo algorithms, low discrepancy point sets require the function to be Riemann integrable (however, in order to derive a convergence rate it also has to be of bounded variation in the sense of Hardy-Krause). Also hybrid techniques in which a deterministic sequence is perturbed by stochastic mechanisms can be considered, such as the randomized nets of Owen ( [23, 24]): in [18], the author applies randomized quasi-Monte Carlo methods to stochastic programming. Example 4. (Quadrature Rules) Faster convergence rates can be obtained using quadrature rules. In this case, the approximating measure is given by: Pn (A) =

n (

wi,n · δyi,n (A)

i=1

for A ∈ B (Y), where the sequence of nodes (yi,n )i=1,...,n and the sequence   1n i,n of weights wi,n i=1,...,n (often constrained to respect = 1) are i=1 w choosen so to optimize a certain measure of accuracy. As an example, n−point Gaussian quadrature is obtained imposing that the integrals with respect to P and Pn coincide for all polynomials of degree 2n − 1 or less. The use of these methods for the approximation of stochastic programming problems has been proposed in [25–27]. Now, we come to our main result on the epi-convergence of approximate Stochastic Programming problems. We start introducing some hypotheses that will be useful in the following. Hypothesis 1: g is a normal integrand on Y × X with values in R+ . Moreover, the function Eg (Y, ·) is not identically +∞ on X. Hypothesis 2: P is a probability measure defined on the measurable space (Y, B (Y)). (Pn )n is a sequence of transition probabilities defined by (7) for each n ≥ 1. Hypothesis 3: For any k ∈ N∗ and any x ∈ X (where X is a dense subset of X):    lim inf En g k (Y, x) (ω) = Eg k (Y, x) , ∀ω ∈ Ω\N1 (x, k) , n→∞

where N1 (x, k) is a negligible set.

Approximation of Stochastic Programming Problems

51

Hypothesis 4: For any x ∈ X : lim sup [En (g (Y, x))] (ω) = Eg (Y, x) , n→∞

∀ω ∈ Ω\N2 (x) ,

where N2 (x) is a negligible set. Theorem 1. Let (X, d) be a metrizable separable space endowed with its Borel σ−algebra B (X), (Y, ρ) a metric space endowed with its Borel σ−algebra B (Y) and g be a normal integrand on Y × X with values in R respecting Hypothesis 1. Then, if Hypotheses 2, 3 and 4 hold, there exists a negligible subset N of Ω such that, for every x ∈ X and ω ∈ Ω\N , one has: Eg (Y, x) = epi− lim [En g (Y, x)] (ω) . n→∞

(8)

Remark 1. Hypothesis 1 can be replaced by one the following two hypotheses: Hypothesis 1’: g is a normal integrand on Y × X with values in R such that there exist a P−integrable function β (·), a constant a > 0 and x0 ∈ X such that, for P−almost any y ∈ Y and any x ∈ X, g (y, x)+β (y)+a·d (x, x0 ) ≥ 0. Moreover, the function Eg (Y, ·) is not identically +∞ on X. Hypothesis 1”: g is a normal integrand on Y×X with values in R such that for any x ∈ X there exists a neighborhood W of x and a P−integrable function β (·) such that, for P−almost any y ∈ Y and any x ∈ W , g (y, x) ≥ β (y). Moreover, the function Eg (Y, ·) is not identically +∞ on X. The main problem with the previous results is to show that Hypotheses 3 and 4 hold true. Hypothesis 4 is usually much simpler to prove. Indeed, algorithms based on random drawings from a distribution P are able to integrate any P−integrable function. On the other hand, the error of quasi-Monte Carlo algorithms based on n points and Gaussian quadrature formulas with n nodes goes to 0 as long as n → ∞ for Riemann integrable functions defined on a bounded interval (see [16], Definition on page 311 and Theorem 7.2.2).6 Hypothesis 3 is often more complex to deal with, since it involves Lipschitz approximations. As an example, suppose that the class of functions {g (·, x) , x ∈ X} is uniformly equicontinuous, that is for every ε > 0, there exists δ > 0 such that for any x ∈ X and all y, y  ∈ Y such that d (y, y  ) < δ, we have |g (y, x) − g (y  , x)| < ε. This implies that the Lipschitz  is  approximation continuous in X (this implies, indeed, that also the class g k (·, x) , x ∈ X is uniformly equicontinuous). This hypothesis does not rule out discontinuities of g arising for a fixed value of x0 and for any y. 6

Using an algorithm based on the probability integral transform (see e.g. [1]) and on its multivariate generalization (see [29]), the result can be extended to Lebesgue-Riemann integrable functions. In [27], the univariate version of this method is dealt with under the name of method of inversion.

52

C. Choirat et al.

4 An Application to ams Transformations The objective of this Section is to derive a functional form of the Birkhoff Ergodic Theorem for asymptotically mean stationary transformations. Consider a probability space (Ω, A, Q) and an A−measurable  transforma tion T : Ω → Ω. T is said to be measure-preserving if Q T −1 A = Q (A) for all A ∈ A. More precisely, we say that T preserves the Q−measure. Equivalently, Q is said to be stationary with respect to T . The sets A ∈ A that satisfy T −1 A = A are called invariant sets and constitute a sub−σ−field I of A. A random variable X is I−measurable if and only if X (ω) = X (T ω) for all ω ∈ Ω: X is said to be an invariant random variable. The measurable and measure-preserving transformation T is said to be ergodic if Q (A) = 0 or 1 for all invariant sets A. Another equivalent formulation is also used: a sequence X1 , X2 , . . . is said to be stationary if the random vectors (X1 , . . . , Xn ) and (Xk+1 , . . . , Xn+k ) have the same distribution for all integers n, k ≥ 1. Any stationary sequence X1 , X2 , . . . can almost surely be rewritten using a measurable and measure-preserving transformation T as Xt (ω) = X0 (T t ω) (see e.g. Proposition 6.11 in [6]). The transformation T : Ω → Ω on (Ω, A, Q) is said 1n−1 to be asymptotically mean stationary (ams) if the sequence n1 j=0 Q T −j A is convergent for all A ∈ A (see [13]). It is known from the Vitali-Hahn-Saks 1n−1 Theorem that lim n1 j=0 QT −j is a probability measure that we indicate n→∞ as P and call asymptotic mean of Q. The interest of ams transformations lies in the fact that asymptotic mean stationarity is a necessary and sufficient condition for a Birkhoff Ergodic Theorem to hold (see Theorem 2 for the Birkhoff Ergodic Theorem for ams processes, and [8,13] for the necessary and sufficient conditions). Moreover, it is possible to show that many processes that appear in Applied Probability (in particular Queueing Theory, see [21, 28, 30, 31], Information Theory, see [11, 13, 17], etc.) are ams and that many real situations give rise to data that are well modelled by ams processes. In particular, the class of asymptotically mean stationary processes includes many subclasses of processes that are relevant for applications, such as stationary processes, cyclostationary processes (i.e. processes with seasonal variations that are stationary in a certain sense, see [5]), Markov processes with an invariant probability measure and even some dynamical systems. Therefore the following result is particularly suitable for approximations of stochastic programs using real data. The following Theorem (see [8]) will be used to derive the main result of this Section. Theorem 2. Let (Ω, A) be a measurable space, T : Ω → Ω be a measurable transformation and X an extended real-valued positive random variable defined on (Ω, A). In addition, suppose that Q is an ams measure with respect to T on (Ω, A) with stationary mean P. Then, for Q− (and P−) almost every ω ∈ Ω, one has

Approximation of Stochastic Programming Problems

53

n−1 1(  i  X T ω = E (X |I ) (ω) , n→∞ n i=0

lim

where each side can be equal to +∞ or −∞, and where E (X |I ) denotes the conditional expectation taken on (Ω, A, P). We introduce the following hypothesis that will be useful in the proof of Corollary 1. Hypothesis 2’: Q is an asymptotically mean stationary probability measure defined on the measurable space (Y, B (Y)) with stationary mean P; P is    an ergodic probability measure. (Yi (ω))i=1,...,n = Y0 T i ω i=1,...,n is a realization of length n from the probability measure Q. Pn is a sequence of transition probabilities defined by 1( 1( i δYi (ω) (A) = δ (A) n i=1 n i=1 Y0 (T ω) n

Pn (ω, A) =

n

for each n ≥ 1. Corollary 1. Let (X, d) be a metrizable separable space endowed with its Borel σ−algebra B (X), (Y, ρ) a metric space endowed with its Borel σ−algebra B (Y) and g be a normal integrand on Y × X with values in R respecting Hypothesis 1. Then, if Hypothesis 2’ holds, there exists a P− and Q−negligible subset N of Ω such that, for every x ∈ X and ω ∈ Ω\N , one has: n−1 1( Eg (Y, x) = epi− lim g (Yi (ω) , x) , n→∞ n i=0

where E denotes the expectation under the probability measure P.

5 Proofs Proof of Theorem 1. For any n ≥ 1, define on Ω × X the function hn (ω, x)
[En g (Y, x)] (ω) . To show (8), it is enough to prove the following two inequalities: epi− lim inf hn (ω, x) ≥ Eg (Y, x) ,

∀ω ∈ Ω\N1 , ∀x ∈ X ,

(9)

epi− lim sup hn (ω, x) ≤ Eg (Y, x) ,

∀ω ∈ Ω\N2 , ∀x ∈ X ,

(10)

n→∞

n→∞

where N1 and N2 are some negligible subsets of Ω that will be specified in the following. Recall that, for any ω ∈ Ω and for any fixed k ≥ 1, the Lipschitz approximation of order k of hn (ω, ·) is defined by:

54

C. Choirat et al.

hkn (ω, x)
inf {hn (ω, x ) + k · d (x, x )}  x ∈X

∀x ∈ X .

Now, using the super-additivity of the infimum operation, we easily obtain:   hkn (ω, x) ≥ En g k (Y, x) (ω) . (11) An appeal to Proposition 4.4 in [15] shows that g k and hkn are A3 ⊗ B (X) −

measurable.  k  for any x ∈ X and k ≥ 1, we can apply Hypoth Consequently, esis 3 to En g (Y, x) n≥1 . This proves the existence of a negligible subset N1 (x, k) such that, for any ω ∈ Ω\N1 (x, k),   lim inf hkn (ω, x) ≥ Eg k (Y, x) (ω) . (12) n

  Set N1
x∈X k≥1 N1 (x, k), where we recall that X is a dense countable subset of X. Inequality (12) is valid for ω ∈ Ω\N1 , k ≥ 1 and x ∈ X ; moreover, it remains valid for any x ∈ X because each side of (12) defines a Lipschitz function of x, with Lipschitz constant k. Then, taking the supremum, with respect to k, in both sides of (12) and using formula (20) together with the monotone convergence theorem, we obtain (9). To prove (10), it is useful to put, for any x ∈ X, φ (x) = Eg (Y, x) and, for any k > 1 and x ∈ X, φk (x) = inf {φ (x ) + kd (x, x ) : x ∈ X}. First, observe that, due to the properness of φ, φk is finite on X. Further, for any x ∈ X , p ≥ 1 and k ≥ 1, one can find x = x (x, p, k) ∈ X such that φ (x ) + kd (x, x ) ≤ φk (x) + p1 . Hence, for each x ∈ X and k ≥ 1, the following equality holds true: φk (x) = inf {φ (x (x, p, k)) + kd (x, x (x, p, k)) : p ≥ 1} .

(13)

Further, applying Hypothesis 4 to the sequence (En [g (Y, x (x, p, k))])n≥1 , we can see that, for every x ∈ X , k ≥ 1 and p ≥ 1, there exists a negligible subset N2 (x, p, k) such that, for every ω ∈ Ω\N2 (x, p, k), lim sup [En g (Y, x (x, p, k))] (ω) ≤ φ (x (x, p, k)) .

(14)

n

   Put N2
x∈X p≥1 k≥1 N2 (x, p, k) and consider ω ∈ Ω\N2 . For any x ∈ X and k ≥ 1, we have: lim sup hkn (ω, x) ≤ inf lim sup [hn (ω, x ) + kd (x, x )] .  x ∈X n→∞

n

Restricting the infimum to the subset {x (x, p, k) : p ≥ 1} and using (14) and (13), we obtain lim sup hkn (ω, x) ≤ inf [φ (x (x, p, k) , ω) + kd (x, x (x, p, k))] = φk (x) . n

p≥1

So, we have proved, for each k ≥ 1 and ω ∈ Ω\N2 ,

Approximation of Stochastic Programming Problems

lim sup hkn (ω, x) ≤ φk (x)

∀x ∈ X .

n

55

(15)

Then, invoking once more the Lipschitz property, we conclude that (15) remains valid for all x ∈ X. Finally, taking the supremum on k in both sides of (15) and using (21), we get (10). Proof of Corollary 1. Hypotheses 3 and 4 of Theorem 1 holds except on P−negligible sets because of Theorem 2; this implies that P (N ) = 0. Moreover, since P is ergodic, [En (g (Y, x) |I )] (ω) = [En g (Y, x)] (ω). The fact that P (N ) = Q (N ) = 0 derives from the properties of ams measures (see [13]).

A Some Results on Epi-convergence We begin by a sequential characterization of epi-convergence (Proposition 1.14 in [3]). Proposition 1. A sequence of functions (φn )n∈N from E into the extended reals epi-converges to a function φ at x ∈ E, if and only if the following two properties (i) and (ii) hold: (i) For each sequence (xn ) converging to x ∈ E one has φ (x) ≤ lim inf φn (xn ) . n→∞

(ii) There exists a sequence (xn ) in E converging to x and such that φ (x) ≥ lim sup φn (xn ) . n→∞

Remark 2. Properties (i) and (ii) are equivalent to (i) and (ii’) where: (ii’) There exists a sequence (xn ) converging to x such that φ (x) = limn→∞ φn (xn ). The variational properties of epi-convergence play a crucial role in the present paper. For this purpose, some notations are useful. For any extended realvalued function φ defined on E, we define the set of (exact) minimizers of φ on E by setting '  Arg min (φ)
x ∈ E : φ (x) = inf φ (y) . y∈E

More generally, for a function φ such that the infimum on E is different from −∞, we define, for any α ≥ 0, the set of α−approximate minimizers by '  α − Arg min (φ)
x ∈ E : φ (x) ≤ inf φ (y) + α . y∈E

Whenever α > 0, the set of α−approximate minimizers is non-empty (unlike the set of exact minimizers which is obtained for α = 0). The variational properties of epi-convergence that we need are stated in the following result (see Corollary 2.10 in [3]).

56

C. Choirat et al.

Proposition 2. The following two results hold: (i) Assume that (φn ) is epi-convergent to φ, that is φ = epi− limn→∞ φn . Then, the following inequality holds inf φ (x) ≥ lim sup inf φn (x) .

x∈E

n→∞ x∈E

(16)

(ii) Let (αn )n∈N be a sequence of positive reals converging to 0. For any n ≥ 1, let xn be an αn −approximate minimizer of φn . If the sequence (xn ) admits a subsequence converging to some x ∈ E, then x belongs to Arg min (φ) and (16) becomes min φ (x) = lim sup inf φn (x) . x∈E

n→∞ x∈E

Now we introduce some further properties of epi-convergence, and in particular its relation with uniform convergence. We recall that the lower (resp. upper ) hypo-limits, as well as the hypo-convergence of a sequence (φn ), can be obtained from the corresponding epigraphical concepts in a symmetric way. Indeed, the sequence (φn ) hypo-converges to φ iff (−φn ) epi-converges to −φ. In order to present the relation between epi-convergence, hypo-convergence and uniform convergence, we remark that replacing (φn ) with (−φn ) and φ with −φ in the statement of Proposition 1, we get similar characterizations of hypo-convergence. Consequently, a sequence (φn ) is both epi- and hypoconvergent to φ if and only if the following property holds: ∀x ∈ E, ∀ (xn ) → x,

φ (x) = lim φn (xn ) . n→∞

(17)

The following simple result shows the connection with uniform convergence. Proposition 3. If φ and (φn ) satisfy (17), then φ is continuous and (φn ) converges uniformly to φ on all compact sets. Proof. First observe that Remark 2 shows that (φn ) is both epi- and hypoconvergent to φ. Thus φ is both lower and upper semi-continuous, hence continuous on E. Further consider a compact subset K of E and suppose that (φn ) does not converge uniformly to φ on K. It is therefore possible to find α > 0 and a subsequence (φm ) of (φn ) satisfying: φm − φu,K = sup |φm (x) − φ (x)| ≥ α > 0

for all m ≥ 1 .

x∈K

But for all m ≥ 1, there exists xm ∈ K such that |φm (xm ) − φ (xm )| ≥ φm − φu,K −

1 . m

(18)

Moreover, by extracting a subsequence (denoted similarly) converging to some x ∈ K, we have

Approximation of Stochastic Programming Problems

|φm (xm ) − φ (x)| ≥ |φm (xm ) − φ (xm )| − |φ (xm ) − φ (x)| .

57

(19)

From (18) and (19), we get |φm (xm ) − φ (x)| ≥ φm − φu,K −

1 − |φ (xm ) − φ (x)| . m

Thus lim inf m→∞ |φm (xm ) − φ (x)| ≥ α > 0, which contradicts property (17). Epi-convergence can be conveniently characterized by means of the Lipschitz approximations (see e.g. Proposition 3.4 of [15]). Given a lsc function φ : E → R and an integer k ≥ 1, the Lipschitz approximation of order k of φ is defined by: φk (x)
inf {φ (y) + k · d (x, y)} , y∈E

k≥1.

Its main properties are listed in the following Proposition (see [15], Proposition 3.3). Proposition 4. Let φ : E → R be a lsc function non identically equal to +∞. Suppose that there exists a > 0, b ∈ R and x0 ∈ E such that, for all x ∈ E, φ (x) + a · d (x, x0 ) + b ≥ 0. Then: (i) ∀k > a and ∀x ∈ E, φk (x) + a · d (x, x0 ) + b ≥ 0, (ii) ∀k ≥ 1, φk < +∞ and φk is Lipschitz of constant k, (iii) ∀x ∈ E, the sequence φk (x) k≥1 is increasing and φ (x) = supk≥1 φk (x). The role of Lipschitz approximations in epi-convergence is explained in the following result (see [15], Proposition 3.4). Proposition 5. Let (φn ) be a sequence of functions from E to R satisfying: there exist a > 0, b ∈ R and x0 ∈ E such that, for every n ≥ 1 and x ∈ E, φn (x) + a · d (x, x0 ) + b ≥ 0. Then for all x ∈ E: epi− lim inf φn (x) = sup lim inf φkn (x)

(20)

epi− lim sup φn (x) = sup lim supφkn (x)

(21)

k≥1 n→∞

k≥1 n→∞

Acknowledgement We would like to thank the editor D. Talay and a referee for providing useful comments and corrections.

58

C. Choirat et al.

References 1. J.E. Angus. The probability integral transform and related results. SIAM Rev., 36(4):652–654, 1994. 2. Z. Artstein and R.J.-B. Wets. Consistency of minimizers and the SLLN for stochastic programs. J. Convex Anal., 2(1-2):1–17, 1995. 3. H. Attouch. Variational convergence for functions and operators. Applicable Mathematics Series. Pitman (Advanced Publishing Program), Boston, MA, 1984. 4. J.R. Birge and F. Louveaux. Introduction to stochastic programming. Springer Series in Operations Research. Springer-Verlag, New York, 1997. 5. R.A. Boyles and W.A. Gardner. Cycloergodic properties of discrete-parameter nonstationary stochastic processes. IEEE Trans. Inform. Theory, 29(1):105–114, 1983. 6. L. Breiman. Probability, volume 7 of Classics in Applied Mathematics. Society for Industrial and Applied Mathematics (SIAM), Philadelphia, PA, 1992. 7. C. Choirat, C. Hess, and R. Seri. A functional version of the Birkhoff ergodic theorem for a normal integrand: a variational approach. Ann. Probab., 31(1):63– 92, 2003. 8. C. Choirat, C. Hess, and R. Seri. Ergodic theorems for extended real-valued random variables. Working paper, 2004. 9. G. Dal Maso. An introduction to Γ -convergence. Progress in Nonlinear Differential Equations and their Applications, 8. Birkh¨ auser Boston Inc., Boston, MA, 1993. 10. K.-T. Fang and Y. Wang. Number-theoretic methods in statistics, volume 51 of Monographs on Statistics and Applied Probability. Chapman & Hall, London, 1994. 11. R.J. Fontana, R.M. Gray, and J.C. Kieffer. Asymptotically mean stationary channels. IEEE Trans. Inform. Theory, 27(3):308–316, 1981. 12. I.I. Gikhman and A.V. Skorokhod. Introduction to the theory of random processes. Translated from the Russian by Scripta Technica, Inc. W. B. Saunders Co., Philadelphia, Pa., 1969. 13. R.M. Gray and J.C. Kieffer. Asymptotically mean stationary measures. Ann. Probab., 8(5):962–973, 1980. 14. C. Hess. Epi-convergence of sequences of normal integrands and strong consistency of the maximum likelihood estimator. Cahiers de Math´ ematiques de la D´ecision, No. 9121, Universit´ e Paris Dauphine, 1991. 15. C. Hess. Epi-convergence of sequences of normal integrands and strong consistency of the maximum likelihood estimator. Ann. Statist., 24(3):1298–1315, 1996. 16. K.L. Judd. Numerical methods in economics. MIT Press, Cambridge, MA, 1998. 17. J.C. Kieffer and M. Rahe. Markov channels are asymptotically mean stationary. SIAM J. Math. Anal., 12(3):293–305, 1981. 18. M. Koivu. Variance reduction in sample approximations of stochastic programs. Working Paper, 2004. 19. L.A. Korf and R.J.-B. Wets. Random-lsc functions: an ergodic theorem. Math. Oper. Res., 26(2):421–445, 2001. 20. J.-J. Liang, K.-T. Fang, F.J. Hickernell, and R. Li. Testing multivariate uniformity and its applications. Math. Comp., 70(233):337–355, 2001.

Approximation of Stochastic Programming Problems

59

21. T. Nakatsuka. Absorbing process in recursive stochastic equations. J. Appl. Probab., 35(2):418–426, 1998. 22. J. Neveu. Bases math´ ematiques du calcul des probabilit´es. Masson et Cie, ´ Editeurs, Paris, 1964. 23. A.B. Owen. Monte Carlo variance of scrambled net quadrature. SIAM J. Numer. Anal., 34(5):1884–1910, 1997. 24. A.B. Owen. Scrambled net variance for integrals of smooth functions. Ann. Statist., 25(4):1541–1562, 1997. 25. T. Pennanen. Epi-convergent discretizations of multistage stochastic programs. Stochastic Programming E-Print Series, Number 2004-03, 2004. 26. T. Pennanen and M. Koivu. Integration quadratures in discretizations of stochastic programs. Stochastic Programming E-Print Series, Number 2002-11, 2002. 27. T. Pennanen and M. Koivu. Epi-convergent discretizations of stochastic programs via integration quadratures. Stochastic Programming E-Print Series, Number 2003-16, 2003. 28. T. Rolski. Queues with nonstationary inputs. Queueing Systems Theory Appl., 5(1-3):113–129, 1989. 29. M. Rosenblatt. Remarks on a multivariate transformation. Ann. Math. Statistics, 23:470–472, 1952. 30. W. Szczotka. Stationary representation of queues. I. Adv. in Appl. Probab., 18(3):815–848, 1986. 31. W. Szczotka. Stationary representation of queues. II. Adv. in Appl. Probab., 18(3):849–859, 1986. 32. M. Valadier. Conditional expectation and ergodic theorem for a positive integrand. J. Nonlinear Convex Anal., 1(3):233–244, 2000. 33. M. Valadier. What differentiates stationary stochastic processes from ergodic ones: a survey. S¯ urikaisekikenky¯ usho K¯ oky¯ uroku, (1215):33–52, 2001.

The Asymptotic Distribution of Quadratic Discrepancies Christine Choirat1 and Raffaello Seri2 1

2

Dipartimento di Economia, Universit` a degli Studi dell’Insubria, Via Ravasi 2, 21100 Varese, Italy [email protected], Dipartimento di Economia, Universit` a degli Studi dell’Insubria, Via Ravasi 2, 21100 Varese, Italy [email protected]

Summary. In Numerical Analysis, several discrepancies have been introduced to test that a sample of n points in the unit hypercube [0, 1]d comes from a uniform distribution. An outstanding example is given by Hickernell’s generalized Lp −discrepancies, that constitute a generalization of the Kolmogorov-Smirnov and the Cram´er-von Mises statistics. These discrepancies can be used in numerical integration by Monte Carlo and quasi-Monte Carlo methods, design of experiments, uniformity and goodness of fit tests. In this paper, after having recalled some necessary asymptotic results derived in companion papers, we show that the case of L2 −discrepancies is more convenient to handle and we provide a new computational approximation of their asymptotic distribution. As an illustration, we show that our algorithm is able to recover the tabulated asymptotic distribution of the Cram´er-von Mises statistic. The results so obtained are very general and can be applied with minor modifications to other discrepancies, such as the diaphony, the weighted spectral test, the Fourier discrepancy and the class of chi-square tests.

1 Introduction Hickernell [18–20] has introduced the generalized Lp −discrepancies Dp (Pn ) d based on the sample of n points Pn in [0, 1] : they extend the KolmogorovSmirnov and Cram´er-von Mises statistics and allow for measuring the degree of nonuniformity of a sample and the efficiency of numerical integration procedures. The links of these figures of merit with goodness-of-fit statistics and with optimal design of experiments have been pointed out in [22]. Liang et al. [35] have started an investigation of the statistical properties of D2 (Pn ) and have derived two alternative statistics, strictly linked to this one, that can be used to test statistically the efficiency of numerical integration procedures. Recently, other authors have investigated similar discrepancies [26–28, 33, 43]. However, the asymptotic statistical properties of generalized Lp −discrepancies

62

C. Choirat and R. Seri

are still largely unknown. In [3], we fill this gap in the literature, providing formulas for the asymptotic distribution of these statistics in various cases of interest. This distribution is nonstandard and not available in closed form (the topic is briefly reviewed in Sect. 2). However, in the special case p = 2, we are able to exploit the structure of the statistics in order to get finer results, among which an alternative representation and rates of convergence (see Sect. 3 for a brief review, [4] for a more detailed treatment). The new distribution is amenable to computational approximation: this constitutes the main topic of the paper and is dealt with in Sect. 4. The results so obtained are very general and can be applied to the generalized L2 −discrepancies of Hickernell [18–21, 23, 24] (see also [35]), the classical ([47], the dyadic [15] and the b−adic [13] diaphony, the weighted spectral test [14, 16, 17]), the χ2 −tests (see e.g. [32]), the discrepancies proposed by Warnock [44], Coveyou [5], Paskov [38], Morokoff and Caflisch [36], Strauch [42] and Grabner et al. [12], Lev [34], Frank and Heinrich [9], Pag`es and Xiao [37], Hoogland and Kleiss [26–28], James et al. [30], van Hameren et al. [43], Hoogland et al. [25] and Grabner et al. [11], the Cram´er-von Mises, Anderson-Darling, ω 2 (q), Watson [45, 46] statistics and several symmetry and independence tests. In the companion paper [4], a large number of examples are studied. We introduce some notation that will be used in the following. For any u index set u ⊂ {1, . . . , d}, we denote by |u| its cardinality, by [0, 1] the |u| −dimensional unit hypercube and by xu a |u| −dimensional vector containing the coordinates of x indexed by the elements of u.

2 Asymptotic Results for Lp −Discrepancies In the case 1 ≤ p ≤ +∞ (see equation (3.8b) in [20]) the generalized discrepancies (see [3]) are defined by the equation: ⎧ ⎡ ( |u| ⎨   1 β · Dp (Pn ) = ⎣ µ (xj ) − u ⎩ n j∈u u =∅ [0,1] ⎫ p ⎤1/p ⎬ (   · µ (xj ) + xj − 1{xj >zj } dxu ⎦ (1) ⎭ z∈Pn j∈u

where summation is over all subsets u ⊂ {1, . . . , d}, β is an arbitrary given positive constant and µ (·) is an 9arbitrary function satisfying µ ∈ 8
1 df f : dx ∈ L∞ ([0, 1]) and 0 f (x) dx = 0 . Remark that in the case p = +∞, the previous formula becomes:  (   1 |u|   D∞ (Pn ) = max ess sup β µ (xj ) + xj − 1{xj >zj } . µ (xj )− u x ∈[0,1]u n j∈u u z∈Pn j∈u

The Asymptotic Distribution of Quadratic Discrepancies

63

In the general case of equation (1), the choices:   β=2, µ (x)  = − 21 x2 − x + 16 , 1 1 1 2 1 µ (x) = − 2 x − 2 − x − 2 + 6 , β = 1 , µ (x) =

1 6

−

x2 2 ,

β=1,

yield respectively the symmetric, the centered and the star discrepancy. In particular, the star L∞ −discrepancy coincides with the Kolmogorov-Smirnov statistic and the star L2 −discrepancy is the Cram´er-von Mises statistic (see [22]). It can be shown that, when n → +∞, the discrepancy Dp (Pn ) converges distributed. Moreover, P−as to 0 if and only if the sample Pn is uniformly √ under the hypothesis of uniform distribution, nDp (Pn ) converges in distribution to a non-degenerate random variable: this means that the average-case error of a Monte Carlo integration procedure decreases as √1n . A worst-case error is given by a Law of the Iterated : Logarithm  (LIL) for the discrepancy Dp (Pn ), stating that Dp (Pn ) = O ln ln n/n P − as. The asymptotic distribution of Dp (Pn ) can be obtained as a function of a stochastic integral with respect to a pinned Brownian Sheet (a multivariate generalization of the Brownian Bridge). This result is quite difficult to use and therefore, in the following, we will investigate how it can be specialized in the case p = 2. On the other hand, under the alternative hypothesis of nonuniformity, the asymptotic distribution is a normal random variable.

3 Asymptotic Results for L2 −Discrepancies Now, we turn to the discrepancy D2 (Pn ) and we generalize some of the results of the previous Section. In [4], we show that the statistic proposed in [35] can be written as the V −statistic 2

[D2 (Pn )] =

n 1 ( h (xk , x ) n2 k,=1

where h (xk , x ) = f (xk , x ) − g1 (xk ) − g1 (x ) + M d is called the kernel and:

(2)

64

C. Choirat and R. Seri

f (xk , x ) =

d 



 M + β 2 µ (xkj ) + µ (xj )

j=1

 1 + B2 (|xkj − xj |) + B1 (xkj ) B1 (xj ) 2 g1 (xk ) =

 ,

d  

 M + β 2 µ (xkj ) ,

j=1

M =1+β



1



2 0

dµ dx

1 B1 (x) = x − , 2

2 dx ,

B2 (x) = x2 − x +

1 . 6 2

The results of the previous Section imply that n · [D2 (Pn )] converges to a well-defined random variable. In this Section, we expose the theoretical results pertaining to the asymptotic distribution of the proposed test statistic. Under the null hypothesis of uniformity, from equation (2), it can be proved that the asymptotic distribution of the V −statistic is given by a weighted infinite mixture of χ2 distributions and is a particular instance of what is 2 called a Gaussian Chaos. Indeed, the statistic [D2 (Pn )] has the asymptotic distribution: ∞ ( 2 D n · [D2 (Pn )] −→ λj Zj2 , (3) j=1

where (Zj ) are independent standard normal random variables and (λj ) are the eigenvalues of the integral operator  A m (xk ) = h (xk , x ) m (x ) dx . (4) [0,1]d

Moreover, the following Berry-Ess´een bound holds with C dependent on some features of the kernel h: ⎧ ⎫; ; ; ∞ ⎨ ⎬; ( ; C ;  2  2 ;P nD2 (Pn ) ≤ y − P ≤ . λj Zj ≤ y ; ; ⎩ ⎭; n ; ; j=1

As an example, we recall, from [20], equations 3.9b and 5.1c, that the star L2 −discrepancy D2 (Pn ) coincides with the Cram´er-von Mises statistic. When d = 1, the asymptotic distribution is: 2

D

n · [D2 (Pn )] −→

∞ ( Zj2 . j 2 π2 j=1

In order to illustrate this result, we have drawn 10, 000 samples Pn of size n (for n ∈ {25, 50, 100, 200, 400}) of uniform independent random variables on

The Asymptotic Distribution of Quadratic Discrepancies

65

2 5

1

n = 25 0

0 0.00

0.25

0.50

0.75

1.00

1.25

0.0

0.5

1.0

1.5

2.0

0.0

0.5

1.0

1.5

2.0

0.0

0.5

1.0

1.5

2.0

0.0

0.5

1.0

1.5

2.0

0.0

0.5

1.0

1.5

2.0

2 5

1

n = 50 0

0 0.00

0.25

0.50

0.75

1.00

1.25 2

5

1

n = 100 0

0 0.00

0.25

0.50

0.75

1.00

1.25 2

5

1

n = 200 0

0 0.00

0.25

0.50

0.75

1.00

1.25 2

5

1

n = 400 0

0 0.00

0.25

0.50

0.75

1.00

1.25

Density

Q−Q Plot

Fig. 1. Star n · [D2 (Pn )]2 for d = 1 and varying n. d

2

[0, 1] . We have calculated n · [D2 (Pn )] for each of the 10, 000 samples for the star discrepancy with d = 1. Then, we have represented in Fig. 1 the density of the star discrepancy (as a histogram and a kernel estimator) and the Q − Q plot with respect to a Gaussian random variable with the same mean and the same variance. It is clear that the density is remarkably stable as reflected by the Berry-Ess´een bound in n1 , and very far from normality. Under the alternative hypothesis of nonuniformity, the quantity 9 √ 8 2 2 n · [D2 (Pn )] − E∗ [D2 (Pn )] converges to a normal random variable whose variance can be calculated(E∗ is the mean under the alternative); the Berry-Ess´een bound is of order O 2

√1 n

.

The simulated example of Fig. 2 shows that [D2 (Pn )] approaches a normal d distribution when the points are not uniformly distributed on [0, 1] . For each graph, we have drawn 10, 000 samples Pn of size n of Beta(2, 2) independent random variables on [0, 1]. The convergence towards a Gaussian random variable is evident, but slower than the convergence towards a second order Gaussian Chaos under the null: this is reflected by the Berry-Ess´een bound. It is interesting to remark that under a set of limiting hypotheses converging to uniformity (but in the alternative), the asymptotic distribution has the 2 D 1∞ 2 form n · [D2 (Pn )] −→ j=1 λj (Zj + aj ) , where (Zj ) are independent standard normal random variables and (aj ) is a series of constants determined by the Pitman drift.

66

C. Choirat and R. Seri 0.50

n = 25

10

0.25

0 −2

0

2

4

6

0.50

n = 50

0.25 0

2

4

6

0.50 0.25

5.0

7.5

10.0

12.5

−2.5

0.0

2.5

5.0

7.5

10.0

12.5

0.0

2.5

5.0

7.5

10.0

12.5

0 0

2

4

6

0.50

−2.5 10

0.25

0 −2

0

2

4

6

0.50

n = 400

2.5

10

−2

n = 200

0.0

0 −2

n = 100

−2.5 10

−2

0

2

4

6

10

0.25

0 −2

0

2

4

6

−2.5

0.0

Density

2.5

5.0

7.5

10.0

12.5

Q−Q Plot

Fig. 2. Convergence of [D2 (Pn )]2 towards a normal random variable.

When d diverges to infinity with n at a certain rate, the statistic converges to a normal distribution. Under some technical conditions on the moments of the kernel h (see the companion paper [4]), we have: √

n3/2 ·{[D2 (Pn )]2 −E[D2 (Pn )]2 }

E(h(X1 ,X1 ))2 −(Eh(X1 ,X1 ))2 +2(n−1)E(h(X1 ,X2 ))2

D

−→ N (0, 1) .

A Berry-Ess´een bound has been obtained, but it is very complex and the rate of decrease to 0 depends on the interplay between d and n. However, it is interesting to remark that this upper bound can be explicitly calculated and used to assess the distance between the two distributions.

4 Practical Aspects of Testing The main problem with the use of generalized discrepancies and related quantities for testing uniformity is the difficulty of computing the cumulative distribution function and of obtaining quantiles of a second order Gaussian chaos. Therefore, in this Section we propose an algorithm for the approximation of the cumulative distribution function of the second order Gaussian chaos random variable in (3). The method we will consider is based on a twofold approximation. First of all, we replace the integral operator Aof equation (4) by an  ˆ : this inapproximate one (say AN ) yielding N eigenvalues λN,j j=1,...,N

The Asymptotic Distribution of Quadratic Discrepancies

67

duces two sources of errors, namely the neglection of the smallest eigenvalues ((λj )j=N +1,...,∞ ) and the numerical approximation of the leading N 1N ˆ 2 ones. Then, we are led to approximate the distribution of j=1 λN,j Zj , where (Zj ) are independent standard normal random variables:  this is a quadratic form in normal random vectors. Indeed, setting Z
Z1 · · · ZN   ˆ N,N (dg (v) is the matrix having v on its diagˆ N,1 · · · λ and AN
dg λ 1N ˆ 2 N  onal and 0’s elsewhere), we get j=1 λN,j Zj = ZA Z where the matrix N A is diagonal. This distribution can be computed through the techniques of [7, 8, 29, 41] for quadratic forms in normal random variables. We use the algorithm in [8] since it allows for a better control of the error term than Imhof’s one and it holds more generally than the one of Sheil and O’Muircheartaigh.3 Related algorithms have been proposed in [40] and [2]. The quantiles of the distribution of X can be obtained using a root finding algorithm, as solutions of the equation P (X < x) = p. In particular, even a simple algorithm such as the bisection method appears in this case to be a good choice. 4.1 The Nystr¨ om Method Consider the operator A defined as:  A m (x) = h (x, y) m (y) dy ,

(5)

for x ∈ X , m ∈ L2 ; let (λj ) and (φj ) be respectively the eigenvalues and eigenfunctions of the operator. The rationale of the Nystr¨ om method is to substitute the integral with an approximate one using a quadrature rule. Consider the quadrature rule given by the sequence of nodes (yj )j=1,...,N and weights (wj )j=1,...,N . Then, the integral operator A of (5) is substituted by the approximate one given by: AN m (x) =

N (

h (x, yj ) m (yj ) · wj .

j=1

The solution of the equation λm (x) = AN m (x) can be obtained through the linear system: λm (yi ) = AN m (yi ) =

N (

h (yi , yj ) m (yj ) · wj ,

j=1 3

In particular, the algorithm of Sheil and O’Muircheartaigh cannot handle negative eigenvalues. However, if the computation of the eigenvalues (λj ) is performed using the version of the Nystr¨ om method recommended in [31], then it is possible that some negative eigenvalues arise.

68

C. Choirat and R. Seri

where yi ∈ (yj )j=1,...,N . This allows one to get a set of N eigenvalues as the  N where H  N is the solution of the matricial eigenvalue problem λ · IN = H matrix with generic element  hij,N = h (yi , yj )·wj , and IN is the identity matrix  N is not symmetric, it is often customary to introduce a new of size N . Since H matrix DN = dg (w1 , . . . , wN ) (we recall that dg (v) is the diagonal matrix with v on the diagonal) and to consider the alternative eigenvalue problem 1/2  −1/2 = HN , where A1/2 is a square root of A and A−1/2 is λ·IN = DN · H N ·DN its inverse. This does not alter the solution of the eigenvalue problem (since the  N and HN coincide), but simplifies the numerical procedure since spectra of H √ now HN is symmetric with generic element given by hij,N = h (yi , yj )· wi wj . q If the variable y takes its values in a Euclidean space R with q > 1, it is possible to use product rules, i.e. rules formed integrating with respect to each component of the vector y through an independent one-dimensional Gauss rule. Nonproduct rules exist but are more difficult to obtain and often limited to low dimensional spaces, so that product rules are often the only reasonable choice. The main problem arising with the use of product Gaussian quadrature is the fact that the number of points increases as N q where N is the order of the one-dimensional Gauss quadrature rule. So, we propose to replace the Gaussian quadrature rule used in the classical Nystr¨ om method with quasiMonte Carlo rules. Indeed, quasi-Monte Carlo points are simpler to obtain and are available also for Rq with very large q.4 Therefore,  if we have a lowˆ N,j discrepancy sequence (yj )j=1,...,N , the set of eigenvalues λ is obj=1,...,N

tained as the spectrum of the matrix HN with generic element hij = h (yi , yj ). 4.2 The Davies Algorithm As explained above, the approximation of the eigenvalues of the integral operator A through the finite spectrum of the matrix HN allows us to replace the second order Gaussian chaos of equation (3) with a quadratic form in Gaussian random variables. Some methods for the approximation of the distribution of this class of random variables have been proposed in the literature, starting from the seminal paper [29]. Here we focus on the method of [7, 8] since it allows for a good control of the error term.  Let us consider a random variable X with characteristic function φ (u) = E eiuX . Under some conditions (i.e. if E |X| < ∞ and, for some c and δ > 0 and for all u > 1, |φ (u)| < cu−δ ; see [7, 10], p. 415), it is possible to express the cumulative distribution function of X as a function of φ:   +∞  φ (u) e−iux 1 P (X < x) = −  du 2 2πu −∞ 4

A rigorous rate of convergence will be derived in a forthcoming companion paper.

The Asymptotic Distribution of Quadratic Discrepancies

69

where  (·) is the imaginary part of a complex number. Using the elementary equality &  ∆" (  +∞ +∞  ∆ +∞ ( f (u) du = f (u + k∆) du = f (u + k∆) du −∞

k=−∞

and setting Φ (u, x)


0

0

φ(u)e−iux , 2πu

1 P (X < x) = + 2 1 = + 2



∆

" −

0



∆

− 0

we can write this as: +∞ ( k=−∞

"

k=−∞

+∞ (

 

φ (u + k∆) e−i(u+k∆)x 2π (u + k∆) &

& du

 (Φ (u + k∆, x)) du.

k=−∞

On the other hand, we have:   +∞  φ (u) e−iux (cos (ut) + i sin (ut)) 1  P (X < x − t) = − du 2 2πu −∞  +∞ 1  (Φ (u, x)) cos (ut) du = − 2 −∞  +∞ −  (iΦ (u, x)) sin (ut) du , −∞

 +∞ 1 P (X < x + t) = −  (Φ (u, x)) cos (ut) du 2 −∞  +∞ +  (iΦ (u, x)) sin (ut) du. −∞

This gives: P (X < x − t) − P (X > x + t)  +∞ = −2  (Φ (u, x)) cos (ut) du −∞ &  ∆" ( +∞  (Φ (u + k∆, x)) cos ((u + k∆) t) du, = −2 0

and taking t =

k=−∞

2πn ∆ :

    2πn 2πn P X <x− −P X >x+ ∆ ∆ &    ∆" ( +∞ 2πnu  (Φ (u + k∆, x)) cos − =2 du. ∆ 0 k=−∞

70

C. Choirat and R. Seri

1+∞ Now, letting S (u)
− k=−∞  (Φ (u + k∆, x)), we can identify in the previous terms the coefficients of the following Fourier expansion in terms of cosines, relative to the function S defined on [0, ∆]:   ∞ ( 2πnu 1 1 a0 (S) + an (S) · cos = · [S (u) + S (∆ − u)] 2 ∆ 2 n=1  
∆ 2 where an (S) = ∆ du for every n ≥ 0. Direct substitution S (u) · cos 2πnu ∆ 0 leads to: 1 · [2P (X < x) − 1] 2∆      ∞   1 ( 2πnu 2πn 2πn + · P X <x− −P X >x+ · cos ∆ n=1 ∆ ∆ ∆ " +∞ & +∞ ( ( 1 −  (Φ (u + k∆, x)) −  (Φ (∆ − u + k∆, x)) . = 2 k=−∞

k=−∞

After some manipulations, this becomes: P (X < x) =

+∞ ( 1 −∆·  (Φ (u + k∆, x)) 2 k=−∞      ∞   ( 2πnu 2πn 2πn − P X <x− −P X >x+ · cos . ∆ ∆ ∆ n=1

A whole set of formulas can be obtained with an appropriate choice of u. The value that seems to be better in applications is u = ∆ 2:      1 −i(k+ 2 )∆x 1 +∞  φ ∆ e k + ( 2 1   P (X < x) = − 2 π k + 12 k=0      ∞ ( 2πn 2πn n − (−1) P X < x − −P X >x+ . ∆ ∆ n=1 Davies [7] proposes to approximate P (X < x) as:     −i(k+ 1 )∆x  1 K 2 1 ( φ k+ 2 ∆ e   P (X < x) ∼ − . 2 π k + 12 k=0 Using the characteristic formula for a finite sum of chi-squared random vari2N −1 ables, i.e. φ (u) = j=1 (1 − 2iλj u) 2 , this formula becomes: '   1N arctan[2(k+ 12 )∆λj ]  1 ∆x sin − k + K j=1 2 2 1 ( P (X < x) ∼ − 8 91 .  2N    2 1 2 2 2 4 k=0 π k + 1 · ∆ λj j=1 1 + 4 k + 2 2

The Asymptotic Distribution of Quadratic Discrepancies

71

The use of this formula induces two sources of error:    1∞ n  − P X > x + 2πn : • an integration error − n=1 (−1) P X < x − 2πn ∆ ∆ this error can be made small choosing adequately ∆. Davies [7] (p. 416)      2π is proposes to choose ∆ so that max P X < x − 2π ∆ ,P X > x + ∆ less than half of the maximum allowable error.   −i k+ 1 ∆x

φ((k+ 12 )∆)e ( 2 ) 1+∞ • a truncation error − k=K+1 : the way in which π (k+ 12 ) this error term can be made small choosing adequately the parameter K is discussed in [29] (p. 423) and more thoroughly in [8] (p. 324) where several upper bounds on this error are given.

5 Results of the Algorithm The algorithm previously exposed has been implemented in an R script ( [39]). The computation of the eigenvalues is performed through the Nystr¨ om method using a Halton sequence. On the other hand, the approximation of the cdf uses a routine recently written by Robert B. Davies in C (indeed, a new version, available at the website http://www.robertnz.net/ftp/qf.tar.gz or qf.zip, of the 1980 program that was originally written in Algol). In order to show the degree of approximation of the procedure, we consider the Cram´er-von Mises statistic as described in Sect. 3. Even if much more efficient methods exist in this case (see [6]), we compute the distribution using the following methods: • the Davies algorithm using 10, 100 and 1, 000 eigenvalues computed using the Nystr¨ om method using a Halton sequence with prime number (2) (est. in the Table); • the Davies algorithm using 10, 100, 1, 000 and 10, 000 real eigenvalues (real in the Table). In Table 1, we compare several quantiles of these distributions with the values given in [1] (p. 203; AD in the Table). Figures 3, 4 and 5 show the cdf of the three discrepancies computed using 1, 000 points from the Halton sequences with prime numbers (2), (2, 3), (2, 3, 5), (2, 3, 5, 7) and (2, 3, 5, 11). This shows that the behavior of the three discrepancies with d is different.

Acknowledgement We would like to thank a referee who helped in improving our paper. Specials thanks go to the editor Prof. H. Niederreiter for his comments and his patience.

72

C. Choirat and R. Seri

Table 1. Quantiles of the Cram´er-von Mises statistic from [1] (AD) and computed through the algorithm described in the text.

AD est. 10 est. 100 est. 1,000 real 10 real 100 real 1,000 real 10,000

0.025 0.03035 0.02240 0.02997 0.03031 0.02091 0.02933 0.03024 0.03034 0.8 0.24124 0.23469 0.24010 0.24116 0.23158 0.24025 0.24116 0.24125

0.05 0.03656 0.02866 0.03615 0.03650 0.02703 0.03552 0.03644 0.03653 0.85 0.28406 0.27848 0.28292 0.28394 0.27443 0.28304 0.28394 0.28403

0.1 0.04601 0.03814 0.04555 0.04594 0.03641 0.04499 0.04589 0.04598 0.9 0.34730 0.34329 0.34626 0.34720 0.33764 0.34630 0.34720 0.34729

0.15 0.05426 0.04640 0.05375 0.05419 0.04464 0.05323 0.05415 0.05424 0.95 0.46136 0.46024 0.46048 0.46127 0.45170 0.46035 0.46126 0.46135

0.2 0.06222 0.05436 0.06167 0.06215 0.05258 0.06121 0.06211 0.06220 0.975 0.58061 0.58241 0.57986 0.58050 0.57094 0.57959 0.58049 0.58059

0.25 0.07026 0.06238 0.06968 0.07021 0.06061 0.06927 0.07018 0.07027 0.99 0.74346 0.74920 0.74293 0.74339 0.73378 0.74247 0.74337 0.74346

0.5 0.11888 0.11090 0.11797 0.11879 0.10921 0.11787 0.11877 0.11886 0.999 1.16204 1.18368 1.16776 1.16779 1.15819 1.16687 1.16778 1.16787

0.4

cdf

0.6

0.8

1.0

AD est. 10 est. 100 est. 1,000 real 10 real 100 real 1,000 real 10,000

0.01 0.02480 0.01681 0.02445 0.02477 0.01551 0.02379 0.02470 0.02479 0.75 0.20939 0.20223 0.20823 0.20929 0.19972 0.20838 0.20929 0.20938

0.0

0.2

d=1 d=2 d=3 d=4 d=5 0

1

2

3

4

x Fig. 3. cdf of the asymptotic distribution for the centered discrepancy with varying d.

73

0.4

cdf

0.6

0.8

1.0

The Asymptotic Distribution of Quadratic Discrepancies

0.0

0.2

d=1 d=2 d=3 d=4 d=5

0

2

4

6

8

10

x

0.4

cdf

0.6

0.8

1.0

Fig. 4. cdf of the asymptotic distribution for the star discrepancy with varying d.

0.0

0.2

d=1 d=2 d=3 d=4 d=5

0

10

20

30

40

50

x Fig. 5. cdf of the asymptotic distribution for the symmetric discrepancy with varying d.

References 1. T.W. Anderson and D.A. Darling. Asymptotic theory of certain “goodness of fit” criteria based on stochastic processes. Ann. Math. Statistics, 23:193–212, 1952. 2. R.H. Brown. The distribution function of positive definite quadratic forms in normal random variables. SIAM J. Sci. Statist. Comput., 7(2):689–695, 1986. 3. C. Choirat and R. Seri. Statistical properties of generalized discrepancies. Working paper, 2004.

74

C. Choirat and R. Seri

4. C. Choirat and R. Seri. Statistical properties of quadratic discrepancies. Working paper, 2004. 5. R.R. Coveyou. Review MR0351035 of MathSciNet, 1975. 6. S. Cs¨ org˝ o and J.J. Faraway. The exact and asymptotic distributions of Cram´ervon Mises statistics. J. Roy. Statist. Soc. Ser. B, 58(1):221–234, 1996. 7. R.B. Davies. Numerical inversion of a characteristic function. Biometrika, 60:415–417, 1973. 8. R.B. Davies. Statistical algorithms: Algorithm AS 155: The distribution of a linear combination of χ2 random variables. Applied Statistics, 29(3):323–333, 1980. 9. K. Frank and S. Heinrich. Computing discrepancies of Smolyak quadrature rules. J. Complexity, 12(4):287–314, 1996. Special issue for the Foundations of Computational Mathematics Conference (Rio de Janeiro, 1997). 10. J. Gil-Pelaez. Note on the inversion theorem. Biometrika, 38:481–482, 1951. 11. P.J. Grabner, P. Liardet, and R.F. Tichy. Average case analysis of numerical integration. In Advances in Multivariate Approximation (Witten-Bommerholz, 1998), volume 107 of Math. Res., pages 185–200. Wiley-VCH, Berlin, 1999. 12. P.J. Grabner, O. Strauch, and R.F. Tichy. Lp -discrepancy and statistical independence of sequences. Czechoslovak Math. J., 49(124)(1):97–110, 1999. 13. V.S. Grozdanov and S.S. Stoilova. The b-adic diaphony. Rend. Mat. Appl. (7), 22:203–221 (2003), 2002. 14. P. Hellekalek and H. Niederreiter. The weighted spectral test: diaphony. ACM Trans. Model. Comput. Simul., 8(1):43–60, 1998. 15. P. Hellekalek. Dyadic diaphony. Acta Arith., 80(2):187–196, 1997. 16. P. Hellekalek. On correlation analysis of pseudorandom numbers. In Monte Carlo and Quasi-Monte Carlo Methods 1996 (Salzburg), volume 127 of Lecture Notes in Statist., pages 251–265. Springer, New York, 1998. 17. P. Hellekalek. On the assessment of random and quasi-random point sets. In Random and Quasi-Random Point Sets, volume 138 of Lecture Notes in Statist., pages 49–108. Springer, New York, 1998. 18. F.J. Hickernell. Erratum: “Quadrature error bounds with applications to lattice rules” [SIAM J. Numer. Anal. 33 (1996), no. 5, 1995–2016;]. SIAM J. Numer. Anal., 34(2):853–866, 1997. 19. F.J. Hickernell. Quadrature error bounds with applications to lattice rules. SIAM J. Numer. Anal., 33(5):1995–2016, 1996. 20. F.J. Hickernell. A generalized discrepancy and quadrature error bound. Math. Comp., 67(221):299–322, 1998. 21. F.J. Hickernell. Lattice rules: how well do they measure up? In Random and Quasi-Random Point Sets, volume 138 of Lecture Notes in Statist., pages 109– 166. Springer, New York, 1998. 22. F.J. Hickernell. Goodness-of-fit statistics, discrepancies and robust designs. Statist. Probab. Lett., 44(1):73–78, 1999. 23. F.J. Hickernell. The mean square discrepancy of randomized nets. ACM Trans. Model. Comput. Simul., 6(4):274–296, 1996. 24. F.J. Hickernell. What affects the accuracy of quasi-Monte Carlo quadrature? In Monte Carlo and Quasi-Monte Carlo Methods 1998 (Claremont, CA), pages 16–55. Springer, Berlin, 2000. 25. J. Hoogland, F. James, and R. Kleiss. Quasi-Monte Carlo, discrepancies and error estimates. In Monte Carlo and Quasi-Monte Carlo Methods 1996 (Salzburg),

The Asymptotic Distribution of Quadratic Discrepancies

26. 27.

28.

29. 30.

31. 32.

33. 34. 35. 36. 37.

38. 39.

40.

41.

42. 43. 44.

75

volume 127 of Lecture Notes in Statist., pages 266–276. Springer, New York, 1998. J. Hoogland and R. Kleiss. Discrepancy-based error estimates for quasi-monte carlo. I: General formalism. Comput. Phys. Comm., 98(1–2):111–127, 1996. J. Hoogland and R. Kleiss. Discrepancy-based error estimates for quasi-monte carlo. II: Results in one dimension. Comput. Phys. Comm., 98(1–2):128–136, 1996. J. Hoogland and R. Kleiss. Discrepancy-based error estimates for quasi-monte carlo. III: Error distribution and central limits. Comput. Phys. Comm., 101(1– 2):21–30, 1997. J.P. Imhof. Computing the distribution of quadratic forms in normal variables. Biometrika, 48:419–426, 1961. F. James, J. Hoogland, and R. Kleiss. Multidimensional sampling for simulation and integration: Measures, discrepancies and quasi-random numbers. Comput. Phys. Comm., 99(2–3):180–220, 1997. V. Koltchinskii and E. Gin´e. Random matrix approximation of spectra of integral operators. Bernoulli, 6(1):113–167, 2000. P. L’Ecuyer and P. Hellekalek. Random number generators: selection criteria and testing. In Random and Quasi-Random Point Sets, volume 138 of Lecture Notes in Statist., pages 223–265. Springer, New York, 1998. H. Leeb. Asymptotic properties of the spectral test, diaphony, and related quantities. Math. Comp., 71(237):297–309, 2002. V.F. Lev. On two versions of L2 -discrepancy and geometrical interpretation of diaphony. Acta Math. Hungar., 69(4):281–300, 1995. J.-J. Liang, K.-T. Fang, F.J. Hickernell, and R. Li. Testing multivariate uniformity and its applications. Math. Comp., 70(233):337–355, 2001. W.J. Morokoff and R.E. Caflisch. Quasi-random sequences and their discrepancies. SIAM J. Sci. Comput., 15(6):1251–1279, 1994. G. Pag`es and Y.-J. Xiao. Sequences with low discrepancy and pseudo-random numbers: theoretical results and numerical tests. J. Statist. Comput. Simulation, 56(2):163–188, 1997. S.H. Paskov. Average case complexity of multivariate integration for smooth functions. J. Complexity, 9(2):291–312, 1993. R Development Core Team. R: A Language and Environment for Statistical Computing. R Foundation for Statistical Computing, Vienna, Austria, 2004. ISBN 3-900051-00-3. S.O. Rice. Distribution of quadratic forms in normal random variables— evaluation by numerical integration. SIAM J. Sci. Statist. Comput., 1(4):438– 448, 1980. J. Sheil and I. O’Muircheartaigh. Statistical algorithms: Algorithm AS 106: The distribution of non-negative quadratic forms in normal variables. Applied Statistics, 26(1):92–98, 1977. O. Strauch. L2 discrepancy. Math. Slovaca, 44(5):601–632, 1994. Number theory (Raˇckova dolina, 1993). A. van Hameren, R. Kleiss, and J. Hoogland. Gaussian limits for discrepancies. I. Asymptotic results. Comput. Phys. Comm., 107(1-3):1–20, 1997. T.T. Warnock. Computational investigations of low-discrepancy point sets. In Applications of Number Theory to Numerical Analysis (Proc. Sympos., Univ. Montreal, Montreal, Que., 1971), pages 319–343. Academic Press, New York, 1972.

76

C. Choirat and R. Seri

45. G.S. Watson. Goodness-of-fit tests on a circle. Biometrika, 48:109–114, 1961. 46. G.S. Watson. Another test for the uniformity of a circular distribution. Biometrika, 54:675–677, 1967. ¨ 47. P. Zinterhof. Uber einige Absch¨ atzungen bei der Approximation von Funktionen ¨ mit Gleichverteilungsmethoden. Osterreich. Akad. Wiss. Math.-Naturwiss. Kl. S.-B. II, 185(1-3):121–132, 1976.

Weighted Star Discrepancy of Digital Nets in Prime Bases Josef Dick1 , Harald Niederreiter2 and Friedrich Pillichshammer3

1

2

3

School of Mathematics, University of New South Wales, Sydney 2052, Australia [email protected] Department of Mathematics, National University of Singapore, 2 Science Drive 2, Singapore 117543, Republic of Singapore [email protected] Institut f¨ ur Finanzmathematik, Universit¨ at Linz, Altenbergerstrasse 69, A-4040 Linz, Austria [email protected]

Summary. We study the weighted star discrepancy of digital nets and sequences. Product weights and finite-order weights are considered and we prove tractability bounds for Niederreiter and Faure-Niederreiter sequences. Further we prove an existence result for digital nets achieving a strong tractability error bound by calculating the average over all generator matrices.

1 Introduction For numerical integration of functions over the s-dimensional unit cube [0, 1)s one needs point sets which are well distributed. What we mean by well distributed can be specified in several ways by various measures. Commonly such measures are based on the discrepancy function ∆ which, for a point set x0 , . . . , xN −1 in the s-dimensional unit cube [0, 1)s , is defined by 2s AN ( i=1 [0, αi )) − α1 · · · αs ∆(α1 , . . . , αs ) := N for 0 < α1 , . . . , αs ≤ 1. Here AN (E) denotes the number of indices n, 0 ≤ n ≤ N − 1, such that xn is contained in the set E. By taking the sup norm of this function, we obtain the star discrepancy ∗ = DN 



sup |∆(z)|

z∈(0,1]s

The first author is supported by the Australian Research Council under its Center of Excellence Program. The third author is supported by the Austrian Science Fund (FWF), Project S8305 and Project P17022-N12.

78

Josef Dick et al.

which appears in the Koksma-Hlawka inequality  N −1 1 ( ∗ f (x)dx − f (xk ) ≤ DN V (f ), [0,1)s N

(1)

k=0

where V (f ) denotes the variation of f in the sense of Hardy and Krause. It has been shown that there exist low-discrepancy sequences, i.e., sequences for which the first N ≥ 2 points satisfy ∗ ≤ Cs DN

(log N )s , N

(2)

with a constant Cs only dependent on the dimension s. For values of N of practical interest, bounds of the form (2), and thus low-discrepancy sequences, seem to be useful only for dimensions up to about 15. In practice, though, lowdiscrepancy sequences have been used successfully in much higher dimensions. In order to understand why low-discrepancy sequences can still work in higher dimensions, Sloan and Wo´zniakowski [15] (see also [5]) introduced a weighted discrepancy. The idea is that in many applications some projections are more important than others and that this should also be reflected in the quality measure of the point set. Before we give the definition of the weighted star discrepancy, let us introduce some notation. Let Is = {1, 2, . . . , s} denote the set of coordinate indices. For u ⊆ Is , u = ∅, let γu,s be a nonnegative real number (the weight), |u| the cardinality of u, and for a vector z ∈ [0, 1]s let z u denote the vector from [0, 1]|u| containing the components of z whose indices are in u. By (z u , 1) we mean the vector z from [0, 1]s with all components whose indices are not in u replaced by 1. Definition 1. For a point set x0 , . . . , xN −1 in [0, 1)s and given weights γ = ∗ is given by {γu,s : u ⊆ Is , u = ∅}, the weighted star discrepancy DN,γ ∗ DN,γ =

sup max γu,s |∆(z u , 1)|.

z∈(0,1]s

u⊆Is u=∅

This is a generalization of the classical star discrepancy which is recovered if we choose γIs ,s = 1 and γu,s = 0 for all ∅ = u ⊂ Is . To avoid a trivial case, we will always assume that not all weights are 0. Furthermore the error bound (1) can also be generalized by replacing the star discrepancy with the weighted star discrepancy and the variation by a weighted version of the variation (see [15] for more details). Now consider for example the case where γu,s = 0 for all |u| ≥ 3 and γu,s > 0 for |u| = 1, 2. In this case it is reasonable to guess that there are point sets which achieve ∗ DN,γ ≤ Cγ,s

(log N )2 , N

Weighted Star Discrepancy of Digital Nets in Prime Bases

79

where the constant depends only on the weights γ and the dimension s (such an example was also discussed in [5]). For such a choice of weights we can obtain useful error bounds also for a large dimension s (say several hundred), far beyond the previous suggestion that low-discrepancy sequences work only for dimensions up to about 15. It is the aim of the paper to show that known point sets and sequences can indeed work well (meaning that we obtain a convergence rate of almost O(N −1 )) in high dimensions under certain conditions on the weights. The results established here are of course for weights of a more general form than the special case considered above. Specifically, we will deal with two important kinds of weights, namely product weights and finite-order weights. 2 • Product weights are weights of the form γu,s = j∈u γj,s , for u ⊆ Is , u = ∅, where γj,s is the weight associated to the j-th component. Sometimes the weights γj,s have no dependence on s, i.e., γj,s = γj . See [5, 15]. • Finite-order weights of order k, k ∈ N fixed, are weights with γu,s = 0 for all u ⊆ Is with |u| > k. See [6, 14]. Similar results for different cases have previously been established in papers such as [2, 14, 17–19], where either different point sets, discrepancies or settings have been considered. In those papers, tractability and strong tractability have been investigated. Tractability can be shown by proving an upper bound on the discrepancy depending at most polynomially on the dimension, whereas strong tractability can be shown by proving an upper bound on the discrepancy which is independent of the dimension (for a formal definition of (strong) tractability see [15] We give a brief outline of the paper. In the following section we will introduce digital (t, m, s)-nets in base p. In Section 3 we introduce the tools needed to obtain bounds on the star discrepancy of digital nets. Section 4 deals with the weighted star discrepancy of digital nets and in Section 5 we obtain improved results for the special case where p = 2. Bounds on the weighted star discrepancy of Niederreiter and Faure-Niederreiter sequences are obtained in Section 6. The last section, Section 7, is concerned with existence results for digital nets satisfying a certain bound on the weighted star discrepancy.

2 Digital (t, m, s)-Nets in Base p A detailed theory of (t, m, s)-nets was developed in Niederreiter [9] (see also [11, Chapter 4] and [12, Chapter 8] for surveys of this theory). We refer to [11] and [12] for the definition of (t, m, s)-nets. The crucial fact is that (t, m, s)nets in a base p provide sets of pm points in the s-dimensional unit cube [0, 1)s which are extremely well distributed if the quality parameter t is ‘small’. From now on let p denote a prime number. We recall the following definition. Definition 2. Let p ≥ 2 be a given prime number and let Zp := {0, 1, . . . , p − 1} be the finite field with p elements. Further let Ci , i = 1, . . . , s, be s given

80

Josef Dick et al.

m × m matrices over Zp . Now we construct pm points in [0, 1)s : represent n ∈ Z, 0 ≤ n < pm , in base p, n = n0 + n1 p + · · · + nm−1 pm−1 , and identify n with the vector n = (n0 , . . . , nm−1 )T ∈ Zm p , where T means the transpose of the vector. For 1 ≤ i ≤ s multiply the matrix Ci by n modulo p, Ci n =: (yi,1 , . . . , yi,m )T ∈ Zm p , and set xn,i :=

yi,m yi,1 + ··· + m . p p

If for some integer t with 0 ≤ t ≤ m the point set consisting of the points xn = (xn,1 , . . . , xn,s ) for 0 ≤ n < pm is a (t, m, s)-net in base p, then it is called a digital (t, m, s)-net in base p (or over Zp ), or shortly a digital net (over Zp ).

3 The Star Discrepancy of Digital Nets In this section we will introduce the tools needed for analyzing the star discrepancy of digital nets. The quantity Rp (C1 , . . . , Cs ), defined in the following, will be useful to obtain bounds on the star discrepancy (compare with [11, Lemma 4.32]) as it captures the essential part of the discrepancy. In detail, we define s ( Rp (C1 , . . . , Cs ) := rp (ki ), k∈D i=1

where the set D is the dual net restricted to {0, 1, . . . , pm − 1}s \ {0}, that is, D = {k ∈ {0, 1, . . . , pm − 1}s \ {0} : C1T k1 + · · · + CsT ks = 0},

(3)

1m−1 (i) (i) (i) k = (k1 , . . . , ks ), ki = (κ0 , . . . , κm−1 )T if ki = j=0 κj pj , 1 ≤ i ≤ s, and

1 if k = 0, 1 rp (k) = if k = κ0 + κ1 p + · · · + κg pg , κg = 0. pg+1 sin( π p κg ) ∗ of the digital net over Zp generated Theorem 1. For the star discrepancy DN by the m × m matrices C1 , . . . , Cs we have, with N = pm ,  s 1 s ∗ DN ≤ 1 − 1 − + Rp (C1 , . . . , Cs ). + Rp (C1 , . . . , Cs ) ≤ (4) N N

Proof. The result follows from the proof of [11, Lemma 4.32] by using [7, Theorem 1] instead of [11, Theorem 3.12].  We will obtain bounds on the star discrepancy by establishing bounds on Rp and using Theorem 1. It proves to be convenient to represent Rp (C1 , . . . , Cs ) in terms of Walsh functions, which are introduced in the following.

Weighted Star Discrepancy of Digital Nets in Prime Bases

81

For an integer b ≥ 2 let ωb = e2πi/b ∈ C. For a nonnegative integer k with base b representation k = κ0 + κ1 b + · · · + κr br , the function b walk : R −→ C, periodic with period 1, is defined by b walk (x)

κ x1 +···+κr xr+1

= ωb 0

,

where x ∈ [0, 1) has base b representation x = x1 /b+x2 /b2 +. . . (unique in the sense that infinitely many of the xj must be different from b − 1). Information on Walsh functions can be found in [1, 13, 16]. In the following we will always consider Walsh functions in base p, and hence we will often write walk instead of p walk . Subsequently we will make use of the following result (see e.g. [3]): for (1) (s) the digital net {x0 , . . . , xpm −1 } with xn = (xn , . . . , xn ), generated by the m × m matrices C1 , . . . , Cs over Zp , we have  pm −1 s 1 (  1 if C1T k1 + · · · + CsT ks = 0, (i) wal (x ) = (5) ki p n m 0 otherwise. p n=0 i=1

In the following lemma we show how Rp can be represented by Walsh functions. (1)

(s)

Lemma 1. Let x0 , . . . , xpm −1 with xn = (xn , . . . , xn ) for 0 ≤ n < pm be a digital net over Zp generated by C1 , . . . , Cs . Then we have   m p( −1 pm −1 s 1 (  (i) Rp (C1 , . . . , Cs ) = −1 + m rp (k)walk (xn ) . 1+ p n=0 i=1 k=1

Proof. We have

  m p( −1 pm −1 s 1 (  (i) −1 + m rp (k)walk (xn ) 1+ p n=0 i=1 k=1

= −1 +

1 pm

1 = −1 + m p

m m −1  −1 p( s p(

n=0 i=1 k=0 m −1 p(

k1 ,...,ks =0

=

m p( −1

k1 ,...,ks =0 (k1 ,...,ks )=(0,...,0)

=

s (

m p( −1

s 

rp (ki )walki (xn(i) )

n=0 k1 ,...,ks =0 i=1

m p( −1

= −1 +

rp (k)walk (xn(i) )

p −1 s 1 (  rp (ki ) m walki (xn(i) ) p n=0 i=1 i=1 s 

m

p −1 s 1 (  rp (ki ) m walki (xn(i) ) p n=0 i=1 i=1 s 

m

rp (ki ) = Rp (C1 , . . . , Cs ),

k∈D i=1

where we used formula (5).



82

Josef Dick et al.

4 Weighted Star Discrepancy of Digital Nets ∗ It follows easily from Definition 1 that for the weighted star discrepancy DN,γ s of a point set x0 , . . . , xN −1 in [0, 1) we have ∗ DN,γ =

sup max γu,s |∆(z u , 1)| ≤ max γu,s

z∈(0,1]s

u⊆Is u=∅

u⊆Is u=∅

sup z u ∈(0,1]|u|

|∆(z u , 1)|

∗ = max γu,s DN (u), u⊆Is u=∅

∗ where DN (u) denotes the star discrepancy of the |u|-dimensional projection of the point set x0 , . . . , xN −1 to the coordinates given by u. If we consider a digital net over Zp , generated by C1 , . . . , Cs , then for u ⊆ Is , u = ∅, from (4) we obtain

 |u| 1 ∗ DN (u) ≤ 1 − 1 − + Rp ((Ci )i∈u ) N and for u = {u1 , . . . , ud }, Rp ((Ci )i∈u ) is given by m 2( −1

d 

k1 ,...,kd =0 (k1 ,...,kd )=(0,...,0) T T k =0 Cu k1 +···+Cu 1 d d

i=1

Rp ((Ci )i∈u ) =

rp (ki ).

This leads to the following result. ∗ Theorem 2. For the weighted star discrepancy DN,γ of a digital net over Zp generated by the m × m matrices C1 , . . . , Cs we have   |u|  1 ∗ p,γ (C1 , . . . , Cs ) +R DN,γ ≤ max γu,s 1 − 1 − u⊆Is N u=∅

≤

1 p,γ (C1 , . . . , Cs ), max(|u|γu,s ) + R s N u⊆I u=∅

where N = pm and p,γ (C1 , . . . , Cs ) := max γu,s Rp ((Ci )i∈u ). R u⊆Is u=∅

Hence we are concerned with the quality of projections of a digital net. Thus, it is useful to define a quality parameter tu , u ⊆ Is with u = ∅, of the projection of a digital net to the coordinates given by the set u, that is, for all u ⊆ Is with u = ∅ the projection of the digital net P to the coordinates given by u is a (tu , m, |u|)-net. The following definition and theorem prove to be useful.

Weighted Star Discrepancy of Digital Nets in Prime Bases

83

(i)

Definition 3. For 1 ≤ i ≤ s let cj ∈ Zm p , 1 ≤ j ≤ m, be the row vectors of the matrix Ci . For u ⊆ Is , u = ∅, let ρu (C1 , . . . , Cs ) be the largest integer d (i) such 1 that any system {cj : 1 ≤ j ≤ di , im∈ u} with 0 ≤ di ≤ m for i ∈ u and i∈u di = d is linearly independent in Zp . (Here the empty system is viewed as linearly independent.) be the generating Theorem 3. Let p be a prime and let C1 , . . . , Cs ∈ Zm×m p matrices of a digital net P . Then P is a digital (t, m, s)-net over Zp and the quality parameter tu of the projection of the net to the coordinates given by u is tu = m − ρu (C1 , . . . , Cs ). Proof. The result follows from [11, Theorem 4.28].



p,γ with the help of the quantities ρu . Now we can give bounds on R Theorem 4. For s ≥ 2 and any prime p, we have γu,s p,γ (C1 , . . . , Cs ) ≤R pρu (C1 ,...,Cs )+1     ρu (C1 , . . . , Cs ) + |u| k(p)|u| 1 |u| ≤ max γu,s 1 − , (m + 1) − u⊆Is p |u| pρu (C1 ,...,Cs ) u=∅

max u⊆Is u=∅

where k(2) = 1 and k(p) = csc(π/p) if p > 2. p,γ (C1 , . . . , Cs ) together with Proof. The result follows from the definition of R the proof of [11, Theorem 4.34].  Corollary 1. Let P be a digital (t, m, s)-net over Zp and for u ⊆ Is , u = ∅, let tu denote the quality parameter of the projection of P to the coordinates given by u. Then we have ∗ DN,γ ≤

1 max(|u|γu,s ) s N u⊆I u=∅      m − tu + |u| 1 1 + max γu,s 1 − k(p)|u| (m + 1)|u| − ptu , s N u⊆I p |u| u=∅

where N = pm and k(p) is defined in Theorem 4. Proof. The corollary follows from Theorem 2, Theorem 3, and Theorem 4. 

5 An Alternative Bound in the Binary Case For p = 2 we present an alternative bound on the weighted star discrepancy of digital nets over Zp (see Corollary 2 below) which is often better than the corresponding bound in Corollary 1.

84

Josef Dick et al.

Lemma 2. Let m ≥ 2 and s ≥ 2 and let C1 , . . . , Cs be the generating matrices of a digital (t, m, s)-net over Z2 . Then we have m 2( −1

s 

k1 ,...,ks =1 T k +···+C T k =0 C1 1 s s

i=1

 r2 (ki ) ≤ 2t−m ms

1 m+1 + 2s m2

 .

Proof. We have m 2( −1

s 

k1 ,...,ks =1 T k +···+C T k =0 C1 1 s s

i=1

Σ :=

m−1 (

r2 (ki ) =

s 

v1 ,...,vs =0 i=1

1

+1 −1 2v1(

2vi +1

k1 =2v1



···

+1 2vs( −1

1.

ks =2vs



C1T k1 +···+CsT ks =0

From the proof of [4, Lemma 7] we find that ⎧ 0 if v1 + · · · + vs ≤ m − t − s, ⎪ v1 +1 vs +1 ⎪ 2 (−1 2 (−1 ⎨ 1 if m − t − s + 1 ≤ v1 + · · · + vs ··· 1≤ ≤ m − t, ⎪ ⎪ k1 =2v1 ks =2vs ⎩ v1 +···+vs −m+t if v1 + · · · + vs > m − t. 2 

 C1T k1 +···+CsT ks =0

Therefore we obtain 1 Σ≤ s 2

m−1 (

1

v1 ,...,vs =0 m−t−s+1≤v1 +···+vs ≤m−t

2v1 +···+vs

1 + s 2

m−1 ( v1 ,...,vs =0 v1 +···+vs >m−t

2t =: Σ1 + Σ2 . 2m

Trivially we have Σ2 ≤ 2t−s−m ms . Further we have 1 Σ1 ≤ s 2

m−t ( l=max(0,m−t−s+1)

 1 m−t+s−1 ≤ s s−1 2

  l+s−1 1 s−1 2l



m−t ( l=max(0,m−t−s+1)

  1 t−m m − t + s − 1 ≤2 . s−1 2l

We obtain Σ = Σ1 + Σ2 ≤ 2t−m 

Therefore Σ ≤ 2t−m ms and using m ≥ 2 and s ≥ 2,

  ms t−m m − t + s − 1 + 2 . s−1 2s

  1 1 m+s−1 + s−1 2s ms

Weighted Star Discrepancy of Digital Nets in Prime Bases

85

   s−1  s−1 1 1 1 1 m+s−1 1  m+i 1  1 1 = + + + + = s−1 2s ms 2s ms i=1 i 2s m i=1 i m   1 1 1 1+ . ≤ s+ 2 m m 

This yields the desired result.

Remark 1. It is clear that for large m and s the 2 bound in Lemma 2 can be s−1 improved by using sharper bounds for the product i=1 (i−1 +m−1 ) appearing in the last part of the proof. Theorem 5. Let m ≥ 2, N = 2m , s ≥ 2 and C1 , . . . , Cs be the generating matrices of a digital (t, m, s)-net over Z2 . For u ⊆ Is , u = ∅, let tu denote the quality parameter of the projection of the net to the coordinates given by u. Then we have 2,γ (C1 , . . . , Cs ) R   |u| m m+1 1 |u| max γu,s 2tu +1 −1+ ((m + 1) − 1) . ≤ s N u⊆I 2 m2 u=∅ Proof. We have 2,γ (C1 , . . . , Cs ) = R

max

u⊆Is u=∅ u={u1 ,...,ud }

d 

k1 ,...,kd =0 (k1 ,...,kd )=(0,...,0) T T k =0 Cu k1 +···+Cu 1 d d

i=1

γu,s

r2 (ki )

(

m 2( −1

e 

w⊆u w=∅ w={w1 ,...,we }

k1 ,...,ke =1 T k +···+C T ke =0 Cw we 1 1

i=1

= max γu,s u⊆Is u=∅

m −1 2(

r2 (ki ).

Now we use Lemma 2 and obtain 2,γ (C1 , . . . , Cs ) ≤ max γu,s R u⊆Is u=∅

(

tw −m

2

m

w⊆u w=∅

|w|



1 2|w|

m+1 + m2

 .

For ∅ = w ⊆ u we have tw ≤ tu and hence ( w⊆u w=∅

tw

2 m

|w|



1 2|w|

m+1 + m2

= 2tu The result follows.



 ≤2

tu

   |u|  ( |u| m d d=1

d

2

m+1 d + m m2



 |u| m m+1 |u| +1 −1+ ((m + 1) − 1) . 2 m2 

86

Josef Dick et al.

Hence by using Theorems 2 and 5 we obtain the following improved upper bound on the weighted star discrepancy of a digital net over Z2 . Corollary 2. Let m ≥ 2, N = 2m and s ≥ 2. Then for the weighted star discrepancy of a digital (t, m, s)-net over Z2 we have ∗ ≤ DN,γ

1 max(|u|γu,s ) s N u⊆I u=∅ 1 + max γu,s 2tu s N u⊆I u=∅



 |u| m m+1 |u| +1 −1+ ((m + 1) − 1) . 2 m2

6 Weighted Star Discrepancy of Niederreiter and Faure-Niederreiter Sequences Weighted types of discrepancies of several types of sequences in a worstcase or randomized setting have previously been studied in papers such as [2, 14, 17–19]. Here we concentrate on the weighted star discrepancy of socalled Niederreiter and Faure-Niederreiter sequences using various types of weights as mentioned in the introduction. Most previously obtained results concentrate on the L2 discrepancy, but a result directly comparable to a result in this paper (see Theorem 6 below) was obtained in [18, Lemma 3]. Further note that the results shown in this section can be used to obtain bounds on other discrepancies, as it was previously done in [14, 18]. In this section we apply the results from the previous section to give estimates for the weighted star discrepancy of Niederreiter sequences in a prime base p. The first pm points of such sequences are a digital (t, m, s)-net over Zp , where the generating matrices are constructed in a special way and the quality parameter t is independent of m. For the general definition of Niederreiter sequences we refer to [10], [11, Section 4.5]. Here it suffices to note that the construction of these sequences depends on the choice of s distinct monic irreducible polynomials q1 , . . . , qs over Zp . Then we get a (t, s)-sequence in base p with s ( (deg(qi ) − 1). t= i=1

Let u ⊆ Is , u = ∅, then we have that the projection of the sequence to the coordinates given by u is a (tu , |u|)-sequence in base p with ( (deg(qj ) − 1). (6) tu = j∈u

As in [11] we consider two different choices for the polynomials q1 , . . . , qs . First we order the set of all monic irreducible polynomials over Zp according to their degree such that deg(q1 ) ≤ deg(q2 ) ≤ . . .. Let logp denote the logarithm in base p. The following lemma was proved in [18, Lemma 2].

Weighted Star Discrepancy of Digital Nets in Prime Bases

87

Lemma 3. The degree of the j-th monic irreducible polynomial qj over the finite field Zp can be bounded by deg(qj ) ≤ logp j + logp logp (j + p) + 2

for j = 1, 2, . . . .

Thus, using (6), the value tu can be bounded by ( ( (deg(qj ) − 1) ≤ (logp j + logp logp (j + p) + 1). tu = j∈u

(7)

j∈u

This yields the following discrepancy bound. Theorem 6. For the weighted star discrepancy of the first N = pm points of a Niederreiter (t, s)-sequence in prime base p we have ∗ ≤ DN,γ

 jp2 logp (j + p) logp (pN ) 1 1 max(|u|γu,s ) + max γu,s . s s N u⊆I N u⊆I 2 u=∅ u=∅ j∈u

Proof. From Corollary 1 we obtain ∗ DN,γ ≤

1 1 max(|u|γu,s ) + m max γu,s (m + 1)|u| k(p)|u| ptu . u⊆Is s pm u⊆I p u=∅ u=∅

From k(p) ≤ p/2 and (7) we obtain ∗ DN,γ ≤

1 max(|u|γu,s ) s pm u⊆I u=∅ +

 p |u|  1 max γu,s (m + 1)|u| plogp j+logp logp (j+p)+1 m s p u⊆I 2 u=∅ j∈u

=

 jp2 logp (j + p) 1 1 |u| max (|u|γ ) + max γ (m + 1) u,s u,s s s pm u⊆I pm u⊆I 2 u=∅ u=∅ j∈u

=

 jp2 logp (j + p) logp (pN ) 1 1 max(|u|γu,s ) + max γu,s , s s N u⊆I N u⊆I 2 u=∅ u=∅ j∈u

which is the desired result.



Remark 2. We remark that for p = 2 the bound in Theorem 6 can be improved if we use Corollary 2 instead of Corollary 1 in the proof. Note that different types of bounds on the weighted star discrepancy of the Niederreiter sequence have previously been shown in [14, 18]. The bound in Theorem 6 can be used to show [18, Lemma 3]. In this case our bound improves the constant explicitly stated in the proof of [18, Lemma 3]. We obtain the following results for the product weight case and the finiteorder weight case.

88

Josef Dick et al.

Corollary 3. Assume that for u ⊆ Is , u = ∅, γu,s is given by γu,s = with nonnegative reals γj,s . Then we have:

2 j∈u

γj,s

1. For the weighted star discrepancy of the first N = pm points of a Niederreiter (t, s)-sequence in prime base p we have ∗ ≤ DN,γ

 jp2 logp (j + p) logp (pN ) 1 1 max(|u|γu,s ) + max . γj,s s s N u⊆I N u⊆I 2 u=∅ u=∅ j∈u

2. If Γ := sup max s∈N

u⊆Is u=∅



γj,s

j∈u

jp2 logp (j + p) < ∞, 2

(8)

then for the weighted star discrepancy of the first N = pm points of a Niederreiter (t, s)-sequence in prime base p we have ∗ ≤ 2Γ DN,γ

(m + 1)s . N

Corollary 4. Let {γu,s } be arbitrary finite-order weights of order k. Then we have: 1. For the weighted star discrepancy of the first N = pm points of a Niederreiter (t, s)-sequence in prime base p we have ∗ DN,γ

k  sk p2 logp (s + p) logp (pN ) , ≤ cγ,s 1 + N 2

where cγ,s = max γu,s . u⊆Is 1≤|u|≤k

2. If Γ := sup max γu,s s∈N

u⊆Is 1≤|u|≤k



j logp (j + p) < ∞,

(9)

j∈u

then for any δ > 0 there exists a Cp,γ,δ > 0, independent of s and m, such that for the weighted star discrepancy of the first N = pm points of a Niederreiter (t, s)-sequence in prime base p we have ∗ DN,γ ≤

Cp,γ,δ . N 1−δ

Thus under condition (9) the upper bound is independent of the dimension which shows that the weighted star discrepancy of Niederreiter sequences achieves strong tractability. Proof. From the proof of Theorem 6 and since we deal with finite-order weights of order k, we obtain

Weighted Star Discrepancy of Digital Nets in Prime Bases

89

 jp2 logp (j + p) logp (pN ) 1 1 max (|u|γu,s ) + max γu,s u⊆Is u⊆Is N 1≤|u|≤k N 1≤|u|≤k 2 j∈u k  2 1 1 p max γu,s s logp (s + p) logp (pN ) . ≤ kcγ,s + u⊆Is N N 1≤|u|≤k 2

∗ DN,γ ≤

The first part of the corollary follows. We prove the second part:  jp2 logp (j + p) logp (pN ) 1 1 max (|u|γu,s ) + max γu,s u⊆Is u⊆Is N 1≤|u|≤k N 1≤|u|≤k 2 j∈u r  2  p 1 Γ + max logp (pN ) max γu,s j logp (j + p) ≤ u⊆I s N N 1≤r≤k 2 |u|=r j∈u k  Γ Γ p2 ≤ + logp (pN ) . N N 2

∗ ≤ DN,γ



The result follows.

The first part of Corollary 4 shows that for finite-order weights of order k we can indeed obtain a convergence rate of O((log N )k N −1 ), as outlined in the example in the introduction. The second part, on the other hand, shows that under a certain condition on the weights we can even obtain an error bound independent of the dimension. The conditions on the weights are of a similar form as obtained in the earlier papers [14, 17–19], with the exception of [2] (see also Section 7 in this paper) where a much weaker dependence on the weights was shown. On the other hand, the construction in [2] requires a computer search and it is not known if a-priori given sequences can achieve such upper bounds on the star discrepancy. If p is prime and s is an arbitrary dimension ≤ p, then for q1 , . . . , qs we can choose the polynomials qi (x) = x − ai , 1 ≤ i ≤ s, where a1 , . . . , as are distinct elements of Zp ; see [11, Remark 4.52]. (Sequences constructed with such polynomials are called Faure-Niederreiter sequences.) From (6) it follows that for any u ⊆ Is , u = ∅, we have tu = 0. This leads to the following result. Theorem 7. For the weighted star discrepancy of the first N = pm points of a Faure-Niederreiter (0, s)-sequence in prime base p ≥ s we have ∗ ≤ DN,γ

p |u| 1 1 max(|u|γu,s ) + max γu,s logp (pN ) . s s N u⊆I N u⊆I 2 u=∅ u=∅

Proof. The result follows from Corollary 1.



Corollary 5. Let {γu,s } be arbitrary finite-order weights of order k. Then we have:

90

Josef Dick et al.

1. For the weighted star discrepancy of the first N = pm points of a FaureNiederreiter (0, s)-sequence in prime base p ≥ s we have  k  p 1 ∗ logp (pN ) ≤ Gγ,s k + , DN,γ N 2 where Gγ,s := max γu,s . u⊆Is 1≤|u|≤k

2. If

sup max γu,s s|u| < ∞, s∈N

u⊆Is 1≤|u|≤k

(10)

then for any δ > 0 there exists a Cγ,δ > 0, independent of s and m, such that for the first N = pm points of a Faure-Niederreiter (0, s)-sequence in prime base p, s ≤ p ≤ 2s, we have ∗ DN,γ ≤

Cγ,δ . N 1−δ

Thus under condition (10) the upper bound is independent of the dimension which shows that the weighted star discrepancy of Faure-Niederreiter sequences achieves strong tractability. 3. If (11) sup max γu,s < ∞, s∈N

u⊆Is 1≤|u|≤k

then for any δ > 0 there exists a Cγ,δ > 0, independent of s and m, such that for the first N = pm points of a Faure-Niederreiter (0, s)-sequence in prime base p, s ≤ p ≤ 2s, we have ∗ DN,γ ≤ Cγ,δ

sk . N 1−δ

Thus under condition (11) the upper bound depends only polynomially on the dimension which shows that the weighted star discrepancy of FaureNiederreiter sequences achieves tractability. Proof. The first statement follows immediately from Theorem 7. For the second and third statement we use again Theorem 7 and note that we choose s ≤ p ≤ 2s, which is possible by Bertrand’s postulate.  The upper bound in Corollary 5 for the weighted star discrepancy of the Faure-Niederreiter sequence shows a stronger dependence on the dimension than the result for the Niederreiter sequence. This is due to the dependence of the base p on the dimension s, that is, we have to demand that p ≥ s. Still, as part 3 of Corollary 5 shows, we can prove that the weighted star discrepancy depends at most polynomially on the dimension under certain conditions on the weights.

Weighted Star Discrepancy of Digital Nets in Prime Bases

Corollary 6. Assume that for u ⊆ Is , u = ∅, γu,s is given by γu,s = with nonnegative reals γj,s . Then we have:

91

2 j∈u

γj,s

1. For the weighted star discrepancy of the first N = pm points of a FaureNiederreiter (0, s)-sequence in prime base p, s ≤ p ≤ 2s, we have ⎛ ⎞   1 1 ∗ DN,γ max ⎝|u| max ≤ γj,s ⎠ + (γj,s s logp (pN )). s s N u⊆I N u⊆I u=∅ u=∅ j∈u j∈u 2. If Γ := sup max s∈N

u⊆Is u=∅



γj,s s < ∞,

j∈u

then for the first N = pm points of a Faure-Niederreiter (0, s)-sequence in prime base p, s ≤ p ≤ 2s, we have ∗ DN,γ ≤ 2Γ

(m + 1)s . N

7 Average Weighted Star Discrepancy In this section we obtain an upper bound on the average of the weighted star discrepancy over all digital nets constructed over Zp in s dimensions and with pm points. We consider only weights of product form where the weights are independent of the dimension, that is, for u ⊆ Is , u = ∅, the weights γu,s are 2 given by γu,s = γu = j∈u γj with nonnegative reals γj independent of s. Define ( 3p,γ (C1 , . . . , Cs ) := R γu Rp ((Ci )i∈u ). u⊆Is u=∅

∗ Then it follows from Theorem 2 that for the weighted star discrepancy DN,γ of a digital net over Zp generated by the matrices C1 , . . . , Cs we have ∗ ≤ DN,γ

1 3p,γ (C1 , . . . , Cs ), max(|u|γu ) + R s N u⊆I u=∅

where N = pm . Lemma 4. We have 3p,γ (C1 , . . . , Cs ) = R

s (

r3p (ki , γi ),

k∈D i=1

where D is defined by (3) and r3p (k, γ) is defined by  1 + γ if k = 0, r3p (k, γ) := γrp (k) if k = 0.

(12)

92

Josef Dick et al.

Proof. Let x0 , . . . , xpm −1 be the digital net generated by C1 , . . . , Cs and write (1) (s) xn = (xn , . . . , xn ) for 0 ≤ n < pm . From Lemma 1 it follows that for u ⊆ Is , u = ∅, we have   m p( −1 pm −1 1 (  (i) 1+ rp (k)walk (xn ) . Rp ((Ci )i∈u ) = −1 + m p n=0 i∈u k=1

Thus, we obtain ( γu Rp ((Ci )i∈u ) u⊆Is u=∅

=−

( u⊆Is u=∅

−1  ( 1 p( γu + γi pm n=0 i∈u u⊆Is m

u=∅



= − −1 +

s 

 1+

m p( −1

 rp (k)walk (xn(i) )

k=1



(1 + γi )

i=1

   m pm −1 −1 p( s  1 ( + m rp (k)walk (xn(i) ) −1 + 1 + γi + γi p n=0 i=1 k=1  pm −1 m p( −1  s s  ( 1 = − (1 + γi ) + m r3p (k, γi )walk (xn(i) ) p n=0 i=1 i=1 k=0

=−

s 

(1 + γi ) +

i=1

=

k1 ,...,ks =0

m p( −1

k1 ,...,ks =0 (k1 ,...,ks )=(0,...,0)

=

m p( −1

s (

p −1 s 1 (  r3p (ki , γi ) m walki (xn(i) ) p n=0 i=1 i=1 m

s 

p −1 s 1 (  r3p (ki , γi ) m walki (xn(i) ) p n=0 i=1 i=1 m

s 

r3p (ki , γi ),

k∈D i=1



where we used formula (5). Let Cp := {(C1 , . . . , Cs ) : Ci ∈ Zm×m for i = 1, . . . , s}. Then we define p Ap (m, s) :=

1 |Cp |

(

3p,γ (C1 , . . . , Cs ), R

(13)

(C1 ,...,Cs )∈Cp

3p,γ taken over all s-tuples of m × m matrices i.e., Ap (m, s) is the average of R over Zp . Theorem 8. Let Ap (m, s) be defined by (13) and N = pm . Then for p = 2 we have

Weighted Star Discrepancy of Digital Nets in Prime Bases

A2 (m, s) =

1 2m



s  

1 + γi

m 2

i=1

93



s   +1 − (1 + γi ) , i=1

and for p > 2 we have  s      s 1 1 1  log p + (1 + γi ) . 1 + 2γi m Ap (m, s) ≤ m − p π 5 i=1 i=1 Proof. We have Ap (m, s) =

=

(

1 pm 2 s

r3p (ki , γi )

(C1 ,...,Cs )∈Cp k∈D i=1 m −1 p(

s 

k1 ,...,ks =0 (k1 ,...,ks )=(0,...,0)

i=1

1 p

s (

m2 s

(

r3p (ki , γi )

1.

(C1 ,...,Cs )∈Cp T k +···+C T k =0 C1 1 s s

(i)

Let cj denote the j-th row vector, 1 ≤ j ≤ m, of the matrix Ci , 1 ≤ i ≤ s. Then for k ∈ {0, 1, . . . , pm − 1}s , k = 0, our condition in the innermost sum of the above expression becomes s m−1 ( (

(i)

cj+1 κi,j = 0,

(14)

i=1 j=0

where ki = κi,0 + κi,1 p + · · · + κi,m−1 pm−1 . Since at least one ki = 0, it follows that there is a κi,j = 0. First assume that κ1,0 = 0. Then for any choice of (1)

(2)

(s)

(2) (s) c2 , . . . , c(1) m , c1 , . . . , cm , . . . , c1 , . . . , cm (1)

we can find exactly one vector c1 such that condition (14) is fulfilled. The (1) (i) same argument holds with κ1,0 replaced by κi,j and c1 replaced by cj+1 . Therefore we get 1 Ap (m, s) = m p 1 = m p 1 = m p 1 = m p

m p( −1

s 

k1 ,...,ks =0 (k1 ,...,ks )=(0,...,0)

i=1



−1 s p( 

r3p (ki , γi )

m

r3p (k, γi ) −

i=1 k=0



s 

i=1



s 

i=1

1 + γi + γi

1 + γi

 (1 + γi )

i=1

 

s 

m −1 p(

k=1 m −1 p(

k=0



rp (k)

i=1



rp (k)

−

s 

−

s  i=1

 (1 + γi ) 

(1 + γi ) .

94

Josef Dick et al.

From the proof of [11, Lemma 3.13] we find that m 2( −1

r2 (k) =

k=0

m +1 2

and that for p > 2, m p( −1

 rp (k) ≤ m

k=0

2 2 log p + π 5

 . 

The result follows. Corollary 7. For any prime p and ε ≥ 1 we have

  8 9 ε 1 3 , (C1 , . . . , Cs ) ∈ Cp : Rp,γ (C1 , . . . , Cs ) ≤ H(p, m, s, γ) ≥ |Cp | 1 − N ε where N = pm , H(p, m, s, γ) :=

s 

(1 + γi h(p, m)) −

i=1

s 

(1 + γi )

i=1

and where h(p, m) = 2m((log p)/π + 1/5) if p = 2 and h(2, m) = m/2 + 1. Proof. From Theorem 8 we obtain 1 1 H(p, m, s, γ) ≥ pm |Cp |

(

3p,γ (C1 , . . . , Cs ) R

(C1 ,...,Cs )∈Cp

1 ε H(p, m, s, γ) × ≥ |Cp | pm  ' 3p,γ (C1 , . . . , Cs ) > ε H(p, m, s, γ) . (C1 , . . . , Cs ) ∈ Cp : R pm Hence we have  ' 1 1 ε 3 (C1 , . . . , Cs ) ∈ Cp : Rp,γ (C1 , . . . , Cs ) > m H(p, m, s, γ) , ≥ ε |Cp | p and the result follows.   1 1 ∞ Let cp := 2 π log p + 15 , N = pm , and assume that i=1 γi < ∞. Then we have   s s  s   cp (1 + γi mcp ) = (1 + γi c log N ), 1 + γi (log N ) ≤ log p i=1 i=1 i=1

Weighted Star Discrepancy of Digital Nets in Prime Bases

95

where c > 1 is an absolute constant. Now we follow the proof of [8, Lemma 3] . Let ∞  S(γ, N ) := (1 + γi c log N ) and define σd := c log S(γ, N ) =

1∞ i=d+1 ∞ (

i=1

γi for d ≥ 0. Then

log(1 + γi c log N )

i=1

≤

d (

∞ (

log(1 + σd−1 + γi c log N ) +

i=1

log(1 + γi c log N )

i=d+1

≤ d log(1 + σd−1 ) +

d (

log(1 + γi σd c log N )

i=1

+

∞ (

log(1 + γi c log N )

i=d+1

≤ d log(1 + σd−1 ) + σd c(log N )

d (

γi + σd log N

i=1

≤ d log(1 + σd−1 ) + σd (σ0 + 1) log N. Hence we obtain

S(γ, N ) ≤ (1 + σd−1 )d pm(σ0 +1)σd .

For δ > 0 choose d large enough to make σd ≤ δ/(σ0 + 1). Then we obtain S(γ, N ) ≤ cγ,δ pδm . Therefore from (12) and Theorem 8 we obtain the following result. 1∞ Corollary 8. If i=1 γi < ∞, then for any δ > 0 there exist a constant  cγ,δ > 0, independent of s and m, and m × m matrices C1 , . . . , Cs over Zp ∗ such that the weighted star discrepancy DN,γ of the digital net generated by C1 , . . . , Cs satisfies ∗ ≤ DN,γ

 cγ,δ N 1−δ

(15)

for all m, s ≥ 1 and where N = pm . Hence there exist digital nets whose weighted star discrepancy achieves a strong tractability error bound as long as the sum of the weights is finite. In [2] the authors introduced an algorithm which shows how matrices C1 , . . . , Cs which satisfy a bound of the form (15) can be found by computer search.

96

Josef Dick et al.

References 1. Chrestenson, H.E.: A class of generalized Walsh functions. Pacific J. Math., 5: 17–31, 1955. 2. Dick, J., Leobacher, G., Pillichshammer, F.: Construction algorithms for digital nets with small weighted star discrepancy. To appear in SIAM J. Num. Anal., 2005. 3. Dick, J., Pillichshammer, F.: Multivariate integration in weighted Hilbert spaces based on Walsh functions and weighted Sobolev spaces. J. Complexity, 21: 149– 195, 2005. 4. Dick, J., Pillichshammer, F.: On the mean square weighted L2 discrepancy of randomized digital (t, m, s)-nets over Z2 . Acta Arith., 117: 371–403, 2005. 5. Dick, J., Sloan, I.H., Wang, X., Wo´zniakowski, H.: Liberating the weights. J. Complexity, 20: 593–623, 2004. 6. Dick, J., Sloan, I.H., Wang, X., Wo´zniakowski, H.: Good lattice rules in weighted Korobov spaces with general weights. Submitted, 2003. 7. Hellekalek, P.: General discrepancy estimates: the Walsh function system. Acta Arith., 67: 209–218, 1994. 8. Hickernell, F.J., Niederreiter, H.: The existence of good extensible rank-1 lattices. J. Complexity, 19: 286–300, 2003. 9. Niederreiter, H.: Point sets and sequences with small discrepancy. Monatsh. Math., 104: 273–337, 1987. 10. Niederreiter, H.: Low-discrepancy and low-dispersion sequences. J. Number Theory, 30: 51–70, 1988. 11. Niederreiter, H.: Random Number Generation and Quasi-Monte Carlo Methods. SIAM, Philadelphia, 1992. 12. Niederreiter, H., Xing, C.P.: Rational Points on Curves over Finite Fields: Theory and Applications. Cambridge University Press, Cambridge, 2001. 13. Rivlin, T.J., Saff, E.B.: Joseph L. Walsh Selected Papers. Springer Verlag, New York, 2000. 14. Sloan, I.H., Wang, X., Wo´zniakowski, H.: Finite-order weights imply tractability of multivariate integration. J. Complexity, 20: 46–74, 2004. 15. Sloan, I.H., Wo´zniakowski, H.: When are quasi-Monte Carlo algorithms efficient for high dimensional integrals? J. Complexity, 14: 1–33, 1998. 16. Walsh, J.L.: A closed set of normal orthogonal functions. Amer. J. Math., 55: 5–24, 1923. 17. Wang, X.: A constructive approach to strong tractability using Quasi-Monte Carlo algorithms. J. Complexity, 18: 683–701, 2002. 18. Wang, X.: Strong tractability of multivariate integration using quasi-Monte Carlo algorithms. Math. Comp., 72: 823–838, 2003. 19. Yue, R.X., Hickernell, F.J.: Strong tractability of integration using scrambled Niederreiter points. To appear in Math. Comp., 2005.

Explaining Effective Low-Dimensionality Andrew Dickinson Mathematical Institute, University of Oxford, 24-29 St Giles’, Oxford, OX1 3LB, UK. [email protected]

Summary. It has been proposed by Owen et al. [CMO97,LO02] that the surprising efficacy of quasi-Monte Carlo methods, when applied to certain high-dimensional integrands arising in mathematical finance, results from the integrands being effectively low-dimensional in the superposition sense. In this paper, mathematical results are presented which relate effective low-dimensionality with the structure of the underlying stochastic differential equation.

The pricing of a contingent claim often reduces to the problem of estimating a quantity of the form E [Φ (X)] where (Xt )t∈[0,T ] is an Itˆo diffusion satisfying a stochastic differential equation dXt = µ (t, Xt ) dt + σ (t, Xt ) dWt ,

(1)

X0 = ξ, (Wt )t∈[0,T ] is a standard Wiener process defined on the filtered probability   space Ω, F, (Ft )t∈[0,T ] , P , µ : [0, T ] × R → R, σ : [0, T ] × R → R and Φ : C [0, T ] → R. There are many financial problems where the functional, Φ, is complex and “path-dependent” for which the value of E [Φ (X)] is not known analytically and numerical techniques must be employed. Quantities of the form E [Φ (X)] may be estimated by the integral over a high-dimensional unit cube of a function formed using the Brownian bridge discretization . For particular problems arising in mathematical finance, there is significant numerical evidence suggesting the superiority of quasi-Monte Carlo methods over the standard Monte Carlo method. The considerable superiority of quasi-Monte Carlo methods over standard Monte Carlo is surprising for such high-dimensional integrals. Insightfully, Caflisch et al. suggested that the surprising efficacy of quasi-Monte Carlo methods when applied to problems in mathematical finance is due to the integrand one forms using the Brownian bridge discretization being effectively low-dimensional in the superposition sense, see [CMO97]. The goal of this paper is to explain effective low-dimensionality of the integrand one forms using the Brownian bridge

98

A. Dickinson

discretization in terms of the structure of the stochastic differential equation (1) and that of the functional, Φ. This paper is divided into three sections: in Sect. 1 we introduce a concise notation to describe the manner in which one constructs an integrand via the Brownian bridge discretization, this notation significantly simplifies the exposition of Sect. 3; in Sect. 2 we outline some examples from mathematical finance to which quasi-Monte Carlo methods have been applied successfully, these examples shall motivate the hypotheses of Sect. 3; in Sect. 3 we present results relating the effective low-dimensionality of the integrand one obtains using the Brownian bridge discretization to the structure of the stochastic differential equation (1) and that of the functional, Φ.

1 Formation of the Integrand Let D [0, T ] denote the collection of all finite subdivisions of [0, T ]. More precisely, let D [0, T ] represent all finite increasing sequences, ∆, of the form ∆ = {0 = t0 < ... < tN = T } where N ∈ N. Given ∆1 , ∆2 ∈ D [0, T ] with ∆1 = {0 = s0 < ... < sM = T } and ∆2 = {0 = t0 < ... < tN = T } we shall write ∆1  ∆2 iff there exists an increasing function i : {0, ..., M } → {0, ..., N } such that i (0) = 0 , i (M ) = N. In other words, “∆1  ∆2 ” denotes the fact that ∆1 is a refinement of ∆2 . Clearly, “” is a partial ordering on D [0, T ] . Given ∆ = {0 = t0 < ... < tN = T } ∈ D [0, T ] we shall write |∆| =

sup

|ti − ti−1 |

i=1,...,N

and #∆ = N . For ∆ = {0 = t0 < ... < t#∆ = T } ∈ D [0, T ] let E ∆ : R#∆ → R#∆+1 be defined by

Explaining Effective Low-Dimensionality

99

⎧ ξ, if i = 1 ⎪ ⎪ ⎨ ξ + µ (0, ξ) t1 +σ (0, ξ) x1 , if i = 2 ∆  E (x)i = (2) ∆ ∆ (t E (x) + µ t , E (x) − t ) ⎪ i−1 i i−1 i−1 i−1 ⎪   ⎩ +σ ti−1 , E ∆ (x)i−1 (xi − xi−1 ) , if i ≥ 3 .   Note that E ∆ Wt1 , ..., Wt#∆ is the Euler-Maruyama approximation to the solution of (1). Next, a function Φ∆ : R#∆+1 → R is chosen such that     E Φ∆ E ∆ Wt1 , ..., Wt#∆ “well-approximates” E [Φ (X)] . Typically, Φ∆ is chosen to be some “natural” discrete time analogue of Φ. In Sect. 3 we find it necessary to introduce a “canonical” choice of Φ∆ . Let G : (0, 1) → R be the inverse of the cumulative distribution of the standard Gaussian distribution, i.e. for each α ∈ (0, 1) let G (α) ∈ R be the unique value such that  G(α) − z2 e 2 √ dz = α. 2π −∞ n

For each n ∈ N, let Gn : (0, 1) → R be the function formed by applying G componentwise: (Gn (x))i = G (xi ) . Note that if X is a random variable having the uniform distribution on #∆ and A∆ ∈ R#∆ × R#∆ is any matrix such that (0, 1) 

A∆



A∆

T

  = Cov Wt1 , ..., Wt#∆

then E ∆ ◦ A∆ ◦ G#∆ (X) has the same distribution as the Euler approximation   E ∆ Wt1 , ..., Wt#∆ . In particular,  (0,1)

#∆

Φ∆ ◦ E ∆ ◦ A∆ ◦ G#∆ (x) Λ#∆ (dx)

    = E Φ∆ E ∆ Wt1 , ..., Wt#∆+1 ≈ E [Φ (X)] where Λ#∆ denotes #∆-dimensional Lebesgue measure. It is in this fashion that we may approximate E [Φ (X)] by an integral of a function defined on an #∆-dimensional unit hypercube.

100

A. Dickinson

Different choices of the matrix A∆ lead to different so-called “discretizations”. In this paper we shall be concerned with the “Brownian bridge discretization” which was introduced in Moskowitz and Caflisch [MC96] and is closely related to the L´evy construction of Brownian motion. Suppose that  ' T 2T ∆ = 0, n , n ..., T 2 2 for some n ∈ N, then we may form the Brownian bridge discretization by choosing A∆ ∈ R#∆×#∆ such that ⎞ ⎛ j   n−1 −1 2( ( √  ∆  i i (3) Sk,j A x i= T⎝ n+ x2j +k+1 ⎠ n 2 2 j=0 k=0

where Sk,n : [0, 1] → R is the Schauder function defined by ⎧ n if t ∈ [k2−n , (2k + 1) 2−n ) ⎨ 2 2 (t − k2−n ) , n −n Sk,n (t) = 2 2 ((k + 1) 2 − t) , if t ∈ [(2k + 1) 2−n , (k + 1) 2−n ] ⎩ 0, otherwise.

(4)

We remark that the assumption that ∆ is equidistant and #∆ is a power of 2 is not strictly necessary to define a Brownian bridge discretization, however, if this is not the case then the construction becomes ungainly and somewhat arbitrary. As such, we favor (3) over full generality.

2 Numerical Examples In this section, we present some concrete examples of functionals of Itˆo diffusions occurring in mathematical finance for which quasi-Monte Carlo methods have been applied successfully. For the purpose of this paper, these examples shall motivate the general abstract conditions used in Sect. 3. We are more interested in the general form of the functional Φ : C [0, T ] → R and the structure of the Itˆ o diffusion, (Xt )t∈[0,T ] , so some irrelevant details have been omitted. These examples fall in to two groups: firstly, problems for which there is no closed-form solution and quasi-Monte Carlo methods provide a valuable tool by which to estimate the solution; and secondly, test problems for which there exist closed-form solutions. Paskov and Traub were the first to apply quasi-Monte Carlo methods to problems occurring in mathematical finance (see [PT95]), they considered the problem of pricing a collateralized mortgage obligation whose value is derived from a series of cash flows from a pool of mortgages through time (the reader should consult this paper for the details). Paskov and Traub assumed that the proportion of mortgage holders prepaying in a particular month is a deterministic function of the prevailing short rate and they assume the short

Explaining Effective Low-Dimensionality

101

rate follows a certain log-normal stochastic process. Ninomiya and Tezuka [NT96] and Caflisch et al. [CMO97] considered a simplified mortgage-backed security that shares the same key features with the more complex mortgagebacked security considered in [PT95]. For simplicity, we restrict our attention to the simplified mortgage-backed security considered in [NT96]. Ninomiya and Tezuka formulated the problem in discrete time, but we take the liberty of reformulating it in continuous time. Ninomiya and Tezuka’s assumptions are equivalent to assuming that the short rate, Xt , satisfies the RendlemanBartter model dXt = µXt dt + σXt dWt , X0 = ξ .

(5)

The value to be approximated is the expected discounted payoff of the cash flows from the pool of mortgages over time under the dynamics (5) which may be written as E [Φ (X)] where the functional Φ : C [0, T ] → R is of the form  T  
t
t (6) Φ (Z) = e− 0 Zt dt ce− 0 w(Zu )du dt, 0 ∞

c ∈ R, w ∈ C (R) is bounded and w (Zu ) du describes the proportion prepaying in the infinitesimal interval [u, u + du] if the trajectory of the short rate is Z ∈ C [0, T ]. Ninomiya and Tezuka [NT96] also apply quasi-Monte Carlo methods to the pricing of a bond in Vasicek’s model. More precisely, they are interested in estimating the value of E [Φ (X)] where Φ : C [0, T ] → C [0, T ] is defined by Φ (Z) = e−


T 0

Zt dt

(7)

and (Xt )t∈[0,T ] represents the short-rate whose risk-neutral dynamics are governed by an Ornstein-Uhlenbeck process of the form dXt = a (b − Xt ) dt + cdWt , X0 = ξ, where a, b, c, ξ ∈ R. Since X is a Gaussian process we see that Φ (X) is lognormal and E [Φ (X)] is known analytically. Many options have a payoff of the form Φ (Z) if the trajectory of the price of the underlying is (Zt )t∈[0,T ] and where Φ : C [0, T ] → R . By the risk-neutral valuation principle (see for example Musiela and Rutkowski [MR97]) the price of such an option at time 0 in the Black-Scholes model equals e−rT E [Φ (X)] where the risk-neutral dynamics (Xt )t∈[0,T ] are governed by the stochastic differential equation dXt = rXt dt + σXt dWt , X0 = ξ,

(8)

102

A. Dickinson

where r ∈ R is the short rate, σ ∈ R is the volatility and ξ ∈ R is the initial price of the underlying. The payoffs of an arithmetic average Asian call option and an arithmetic average Asian put option have the form +  T Zt ν (dt) − K (9) Φ (Z) = 0

and

 Φ (Z) =



+

T

K−

Zt ν (dt)

(10)

0

respectively, where K ∈ R is the strike and ν is a probability measure on ([0, T ] , B [0, T ]) (taking ν to be normalized Lebesgue measure corresponds to “continuous-averaging” and taking ν to be a convex combination of Dirac masses corresponds to “discrete-averaging”). The payoffs of a geometric average Asian call option and a geometric average Asian put option have the form  +   T log (Zt ) ν (dt) − K (11) Φ (Z) = exp 0

and

 Φ (Z) =

 K − exp

+

T

log (Zt ) ν (dt)

(12)

0

respectively, where K ∈ R and ν is a probability measure on ([0, T ] , B [0, T ]) . There is no closed-form expression for the value of an arithmetic average Asian call option in the Black-Scholes model, but one may obtain a closedform expression for the price of a geometric average Asian call option since   T

exp

log (Zt ) ν (dt) 0

has a log-normal distribution under the risk-neutral probability measure. Quasi-Monte Carlo methods have been applied to Asian options (with geometric averaging and arithmetic averaging) by various authors, including Acworth et al. [AB97] and Joy et al. [JBT96]. We remark that the functionals (6) and (7) are Fr´echet differentiable and the functionals (9),(10),(11) and (12) are Fr´echet differentiable except on a null set (by null set we mean a set of measure zero with respect to the law of the asset price under the risk neutral probability measure). This observation will motivate the condition on Φ investigated in the next section. Broadly speaking, the results of the numerical experiments in the literature tend to suggest that, for many functionals of Itˆ o diffusions occurring in finance, quasi-Monte Carlo methods applied to the integrands formed using the Brownian bridge discretization often require many-fold fewer quasi-random sample outcomes than the number of pseudo-random sample outcomes required by standard Monte Carlo methods to achieve the same level of accuracy.

Explaining Effective Low-Dimensionality

103

3 Sufficient Conditions For Effective Low-Dimensionality In [CMO97], Caflisch et al. describe two notions of effective low-dimensionality in terms of the functional ANOVA decomposition. We shall find it convenient to reformulate the ANOVA decomposition in the language of the Hilbert space structure of the square integrable functions defined on the unit cube. The advantage of this mathematically equivalent formulation is that it makes the proofs in the sequel simpler and more transparent. Consider the probability space     d d [0, 1] , B [0, 1] , Λd where Λd denotes d-dimensional Lebesgue measure. For i ∈ {1, ..., d} write d xi : [0, 1] → R for the projection onto the ith coordinate: xi (a) = ai . Write I for the power set of {1, ..., d} (the set of all subsets of {1, ..., d} including the empty set, ∅). Note the natural bijection between elements of I d and the subfaces of the unit hypercube [0, 1] . For each I ∈ I, define the σ-algebras

8 9 d ∅, [0, 1] , if I = ∅ FI = σ (xi1 , ..., xik ) , if I = {i1 , ..., ik } and the vector spaces of random variables   d FI = L2 [0, 1] , FI , Λd and

⎧ if I = ∅ ⎪ ⎨ F∅ , ( FJ , if I = {i1 , ..., ik } GI = FI  ⎪ ⎩ J⊂I J =I

where  denotes orthogonal complement with respect to the inner product < ., . >L2 ([0,1]d ,B([0,1]d ),Λd ) . One can show that   < d d L2 [0, 1] , B [0, 1] , Λd = GI I∈I

  d d is an orthogonal decomposition of the Hilbert space L2 [0, 1] , B [0, 1] , Λd .   d d Given a subspace, H, of L2 [0, 1] , B [0, 1] , Λd we shall write   d d PH : L2 [0, 1] , B [0, 1] , Λd → R for the orthogonal projection onto H.

104

A. Dickinson

  d d Given f ∈ L2 [0, 1] , B [0, 1] , Λd we define the ANOVA decomposition of f to be ( f= fI I∈I

where (fI )I∈I are the ANOVA components of f defined by fI = PGI (f ) . One can show that the above definitions are mathematically equivalent to those presented in [Owe92]. Caflisch et al. [CMO97] proposed the following notion of effective dimension based upon the ANOVA decomposition: s

Definition 1. Let s ∈ N and suppose f : [0, 1] → R. f is effectively kdimensional in the superposition sense if ( V arΛs (fI ) ≥ (1 − ε) V arΛs (f ) #I≤k

where ε ∈ (0, 1) is some small value (eg ε = 10−2 ). Loosely speaking, a function is effectively k-dimensional in the superposition sense if it is well-approximated (in ·L2 (Λs ) ) by a linear combination of functions, each of which is a function of at most k of the coordinate variables. One might anticipate quasi-Monte Carlo methods to be effective when applied to functions that are effectively low-dimensional in the superposition sense due to the fact that low-discrepancy sequences have well-distributed projections onto low-dimensional faces of the unit cube. In [CMO97] and [LO02], Owen and his collaborators present numerical evidence suggesting that certain integrands formed using the Brownian bridge discretization are effectively low-dimensional in the superposition sense. For a given function, f , and ε > 0 finding the minimal k such that ( V arΛs (fI ) ≥ (1 − ε) V arΛs (f ) #I≤k

is considerably more involved than the original goal of calculating Instead, we study how ( V arΛs (fI ) k → V arΛs (f ) −


[0,1]s

f dΛs .

(13)

#I≤k

decays with k. We provide conditions on Φ, µ and σ which will allow us to prove that the mapping (13) decays approximately quadratically with k. We shall suppose that Φ satisfies the following technical condition:

Explaining Effective Low-Dimensionality

105

Condition 1 Φ : C [0, T ] → R is Fr´echet differentiable with derivative DΦ. Further, there exists KΦ , rΦ > 0 such that for every φ, δ ∈ C [0, T ] Φ (φ + δ) = Φ (φ) + (DΦ) (φ) (δ) + RΦ (φ, δ) where

  rΦ (DΦ) (φ)C[0,T ]∗ ≤ KΦ 1 + φC[0,T ]   rΦ rΦ 2 RΦ (φ, δ) ≤ KΦ 1 + φC[0,T ] + δC[0,T ] δC[0,T ]

(14)

(15) (16)

∗

and ·C[0,T ]∗ represents the operator norm on C [0, T ] . The above condition is a slight idealization, but the examples presented in Sect. 2 hopefully convince the reader that this condition is not contrived. We shall consider the following technical condition on the drift and diffusion coefficient of the Itˆo diffusion: Condition 2 µ and σ are such that the collections of functions  '  ∂σ ∂σ , σ, σ A= µ−σ ∂x ∂x and

      1 ∂σ ∂ 1 ∂σ ∂ 1 ∂σ B= µ− σ µ− σ ,σ µ− σ , 2 ∂x ∂x 2 ∂x ∂x 2 ∂x        '  ∂σ ∂ ∂σ 1 ∂σ ∂σ 1 ∂σ ∂ σ ,σ σ , µ− σ . µ− σ 2 ∂x ∂x ∂x ∂x ∂x 2 ∂x ∂x

exist. Further, every function h ∈ A satisfies the Lipschitz condition |h (t, x) − h (t, y)| ≤ K |x − y| for some K > 0 and every t ∈ [0, T ] and x, y ∈ R; every function h ∈ A satisfies the H¨ older condition 1

|h (s, x) − h (t, x)| ≤ K (1 + x) |s − t| 2 for some K > 0 and every s, t ∈ [0, T ] and x ∈ R; and every function k ∈ A ∪ B satisfies the linear growth condition |k (t, x)| ≤ K (1 + |x|) for some K > 0 and every t ∈ [0, T ] and x ∈ R. The reader may recognize the above conditions as being precisely those required for proving that Milstein’s scheme has a linear rate of strong convergence (see [KP92]). In fact, a suitable modification of the Milstein scheme

106

A. Dickinson

yields a convenient approach to explicitly construct a function that is lowdimensional (in the superposition sense) which well-approximates the integrand one constructs when one applies the Brownian bridge discretization to a smooth functional of a one-dimensional Itˆ o diffusion. Given ∆ = {0 = t0 , ..., tN = T } ∈ D [0, T ] we shall write L∆ : C [0, T ] → C [0, T ] for the operator that linearly interpolates a function between times t0 , ..., tN . More precisely, given f ∈ C [0, T ] let L∆ f ∈ C [0, T ] be defined by   (t − ti ) fti+1 − fti for t ∈ [ti , ti+1 ] . (L∆ f )t = fti + ti+1 − ti We remark that L∆ is a linear contraction. Proposition 1. Suppose that Φ satisfies Condition 1 and that µ and σ satisfy Condition 2. Suppose ∆c , ∆f ∈ D [0, T ] with ∆c = {0 = s0 < ... < sM = T } , ∆f = {0 = t0 < ... < tN = T }  ∆ and ∆f  ∆c . Define Ut f to be the Euler-Murayama approximation: 

t∈[0,T ]

∆ Ut f

=

ξ,

if t = 0     ∆ ∆ ∆ Uti f + µ ti , Uti f (t − ti ) + σ ti , Uti f (Wt − Wti ) , if t ∈ (ti , ti+1 ]

  define Zt∆c

Zt∆c

to be the linear interpolant of the Milstein scheme: t∈[0,T ]

⎧ ξ, if t = 0 ⎪ ⎪ ∆c ⎨ ∆   σ s ,Z W −W (t−s ) ( )( ) i s s i s i+1 i ∆c i c (t − si ) + = Zsi + µ si , Zs si+1 −si i   ⎪ ∆ 2 2 ∂σ c ⎪ (σ ∂x )(si ,Zsi ) Wsi+1 −si+1 −(Wsi −si ) (t−si ) ⎩ + , if t ∈ (si , si+1 ] 2(si+1 −si )

  and define Vt∆c

to be the scaled Brownian bridge: t∈[0,T ]

    Vt∆c = σ Zs∆i c Wt − L∆c W t , if t ∈ [si , si+1 ] . Then,        Φ L∆f U ∆f = Φ L∆c Z ∆c + (DΦ) L∆c Z ∆c L∆f V ∆c + εΦ,∆c ,∆f where       2 2 E ε2Φ,∆c ,∆f ≤ K (ξ, µ, σ, T, Φ) |∆c | 1 ∨ |log |∆c || + |∆f | (1 ∨ |log |∆f ||) (17) for some K (ξ, µ, σ, T, Φ) > 0 depending on ξ, µ, σ,T and Φ only.

Explaining Effective Low-Dimensionality

107

The proof of the above result is too lengthy to reproduce here. Recall that, in order to construct an integrand on the unit hypercube, one must choose a “discrete-time analogue”, Φ∆ : R#∆+1 → R, of the functional Φ : C [0, T ] → R. Before proceeding, we must describe a canonical way of choosing Φ∆ given Φ. Given Φ : C [0, T ] → R and ∆ = {0 = t0 < ... < t#∆ = T } we shall choose Φ∆ : R#∆+1 → R to be the function Φ∆ (x0 , ..., x#∆ ) = Φ (F x )

(18)

where F x ∈ C [0, T ] is the piecewise linear function such that F x (t) = xi for every i ∈ {0, ..., #∆} and is linear on each interval [ti , ti+1 ]. Theorem 3. Suppose that N ∈ N is a power of 2. Suppose that Φ satisfies Condition 1 and µ, σ satisfy Condition 2. Write f : RN → R for the function constructed by using the Brownian bridge discretization on the functional Φ applied to the linearly interpolated Euler-Maruyama scheme using N equally sized time steps: f = Φ∆ ◦ E ∆ ◦ A∆  T  where ∆ = 0, N , ..., T , E ∆ is as in Sect. 1, A∆ is as in (3) and Φ∆ is as in (18). Then there exists K (ξ, µ, σ, T, Φ) > 0 depending on ξ, µ, σ, T and Φ only such that the ANOVA components of f satisfy ⎛ V arΛM (f ) − V arΛM ⎝  ≤ K (ξ, µ, σ, T, Φ)

⎞

(

fI ⎠

I:#I≤M +1

1 ∨ log M M



2 +

1 ∨ log N N

 .

for every power of two M ∈ N with M < N . Proof. As in Sect. 1, let G : (0, 1) → R be the inverse of the cumulative distribution of the standard Gaussian distribution and let Sk,n : [0, 1] → R denote the Schauder function given by (3). Fix M = 2m and N = 2n with M < N . For i ∈ {0, ..., 2m } define si = 2iTm and for j ∈ {0, ..., 2n } define tj = jT 2n . Let ∆c = {0 = s0 < ... < s2m = T } and let ∆f = {0 = t0 < ... < t2n = T } n

˜ ∆f : (0, 1)2 → C [0, T ] by note that ∆f  ∆c . Define W

108

A. Dickinson

˜ ∆f W

⎛ ⎞ j   n−1 −1 ( 2( √ t t ⎠ (x) (t) = T ⎝G (x1 ) + G (x2j +k+1 ) Sk,j T T j=0 k=0

n

2 ˜ ∆f has the law of a Wiener process and note that if X ∼ U (0, 1) then W ˜ ∆c ,∆f : on [0, T ] linearly interpolated between the times t0 , ..., t2n . Define U 2n 2n (0, 1) → C [0, T ] to be the function that maps x ∈ (0, 1) to the path ˜ ∆c ,∆f (x) ∈ C [0, T ] which is linear on each interval [ti , ti+1 ] and the values U ˜ ∆c ,∆f (x) (ti ) are determined by the recurrence relation U

˜ ∆c ,∆f (x) (ti+1 ) U

  ˜ ∆c ,∆f (x) (ti ) + µ ti , U ˜ ∆c ,∆f (x) (ti ) (ti+1 − ti ) =U    ˜ ∆c ,∆f (x) (ti ) W ˜ ∆f (x) (ti+1 ) − W ˜ ∆f (x) (ti ) . + σ si , U

for i = 0, ..., M − 1 and

˜ ∆c ,∆f (x) (t0 ) = ξ . U n

2 Define Z˜ ∆c ,∆f : (0, 1) → C [0, T ] such that Z˜ ∆c ,∆f (x) is linear on each interval [si , si+1 ] and the values of Z˜ ∆c ,∆f (x) (si ) satisfy the recurrence relation

Z˜ ∆c ,∆f (x) (si+1 )

  = Z˜ ∆c ,∆f (x) (si ) + µ si , Z˜ ∆c ,∆f (x) (si ) (si+1 − si )    ˜ ∆f (x) (si+1 ) − W ˜ ∆f (x) (si ) + σ si , Z˜ ∆c ,∆f (x) (si ) W     2 1 ∂σ  ˜ ∆c ,∆f ˜ ∆f (x) (si+1 ) − W ˜ ∆f (x) (si ) si , Z (x) (si ) W + σ 2 ∂x

− (si+1 − si )) for i = 0, ..., N − 1 and

Z˜ ∆c ,∆f (x) (s0 ) = ξ .

n

2 Define V˜ ∆c ,∆f : (0, 1) → C [0, T ] by       ˜ ∆f (x) (t) ˜ ∆f (x) (t) − L∆ W V˜ ∆c ,∆f (x) (t) = σ si , Z˜ ∆c ,∆f (x) (si ) W c



= σ si , Z˜ ∆c ,∆f (x) (si )

j −1  n−1 ( 2(

j=m k=0

  t G (x2j +k+1 ) Sk,j T

˜ ∆f (x) (si ) − for [si , si+1 ] (note that this is well-defined since W  t ∈  ˜ ∆f (x) (si ) = 0 for every i). L∆c W In terms of this more involved notation, we may express f as

Explaining Effective Low-Dimensionality

109

˜ ∆c ,∆f (x) . f (x) = Φ ◦ U Define g : (0, 1)

2n

(19)

→ R by

     g (x) = Φ Z˜ ∆c ,∆f (x) + (DΦ) Z˜ ∆c ,∆f (x) V˜ ∆c ,∆f (x) .

(20)

  One sees from (20) and the linearity of (DΦ) Z˜ ∆c ,∆f (x) that <

g∈

GI .

|I|≤M +1

In terms of the notation Proposition 1, one can see that the pair (f, g) have the same joint distribution under Lebesgue measure, Λ#∆f , as the pair         Φ L∆f U ∆f , Φ L∆c Z ∆c + (DΦ) L∆c Z ∆c L∆f V ∆c <

under P. Therefore, using Proposition 1 and the fact that g ∈

GI

|I|≤M +1

⎛ V ar (f ) − V ar ⎝  =  ≤

⎞

(

fI ⎠

I:#I≤M +1

⎛ ⎜ ⎜f − P ⎝

< GI

⎞2

⎟ (f )⎟ ⎠ dΛN

#I≤M +1 2

(f − g) dΛN         2  = E Φ L∆f U ∆f − Φ L∆c Z ∆c + (DΦ) L∆c Z ∆c L∆f V ∆c  2   1 ∨ log M 1 ∨ log N ≤ K (ξ, µ, σ, T, Φ) + M N for some K (ξ, µ, σ, T, Φ) > 0 depending on ξ, µ, σ,T and Φ only.



We discuss how one should interpret the above technical result at an intuitive level. Using the notation and assumptions of Theorem 3 write ⎛ ⎞ ( fI ⎠ ρk,N = V arΛM (f ) − V arΛM ⎝ I:#I≤k

where N ∈ N is a power of two and represents the number of steps in our Euler approximation. Essentially, the notion of effective low-dimensionality in

110

A. Dickinson

the superposition sense proposed by Caflisch et al. [CMO97] states that ρk,N is small compared to V arΛM (f ). Typically, N is large and V arΛN (f ) ≈ V arP [Φ (X)] . Further, ρ1,N ≤ ρ0,N = V arΛN (f ) . Therefore, if M is a power of two (and N > M 2 ) we would expect ρM +1,N ρ1,N to have the same order of magnitude as M 2 . So, we would expect that the proportion of the variance of f explained by the ANOVA components   {fI : #I ≤ M + 1} to be at least 1 − M12 . For example, taking M = 8 would suggest that over 98% of the variance of f is explained by the ANOVA components {fI : #I ≤ 9}. So, Theorem 3 strongly suggests that the integrands one forms using the Brownian bridge discretization are effectively low-dimensional in the superposition sense, in turn this gives a partial explanation for the surprising efficacy of quasi-Monte Carlo methods when applied to the highdimensional integrands formed in connection with problems in mathematical finance. As remarked earlier, Condition 1 is a slight idealization. For several examples of interest in mathematical finance (for example, the pricing of an Asian option or the mortgage backed security problem considered in [PT95]) the associated functional, Φ, has points (paths) of non-differentiability, although the measure (with respect to the law of the Itˆ o diffusion, X) of these points of non-differentiability is zero. However, Theorem 3 allows us to give heuristic explanation for the efficacy of quasi-Monte Carlo methods to problems that “almost” satisfy Condition 1. Essentially, the property of a function N f : [0, 1] → R being effectively low-dimensional in the superposition sense relates to the fact that ⎞2 ⎤ ⎡⎛ ⎢⎜ ⎜ < EΛN ⎢ ⎣⎝f − P

⎟ ⎥ ⎥ f⎟ GI ⎠ ⎦

V arΛN (f )

#I≤k

for modest k (k < 10, say). If the functional of interest, Φ : C [0, T ] → C [0, T ], “almost” satisfies Condition 1 in that there exists Φ˜ : C [0, T ] → C [0, T ] satisfying Condition 1 with  2  ˜ EP Φ (X) − Φ (X) “small” and if f and f˜ are constructed from Φ and Φ˜ (respectively) using the Brownian bridge discretization using N = 2n steps in the Euler approximation : L2 (ΛN ) → L2 (ΛN ) is a then using the fact that the projection P < GI

#I≤k

linear contraction one may argue that

Explaining Effective Low-Dimensionality

⎞2 ⎤

⎡⎛ ⎢⎜ ⎜˜ < EΛN ⎢ ⎣⎝f − P #I≤k

111

⎞2 ⎤

⎡⎛

⎟ ⎥ ⎢⎜ ⎥ ≈ EΛN ⎢⎜f − P < f˜⎟ ⎠ ⎦ ⎣⎝ GI

⎟ ⎥ ⎥ f⎟ GI ⎠ ⎦

#I≤k

  f˜ ≈ V arΛN [f ] . So, one might expect the effective dimension of

and V arΛN f˜ to be comparable to the effective dimension of f . There remains a few issues worth mentioning. First of all, if the number of steps, N = 2n , in our Euler scheme is large (N > 100, say) then, even for modest k (k ≥ 5), there are an astronomical number of k-dimensional subfaces of the N -dimensional unit cube on which the integrand is defined. However, the proof of Theorem 3 reveals that only relatively few of these subfaces actually “matter”. Secondly, the proof of Theorem 3 implicitly depends on the fact that the diffusion coefficient of a one-dimensional Itˆo diffusion (trivially) satisfies the commutativity condition and the fact that Brownian motion has deterministic quadratic variation. Whether the integrand one obtains is effectively low-dimensional when one applies the Brownian bridge discretization to a functional of a multi-dimensional Itˆ o diffusion whose diffusion coefficient does not satisfy the commutativity condition is far from clear. For such diffusions, the Itˆo map (the mapping from the underlying Wiener process to the associated solution of the stochastic differential equation) may become pathological (see [LQ02]) and the integrands one obtains may potentially not be effectively low-dimensional.

References [AB97]

Acworth, P., Broadie, M., Glasserman, P.: A comparison of some Monte Carlo and quasi Monte Carlo techniques for option pricing, In: Niederreiter, H., Hellekalek, P., Larcher, P., Zinterhof, P. (ed) Monte Carlo and Quasi-Monte Carlo Methods 1996. Springer-Verlag, Berlin-New York, 1997. [CMO97] Caflisch, R. E., Morokoff, W. J., Owen, A. B., Valuation of mortgage backed securities using Brownian bridges to reduce effective dimension, Journal of Computational Finance, 1, 27–46, 1997 [JBT96] Joy, C., Boyle, P. P., Tan, K. S., Quasi-Monte Carlo methods in numerical finance, Management Science, 42, 926–938, 1996 [KP92] Kloeden, P. E., Platen, E., Numerical Solution of Stochastic Differential Equations. Springer-Verlag, Berlin-New York, 1992 [LO02] Lemieux, C., Owen, A. B.: Quasi-regression and the relative importance of the ANOVA components of a function, In: Fang, K., Hickernell, F. J., Niederreiter, H. (ed) Monte Carlo and Quasi-Monte Carlo methods 2000, Springer-Verlag. Berlin-New York, 2002 [LQ02] Lyons, T. J. and Qian, Z.: System Control and Rough Paths. Oxford University Press, Oxford, 2002

112 [MC96]

[MR97] [NT96] [Owe92] [Owe02] [PT95]

A. Dickinson Moskowitiz, B. and Caflisch,R. E., Smoothness and dimension reduction in quasi-Monte Carlo methods, Mathematical and Computer Modelling, 23, 37–54 (1996) Musiela, M., Rutkowski, M.: Mathematical Methods in Financial Modelling. Springer-Verlag, Berlin-New York, 1997 Ninomiya, S., Tezuka, S., Toward real-time pricing of complex financial derivatives, Appl. Math. Finance, 3, 241–273, 1996 Owen, A. B., Orthogonal arrays for computer experiments, integration and visualization, Statistica Sinica, 2, 439–452 (1992) Owen, A. B., Necessity of low effective dimension, working paper, Stanford University (2002) Paskov, S., Traub, J., Faster valuation of financial derivatives, Journal of Portfolio Management, 22, 113–120, 1995

Selection Criteria for (Random) Generation of Digital (0,s)-Sequences Henri Faure Institut de Math´ematiques de Luminy, UMR 6206 CNRS, 163 Av. de Luminy, case 907, 13288 Marseille cedex 9, France. [email protected] Summary. Digital (0, s)-sequences in an arbitrary prime base b ≥ s may be randomly scrambled in various ways. Simple and widely used scramblings are obtained by multiplying on the left the upper triangular generator matrices by nonsingular lower triangular (NLT) matrices whose entries are randomly chosen in the set of digits Zb = {0, 1, . . . , b − 1} [Tez94]. From our recent results [Fau05], we are able to propose subsets of Zb of various sizes for the random selection of the entries of the NLT matrices above. Moreover, since multiplications are permutations, our selection criteria are part of the general framework of Owen [Owe95] and may be applied to any kind of digital sequences, like Halton or Niederreiter sequences.

1 Introduction The selection criteria we propose are deduced from a recent study in one dimension [Fau05] and may be applied to each one-dimensional coordinate projection of any digital (0, s)-sequence in prime base [Fau82] (or even digital (t, s)-sequence in prime base [15]). Among the very general scramblings introduced by A. Owen [Owe95], simple and widely used scramblings are obtained by S. Tezuka [Tez94, Tez95] by multiplying on the left the upper triangular generator matrices by nonsingular lower triangular (NLT) matrices whose entries are randomly chosen in the set of digits {0, 1, . . . , b − 1} identified to Fb . For these special scramblings we propose to select subsets of Fb of various sizes for the random choice of the entries of the NLT matrices (eventually diagonal). Basically, these subsets are obtained in the following way: ◦ From our study, the extreme discrepancy and the diaphony of onedimensional sequences generated by nonsingular upper triangular (NUT) matrices are equal to those obtained with the diagonal matrix having the same diagonal. ◦ On the other hand, from our results on generalized van der Corput sequences [CF93, Fau81], we have good estimates for the extreme discrep-

114

H. Faure

ancy and the diaphony of sequences generated by diagonal matrices (these sequences are special cases of generalized van der Corput sequences). ◦ Therefore, by computing bounds for the extreme discrepancy and the diaphony of the generalized van der Corput sequences with permutations defined by digit multiplication, we obtain a classification of the digits f of Fb : the smaller the discrepancy (or diaphony) with the permutation defined by f , the better the multiplicative factor f for a row of the generator matrix of a coordinate of the digital (0, s)-sequence. After random multiplication of all rows of all generator matrices by good selected factors f , we obtain a scrambled (0, s)-sequence whose onedimensional coordinate projections have far better discrepancy (or diaphony) than the original ones generated by the powers of the Pascal matrix. For instance with b = 367, if we choose f at random in the 183 elements of the subset selected by means of the extreme discrepancy, we get discrepancies about 20 times less for the one-dimensional coordinate sequences (compared to Pascal power generator matrices). Since at present we only have results for one-dimensional sequences generated by NUT matrices, strictly speaking, only left multiplication by diagonal matrices with selected entries gives such gains; but we think the efficiency of random scramblings in many simulation experiments comes from the fact, among others, that they often avoid the use of bad digits like 1 and b − 1; if the randomization by NLT matrices is carried out from a subset of Fb selected by our methods, it should perform better since it avoids systematically a subset of worst digits. Sections 2 and 3 contain basic definitions and Sect. 4 the exact formulas for the discrepancy D and the diaphony F . The Sect. 5 is devoted to the classification of the digits by means of two figures of merit for D and F . Finally, some results of computations are given in Sect. 6 and a conclusion in Sect. 7.

2 Discrepancy and Diaphony Let X = (xn )n≥1 be an infinite sequence in [0, 1], N ≥ 1 an integer and [α, β[ an interval of [0, 1]; the error to ideal distribution is the difference E([α, β[; N ; X) = A([α, β[; N ; X) − N l([α, β[) where A([α, β[; N ; X) is the number of indices n such that 1 ≤ n ≤ N and xn ∈ [α, β[ and where l([α, β[) is the length of [α, β[. Definition of the extreme discrepancy: D(N, X) =

sup

|E([α, β[; N ; X)| .

0≤α<β≤1

Definition of the L2 -discrepancy and of the diaphony:

Selection Criteria



1

T (N, X) =



115

2  12 E([0, α[; N ; X) dα ,

0

N 2  12  ( ∞ 1 ( exp(2iπmxn ) . F (N, X) = 2 2 m n=1 m=1 The diaphony F and the L2 -discrepancy T are related by the formula of Koksma [KN74, Chap. 2, Lemma 2.8] , 2

T (N, X) =

( N  n=1

1 − xn 2

 2 +

1 F 2 (N, X) . 4π 2

 1 N ( 1 Note that ( − xn ) = E([0, α[; N ; X)dα; therefore, another defini2 0 n=1 tion for the diaphony could be  1 1   F 2 (N, X) = 2π 2 E 2 [α, β[; N ; X dα dβ , 0

0

that is, according to some authors, the extreme L2 -discrepancy as opposed to the usual (star) L2 -discrepancy.

3 Van der Corput Sequences, Digital (0,1)-Sequences, and Related Functions For integers n and N with n ≥ 1 and 1 ≤ N ≤ bn , write N −1=

∞ (

ar (N ) br

r=0

in the b-adic system (so that ar (N ) = 0 if r ≥ n) and let Σ = (σr )r≥0 be a sequence of permutations of {0, 1, . . . , b − 1}. Then the generalized van der Corput sequence SbΣ in base b associated with Σ is defined by   ∞ ( σr ar (N ) Σ · Sb (N ) = br+1 r=0 The digital (0, 1)-sequences in prime base b we consider can be simply described as follows: XbC = (xN )N ≥1 with

xN =

∞ ( xN,r r=0

br+1

in which xN,r =

∞ ( k=r

ckr ak (N ) ,

  the generator matrix C = ckr r≥0,k≥0 being an infinite nonsingular upper triangular (NUT) matrix with entries ckr ∈ Fb . From the definitions of the

116

H. Faure

ak (N ) and of C, the summations above are finite. In brief, we name these sequences NUT digital (0, 1)-sequences. Functions ϕσb,h related to a pair (b, σ). In the sequel, σ is a permutation of Fb identified as a set to {0, 1, . . . , b − 1}.  σ(0) σ(b − 1)  Set Zbσ = ,··· , . For any integer h with 0 ≤ h ≤ b − 1, the b b real function ϕσb,h is defined as follows: let k be an integer with 1 ≤ k ≤ b, then for every x ∈ [ k − 1 , k [ we set: b b ϕσb,h (x)

   h σ = A 0, ; k; Zb − hx b

if 0 ≤ h ≤ σ(k − 1)

and

   h , 1 ; k; Zbσ if σ(k − 1) < h < b; ϕσb,h (x) = (b − h)x − A b finally the function ϕσb,h is extended to the reals by periodicity. Note that ϕσb,0 = 0. In the very special case b = 2, we only have two permutations which give either ϕσ2,1 =  · , if σ is the identical permutation or ϕσ2,1 = − · , if σ = (0 1), where  ·  is the distance to the nearest integer function. Actually, the functions ϕσb,h give rise to other functions, depending only on (b, σ), according to the involved notion of discrepancy as we shall see in the next section.

4 Exact Formulas, Application We restrict to D and F , the only measures we need for the sequel, but we have also exact formulas for the star discrepancy and the L2 -discrepancy. 4.1 Results for Generalized van der Corput Sequences These results come from our preceding studies [CF93] and [Fau81]. Set ψbσ = and

max

0≤h
(

χσb =

|ϕσb,h − ϕσb,h |

(ϕσb,h − ϕσb,h )2 .

0≤h
Then, for all N ≥ 1, we have D(N, SbΣ ) =

∞ ( j=1

and

 σ

ψb j−1

N bj



Selection Criteria



∞ 1 2 1 ( σj−1 N Σ F (N, S ) = χ b 4π 2 b2 j=1 b bj

117

 .

In the series above the end of the summation is geometric, so we have finite sums for a given N (as we shall see in the next corollary). 4.2 Results for NUT Digital (0,1)-Sequences These results come from the article [Fau05] to appear. We recall that we deal only with sequences XbC in prime base b, generated by NUT matrices C. If the generator matrix C is diagonal, we have XbC = Sb∆ where ∆ = (δr )r≥0 is the sequence of permutations of Fb defined by δr (i) = crr i i.e. δr is the multiplication in Fb by the diagonal entry crr . Now, if C is not diagonal, the diagonal entries will still determine the same permutations δr , but the permutations σr that will give exact formulas for the star discrepancy and the L2 -discrepancy and T will be translated permutations, depending on N ≥ 1, of the δr ’s. On the contrary, for D and F , we have the remarkable property that digital (0,1)-sequences generated by NUT matrices having the same diagonal have the same extreme discrepancy and the same diaphony : Theorem 1. For all NUT matrices C and all integers N ≥ 1, we have D(N, XbC ) =

∞ (

 δ

ψb j−1

j=1

and

N bj



  ∞ 1 2 1 ( δj−1 N C F (N, X ) = χ · b 4π 2 b2 j=1 b bj

Corollary 1. For all integers n and N with 1 ≤ N ≤ bn , we have D(N, XbC ) =

n ( j=1

and

 δ

ψb j−1

N bj

 +

N bn

  n 1 2 1 ( δj−1 N N2 C F (N, X ) = χ · + b 12 b2n 4π 2 b2 j=1 b bj

This corollary is the starting point for the research of bounds and for the study of the asymptotic behaviour of D and F .

118

H. Faure

Corollary 2. Denote Pµ :=

  k r

µk−r

 r≥0,k≥0

the µ-th power (1 ≤ µ ≤ b−1)

of the Pascal matrix and let ∆Pµ be the matrix obtained by left multiplication with the diagonal matrix (crr ). Then, for all N ≥ 1, we have P

D(N, Xb µ ) =

∞ (

 ψbI

j=1

P F (N, Xb µ )

=

∞ (

 δ ψb j−1

j=1 ∆Pµ

 = D(N, SbI ) ,

  ∞ 4π 2 ( I N = 2 χ = F (N, SbI ) , b j=1 b bj

∆P D(N, Xb µ )

F (N, Xb

N bj

)=

N bj

 = D(N, Sb∆ ) ,

  ∞ 4π 2 ( δj−1 N χ = F (N, Sb∆ ) b2 j=1 b bj

with I the identical permutation and ∆ = (δr ) the sequence of multiplications by crr identified with the diagonal matrix ∆ = (crr ). Remark 1. This corollary applies also to one-dimensional projections of (t, s)sequences whose generator matrices are NUT and have only the digit 1 on the diagonal.

5 Estimates and Asymptotic Behaviour According to Corollary 1, if we want to obtain selection criteria for the entries n ( x g j of the matrix ∆ of Corollary 2, we have to study sums of the type b j=1 where g is a bounded, 1-periodic real function. In our preceding papers [CF93] and [Fau81], we have already focused on such sums and results are available, mainly for the discrepancy D (with g = ψbσ ) and, to a smaller extent, for the diaphony (with g = χσb ). We gather the necessary material in the two propositions below, the second one needing more development according to the preceding sentence. 5.1 Estimations for the Discrepancy D Proposition 1. For any integer n ≥ 1 and any permutation σ of Fb , set dσb (n)

= sup

n (

x∈R j=1

ψbσ

x b

j

and

dσb (n) · n≥1 n

αbσ = inf

Selection Criteria

119

dσb (n) and there exists βn with 0 ≤ βn ≤ 1 such that n→∞ n

Then αbσ = lim

dσb (n) = αbσ n + βn , so that 0 ≤ dσb (1) − αbσ ≤ 1 . Moreover, dσb (1) = sup ψbσ (x) = max

max

1≤k≤b 0≤h
x∈R

Remark 2. We recall that lim sup N →∞

D(N, Sbσ ) ≤

|E([h /b, h/b[; k; Zbσ )| .

ασ D(N, Sbσ ) = b and that log N log b

αbσ log N + αbσ + 2 for all N ≥ 1 . log b

Therefore, the quantity αbσ is a good quality parameter for estimating the discrepancy and according to Proposition 1, dσb (1) is a good approximation of αbσ as b increases. Another advantage of dσb (1) is that it is easy to compute contrary to αbσ which needs to manage with the huge functions ψbσ when b is large. Proof. These results are scattered in [Fau81] (Lemma 4.2.2 and Property 3.2.2 (ii)). We give here the guidelines for the sake of completeness and to prepare the proof of Proposition 2. m = qn + r with 0 ≤ r < n and breaking down the sum writing  1mFirst, σ x into q + 1 sums, we get dσb (m) ≤ qdσb (n) + dσb (r). Then, dividing ψ j=1 b bj dσ (n) · by m and getting m to infinity, we obtain αbσ = lim b n→∞ n σ σ Secondly, for the proof of db (n) = αb n + βn , we need a further property of the functions ψ due to their convexity (they are maxima of affine functions): for 1 ≤ J < K, max x∈R

( J j=1

ψbσ

 x  bj

+ max x∈R

 ( K j=J+1

ψbσ

 x  bj

− max x∈R

( K j=1

ψbσ

 x  bj

≤1,

see [Fau81, Lemma 4.2.1]. By iterative application of this property, we get dσ (n) − αbσ = dσb (m) = qdσb (n) + dσb (r) − sm,n with 0 ≤ sm,n ≤ q, so that b n 1 sm,n sm,n ≤ and βn = n lim is suitable. lim m→∞ m m→∞ m n σ Finally, the last property, i.e. db (1) is the maximum of the (so-called) discrete discrepancy of the net Zbσ , is also a consequence of the convexity  (and the continuity) of ψbσ . With Proposition 1, we are able to classify the multiplicative factors f to be used for the entries of the diagonal matrix ∆ of Corollary 2.

120

H. Faure

First, if we consider a NUT digital (0, 1)-sequence generated by a matrix C whose diagonal entries determine the sequence of permutations ∆ = (δr ) with dδbr (1) ≤ dδb (1) for some δ, we have (by Corollary 1 with bn−1 < N ≤ bn ) D(N, XbC ) = D(N, Sb∆ ) ≤

n (

dδ−j−1 (1) + 1 b

j=1

dδb (1) log N + dδb (1) + 1 . log b Then, to classify the multiplicative factors f for a given base b, it suffices to compute dfb (1) (where the permutation δ consisting in the multiplication by f is identified with f ) and arrange the results in increasing order. Finally, taking digits among the first c terms (1 ≤ c ≤ b − 1) of the list will give a constant less than dfb c (1), where fc is the digit at the place c. We have performed a program to compute and classify dfb (1), the maximum 0 f f (b − 1)  of the discrete discrepancy of the digital net Zbf = , ,··· , , for b b b all factors f = 1, . . . , b − 1 and any arbitrary prime base b. Examples of these computations are given in the next section. To end with D, we recall some formulas we obtained for the van der Corput sequences SbI which give the same discrepancy as sequences generated by NUT matrices having only the digit 1 on the diagonal: ≤ ndδb (1) + 1 ≤

dI2 (1) = 1/2 dIb (1) =

and α2I = 1/3 ; and for b ≥ 3

b−1 b−1 b−1 + and αbI = , so that 4 4b 4 dI3 (1) = 2/3 and α3I = 1/2 ,

dI5 (1) = 6/5 and α5I = 1 , 33672 366 I ≈ 91.75 and α367 = 91.5 . = dI367 (1) = 367 4 In these examples, we see that dIb (1) is really a good approximation of αbI especially for large b. 5.2 Estimations for the Diaphony F The functions χ are more difficult to handle than the functions ψ and contrary to the study for D, we only have the first part of Proposition 1 from our study for F [CF93]. So, we give a lemma before we state Proposition 2. Lemma 1. For 1 ≤ J < K, max x∈R

( J j=1

χσb

 x  bj

+ max x∈R

 ( K j=J+1

χσb

 x  bj

− max x∈R

( K j=1

χσb

 x  bj

≤ 5b3 .

Selection Criteria

121

Proof. The proof is similar to that for ψ ( [Fau81, Lemma( 4.2.1]) and needs to σ bound the derivative of χ. For this, recall that χb = (ϕσb,h − ϕσb,h )2 0≤h
and that ϕσb,h has the form ϕσb,h (x) = µh x + νh on each interval µh and νh integers such that |µh | ≤ b − 1 and |νh | ≤ b − 1. Moreover, the leading term of the trinomial is b2 (b2 − 1) 3.5]). Consequently, (χσb ) (x) = x+2 6



k−1 k b , b

 , with

b2 (b2 − 1) ( [CF93, Property 12 ( (µh −µh )(νh −νh )

0≤h
and since

b(b − 1) b2 (b2 − 1) 25b3 + b2 +2 4(b − 1)2 ≤ (b − 1) , 6 2 6

we obtain

|(χσb ) (x)| < 5b3 (b − 1) . Then, we claim the lemma results 1J fromthe following three properties: ◦ the real function F (x) = j=1 χσb bxj is bJ -periodic,   1K ◦ the real function G(x) = j=J+1 χσb bxj is bK -periodic and convex on intervals of the form [kbJ , (k + 1)bJ ], ◦ the derivative of G is bounded by 5b3−J (since 5b3 (b − 1)(b−J−1 + · · · + −K b ) = 5b3 (b−J − b−K ) < 5b3−J ). Indeed, from the convexity of G on [kbJ , (k + 1)bJ ] and from  the bound on its derivative, we get max G := max[kbJ ,(k+1)bJ ] G(x) = max G(kbJ ), G((k +  1)bJ ) and G(x) ≥ max G − 5b3−J × bJ , so that for all x ∈ [kbJ , (k + 1)bJ ], we have max G − 5b3 + F (x) ≤ G(x) + F (x) ≤ max(F + G). Therefore max F + max G − 5b3 ≤ max(F + G) and the difference between  sum of the maxima and maximum of the sum is at most 5b3 . Proposition 2. For any integer n ≥ 1 and any permutation σ of Fb , set   n ( x θσ (n) 1 . χσb j and γbσ = inf b θbσ (n) = 2 sup n≥1 b x∈R j=1 n b θbσ (n) and there exists βn with 0 ≤ βn ≤ 5b such that n→∞ n

Then γbσ = lim

θbσ (n) = γbσ n + βn , so that 0 ≤ θbσ (1) − γbσ < 5b . Moreover, θbσ (1) =

 F 2 (N, S σ ) 1 N2  σ b . sup χ (x) = max − b 1≤N ≤b b2 x∈R 4π 2 12b2

Remark 3. The interest of writing θbσ (1) by means of F is that the diaphony is easy to compute via the formula of Koksma and the L2 -discrepancy. But according to the last inequality, the approximation of γbσ by θbσ (1) seems not good; this is due to the rough bounds we used for the derivative of χ in Lemma 1. The results for σ = I below show a better behaviour.

122

H. Faure

Proof. The proof of the first part is quite similar to that of the first part of Proposition 1, with Lemma 1 instead of Lemma 4.2.1 of [Fau81]. The second part is a consequence of Corollary 1, with n = 1 and the permutation σ instead k of δ0 : indeed, χ is convex on intervals of the form [ k−1 b , b ] and so its maximum is reached at the ends of such intervals.  As for D, with any NUT digital (0, 1)-sequence generated by a matrix C whose diagonal entries determine the sequence of permutations ∆ = (δr ), with θbδr (1) ≤ θbδ (1) for some δ, we get F 2 (N, XbC ) F 2 (N, Sb∆ ) ( δj−1 1 = ≤ θb (1) + 2 2 4π 4π 12 j=1 n

≤ nθbδ (1) +

θδ (1) 1 1 ≤ b log N + θbδ (1) + · 12 log b 12

Therefore, as we did for the extreme discrepancy, re-ordering the values of θbf (1) in increasing order gives a new classification of the multiplicative factors f for a given prime b by means of the diaphony. Finally, we recall some values for θ and γ obtained in [CF93]: θ2I (1) = 1/16 θbI (1) =

and γ2I = 1/36 ; and for b ≥ 3

b4 + 2b2 − 3 b2 − 1 + 192 b2 48 b2

and γbI =

b4 + 2b2 − 3 . 192 b2

6 Computations In this section, see Tables 1 and 2, we give the results of our computations of dfb (1) and θbf (1) for few prime bases b in order to shorten the paper. For b = 367, we have kept only the 16 best and worst factors. Reals are rounded to 3 decimals.

7 Conclusion The digital (0, s)-sequences in prime bases b ( [Fau82]) are generated by powers of Pascal matrices whose diagonal entries are all equal to f = 1. In the same way, consider digital s-dimensional sequences in base 2 : the coordinate projections which are generated by NUT matrices (for instance the Sobol’ ones) have the same discrepancy and the same diaphony as the van der Corput sequence corresponding to f = 1: 1 α2I = ≈ 0.48 log 2 3 log 2

bounded by

dI2 (1) 1 = ≈ 0.72 for D . log 2 2 log 2

Selection Criteria

123

Table 1. Bases 2 to 11 Discrepancy

Diaphony

base place f bdfb (1)

f

bdb (1) log b

base place f 12 b2 θbf (1)

f

θb (1) log b

2

1

1

1

0.721

2

1

1

3

0.090

3

1 2

1 2

2 2

0.606 0.606

3

1 2

1 2

8 8

0.067 0.067

5

1 2 3 4

2 3 1 4

4 4 6 6

0.497 0.497 0.745 0.745

5

1 2 3 4

2 3 4 1

24 24 48 48

0.049 0.049 0.099 0.099

7

1 2 3 4 5 6

2 3 4 5 1 6

8 8 8 8 12 12

0.587 0.587 0.587 0.587 0.881 0.881

7

1 2 3 4 5 6

2 3 4 5 1 6

72 72 72 72 168 168

0.062 0.062 0.062 0.062 0.146 0.146

11

1 3 2 4 3 7 4 8 5 2 6 5 7 6 8 9 9 1 10 10

14 14 14 14 18 18 18 18 30 30

0.530 0.530 0.530 0.530 0.682 0.682 0.682 0.682 1.137 1.137

11

1 4 2 7 3 8 4 3 5 6 6 2 7 5 8 9 9 1 10 10

216 216 216 216 312 312 312 312 960 960

0.062 0.062 0.062 0.062 0.089 0.089 0.089 0.089 0.275 0.275

The two classifications we have given show the dramatic bad quality, at least, of f = 1 and b − 1 which correspond to the identical permutation, that is to the classical NUT digital (0, 1)-sequences (Sobol’, Faure and as far as we know some of Niederreiter-Xing). Of course, when b = 2 or 3, it’s of no importance since 1 and b − 1 are the only nonzero digits, but for sequences in base b > 3, the larger the base, the worse the quality of the sequence with f = 1 or b − 1. This is probably the (main) reason of the relatively bad behaviour of our sequences compared to the scrambled ones (GFaure) for which the random scrambling avoids inevitably the digits 1 and b − 1 rather often. On the other hand, our classifications show that there always exist very good multipliers which give upper bounds still better than the upper bounds for the base 2 (≈ 0.72 for D) and often very close to the exact value of the NUT digital (0, 1)-sequences in base 2 (≈ 0.48 still for D).

124

H. Faure Table 2. Base 367 Discrepancy base place f bdfb (1) 367

Diaphony f

bdb (1) log b

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16

97 140 227 270 155 161 206 212 112 154 213 255 101 109 258 266

1088 1088 1088 1088 1110 1110 1110 1110 1142 1142 1142 1142 1156 1156 1156 1156

0.502 0.502 0.502 0.502 0.512 0.512 0.512 0.512 0.526 0.526 0.526 0.526 0.533 0.533 0.533 0.533

351 352 353 354 355 356 357 358 359 360 361 362 363 364 365 366

220 362 4 92 275 363 3 112 245 364 2 183 184 365 1 366

7028 7028 8648 8648 8648 8648 11468 11468 11468 11468 16928 16928 16928 16928 33672 33672

3.242 3.242 3.990 3.990 3.990 3.990 5.291 5.291 5.291 5.291 7.810 7.810 7.810 7.810 15.536 15.536

base place f 367

f

12 b2 θbf (1)

θb (1) log b

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16

155 161 206 212 112 154 213 255 99 152 215 268 101 109 258 266

503352 503352 503352 503352 508200 508200 508200 508200 515136 515136 515136 515136 530856 530856 530856 530856

0.052 0.052 0.052 0.052 0.053 0.053 0.053 0.053 0.054 0.054 0.054 0.054 0.055 0.055 0.055 0.055

351 352 353 354 355 356 357 358 359 360 361 362 363 364 365 366

5 220 4 92 275 363 3 122 245 364 2 183 184 365 1 366

45516816 45516816 71053992 71053992 71053992 71053992 126120672 126120672 126120672 126120672 283543632 283543632 283543632 283543632 1133870928 1133870928

4.768 4.768 7.444 7.444 7.444 7.444 13.213 13.213 13.213 13.213 29.707 29.707 29.707 29.707 118.796 118.796

Selection Criteria

125

For random scrambling, there is a large choice with many possibilities: select a small set with good bounds or select a large set with more or less loose bounds; but avoiding the worst digits seems a good strategy in any case. In our eyes, implementations could be performed and tested in at least three situations (in the terminology of J. Matouˇsek [Mat99]) : ◦ Random linear digit-scrambling, but without additive term, i.e. left multiplication by diagonal matrices with entries (randomly) chosen in a selected set of nonzero multiplicative factors. ◦ Random linear scrambling, but without additive term (from S. Tezuka [Tez94]), i.e. left multiplication by NLT matrices with entries (randomly) chosen in a selected set of nonzero multiplicative factors, or alternatively including zero in the selected set for the nondiagonal entries. ◦ The two preceding ones with an additive term as proposed by J. Matouˇsek. Next, more specific scramblings should be considered (see [FT02] and [TF03]). Finally, since multiplications are permutations, our selection criteria are applicable to any kind of (random) scramblings for any kind of digital sequences like Halton or Niederreiter sequences. In relation with that, we refer to our paper [Fau92] where permutations are constructed and compared to other ones from Braaten and Weller [BW79]: the permutations resulting from our best factors f are better than the Braaten and Weller ones and comparable to our construction for b < 20. It should be interesting to continue the comparison for larger b by computing the permutations and the discrepancy for these constructions.

References [BW79] Braaten, E., Weller, G.: An improved low-discrepancy sequence for multidimensional quasi-Monte Carlo integration. J. Comput. Phys., 33, 249–258 (1979). [CF93] Chaix, H., Faure, H.: Discr´epance et diaphonie en dimension un. Acta Arith., 63, 103–141 (1993). [Fau81] Faure, H.: Discr´epance de suites associ´ees ` a un syst`eme de num´eration (en dimension un). Bull. Soc. Math. France, 109, 143–182 (1981). [Fau82] Faure, H.: Discr´epance de suites associ´ees ` a un syst`eme de num´eration (en dimension s). Acta Arith., 41, 337– 351 (1982). [Fau92] Faure, H.: Good permutations for extreme discrepancy. J. Number Theory, 41, 47–56 (1992). [Fau05] Faure, H.: Discrepancy and diaphony of digital (0,1)-sequences in prime base. Acta Arith. (to appear). [FT02] Faure, H., Tezuka, S.: Another random scrambling of digital (t, s)sequences. In: Fang, K.T., Hickernell, F.J., Niederreiter, H. (eds.) Monte Carlo and Quasi-Monte Carlo Methods 2000. Springer, Berlin Heidelberg New York (2002). [KN74] Kuipers, L., Niederreiter, H.: Uniform Distribution of Sequences. Wiley, New York (1974).

126

H. Faure

[Mat99] Matouˇsek, J.: Geometric Discrepancy: An Illustrated Guide. Springer, Berlin Heidelberg New York (1999). [Nie92] Niederreiter, H.: Random Number Generation and Quasi-Monte Carlo Methods. CBMS-NSF Regional Conference Series in Applied Mathematics. SIAM, Philadelphia (1992). [Owe95] Owen, A.B.: Randomly permuted (t, m, s)-nets and (t, s)-sequences. In: Niederreiter, H., Shiue, P. (eds.) Monte Carlo and Quasi-Monte Carlo Methods in Scientific Computing. Springer, Berlin Heidelberg New York (1995). [Tez94] Tezuka, S.: A generalization of Faure sequences and its efficient implementation. Research Report IBM RT0105 (1994). [Tez95] Tezuka, S.: Uniform Random Numbers: Theory and Practice. Kluwer Academic Publishers, Boston (1995). [TF03] Tezuka, S., Faure, H.: I-binomial scrambling of digital nets and sequences. J. Complexity, 19, 744–757 (2003).

Imaging of a Dissipative Layer in a Random Medium Using a Time Reversal Method Jean-Pierre Fouque1 , Josselin Garnier2 , Andr´e Nachbin3 , and Knut Sølna4 1

2

3

4

Department of Mathematics, North Carolina State University, Raleigh NC 27695-8205 [email protected] Laboratoire de Probabilit´es et Mod`eles Al´eatoires & Laboratoire Jacques-Louis Lions, Universit´e Paris VII, 2 Place Jussieu, 75251 Paris Cedex 5, France [email protected] Instituto de Matem´ atica Pura e Aplicada, Est. D. Castorina 110, Jardim Botˆ anico, Rio de Janeiro, RJ 22460-320, Brazil [email protected] Department of Mathematics, University of California, Irvine CA 92697 [email protected]

Summary. In this paper time reversal of acoustic waves in a dissipative random one-dimensional medium is analyzed. It is shown that time reversal can be used as an efficient and statistically stable method to image a dissipative layer embedded in a random scattering medium. The quantities needed to achieve this goal appear as the solutions of a system of transport equations which are solved by a Monte Carlo method.

1 Introduction Time-reversal refocusing for waves propagating in inhomogeneous media have been recently observed and studied experimentally in various contexts, e.g. ultrasound, underwater acoustics, see for instance the review [Fin99]. Important potential applications have been proposed in various fields, for instance imaging [PKC02, FMT03] and communication [DTR03]. A time-reversal mirror is, roughly speaking, a device which is capable of receiving a signal in time, keeping it in memory and sending it back into the medium in the reversed direction of time. The main effect is the refocusing of the scattered signal after time-reversal in a random medium. Surprisingly, the refocused pulse shape only depends on the statistical properties of the random medium, and not on the particular realization of the medium. The full mathematical understanding, meaning both modeling of the physical problem and derivation of the time-reversal effect, is a complex problem. The study of the one-dimensional linear case is now well understood [CF97, Sol03, FS03] as well as the threedimensional waves in the parabolic or paraxial regime [BPR02,BPZ02,PRS03].

128

J.-P. Fouque et al.

In this paper we generalize the model considered so far by introducing absorption modeled by a linear dissipative term in the acoustic equations. We show that time-reversal is still efficient in refocusing waves despite the loss of energy due to absorption. The paper is organized as follows. In Sections 2-4 we derive integral representations of the refocused pulse shapes in terms of reflection coefficients for the wave modes. Section 5 is devoted to the thorough analysis of timereversal experiments in reflection in random and dissipative media. In Sect. 6 we use a probabilistic interpretation of the limit solution to analyze different configurations by Monte Carlo (MC) simulations.

2 The Acoustic Model We consider the acoustic wave equation 1 ∂p ∂u + =0, K ∂t ∂z ∂u ∂p + + σu = 0 , ρ ∂t ∂z

(1) (2)

where p is the pressure and u is the velocity, z and t are the space and time coordinates, respectively, σ is the dissipation of the medium, ρ is the density, and K is the bulk modulus. The fluctuations of the medium parameters are described by z

1 + ν 2 if − L < z < 0 , (3) ρ= ε 1 if z > 0 or z < −L , z

1 1 + η 2 if − L < z < 0 , = (4) ε K 1 if z > 0 or z < −L ,

 z σ z, 2 if − L < z < 0 , (5) σ= ε 0 if z > 0 or z < −L . The dimensionless small parameter ε2 characterizes the ratio of the correlation radius of the fluctuations of the medium and the typical size of the medium. The fluctuations of the density and the bulk modulus are modeled by the zero-mean stationary random process (η, ν)(z). The processes η and ν are assumed to be bounded by a deterministic constant less than 1 and to have strong ergodic properties. We may think for instance that (η, ν) is a function of a Markov, stationary, ergodic process on a compact space satisfying the Fredholm alternative [Pap78]. We can also work with the class of the φ-mixing processes with φ ∈ L1/2 [Kus84]. The dissipation coefficient is a nonnegative function. It may vary rapidly at scale ε2 , but it may also have slow variations at scale one. More exactly, we assume that, for any z ∈ [−L, 0], ζ → σ(z, ζ)

Imaging of a Dissipative Layer in a Random Medium

129

is a nonnegative stationary mixing process with mean σ ¯ (z) = E[σ(z, ζ)]. We think in particular at the two following configurations. 1. The dissipation coefficient has a stationary statistical distribution (this will be our reference framework):  z z σ z, 2 = σ0 2 , ε ε where σ0 is a nonnegative stationary ergodic random process with mean σ ¯0 = E[σ0 (ζ)]. 2. The dissipation coefficient has a different behavior inside some embedded layer [z0 , z1 ], −L < z0 < z1 < 0, ⎧ z  z  ⎨ σ1 if z0 < z < z1 ,  εz2  σ z, 2 = ⎩ σ0 ε if z1 ≤ z ≤ 0 or − L ≤ z ≤ z0 , ε2 where σ0 and σ1 are two nonnegative stationary ergodic random processes. We denote by σ ¯j their respective means σ ¯j = E[σj (ζ)] , and we assume that σ ¯1 = σ ¯0 . One goal of the paper is to detect the layer [z0 , z1 ]. We consider the problem on the finite slab −L ≤ z ≤ 0 where boundary conditions are imposed at −L and 0 corresponding to a left-going pulse entering the slab from the right at z = 0. We consider an incoming pulse whose typical wavelength is of order ε, that is to say small compared to the size of the medium, but large with respect to the correlation length of the random medium:   t f ε (t) = f . ε

3 Propagator Formulation In this section we first express the scattering problem as a two point boundary value problem in the frequency domain, and then rewrite it as an initial value problem in terms of the propagator. This study follows the lines of the analysis carried out in [AKP91]. 3.1 Mode Propagation in the Frequency Domain We consider the random acoustic equation (1-2) and take the scaled time Fourier transform

130

J.-P. Fouque et al.

  iωt iωt 1 1 u(t, z)e− ε dt , pˆε (ω, z) = p(t, z)e− ε dt , 2πε 2πε so that the system reduces to a set of ordinary differential equations:  z  ∂u ˆε iω  + pˆε = 0 , 1+η 2 ∂z ε ε  z  ∂ pˆε iω  u ˆε + σ u + 1+ν 2 ˆε = 0 . ∂z ε ε We decompose the wave into right-going modes aε and left-going modes bε iωz 1 ε p (ω, z) + u aε (ω, z) = (ˆ ˆε (ω, z)) e ε , (6) 2 iωz 1 ε p (ω, z) − u bε (ω, z) = (ˆ ˆε (ω, z)) e− ε , (7) 2 which satisfy the linear equation  ε   ∂ aε a ε (ω, z) (ω, z) = Q (ω, z) , (8) bε ∂z bε u ˆε (ω, z) =

where the complex 2 × 2 matrix Qε is given by:   2iωz  2iωz  iω σ ε (z) −mε (z) −nε (z)e ε −1 e ε ε Q (ω, z) = + , (9) 2iωz 2iωz 2 nε (z)e− ε 2 mε (z) −e− ε 1 with m(z) = η(z) + ν(z), 1 z mε (z) = m 2 , ε ε

n(z) = η(z) − ν(z), (10) z  z 1 nε (z) = n 2 , σ ε (z) = σ z, 2 . (11) ε ε ε

3.2 Boundary Values We assume that a left-going pulse is incoming from the right and is scattered into a reflected wave at z = 0 and a transmitted wave at z = −L (see Fig. 1). The incoming pulse shape is given by the pressure function f (t/ε) where f is assumed to be a L1 function compactly supported in the Fourier domain:   iωt iωt ˆ ε uinc (t, z = 0) = − f (ω)e dω, pinc (t, z = 0) = fˆ(ω)e ε dω, where fˆ is the standard Fourier transform  1 fˆ(ω) = f (t)e−iωt dt . 2π We also impose a radiation condition at −L corresponding to the absence of right-going wave at the left hand-side of the slab [−L, 0]. The two-point boundary value problem consisting of the system (8) for z ∈ [−L, 0] together with the conditions: bε (ω, z = 0) = fˆ(ω), aε (ω, z = −L) = 0 is then well-posed.

Imaging of a Dissipative Layer in a Random Medium

131

(uinc , pinc )(t)

0 (utr , ptr )(t)

(uref , pref )(t)

−L

0

z

Fig. 1. Scattering problem.

3.3 Propagator It is convenient to transform the two-point boundary value problem into an initial value problem by introducing the propagator Y ε (ω, −L, z) which is a complex 2 × 2 matrix solution of ∂Y ε (ω, −L, z) = Qε (ω, z)Y ε (ω, −L, z), ∂z 

such that Y ε (ω, −L, z)

aε (ω, −L) bε (ω, −L)

Y ε (ω, −L, z = −L) = IdC2 ,



 =

aε (ω, z) bε (ω, z)

 .

The propagator matrix Y ε has the form  ε  a1 (ω, z) aε2 (ω, z) ε Y (ω, −L, z) = , bε1 (ω, z) bε2 (ω, z) where the column vector (aε1 , bε1 )T , resp. (aε2 , bε2 )T , is solution of equation (8) with the initial conditions: aε1 (ω, z = −L) = 1, aε2 (ω, z = −L) = 0,

bε1 (ω, z = −L) = 0, bε2 (ω, z = −L) = 1 ,

(12) (13)

respectively. We can now define the transmission and reflection coefficients T ε (ω, −L, z) and Rε (ω, −L, z), respectively, for a slab [−L, z] by (see also Fig. 2):

0

1

T ε (−L, z)

Rε (−L, z)

L

z

0

Fig. 2. Reflection and transmission coefficients.

132

J.-P. Fouque et al.

 Y ε (ω, −L, z)

0 T ε (ω, −L, z)



 =

Rε (ω, −L, z) 1

 .

(14)

In terms of the propagator entries they are given given by: Rε (ω, −L, z) =

aε2 (ω, z), bε2

T ε (ω, −L, z) =

1 (ω, z) , bε2

(15)

and the reflection coefficient satisfies the closed form nonlinear Riccati differential equation: ∂Rε iω 2iωz iω 2iωz = −iωmε (z)Rε − e− ε nε (z)(Rε )2 − e ε nε (z) ∂z 2 2   2iωz ε 2iωz (z) σ e ε + e− ε (Rε )2 , −σ ε (z)Rε + 2

(16)

with the initial condition at z = −L: Rε (ω, −L, z = −L) = 0. The reflected wave at time t, denoted by (uref , pref ), is a right-going wave whose pressure field admits the following integral representation in terms of the reflection coefficient:  iωt (17) pref (t, z = 0) = fˆ(ω)Rε (ω, −L, 0)e ε dω. Remark: We can give a rapid comment on the role of dissipation. The 2   vectors satisfy ∂z |aεj |2 − |bεj |2 = −σ ε (z) aεj e−iωz/ε − bεj eiωz/ε and thus |aε2 |2 (z) + 1 ≤ |bε2 |2 (z). By (15) this implies the relation expressing the dissipation of energy (18) |Rε |2 + |T ε |2 ≤ 1. A strict inequality means that some energy is lost during the scattering of the wave by the random dissipative slab. Equation (18) also implies the uniform boundedness of the reflection coefficient.

4 Time Reversal In the first step of the time reversal procedure, the time-reversal mirror (TRM) is used as a passive device to record the reflected signal at z = 0 up to a certain time. It turns out that as ε → 0 the interesting asymptotic regime arises when the signal is recorded up to a large time of order 1 which we denote by t1 with t1 > 0. Thus, the recorded signal is of the form prec (t) = pref (t)Gt1 (t) , where the the support of the cut-off function Gt1 is included in [0, t1 ] and pref is given by (17). In the second step the TRM is used as an active device: it time reverses the recorded signal and sends it back into the medium. Observe that the part of the wave which came in last will go out first. It also changes

Imaging of a Dissipative Layer in a Random Medium

133

the sign of the recorded velocity so as to generate a pure left-going wave. Thus, the time-reversed pressure wave sent back into the medium is given by: pinc(T R) (t, z = 0) = prec (t1 − t)      iω(t1 −t) 1 ˆ t ω − ω dω  dω , = e ε pˆεref (ω  )G 1 ε ε ˆ t is the Fourier transform of Gt . where TR stands for “Time Reversal” and G 1 1 Using the fact that pinc(T R) is real valued, we can also write:     iω(t−t1 ) ω − ω 1  ˆ ε ε pˆref (ω )Gt1 pinc(T R) (t, z = 0) = e dω  dω . ε ε This new incoming signal re-propagates into the same medium and generates a new reflected signal which we observe at the time t2 + εt, that is, around the time t2 in the scale of the initial pulse. In terms of the reflection coefficients Rε the observed reflected signal is given by  iωt2 pref (T R) (t2 + εt, z = 0) = pˆεinc(T R) (ω)Rε (ω, −L, 0)e ε +iωt dω . Substituting the expression of pˆεinc(T R) into this equation yields the following representation of the reflected signal:      iω(t2 −t1 ) 1 ˆt ω − ω ε pref (T R) (t2 + εt, z = 0) = eiωt e fˆ(ω  )G 1 ε ε ×Rε (ω, −L, 0)Rε (ω  , −L, 0)dω  dω . ˆ t we get by the change of variable Motivated by the scaled argument of G 1 ω  = ω − εh:   iω(t2 −t1 ) ˆ t (h) ε pref (T R) (t2 + εt, z = 0) = eiωt e fˆ(ω − εh)G 1 ×Rε (ω, −L, 0)Rε (ω − εh, −L, 0)dh dω . (19) The rapid phase exp(iω(t2 − t1 )/ε) averages out this integral except when t2 = t1 . This means that refocusing can be observed only at the time t2 = t1 . The precise description of the refocused pulse will be carried out in the next section.

5 The Refocused Pulse The refocused pulse observed at z = 0 and at time t1 + εt is given by   ˆ t (h) pref (T R) (t1 + εt) = eiωt eiεht/2 fˆ(ω − εh/2)G 1 ×Rε (ω + εh/2, −L, 0)Rε (ω − εh/2, −L, 0)dh dω .

(20)

This section is devoted to the proof of the convergence of the refocused pulse shape to a deterministic shape as ε → 0.

134

J.-P. Fouque et al.

5.1 Tightness We first address the tightness of the process.   Lemma 1. The refocused pulse (pεref (T R) (t1 + εt))−∞
is a tight ε>0

(i.e. weakly compact) family in the space of continuous trajectories equipped with the sup norm. Proof. We must show that, for any δ > 0, there exists a compact subset K of the space of continuous bounded functions such that: sup P(pεref (T R) (t1 + ε·) ∈ K) ≥ 1 − δ ε>0

On the one hand the dissipation relation (18) yields that |Rε | ≤ 1 and pεref (T R) (t1 + εt) is uniformly bounded by:   ˆ t (h)|dh |pεref (T R) (t1 + εt)| ≤ |fˆ(ω)|dω × |G (21) 1 On the other hand the modulus of continuity M ε (δ) =

sup |s1 −s2 |≤δ

|pεref (T R) (t1 + εs1 ) − pεref (T R) (t1 + εs2 )|

is bounded by  M ε (δ) ≤

 sup |s1 −s2 |≤δ

|1 − exp(iω(s1 − s2 ))||fˆ(ω)|dω ×

ˆ t (h)|dh |G 1

which goes to zero as δ goes to zero uniformly with respect to ε. 5.2 Convergence of the Finite-dimensional Distributions The uniform boundedness (21) implies that the finite-dimensional distributions of the process pεref (T R) (t1 + ·) will be characterized by the moments E[pεref (T R) (t1 + εs1 )p1 . . . pεref (T R) (t1 + εsk )pk ]

(22)

for every real numbers s1 < . . . < sk and every integers p1 , . . . , pk . First Moment Let us first address the first moment. Using the representation (20) the expectation of pεref (T R) (t1 + εt) reads:   ˆ t (h) E[pref (T R) (t1 + εt)] = eiωt eiεht/2 fˆ(ω − εh/2)G 1 ×E[Rε (ω + εh/2, −L, 0)Rε (ω − εh/2, −L, 0)]dh dω . (23)

Imaging of a Dissipative Layer in a Random Medium

135

As shown by (23), the statistical distribution of the refocused pulse will depend on the frequency autocorrelation function of the reflection coefficient. We denote:     εh εh ε ε ε , −L, z R ω − , −L, z , U1,1 (ω, h, z) = R ω + 2 2 and we extend the approach developed in [AKP91]. It is necessary to consider a family of moments so as to get a closed system of equations. We introduce for q, p ∈ N q   p   εh εh ε Uq,p , −L, z , −L, z (ω, h, z) = Rε ω + Rε ω − . (24) 2 2 Using the Riccati equation (16) satisfied by Rε , we deduce ε  ∂Uq,p 2iωz ω  ε ε ε = iω(p − q)mε Uq,p −qnε eihz Uq−1,p + ie ε + pnε e−ihz Uq,p+1 ∂z 2  ω  ε ihz ε − 2iωz ε ε pn e Uq,p−1 − qnε e−ihz Uq+1,p +ie 2 ε   2iωz σ ε ε ε qeihz Uq−1,p −(p + q)σ ε Uq,p +e ε + pe−ihz Uq,p+1 2 ε   2iωz σ ε ε peihz Uq,p−1 , (25) + qe−ihz Uq+1,p +e− ε 2 ε starting from Uq,p (ω, h, z = −L) = 10 (q)10 (p), where 10 (q) = 1 if q = 0 and 0 otherwise. Taking a shifted scaled Fourier transform with respect to h  1 ε ε (ω, τ, z) = (ω, h, z)dh , (26) Vq,p eih(τ −(q+p)z) Uq,p 2π

we get ε ε  ∂Vq,p ∂Vq,p ω 2iωz  ε ε ε = −(q + p) + iω(p − q)mε Vq,p + i e ε nε −qVq−1,p + pVq,p+1 ∂z ∂τ 2  ω ε ε − 2iωz ε ε ε n pVq,p−1 − qVq+1,p − (p + q)σ ε Vq,p +ie 2 ε  ε    2iωz σ 2iωz σ ε ε ε ε qVq−1,p pVq,p−1 + e− ε , (27) +e ε + pVq,p+1 + qVq+1,p 2 2 ε starting from Vq,p (ω, τ, z = −L) = δ0 (τ )10 (q)10 (p). A diffusion-approximation ε converge to diffutheorem [AKP91, Sect. 3] establishes that the processes Vq,p ε (ω, τ, z)], p ∈ N, sion processes as ε → 0. In particular the expectations E[Vp,p converge to Wp (ω, τ, z) which obey the closed system of transport equations:

∂Wp ∂Wp + 2p = (Lω W )p − 2¯ σ (z)pWp , ∂z ∂τ 1 (Lω φ)p = αn ω 2 p2 (φp+1 + φp−1 − 2φp ) , 2

(28) (29)

136

J.-P. Fouque et al.

starting from Wp (ω, τ, z = −L) = δ0 (τ )10 (p). The parameters αm and αn are the integrated covariances of the processes m and n:  ∞  ∞ αm = E[m(0)m(z)]dz, αn = E[n(0)n(z)]dz. (30) 0

0

We then get the limit of the autocorrelation function of the reflection coefficient:    εh εh ε→0 E Rε (ω + , −L, 0)Rε (ω − , −L, 0) −→ W1 (ω, τ, 0)e−ihτ dτ. (31) 2 2 The quantity W1 (ω, τ, 0) is obtained through the system of transport equations (28). Substituting this limit into (23) yields    ε→0 ˆ t (h)W1 (ω, τ, 0)dhdτ dω E[pref (T R) (t1 + εt)] −→ eiωt e−ihτ fˆ(ω)G 1   = eiωt fˆ(ω)Gt1 (τ )W1 (ω, τ, 0)dτ dω . (32) Remark 1. The existence and uniqueness of the solution z → W· (·, ·, z) to the transport equations (28) can be rigorously established in the space   ) where SH is the space of distributions introduced in [PW94] to C([−L, 0], SH  can be identified as the study an analogous system without absorption. SH dual of the space SH of the test functions λ = (λp (ω, τ ))p∈N,ω∈R,τ ∈R , where the λp ’s are infinitely differentiable in ω and τ and are rapidly decaying as functions of ω, τ , and p. Higher Order Moments Let us now consider the general moment (22). Using the representation (20) for each factor pεref (T R) (t1 + εsj ), these moments can be written as multiple 1k integrals over p = j=1 pj frequencies: ⎡ E⎣

k 

⎤ pεref (T R) (t1 + εsj )pj ⎦ =

j=1

×





 ...

⎡ ⎢ E⎢ ⎣

⎤  1 ≤ j ≤ k 1 ≤ l ≤ pj

⎥ ε U1,1 (ωj,l , hj,l , 0)⎥ ⎦

ˆ t (hj,l )dωj,l dhj,l . fˆ(ωj,l )eiωj,l sj eiεhj,l sj /2 G 1

1 ≤ j ≤ k 1 ≤ l ≤ pj

2  ε The important quantity is E U (ω , h , 0) . Our problem is now to j,l j,l j,l 1,1 find the limit, as ε goes to 0, of these moments for k distinct frequencies. This limit will be deduced from the study of the convergence of the distribution of

Imaging of a Dissipative Layer in a Random Medium

137

(Uqε1 ,p1 (ω1 , h1 , z), . . . , Uqεk ,pk (ωk , hk , z)) which results once again from the application of a diffusion-approximation theorem. Introducing V ε as in (26) it is found that (Vqε1 ,p1 (ω1 , τ1 , z), . . . , Vqεk ,pk (ωk , τk , z)) converges as ε → 0 to a diffusion process. In particular, ⎡ ⎤  vp1 ,...,pk (ω1 , ..., ωk , τ1 , ..., τk , z) := lim E ⎣ Vpεj ,pj (ωj , τj , z)⎦ ε→0

j

is solution of ⎛ ⎞ ( ( ∂vp ,...,p ( ∂vp1 ,...,pk 1 k +2 pj = Lωj vp1 ,...,pk − 2¯ σ (z) ⎝ pj ⎠ vp1 ,...,pk , ∂z ∂τ j j j j  vp1 ,...,pk (ω1 , ..., ωk , τ1 , ..., τk , z = −L) = δ0 (τj )10 (pj ). j

Using the families of processes W introduced 2 in (28) defined for every frequency ω, a direct calculation shows that j Wpj (ωj , τj ) satisfies the above system, so that we have  Wpj (ωj , τj , z) , vp1 ,...,pk (ω1 , ..., ωk , τ1 , ..., τk , z) = j

and consequently,   ε→0 E pεref (T R) (t1 + εs1 )p1 . . . pεref (T R) (t1 + εsk )pk −→

2 2 iωj,l sj ˆ ... Gt1 (τj,l )dωj,l dτj,l 1 ≤ j ≤ k W1 (ωj,l , τj,l , 0) 1 ≤ j ≤ k f (ωj,l )e 1 ≤ l ≤ pj 1 ≤ l ≤ pj  pj
2 W1 (ω, τ, 0)fˆ(ω)eiωsj Gt1 (τ )dωdτ = 1≤j≤k . This shows the convergence of the finite-dimensional distributions of (pεref (T R) (t1 + εs))s∈(−∞,∞) to the ones of the deterministic function  W1 (ω, τ, 0)fˆ(ω)eiωs Gt1 (τ )dωdτ . 5.3 Convergence of the Refocused Pulse We have just proved the tightness of the process pref (T R) (t1 + ε·) as well as the convergence of its finite-dimensional distributions. Accordingly this proves the following theorem.   Theorem 1. The refocused signal pref (T R) (t1 + εt) t∈(−∞,∞) converges in probability as ε → 0 to the deterministic function

138

J.-P. Fouque et al.

 Pref (T R) (t) =

Λ(ω, τ )fˆ(ω)eiωt Gt1 (τ )dωdτ ,

(33)

where Λ(ω, τ ) = W1 (ω, τ, 0) is the density given by the system (28). We can also write that (34) Pref (T R) (t) = (f (− ·) ∗ KT R (·)) (t) . The Fourier transform of the refocusing kernel KT R is given by  ˆ T R (ω) = Gt (τ )Λ(ω, τ )dτ . K 1

(35)

We can give a probabilistic representation of the density Λ in terms of a jump Markov process. Let us introduce the process (Xt )t≥−L with state space N and infinitesimal generator Lω given by (29). When the jump process reaches the state x ∈ N∗ , a random clock with exponential distribution and parameter x2 αn ω 2 starts running. When the clock strikes, the process jumps to x ± 1 with probability 1/2. Finally, 0 is an absorbing state. The representation of a system of transport equations in terms of a canonical Markov process was first introduced in [AKP91], and it turns out that we can extend this idea to the system (28) by means of a Feynman-Kac formula. We get  τ1  
0 Wp (ω, τ, 0)dτ = EX−L =p 1
0 2Xs ds∈[τ0 ,τ1 ] 1X0 =0 e− −L 2¯σ(−L−s)Xs ds , −L

τ0

(36) where EX−L =p stands for the expectation over the distribution of the jump process starting from X−L = p. In the particular case of a constant mean dissipation, σ ¯ (z) = σ ¯0 , the probabilistic interpretation of the densities Wp reads:  
0 Wp (ω, τ, 0)dτ = EX−L =p 1
0 2Xs ds∈[τ,τ +dτ ] 1X0 =0 e−2¯σ0 −L Xs ds . −L

We can then note that the argument of the exponential is deterministic be
0 cause the value of the integral −L Xs ds is constrained to be equal to τ /2. As a result, (37) Wp (ω, τ, 0) = Wp(0) (ω, τ, 0)e−¯σ0 τ , (0)

where Wp is the solution of the transport equations (28) in absence of dissipation. We can also verify directly that (37) is indeed a solution of the transport equations (28) with σ ¯ (z) = σ ¯0 .

6 Application to the Detection of a Dissipative Layer In this section we prove that time reversal can be used as an efficient and statistically stable method to image a dissipative layer embedded in a randomly scattering medium. We consider the same configuration as the one studied in the previous section. We compute explicitly the refocusing kernel, we show that it contains information about the presence of an embedded dissipative layer, and we show also how this information can be extracted.

Imaging of a Dissipative Layer in a Random Medium

139

6.1 Constant Mean Dissipation Here we consider the case of a half space and we assume that there is no embedded layer. The mean dissipation is constant σ ¯ (z) ≡ σ ¯0 . In this configuration we can compute explicitly the solution for the system of transport equations by using (37) and the explicit solution computed in [AKP91] in absence of dissipation:   p αn ω 2 τ ∂ 1[0,∞) (τ ) e−¯σ0 τ . (38) Wp (ω, τ, 0) = ∂τ 4 + αn ω 2 τ We thus get a closed form expression for the density in case of constant mean dissipation: 4αn ω 2 −¯ σ0 τ . (39) Λ0 (ω, τ ) = 2e (4 + αn ω 2 τ ) Let us further assume that we record everything at the mirror, so that Gt1 ≡ 1. In these conditions     4¯ σ0 4¯ σ0 σ0 ˆ T R (ω) = 1 − 4¯ K exp E , i αn ω 2 αn ω 2 αn ω 2
∞ where Ei is the exponential integral function Ei (x) = 1 exp(−xt)/tdt. The Fourier transform of this refocusing kernel is plotted in Fig. 4. Note that ˆ T R (ω) = 1. This case is of course expected: the pulse σ ¯0 = 0 implies that K is completely scattered back by the random half-space due to Anderson localization [Pap71], and we send back everything that is recorded, so we get a perfect refocusing as a result of the time-reversibility of the wave equation. In presence of dissipation, the refocusing kernel looks like a high-pass filter. This can be explained by simple arguments based on the fact that the localization length decays with increasing frequencies [Pap71]. Thus, high-frequencies : ¯0 /αn ) are well recovered, because they (above the cut-off frequency ωc = 2 σ are very quickly scattered back by the localization effect and they spend a short time in the random dissipative half-space. On the contrary, low frequencies are highly dissipated, because they penetrate deeper and spend a longer time in the medium before being scattered back. σ ¯ (z) σ ¯1 σ ¯0 z0

z1

0

Fig. 3. Profile of the mean dissipation.

z

140

J.-P. Fouque et al. 1

0.8

K

0.6

0.4

0.2

0

0

2

4

ω/ω

6

8

10

c

Fig. 4. Fourier transform of the refocusing kernel for : a random half space with a ¯0 /αn . constant mean dissipation. Here Gt1 ≡ 1 and ωc = 2 σ

6.2 Thin Dissipative Layer We consider here a configuration where a thin layer with mean dissipation σ ¯1 lying in [z0 , z1 ] is embedded into a half plane with constant mean dissipation σ ¯0 (see Fig. 3). The layer is thin in the sense that z0 − z1 → 0, but its dissipation coefficient is high so that σ ¯1 (z0 − z1 ) → λ. We will discuss the domain of validity of this limit in the next section. First, we solve the system of transport equations (28) from −∞ to z0 , so that we get the stationary solution (38):   p αn ω 2 τ ∂ Wp (ω, τ, z = z0 ) = 1 (τ ) e−¯σ0 τ . [0,∞) ∂τ 4 + αn ω 2 τ Second, we solve the system across the layer from z0 to z1 . The layer is very thin, so that the Markov process does not jump. As a result, we get   p αn ω 2 τ ∂ Wp (ω, τ, z = z1 ) = 1[0,∞) (τ ) e−¯σ0 τ −2λp . ∂τ 4 + αn ω 2 τ Third, we solve the system from z1 to 0 with the initial condition given by Wp (ω, τ, z = z1 ). Using the probabilistic interpretation of the solution of the system transport equations, we obtain that the density Λ(ω, τ ) = W1 (ω, τ, z = 0) is    0 Xs ds, z1 ) Λ(ω, τ ) = EXz1 =1 WX0 (ω, τ − 2 z1

This expression would be equal to the stationary solution (38) if the multiplicative factor exp(−2λX0 ) was absent. We then expand this factor as 1 − (1 − exp(−2λX0 )) so that we obtain

Imaging of a Dissipative Layer in a Random Medium

   (0) ˜ Λ(ω, τ ) = Λ0 (ω, τ ) − EXz1 =1 WX0 ω, τ − 2



0

Xs ds



1 − e−2λX0





141

e−¯σ0 τ ,

z1

(40) where the expectation E1 is taken with respect to the distribution of the jump Markov process Xz starting from Xz1 = 1, and ⎧ if p = 0 , ⎨ δ0 (τ ) 2 p−1 ˜ p(0) (ω, τ ) = W (41) 2 (αn ω τ ) 1[0,∞) (τ ) otherwise . ⎩ 4pαn ω (4 + αn ω 2 τ )p+1 Let us have a look at the expectation in the right-hand side of (40). If X0 = 0, then the second term inside the expectation is zero. If X0 ≥ 1, then Xz ≥ 1 for all z ∈ [z1 , 0] because 0 is an absorbing state. This means that only the paths which never reach zero can contribute to the value of the expectation. These
0 ˜ p(0) (ω, τ ) = 0 for τ < 0. This paths satisfy z1 Xs ds ≥ |z1 |. Furthermore, W shows that the expectation is zero for any τ < 2|z1 |. Accordingly, the density Λ is indistinguishable from the density Λ0 corresponding to a constant mean dissipation for any τ ≤ 2|z1 |. When τ crosses this critical value corresponding to a round trip form the surface to the layer, a density jump occurs. Indeed a set of paths suddenly contributes to the expectation in the right-hand side of (40). This is the set of paths where no jump occurs (i.e. Xz = 1 for all z ∈ [z1 , 0]). The density then jumps from Λ(ω, τ = 2|z1 |− ) = Λ0 (ω, 2|z1 |) to Λ(ω, τ = 2|z1 |+ ) = Λ0 (ω, 2|z1 |)   ˜ (0) (ω, 0) 1 − e−2λ P1 (no jump before 0) e−¯σ0 τ −W 1 = Λ0 (ω, 2|z1 |) [1 − ∆Λ] , where the relative amplitude of the jump is 

−2λ

∆Λ = 1 − e



2  2 αn ω 2 |z1 | e−αn ω |z1 | . 1+ 2

(42)

Summary: in order to detect the depth and the dissipation strength of the layer from a measured density Λ, one can plot the ratio of the measured density Λ over Λ0 . The ratio is 1 up to τ = 2|z1 |. A density jump occurs at 2|z1 |. The amplitude of the jump is (42) and allows us to recover the dissipation strength λ. We have carried out MC simulations of the jump Markov process X to compute Λ from the expression (40). The results for several sets of parameters are plotted in Fig. 5 where it can be checked that the jump of the density can be clearly detected. The computation of each density profile requires N = 106 simulations.

J.-P. Fouque et al. 0.25

λ=0 λ=0.1 λ=1 λ=1000

0.15

1 0.8 0

Λ(τ)

0.2

Λ(τ)/Λ (τ)

142

0.1 0.05

(a)

0 0

0.6

λ=0 λ=0.1 λ=1 λ=1000

0.4 0.2

5

10 τ

15

20

(b)

0 0

5

10 τ

15

20

Fig. 5. Picture a: Density τ → Λ(ω, τ ). Picture b: Ratio of the densities τ → Λ(ω, τ )/Λ0 (ω, τ ). Here we assume αn ω 2 = 1, z0 = 1.

Remark 2. We can estimate the confidence interval for the MC simulations. Indeed the quantity to be estimated is roughly speaking the probability of a −1 rare event p ! 10−2 − 10 √ . In such a case the relative error of a standard MC method is about 1/ N p which is of the order of a few percents.

6.3 Thick Dissipative Layer We revisit the previous configuration without assuming that the layer is thin. Accordingly we consider a configuration where a layer with mean dissipation ¯0 . σ ¯1 lying in [z0 , z1 ] is embedded into a half plane with mean dissipation σ We proceed as above and establish that     0 Xs ds, z0 Λ(ω, τ ) = EXz0 =1 WX0 ω, τ − 2 z0





¯1 ) × exp 2(¯ σ0 − σ

'

0 z0 −z1

Xs ds

   ˜ (0) ω, τ − 2 = Λ0 (ω, τ ) − EXz0 =1 W X0 



× 1 − exp 2(¯ σ0 − σ ¯1 )



e−¯σ0 τ 

0

Xs ds z0

'

0 z0 −z1

Xs ds

e−¯σ0 τ . (43)

If Xz0 −z1 = 0, then Xz = 0 for all z ∈ [z0 − z1 , 0] and the second term inside the expectation is zero. If Xz0 −z1 ≥ 1, then Xz ≥ 1 for all z ∈ [z0 , z0 − z1 ], so
0 ˜ p(0) (ω, τ ) is zero for τ < 0 then shows that 2 z0 Xs ds ≥ 2|z1 |. The fact that W that the first term inside the expectation is zero if τ ≤ 2|z1 |. Accordingly Λ(ω, τ ) = Λ0 (ω, τ ) for τ ≤ 2|z1 |. There is a jump of the derivative of the density at τ = 2|z1 |. Indeed, for τ just above 2|z1 | a path contributes to the value of the expectation that appears in the right-hand side of (43), namely the path where Xz = 1 for

Imaging of a Dissipative Layer in a Random Medium

143

all z ∈ [z0 , z0 − z1 ] and a jump from state 1 to state 0 occurs at z0 + τ /2. Accordingly, the derivative goes from Λ (ω, τ = 2|z1 |− ) = Λ0 (ω, 2|z1 |)   αn ω 2 2 =− ¯0 (2 + αn ω 2 |z1 |) e−2¯σ0 |z1 | 3 αn ω + σ 2 (2 + αn ω |z1 |) to 2 1 Λ (ω, τ = 2|z1 |+ ) = Λ0 (ω, 2|z1 |) − (¯ σ1 − σ ¯0 )αn ω 2 e−αn ω |z1 | e−2¯σ0 |z1 | 4 = Λ0 (ω, 2|z1 |) [1 + ∆Λ ] ,

where the relative amplitude of the jump is ∆Λ =

2 (2 + αn ω 2 |z1 |)3 ¯0 σ ¯1 − σ e−αn ω |z1 | . 2 2 4 αn ω + σ ¯0 (2 + αn ω |z1 |)

(44)

There is a second jump in the derivative of Λ at τ = 2|z0 |. Indeed the mechanism described just above fails precisely when τ become larger than τ = 2|z0 |, because the jump of the Markov process at z0 + τ /2 > 0 has no influence. Summary: in order to detect the depth, the thickness, and the dissipation coefficient of the layer from a measured density Λ, one can plot the ratio of the measured density Λ over Λ0 . The ratio is 1 up to τ = 2|z1 |. The position of the first jump in the derivative of the density is 2|z1 |, the position of the second jump is 2|z0 |. The amplitude of the first jump is (44) and allows us to ¯0 to get recover σ ¯1 . Note that we need to know the background dissipation σ σ ¯1 , as we detect the difference σ ¯1 − σ ¯0 . We have carried out Monte Carlo simulations of the jump Markov process X to compute Λ from the expression (43). The results for several sets of parameters are plotted in Fig. 6. Each density profile requires 107 simulations. It can be checked that the first jump of the derivative density can be clearly detected. It may be more difficult to detect the second jump if the dissipative layer is very thick. Note that the case where the thickness of the layer is very small z1 − z0 = 0.1 is very similar to the approximation of a thin layer with λ = 0.1 presented in Fig. 5. The thin layer approximation developed in Sect. 6.2 can now be discussed more quantitatively. The interpretation in terms of the jump Markov is helpful for this discussion. Considering expression (43), it can be seen that the approximation holds true if the event “the process jumps between z0 − z1 and 0” is negligible. The brackets [.] in the right-hand side of (43) then simpli¯1 )(z1 − z0 )X0 )] and we recover precisely (40). This fies into [1 − exp(2(¯ σ0 − σ 1 and this condition turns out to be the event is negligible if αn ω 2 |z1 − z0 | criterion for the validity of the thin layer approximation.

J.-P. Fouque et al.

0.25

z1−z0=0 z −z =0.1 1 0 z −z =0.5 1 0 z −z =1

0.2

1

0

Λ(τ) 0.1 0.05

(a)

0 0

z −z =0 1 0 z1−z0=0.1 z −z =0.5 1 0 z −z =1

0.8 0

0.15

1

Λ(τ)/Λ (τ)

144

0.6

1

0

0.4 0.2

5

10 τ

15

20

(b)

0 0

5

10 τ

15

20

Fig. 6. Picture a: Density τ → Λ(ω, τ ). Picture b: Ratio of the densities τ → ¯0 = 0, σ ¯1 = 1, and the Λ(ω, τ )/Λ0 (ω, τ ). Here we assume αn ω 2 = 1, z1 = 1, σ thickness of the layer z1 − z0 goes from 0 (absence of dissipative layer) to 1.

Acknowledgments J.-P. Fouque was supported by the ONR grant N00014-02-1-0089. J. Garnier acknowledges support from the French program ACI-NIM-2003-94. A. Nachbin was supported by CNPq/Brazil under grant 300368/96-8. K. Sølna was supported by the Darpa grant N00014-02-1-0603 and by the NSF grant 0307011.

References [AKP91] Asch, M., Kohler, W., Papanicolaou, G., Postel, M., White, B.: Frequency content of randomly scattered signals. SIAM Rev., 33, 519–625 (1991). [BPR02] Bal, G., Papanicolaou, G., Ryzhik, L.: Self-averaging in time reversal for the parabolic wave equation. Stochastics and Dynamics, 2, 507–531 (2002). [BPZ02] Blomgren, P., Papanicolaou, G., Zhao, H.: Super-resolution in timereversal acoustics. J. Acoust. Soc. Am., 111, 230–248 (2002). [CF97] Clouet, J.F., Fouque, J.P.: A time-reversal method for an acoustical pulse propagating in randomly layered media. Wave Motion, 25, 361–368 (1997). [DTR03] Derode, A., Tourin, A., de Rosny, J., Tanter, M., Yon, S., Fink, M.: Taking advantage of multiple scattering to communicate with time reversal antennas. Phys. Rev. Lett., 90, 014301 (2003). [Fin99] Fink, M.: Time reversed acoustics. Scientific American, 281:5, 91–97 (1999). [FMT03] Fink, M., Montaldo, G., Tanter, M.: Time reversal acoustics in biomedical engineering. Annual Review of Biomedical Engineering, 5, 465–497 (2003). [FS03] Fouque, J.P., Sølna, K.: Time-reversal aperture enhancement. SIAM Multiscale Modeling and Simulation, 1, 239–259 (2003). [Kus84] Kushner, H.J.: Approximation and Weak Convergence Methods for Random Processes. MIT Press, Cambridge (1984).

Imaging of a Dissipative Layer in a Random Medium [Pap71]

145

Papanicolaou, G.: Wave propagation in a one-dimensional random medium. SIAM J. Appl. Math., 21, 13–18 (1971). [Pap78] Papanicolaou, G.: Asymptotic analysis of stochastic equations. In: Rosenblatt, A. (ed), MAA Stud. in Math., 18, 111–179 (1978). [PRS03] Papanicolaou, G., Ryzhik, L., Sølna, K.: Statistical stability in time reversal. SIAM J. Appl. Math., 64, 1133–1155 (2004). [PW94] Papanicolaou, G., Weinryb, S.: A functional limit theorem for waves reflected by a random medium. Appl. Math. Optim., 30, 307-334 (1994). [PKC02] Prada, C., Kerbrat, E., Cassereau, D., Fink, M.: Time reversal techniques in ultrasonic nondestructive testing of scattering media. Inverse Problems, 18, 1761–1773 (2002). [Sol03] Sølna, K.: Focusing of time-reversed reflections. Waves in Random Media, 12, 365–385 (2003).

A Stochastic Numerical Method for Diffusion Equations and Applications to Spatially Inhomogeneous Coagulation Processes Flavius Guia¸s University of Dortmund, Department of Mathematics, Vogelpothsweg 87, 44221 Dortmund, Germany [email protected] Summary. We propose a stochastic particle method for diffusive dynamics which allows coupling with kinetic reactions. This is realized by constructing and simulating the infinitesimal transitions of a Markov process which models the elementary processes taking place in the system. For this, we divide the domain into a finite number of cells. The simulation of the diffusive motion of the particles is based on assigning to the particles a velocity vector. Instead of considering jumps in all directions, we compute only the flux between neighbouring cells. This approach is a strong and effective improvement on the use of random walks. It allows also the approximation of coagulation equations with diffusion in a bounded domain: in every cell we simulate a coagulation process according to a usual method (direct simulation or mass flow algorithm) and we couple these dynamics with the spatial motion of particles.

1 Introduction We present a stochastic method for simulating diffusion equations with an application to the the following spatially inhomogeneous coagulation equations on the interval (0, 1): ∞ ( d 1 1 1 u (t) = d1 ∆u − u K(1, i)ui dt i=1

(1) ∞

( d k 1( u (t) = dk ∆uk + K(i, k − i)ui uk−i − uk K(k, i)ui dt 2 i=1 i=1 k−1

k = 2, 3, . . .

with dk ≥ 0, k = 1, 2, . . . The initial condition considered here is given by

148

F. Guia¸s

u1 (0, x) = u10 (x) ≥ 0 on (0, 1) uk (0, x) = 0 for k ≥ 2 and we consider Neumann boundary conditions: ukx (0) = ukx (1) = 0 for all k. The coagulation equations describe the time evolution of a spectrum of particles of sizes 1, 2, . . . , k, . . . with concentrations uk (t, ·), which undergo coalescing processes. That is, particles of sizes i and j can merge at the rate K(i, j) and form a new particle of size i + j. In addition to these reactions, the spatially inhomogeneous model has also a diffusion component for the particles. We consider here only monodisperse initial conditions, that is at t = 0 only monomers (particle of size 1) are present. The main challenge of this type of equation system is the fact that it is infinite-dimensional. Often it is considered also in an integral formulation, where the particle sizes form a continuous spectrum. In the spatial homogeneous situation one can construct a direct simulation algorithm which is based on simulating the binary coalescences with the rates K(i, j). If this coagulation kernel has a multiplicative form, the method can be considerably fastened by using certain recursive relations in computing the exponential waiting times. Otherwise, one has to take a multiplicative majorant of the kernel and introduce fictitious jumps, the frequency of which depends on the distance between the actual kernel and its multiplicative majorant. A more efficient particle scheme for solving the coagulation equation, the mass flow algorithm, was proposed in [EW01]. The measure which corresponds to the mass spectrum of the particles is constructed from the distribution of a finite number of numerical particles. Starting from the original coagulation dynamics, by defining jumps of these numerical particles, one can construct a Markov process which describes the time evolution of the discrete measure which approximates the mass spectrum. This method proves also very suited to describe the gelation phenomenon. From the construction of the sim1direct ∞ ulation algorithm one has formally that the total mass M (t) = k=1 kuk (t) is conserved, since particles are neither created, nor destroyed. However, for large coagulation rates, e.g. K(i, j) ≥ (ij)q with q > 12 , one can observe a decay of the total mass, that is M (t) < M (0) for t > tgel (the gelation time). This is related to the formation in finite time of infinitely large clusters, which are not described by the variables uk . In the spatial inhomogeneous situation we refer to [BW97] and [LM02], where existence of solutions for certain classes of coagulation coefficients is proved. A stochastic approach for diffusive coagulation equations is considered for example in [DF02] and [Gui01]. In the latter a stochastic approximation method for the coagulation equations with diffusion is presented. It is based on direct simulation of the coagulation processes and by a random-walk approximation of the diffusion processes. Convergence is proved in the case of bounded coagulation coefficients. The results have only a theoretical relevance, since from the numerical point of view the random walk method is very

A Stochastic Numerical Method for Coagulation-Diffusion Equations

149

imprecise at coarse spatial resolutions, while a combination with coagulation processes in the situation of a fine resolution is not feasible. The approach presented here intends to fill this gap, by presenting first results of a combination of a particle method for diffusive dynamics with the mass flow algorithm, in order to approximate the solution of (1). We consider a finite number of cells and in each cell we consider a spectrum of particles which evolve according to the mass-flow dynamics presented in [EW01]. The mass exchange between the cells is realized by a discrete stochastic version of the so-called deterministic particle method presented in [Ru90] and [LM01]. The idea is to assign to the particle located in x at time t the velocity vector ∇uN (t, x) (2) vN (t, x) = − uN (t, x) where uN is a density function constructed from the N -particle system. This means that the diffusion equation is interpreted as a transport equation. Consider in a volume element U ⊂ Rd the equation: ut = −∇ · (uv). If we put for∇u we obtain nothing else than the diffusion equation: ut = ∆u. mally v = − u The rigorous justification of this heuristic argument is presented in [LMG01]. The method proposed here is based on the same idea. The particles can occupy only a discrete set of locations and the density function in each location is proportional to the number of particles. All particles in one location have assigned the same velocity vector, computed by a discretized version of (2). The motion is given by the transition rates of a Markov jump process, i.e. the jump times are random, the jump intensities are proportional to the length of the velocity vector and the target cell is determined uniquely by the direction of the velocity. Note that in the case of random walks, the target cell of a particle which is selected in order to be moved is chosen at random. Section 2 contains the basic steps of this approach. Since we are interested in numerical simulations with a finite number of spatial locations, we let first only the particle number tend to infinity, while keeping the spatial discretization step fixed. Standard convergence results of density dependent Markov processes from [Ku71] imply the convergence towards a finite-dimensional system of ODE’s, its components being the limit particle concentrations in the single cells. This turns out to be the usual finite-difference scheme for the diffusion equation. The numerical solution of this system is simply the product of the stochastic simulations and it is not indended to be computed by a deterministic solver. Finally, the last section of this paper contains numerical results obtained by coupling the particle scheme for diffusion equations introduced in Sect. 2 (which describes the motion between the cells) with the mass flow algorithm for coagulation processes which take place in every cell. The general case of coagulation-diffusion systems of this semi-discrete type together with a convergence analysis towards the continuous models will be the subject of further investigations.

150

F. Guia¸s

We note that the convergence result obtained in [Gui01] (using random walks, direct simulation algorithm and considering bounded coagulation coefficients) can be easily modified in order to show convergence of the present method towards the solution of the deterministic coagulation-diffusion equations, under the same boundedness property of the coagulation coefficients.

2 The Particle Scheme Approximating the Diffusion Equation We will present next an approach to approximate the diffusion equation in the one-dimensional case by a stochastic particle method which can be extended easily to higher dimensions. The goal is to approximate on the interval (0, 1) the solution of: ⎧ ⎨ u(0) = u(1) = 0 (D) or (3) ut = uxx , with the boundary condition: ⎩ ux (0) = ux (1) = 0 (N ) and initial condition u(0, x) = u0 (x) ≥ 0 ∈ H 1 (0, 1) for all x ∈ [0, 1]. Let M be an integer, denote ε = M −1 and consider the discrete set of sites Gε = {kε, k = 1, M − 1}. Assume that we have N particles distributed in the locations of Gε and denote by nk (t) the number of particles present at the moment t in location kε. We introduce the scaling parameter h = M/N = ε−1 N −1 , which means that h−1 is the average number of particles per site. The density function corresponding to this particle system is defined in the points kε of the discretization grid by uk (t) = hnk (t). In the following we will identify the sites only by the value of k, skipping the ε for sake of simplicity. At the boundary sites k = 0 and k = M (which contain no particles), we will consider formal values, in order to model the boundary conditions. This is needed in order to compute the transitions related to the “near boundary” sites k = 1 and k = M − 1. The next step is assigning to the particles located in site k, k = 1, M − 1 a velocity vector, in analogy to (2). Since our interest is to follow the time evolution of the density function, we assume that all particles present in the same location behave identical and are indistinguishable. That is, if any particle from the site k jumps into a neighbouring site, the density function in the new state will be the same, regardless which particle has jumped. We can thus consider a single transition of this type and multiply the rate with nk (t), i.e. with the number of identical particles present at time t at the site k. Taking the symmetric finite difference scheme we obtain the following form of the velocity: uk−1 (t) − uk+1 (t) . v k (t) = 2εuk (t) This approach turns out to be not satisfactory, because it leads to strongly oscillating patterns. While the discrete derivative of the density function is

A Stochastic Numerical Method for Coagulation-Diffusion Equations

151

approximated correctly, the density values in the odd sites and even sites may show large differences. The reason is that for this velocity, the jump rates do not depend on the particle density in the site k itself, but only on the neighbours, since the multiplication of the transition rates with nk (t) will cancel the dependence on uk (t). It turns out that it is better to consider the left and right discrete derivatives of the particle density function and to simulate in this way the flux between a pair of cells. 2.1 Construction of the Markov Jump Process Based on the previous considerations, we will construct an RM −1 -valued Markov jump process as follows. −1 Given the time moment t and the state u(t) = (uk (t))M k=1 , we define the transitions: 1 −1 −2 k h ε |w+ (t)| 2 1 k k u(t) → u(t) − h · ζ(w− (t)) · (ek − ek−1 ) at rate h−1 ε−2 |w− (t)| 2

k u(t) → u(t) + h · ζ(w+ (t)) · (ek − ek+1 ) at rate

(4)

k k where w+ (t) = uk+1 (t) − uk (t) and w− (t) = uk (t) − uk−1 (t). ek denotes the k-th unit vector in RM −1 and ζ(·) denotes the signum function. The first transition corresponds to the jump of a particle from site k to k + 1 or from site k + 1 to k, depending on the sign of the difference of the densities. The second transition describes analogously the particle exchange between the sites k and k − 1. If we simulate the particle diffusion by random walks, we have to replace k k k (t) in the transition rates by uk (t), ζ(w+ (t)) by -1 and ζ(w− (t)) the terms w± by 1. Taking as initial value of the density function a discretization of a suffik (t) will be of magnitude O(ε) and ciently regular function, the differences w± the corresponding transition rates will be thus of magnitude O(h−1 ε−1 ), while in the case of random walks they are of order O(h−1 ε−2 ). This leads to an improvement of O(ε−1 ) in the computation time, since the number of simulated jumps between two adjacent sites in a time interval ∆t is now proportional with the difference of the densities, while in the case of random walks it is proportional to the density values. We will discuss next the situation at the boundary. In the case of zero boundary conditions (D), we consider formally u0 (t) = M u (t) = 0 in all expressions from (4). This value does not change after any possible transition, that is, the particle which leaves the interior of the domain is “killed”. In the case of Neumann boundary conditions (N) we take formally u0 (t) = 1 u (t) and uM (t) = uM −1 (t).

152

F. Guia¸s

We will express the dynamics of the Markov process u given by the transitions (4) in terms of its generator, by using the characterization from [EK86], p.162 f. If we have an E-valued Markov jump process with a set of transitions {x(·) → y(·)} and 1the corresponding rates rx→y , the waiting time parameter function λ(t) = rx→y is given by the sum of all possible transition rates. The infinitesimal generator Λ is an operator acting on the bounded, measurable functions on E and is given by ( (f (y) − f (x))rx→y . (Λf )(x) = x→y

We can note that for fixed M and N we have a bounded total number of particles (there is no source). The process u has thus bounded components, i.e. there exists a constant LM,N such that max|uk | ≤ LM,N for all times. Moreover, the waiting time parameter function λ is also bounded, which implies that the process is well-defined for all t, i.e. the jumps do not accumulate. For a fixed element φ ∈ RM −1 consider on RM −1 a bounded smooth function fφ which on the set {x : max|xk | ≤ LM,N } has the form fφ (x) = x, φ =

M −1 (

xi φi .

(5)

i=1

Outside this set the values of the function are in our case not of interest, only the boundedness is essential. From [EK86], p.162 we have that the process u satisfies the identity  t (Λu fφ )(u(s))ds + Mφ (t) (6) fφ (u(t)) = fφ (u(0)) + 0

where Mφ (·) is a martingale with respect to the filtration generated by the process u and Λu is the infinitesimal generator. The value Λu fφ is given by: ( (Λu fφ )(u(t)) = u − u, φru→u u→u

=

1 (8 k k hζ(w+ ) ek − ek+1 , φ h−1 ε−2 |w+ (t)| 2 k

9 k k ) −ek + ek−1 , φ h−1 ε−2 |w− (t)| +hζ(w− 1 ( k k k (φ − φk+1 )w+ (t) + (−φk + φk−1 )w− (t). = 2 2ε k

For φ = ei we obtain:

A Stochastic Numerical Method for Coagulation-Diffusion Equations

(Λu fei )(u(t)) =

153

i+1 1 ( k k k (ei − ek+1 )w+ (t) + (−eki + ek−1 )w− (t) i i 2ε2 k=i−1

1 i−1 i+1 i i = 2 (−w+ (t) + w+ (t) − w− (t) + w− (t)) 2ε 1 i i = 2 (2w+ (t) − 2w− (t)) 2ε 1 = 2 (ui−1 (t) − 2ui (t) + ui+1 (t)) ε = ∆ε ui (t). Equation (6) becomes thus  i

i

t

∆ε ui (s)ds + Mi (t) .

u (t) = u (0) +

(7)

0

Standard results for convergence of density dependent Markov processes towards deterministic ODE’s from [Ku71] imply the convergence in probability for N → ∞ and fixed ε of the family of Markov processes u = uN,ε (t) towards the solution of the deterministic ODE-system d i u (t) = ∆ε ui (s), i = 1, M − 1 . dt

(8)

This is the usual one-dimensional finite difference scheme for the heat equation, with ∆ε the well-known discrete Laplacian on the ε-grid. In higher dimensions we obtain a similar result. The numerical solution of this ODEsystem (for fixed ε) is produced directly by the stochastic simulations, and no further application of a deterministic solver based on some time-discretization scheme is necessary. 2.2 Numerical Examples This section is dedicated to numerical examples which compare the method proposed here with the random walk method on one hand, and with the exact solution on the other hand. We mention that for one realization of the algorithm up to the time t = 0.15, a UNIX workstation needed in the case of 20 sites and 104 particles per site about 5 seconds of computing time. If we increase the total number of particles or the number of sites by a given factor, the computing time scales basically by the same factor. Figures 1 and 2 show a comparison of the method presented here (left pictures) with the random walk method (right pictures). By this we mean the algorithm which computes the density function as proportional to the number of (independent) particles which are in the same site, while the trajectories of the particles are random walks. We considered here Neumann boundary conditions, that is the particles do not leave the domain.

154

F. Guia¸s

The initial data is  u(0, x) =

2.5 for x ∈ [0.3, 0.7] 0 else

(9)

and we compare the different solutions at the time steps t = 0.05, 0.10, 0.15. The horizontal axis represents the spatial domain while the vertical axis stands for the density values. At the time moments where the profile of the solution is almost flat, but still retains a certain curvature pattern, the random walk method is practically useless, as one can see in Fig. 1, while averaging over more realizations is also not satisfactory, as Fig. 2 shows. Regarding the computing time, the things are also clearly in advantage of the method proposed here. As we noticed in Sect. 2, the present method is faster by factor of O(ε−1 ) than the random walk method. The accuracy of the approximation is also considerably improved compared to the random walk approach. The smoothness of the density profile is a t=0.05, 0.10, 0.15

1.2

1.2

1.15

1.15 1.1

1.1 1.05 1 0.95

1.05 1 0.95 0.9

0.9

0.85

0.85 0.8

t=0.05, 0.10, 0.15

1.25

density

density

1.25

0.8 0

0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9

0.75

1

0

0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9

1

Fig. 1. Comparison of the new method (left) with the random walk method (right). 10 realizations, ε = 0.02, h−1 = 104 . t=0.05, 0.10, 0.15

1.25

1.2

1.2

1.15

1.15

1.1

density

density

1.25

1.05 1 0.95

1.1 1.05 1 0.95

0.9

0.9

0.85

0.85

0.8

t=0.05, 0.10, 0.15

0

0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9

1

0.8 0

0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9

1

Fig. 2. Comparison of the new method (left) with the random walk method (right) Average over 10 realizations, ε = 0.02, h−1 = 104 .

A Stochastic Numerical Method for Coagulation-Diffusion Equations

155

consequence of the orientation of the jumps of the particles, while in the case of random walks we observe significant fluctuations around the mean value. By taking a larger particle number per cell (h−1 = 105 ) the tightness pattern of the curves delivered by our method is strongly improved. Practically they coincide with the average curves plotted in the left picture of Fig. 2. When using such a particle number, it is sufficient to run only one simulation of the stochastic process in order to get a good approximation of the solution of the deterministic finite-difference scheme (8). Figure 3 shows a comparison with the exact solution in the case of zero boundary conditions. The initial condition is given again by (9). The plot represents the results produced by one realization of the method proposed here for ε = 0.05 and ε = 0.02 and the exact solution computed by Fourier series. The average number of particles per site was taken h−1 = 105 and the curves represent the density profiles at the time steps t = 0.05, 0.10, 0.15, 0.20, 0.25. t=0.05, 0.10, 0.15, 0.20, 0.25 1.4

ε=0.05 ε=0.02 exact solution

1.2

density

1

0.8

0.6

0.4

0.2

0 0

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

Fig. 3. Comparison with the exact solution. h

0.9 −1

1 5

= 10 .

3 Applications to Coagulation Processes with Diffusion In this section we consider applications of the method to the spatially inhomogeneous coagulation equations (1). The initial condition is given by  2.5 for x ∈ [0.3, 0.7] u1 (0, x) = 0 else uk (0, x) = 0 for k ≥ 2

156

F. Guia¸s

The total mass of the system is defined in the spatially inhomogeneous setting as  1( ∞ kuk (t, x)dx . (10) M (t) = 0 k=1

Formally it is conserved for all times, but for certain rates one expects a gelation phenomenon similar as in the spatial homogeneous situation. Our numerical simulations show that this is indeed the case. Throughout the simulations we consider a number M = 20 of sites, with an average number of h−1 = 104 particles per site. The coagulation kernel is taken as K(i, j) = ij. In the spatial homogeneous situation it is well known that for this product kernel we have tgel = 1, see for example [EW01], where this property is verified also numerically. The data are saved at time intervals ∆t = 0.05 and are plotted only for time values t ≥ 0.05, in order to have a better view of the pictures. 3.1 Simulation Results in the Case dk = 0 for k ≥ 2 We consider first the situation than only the monomers can move, that is we take dk = 0 for k ≥ 2. The next figures show the time evolution of the total mass function ∞ ( M (t, x) = kuk (t, x) (11) k=1

for different values of d1 . In Fig. 4 is depicted the gelation phenomenon for d1 = 1. The continuous surface represents the time evolution of the total mass function (11) while the mesh below gives us the mass of the gel, as computed in every site by the mass flow algorithm. In this case the numerical simulations give a value of tgel ≈ 0.9. This value depends both on the initial data and on the diffusion coefficient. In the regions where the density is larger than the average the gelation will occur earlier, while in low-density regions it will occur later, if we take as reference the gelation time of a process with uniform spatial density and the same total mass. There is a competition between the reaction and the diffusion processes. If we inhibit the diffusion, taking for example d1 = 0.1, the mass from the regions of high concentration will spread out slower, ceding the dominance to the coagulation dynamics. Figure 5 shows clearly that for a small diffusion coefficient the gelation time is considerably reduced, being in this case tgel ≈ 0.55. In the case of large diffusion coefficients, like d1 = 10, the concentrated mass spreads out very quickly and diffusion becomes dominant. This can be seen in Fig. 6. The gelation occurs at tgel ≈ 1.1 practically simultaneous in all sites and not preponderant where the mass was initially concentrated, as in the previous situations. We can also observe an oscillating pattern of the

A Stochastic Numerical Method for Coagulation-Diffusion Equations

157

Fig. 4. d1 = 1, dk = 0 for k ≥ 2, K(i, j) = ij, ε = 0.05, h−1 = 104 .

Fig. 5. d1 = 0.1, dk = 0 for k ≥ 2, K(i, j) = ij, ε = 0.05, h−1 = 104 .

total mass function which appears due to a phenomenon often encountered in reaction-diffusion dyanmics (a typical example being the Brusselator). In the regions with higher total concentration the reactions occur with a larger intensity and the monomers are “consumed” more rapidly. Since the dynamics of the monomers has also a significant diffusive component (in the case d1 = 10), this means that monomers will move from regions with less total concentration to regions with higher total concentration, but lower concentration of monomers. In this way the fluctuations in the total concentration are

158

F. Guia¸s

Fig. 6. d1 = 10, dk = 0 for k ≥ 2, , K(i, j) = ij, ε = 0.05, h−1 = 104 .

amplified, leading to the oscillatory pattern which can be observed in Fig. 6. After gelation this phenomenon becomes even more pregnant and can be observed with a small amplitude even for d1 = 1. In the case d1 = 0.1 however it does not appear during the computed time interval. 3.2 Simulation Results in the Case dk = 1/k, k ≥ 1 Figure 7 shows the time evolution of the total mass function for the diffusion coefficients dk = 1/k, k ≥ 1. Since in this case all particles are allowed to move, after a short time the total mass is spread out almost uniformly across the spatial domain. This happens mainly due to the monodisperse initial condition and due to the fact that at early times only particles of relatively small sizes are present, which can diffuse faster. Gelation can be detected numerically at tgel ≈ 1.05. Regarding the physical time needed to cover a computed time interval of, say, ∆t = 0.05, we make the observation that it is increases drastically with the increase of the complexity of the particle system. If at the early stages of the simulation one needs only a few seconds or minutes in order to advance computationally with the given time step, near the gelation time it can take even days. Recall that the expectation of one “infinitesimal” time step of the simulation is inverse proportional to the sum of all possible transition rates. At the beginning only monomers are present, but with the increase of the number of different particle types, the total rate may become very large. This means that one needs more iterations in order to cover a given time interval. Depending on the goal of the simulation, one may use a particle reduction procedure in order to speed up the program. Namely, after reaching the time

A Stochastic Numerical Method for Coagulation-Diffusion Equations

159

Fig. 7. dk = 1/k, K(i, j) = ij, ε = 0.05, h−1 = 104 .

steps where the computed data are saved (in our case multiples of ∆t = 0.05), one can reduce the number of different particle types as follows. Fix a given maximal number N0 (typically between 30-50), and perform in every cell the following procedure. Let j1 ≤ j2 ≤ · · · ≤ jm be the indices where the mass is concentrated, i.e. mj > 0 if and only if j ∈ {j1 , j2 , . . . , jm }, where mj denotes the mass of (numerical) particles of size j present in the given cell. If N0 < m, search the index jk with minimal strictly positive mass, excepting the largest particle, i.e. mjk ≤ mj , j1 ≤ j < jm and k < m. If jk is not unique, take the largest jk possible. In the next step we merge the particle of size jk with the particle which is closest to it in the mass space. That is, we let mjk = 0 and, if jk − jk−1 ≤ jk+1 − jk , then we let mjk−1 = mjk−1 + mjk , otherwise mjk+1 = mjk+1 + mjk . Repeat this procedure until the number m of different particle types becomes smaller than N0 . Note that the maximal particle size remains unchanged. Only the mass concentrated at this value may increase if we merge the numerical particle corresponding to the second largest size with the largest particle. The total mass in the cell remains also unchanged. An important aspect is how does this modification influence the behaviour of the system. If one is interested only in quantities like the number of particle types or moments of different order, a comparison of the simulation with and without the reduction procedure shows that these quantities are not significantly affected. In Fig. 8 are plotted at different time steps the number of different particle types (left) and second moments (right), that is the discrete version of ∞ ( k 2 uk (t, x) . M2 (t, x) = k=1

160

F. Guia¸s 7

original algorithm with reduction procedure

original algorithm with reduction procedure

6

120 t=0.90

t=0.90

100 t=0.85

80 t=0.80

60 t=0.75 t=0.70

40 20 0

0

5

second moment

number of particle types

140

0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9

t=0.85

4 t=0.80 t=0.75

3

t=0.70

2 1 0

1

space

0

0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9

1

space

Fig. 8. a) Number of particle types.

b) Second moment.

The profiles of these quantities coincide up to a certain time, since reduction is applied only if we have more than N0 particle types in a cell. One can note that between two consecutive applications of this procedure the number of particle types increases again and at the end of the time increment it reaches practically the correct value. At this moment the data is saved, after which the reduction procedure is applied again. The values of the second moment are also not significantly perturbed by this modification of the algorithm. The effect on the gelation time is shown in Fig. 9. At t = 1.05 we observe in both situations the appartion of the gel and a blow-up of the size of the largest particle (Fig.9.a) ) and of the second moments (Fig.9.b) ). The same profile with two peaks is encountered, with slight variations in position and height. We recall that in the mass flow algorithm the particles which exceed a given size (taken here as 106 ) are eliminated from the system and account for the gel phase. This particle reduction method used in our simulations shortened the computational effort by up to 3 times. Smaller values of N0 and of ∆t will increase x 105

t=1.05

t=1.05

1200 original algorithm with reduction procedure

original algorithm with reduction procedure

5

1000

second moment

size of the largest particle

6

4 3 2 1 0

800 600 400 200

0

0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9

1

space

Fig. 9. a) Size of the largest particle.

0

0

0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9

space

b) Second moment.

1

A Stochastic Numerical Method for Coagulation-Diffusion Equations

161

accordingly the improvement factor, but also the statistical error. The optimal compromise between speed and precision has to be found in each application.

References [BW97]

B´enilan, P., Wrzosek, D.: On an infinite system of reaction-diffusion equations. Adv.Math.Sci.Appl., 7(1), 351–366 (1997) [DF02] Deaconu, M., Fournier, N.: Probabilistic approach of some discrete and continuous coagulation equations with diffusion. Stochastic Processes Appl. 101(1), 83–111 (2002) [EK86] Ethier, S., Kurtz, T.G.: Markov Processes: Characterization and Convergence. Wiley, New York, 1986. [EW01] Eibeck, A., Wagner, W.: Stochastic particle approximations for Smoluchowski’s coagulation equation. Ann.Appl.Prob., 11(4), 1137–1165 (2001) [Gui01] Guia¸s, F: Convergence properties of a stochastic model for coagulationfragmentation processes with diffusion. Stochastic Anal. Appl., 19(2), 254–278 (2001) [Ku71] Kurtz, T.G.: Limit theorems for sequences of jump Markov processes approximating ordinary differential processes, J. Appl. Prob., 8(1971) 344–356 [LMG01] Lions, P.L., Mas-Gallic, S.: Une m´ethode particulaire de´terministe pour des ´equations diffusives non lin´eaires. C.R. Acad. Sci. Paris, 332(1), 369– 376 (2001) [LM02] Lauren¸cot, P., Mischler, S.: Global existence for the discrete diffusive coagulation-fragmentation equations in L1 . Proc.R.Soc.Edinb., Sect.A, Math. 132(5), 1219–1248 (2002) [Ru90] Russo, G.: Deterministic diffusion of particles. Comm.Pure Appl.Math., XLIII, 697–733 (1990)

Non-Uniform Low-Discrepancy Sequence Generation and Integration of Singular Integrands J¨ urgen Hartinger1 and Reinhold Kainhofer2 1

2

Graz University of Technology, Department of Mathematics, Steyrergasse 30, A-8010 Graz, Austria [email protected] Vienna University of Technology, Department of Math. Methods in Economics, Wiedner Hauptstr. 8-10/105-1, A-1040 Wien, Austria [email protected]

Summary. In this article, we will first highlight a method proposed by Hlawka and M¨ uck to generate low-discrepancy sequences with an arbitrary distribution H and discuss its shortcomings. As an alternative, we propose an interpolated inversion method that is also shown to generate H-distributed low-discrepancy sequences, in an effort of order O(N log N ). Finally, we will address the issue of integrating functions with a singularity on the boundaries. Sobol’ and Owen proved convergence theorems and orders for the uniform distribution, which we will extend to general distributions. Convergence orders will be proved under certain origin- or corner-avoidance conditions, as well as growth conditions on the integrand and the density. Our results prove that also non-uniform quasi-Monte Carlo methods can be well applied to integrands with a polynomial singularity at the integration boundaries.

1 Introduction The numerical solution of several problems arising in financial mathematics requires the use of non-uniformly distributed point sequences. In many cases, (pseudo-) random sequences for a given density are generated by some kind of transformation, possibly involving two or more independent random variables. As quasi-Monte Carlo sequences follow a given construction scheme, subsequent elements of the sequence do not satisfy the requirement of independence. A good overview over several other ways to generate non-uniformly distributed (pseudo-) random sequences can be found in Devroye’s monograph [2]. Unfortunately, almost none of them can be applied to QMC. In 1972, Hlawka and M¨ uck [7] proposed a method to generate H-distributed sequences with low discrepancy by using the (quasi)-empirical distribution

164

J. Hartinger and R. Kainhofer

function instead. Later, they also extended the method to multi-dimensional sequences [8]. The case of dependent random variates is more involved, and hardly anything about this case is known for quasi-Monte Carlo methods. In this article, we will first investigate the Hlawka-M¨ uck method and highlight its shortcomings. We will then propose some adaptions to make the generated sequences more suitable in many practical cases and investigate sequences generated by an approximated inversion of the distribution function. In the second part we will point our view to the non-uniform integration of singular integrands. QMC integration of functions with a singularity at the integration boundaries were already investigated by Sobol’ [17], and later as non-uniform integration problems by Hartinger, Kainhofer, and Tichy [4]. Both publications give criteria for convergence of the singular integral, but do not explicitly prove the order of convergence. Owen [15] proved these for uniform integration using certain growth conditions on the function near the singularity. In this paper we will expand Owen’s results to integration with respect to arbitrary densities.

2 Basic Definitions Remark 1. Although all results in the article will be formulated on the unit s cube U s = [0, 1] , they are valid on any compact subinterval [a, b] ⊂ Rs by a simple affine transformation of the sequence and all corresponding entities. 2.1 Discrepancy and Koksma-Hlawka Inequality When dealing with quasi-Monte Carlo sequences, the most common measure of their distribution properties is the discrepancy. For uniformly distributed sequences on U s it measures the maximum error one can obtain on intervals parallel to the axes: Definition 1 (uniform discrepancy). The discrepancy DN (ω) of a sequence ω = (x1 , x2 , . . .) is defined as 1 DN (ω) = sup AN (J, ω) − λ(J) , J⊆U s N where AN counts the number of elements of (x1 , . . . , xN ) falling into the in1N terval J, i.e. AN (J, ω) = n=1 χJ (xn ), and λ denotes the Borel-measure of the interval J. The best sequences known to date (e.g. Halton, Sobol’, Faure sequences, and (t, s)-sequences) have a discrepancy order of O ((log N )s /N ), which is also conjectured to be optimal.

Non-Uniform Low-Discrepancy Sequences and Singular Integration

165

The notion of discrepancy is especially important in view of the famous Koksma-Hlawka inequality, which allows to bound the quasi-Monte Carlo integration error by the variation of f multiplied by the discrepancy of the sequence ω. A good discussion of variation can be found in [14], and a detailed overview on discrepancy and low-discrepancy sequences is given in the monographs by Niederreiter [13] and Drmota and Tichy [3]. A similar concept of discrepancy can be defined for non-uniformly distributed sequences, i.e. sequences with density h or distribution function H: Definition 2 (non-uniform discrepancy). The H-discrepancy of the sequence ω ˜ = (y 1 , y 2 , . . .) measures its distribution properties with respect to the measure H on U s . It is defined as 1 DN,H (˜ ω ) = sup AN (J, ω ˜ ) − H(J) . J⊆U s N Theorem 1 (non-uniform Koksma-Hlawka Inequality, [1]). Let f be a function of bounded variation on U s , H a probability distribution with contin˜ = (y 1 , y 2 , . . .) a sequence on U s . Then the QMC uous density on U s and ω integration error can be bounded by  N 1 ( f (x)dH(x) − f (y n ) ≤ V (f )DN,H (˜ ω) . (1) Us N n=1

Chelson proves this as a direct consequence of a more general version of the Koksma-Hlawka inequality for importance sampling. Unfortunately, this proof has never been published except in Chelson’s Ph.D. thesis, and it is too complicated to be repeated here. 2.2 Existing Methods for the Generation of Non-Uniform Sequences, and Their Problems The Koksma-Hlawka inequality gives a convergence criterion for QMC integration, and shows that asymptotically QMC methods have to be preferred over Monte√Carlo methods, due to their error order of O ((log N )s /N ) compared to 1/ N for Monte Carlo integration. For large values of s, this bound leads to impractically large values for N . However, empirical studies (e.g. [16]) show that in many cases QMC methods still perform better than MC methods, even for larger values of s. For generating H-distributed low-discrepancy sequences it would be most desirable to transform the uniformly distributed sequence using the inverse H −1 of the distribution function (the conditional distribution functions or the marginal distributions for multi-dimensional sequences). Such a transformation preserves the discrepancy in one dimension, i.e. DN (ω) = DN,H (H −1 (ω)) ,

(2)

166

J. Hartinger and R. Kainhofer

and is independent of the value of N , so that it can be used to generate an arbitrary number of points. In most cases, however, H −1 is not explicitly available, so this method is not applicable. In the multi-dimensional case with dependencies between the dimensions, the discrepancy is not preserved, as intervals are not transformed to intervals. The most common practice for pseudo-random variates, the acceptancerejection method, also fails for quasi-Monte Carlo sequences. The main reason is that the rejection effectively introduces discontinuities into the integrand, which leads to functions of unbounded variation and bad results of QMC methods, as several numerical investigations show (e.g. [11, 18]). To tackle the problem of integration with other densities, Wang proposed a smoothed rejection sampling method [18] by adapting the integrand. Thus, while his method avoids the jumps in the integrand, it cannot be used to directly generate H-distributed sequences. The same shortcoming appears with the approach of stratified sampling, where the integration domain is split into various areas, and in each area ni uniformly distributed points are generated. While this methods works well for integration, the sequences used therein do not display very good distribution properties.

3 The Hlawka-M¨ uck Method The idea behind the Hlawka-M¨ uck transformation [7, 8] is to use an approximation of the distribution function in the inversion method. Instead of directly taking this value, they again use the original sequence to count the relative number of elements below that value and use this number as the new point. This way, the quasi-Monte Carlo error will not only involve the discrepancy of the original sequence, but also the quality of the approximation. They prove a ω ) ≤ (1 + 4M )s DN (ω), where M denotes bound on the discrepancy of DN,H (˜ the supremum of the density. In 1997, Hlawka [6] gave a modification of the Hlawka-M¨ uck method using the one-dimensional marginal distributions instead of the conditional distributions for the transformation. 3. Let h(x) be a density function on [0, 1]s . For a point x = Definition (1) (s) ∈ [0, 1]s we define the marginal distribution functions as x ,...,x  x(1)  1  1 (1) H1 (x ) = ··· h(u)du 0



1



0

0



x(2)

1

···

H2 (x(2) ) = 0

h(u)du 0

0

.. . 

1





1

···

Hs (x(s) ) = 0

0

x(s)

h(u)du . 0

Non-Uniform Low-Discrepancy Sequences and Singular Integration

167

As each of the functions Hi is invertible, Hlawka defines a transformation and bounds the discrepancy of the transformed sequence as follows: Lemma 1 (Hlawka [6]). Let H(x) denote a cumulative distribution function with density h(x) = h1 (x(1) )h2 (x(2) ) · · · hs (x(s) ) defined on U s and Mh = sup h(x). Let furthermore ω = (x1 , x2 , . . . , xN ) be a sequence in U s with ˜ = (y 1 , . . . , y N ) with discrepancy DN (ω). Then the point set ω (j)

yk =

N N  @    1 ( 1 (? (j) 1 + xk − Hj xr(j) = χ[0,x(j) ] Hj xr(j) k N r=1 N r=1

(3)

has an H-discrepancy of DN,H (˜ ω ) ≤ 2(1 + 3Mh )s DN (ω) . The advantage of this approach is that the approximation quality of the distribution function increases with the number of sampled points N , so that the resulting H-distributed sequence is again a low-discrepancy sequence ω ˜, at least for independent marginals. For dependent marginals, i.e. if the distribution function does not factor, the discrepancy can only be proved to satisfy ω ) ≤ c(DN (ω))1/s . the inequality DN,H (˜ Observe, however, that in these integration problems at least in principle one can always avoid dependent sequences by hiding the dependence in the integrand through the incorporation of an appropriate copula (see [12] for an introduction into copulas). Applying the Hlawka-M¨ uck method to singular integrands, we found [4] that these low-discrepancy sequences also work well with singular integrands, (j) but only with an additional shift of all components with yk < 1/N to a value of 1/N . By this shift the order of the discrepancy is preserved, and the resulting sequence is a low-discrepancy sequence with density h. However, the Hlawka-M¨ uck method also has several disadvantages: 1. The resulting sequence is generated only on a grid with spacing N1 . While the resulting sequence displays the required distribution properties, for several applications finer-grained sequences are needed. 2. Several points might have identical coordinates, in particular for highly peaked distributions. Consequently, the minimum distance principle, which is desired in several applications in computer graphics (see e.g. [10]), is no longer fulfilled. 3. The construction of each point involves a sum over all other points, so the cost is O(N 2 ), and the (numerically expensive) distribution function has to be evaluated sN times. 4. One has to fix the number N beforehand, and the resulting set will heavily depend on it. This also means that when adding some points to the sequence, all other elements have to be regenerated. In the sequel we will present several ways to solve or at least considerably ease these problems for most practical uses.

168

J. Hartinger and R. Kainhofer

4 Interpolation For some applications the Hlawka-M¨ uck methods have the drawback that all points of a set with cardinality N lie on the lattice {x ∈ U s x(l) = i/N for 0 ≤ i ≤ N, 1 ≤ l ≤ s}. Rather than using the non-continuous quasi-empirical distribution, we therefore propose to use a smoothed approximation, where the values between the jumps are interpolated in the empirical distribution function. The idea is to avoid the lattice structure and improve the approximation of the inverse distribution function. Theorem 2. Let ωN = (x1 , . . . , xN ) be a sequence in U s with discrepancy function on U s with bounded, continuous DN (ωN ), and H(x) 2s a distribution (i) density h(x)
= i=1 hi (x ) and hi (x(i) ) ≤ M < ∞ for all i. Furthermore, x let Hi (x) = 0 hi (u)du and define for k = 1, . . . , N and l = 1, . . . , s the values (l)−

xk

(l)+

xk

= A=

(l)

and

xk

(l)

and

xk

8

max 9 xi , (l) (l) xi ∈ ωN Hl (xi )≤xk

8

min 9 xi , (l) (l) xi ∈ ωN Hl (xi )≥xk

= B=

(l)−

= 0 for A = ∅ ,

(l)+

= 1 for B = ∅ .

Then the discrepancy of the set ω ¯ N = (y k )1≤k≤N generated by     (l)+ (l) (l) (l)− − xk Hl xk xk − Hl xk (l)     xk(l)− +     xk(l)+ yk = (l)+ (l)− (l)+ (l)− − Hl xk − Hl xk Hl xk Hl xk

(4)

can be bounded by DN,H (¯ ωN ) ≤ (1 + 2sM )DN (ωN ) . For the proof, we need to recall a slightly improved lemma from the original paper of Hlawka and M¨ uck [8]: Lemma 2. Let ω1 = (u1 , . . . , uN ) and ω2 = (v 1 , . . . , v N ) be two sequences in U s and ε > 0. If for all 1 ≤ j ≤ s and all 1 ≤ i ≤ N the condition (j) (j) |ui − vi | ≤ ε holds, we get the following bound on the difference of the discrepancies: (5) |DN (ω1 ) − DN (ω2 )| ≤ 2sε . Sketch of proof. The proof follows the lines of Lemma 2 of [8], but utilizes the following inequality s s   γi ≤ 2sε (γi ± 2ε) − i=1

i=1

with 0 ≤ γi − 2ε < γi < γi + 2ε ≤ 1. This inequality can easily be seen by induction on s.

Non-Uniform Low-Discrepancy Sequences and Singular Integration

169

Proof (of Theorem 2). We first start with the one-dimensional case. From the definitions and the assumptions on H it follows that ω ¯ is Hω ) = DN (H(¯ ω )). We want to apply distributed with discrepancy DN,H (¯ ωN ), so for 1 ≤ k ≤ N we obtain Lemma 2 with ω1 = ωN and ω2 = H(¯ |H(yk ) − xk | = H(yk ) − H(H −1 (xk )) =  yk h(t)dt ≤ M yk − H −1 (xk ) ≤ M DN (ωN ). H −1 (xk )

(6)

The last inequality can be proved as follows: By the definition of x− k and + we have H(x− k ) ≤ xk ≤ H(xk ), and by the monotonicity of H (assuming it is invertible, otherwise similar arguments can be used) we get x+ k

−1 x− (xk ) ≤ x+ k ≤H k .

Furthermore, yk as constructed in (4) is just a linear interpolation between + − + x− k and xk , so we have the same bounds: xk ≤ yk ≤ xk . Subtracting these two, we get the estimate −1 H (xk ) − yk ≤ x+ − x− ≤ max min |xi − xj | ≤ DN (ωN ) , (7) k k 1≤j≤N i =j

where the last inequality can easily be seen via the definition of the discrepancy, or via the notion of dispersion (see e.g. [3]). Applying Lemma 2 with ε = M DN (ωN ) finally gives: |DN (ωN ) − DN,H (¯ ωN )| ≤ 2M DN (ωN ) and thus DN,H (¯ ωN ) ≤ (1 + 2M )DN (ωN ). For the multi-dimensional version we can bound the one-dimensional projections like in the one-dimensional case (6), so applying Lemma 2 we get DN,H (¯ ωN ) ≤ DN (ωN ) + 2sM DN (ωN ) = (1 + 2sM )DN (ωN ).



Remark 2. Theorem 2 also holds for the star discrepancy, defined as 1 ∗ DN,H (¯ AN (J, ω ωN ) = sup ¯ N ) − H(J) . J=[0,a)⊆U s N Lemma 2 holds without the factor 2, while equation (6) has an additional factor of 2 in the one-dimensional case. Since the multi-dimensional case only uses the one-dimensional projections, this factor does not increase to 2s but stays constant. Combined, one obtains a bound of the same form as for the (extreme) discrepancy DN,H (ωN ).

170

J. Hartinger and R. Kainhofer

Remark 3. From the proof, it can readily be seen that this bound holds for every construction that leads to   (l) (l)− (l)+ yk ∈ xk , xk . Thus the kind of interpolation is not relevant for the discrepancy bound, as the smoothness of the interpolation is not taken into account. Using some additional restrictions on the interpolation, one might find even better bounds. (l)

(l)−

x

(l)+

+x

For example, if one chooses yk = k 2 k , equation (7) holds with an ωN ) ≤ (1 + sM )DN (ωN ). upper bound of DN (ωN )/2, and thus DN,H (¯ In order to integrate functions with singularities at the boundary it will be convenient to shift the interpolated sequence in an appropriate way to avoid regions that lie too close to the singularity. Corollary 1. Let (¯ ωN ) = {y1 , . . . , yn } be constructed as in Theorem 2. Then y1 , . . . , yˆn } defined by the sequence ω ˆ N = {ˆ ⎧ (l)+ ⎪ if A = ∅, ⎨xk (l) yˆk = xk(l)− if B = ∅, ⎪ ⎩ (l) yk otherwise, has an H-discrepancy of order DN,H (ˆ ω ) ≤ (1 + 2M )s DN (ω) (j)

(j)

and the same distance mink=1,...,N min1≤j≤s min(ˆ yk , 1 − yˆk ) to the boundaries as the original sequence ω.  the construction one might question why one does not use the point set  k In N 0≤k≤N to approximate the distribution function. However, then adding one single point to the set would change the whole set of support points. As a result, the distribution function for all support points would have to be reevaluated. If one uses the points of the original low-discrepancy sequence, the distribution function only has to be evaluated at the new point, although all y k will still have to be readjusted. 4.1 Using Different Sequences for Approximation and Inversion In the transformation described in the previous section the distribution function has to be evaluated sN times. As this evaluation is the numerically expensive part of the generation, for moderate values of N it is of advantage for practical applications to lower the number of evaluations. The idea now is to use the same one-dimensional low-discrepancy sequence ω ˆ = (zi )0≤i≤N for all dimensions to approximate the distribution function. If two or more dimensions share the same marginal distribution, the cumulative distribution function has to be evaluated only N times instead of a multiple of N . Again, the resulting sequence displays the low-discrepancy property:

Non-Uniform Low-Discrepancy Sequences and Singular Integration

171

Theorem 3. Let ω ˆ = (zi )1≤i≤N be a one-dimensional sequence with discrepω ), and ω = (xi )1≤i≤N an s-dimensional sequence with discrepancy ancy DN (ˆ DN (ω). Let furthermore H(x) like in Theorem 2, and similarly define (l)−

zk

= A=

8

max 9 zi (l) zi ∈ ω ˆ Hl (zi )≤xk (l)−

and

(l)+

zk

=

min 9 8 (l) B= zi ∈ ω ˆ Hl (zi )≥xk

zi .

(l)+

Again, we set zk = 0 if A = ∅ and zk = 1 if B = ∅. Then the H-discrepancy of any transformed sequence ω ¯ = (y k )1≤k≤N with   (l) (l)− (l)+ for all 0 ≤ k ≤ N and 0 ≤ l ≤ s can be the property yk ∈ zk , zk bounded by DN,H (¯ ω ) ≤ DN (ω) + DN (ˆ ω )(1 + 2M )s . Proof. Similar to the proof of Theorem 2 we obtain in one dimension yk − H −1 (xk ) ≤ z + − z − ≤ DN (ˆ ωN ) , k k and from this |H(yk ) − xk | ≤ M DN (ˆ ωN ). As a result, we have DN,H (¯ ωN ) = ωN )) ≤ 2M DN (ˆ ωN ) + DN (ωN ). DN (H(¯ Applying the same steps to the one-dimensional projections, and using the same relations as in the previous theorem, we get the bound s

ωN ) ≤ DN (ωN ) + DN (ˆ ωN ) (1 + 2M ) DN,H (¯ for the multi-dimensional case.



Remark 4. Using low-discrepancy sequences, one can obtain discrepancies of order DN (ωN ) = O ((log N )s /N ) and DN (ˆ ω ) = O (log N/N ). To get a better understanding of the differences in the Hlawka-M¨ uck method and our interpolation method, both are depicted in Figure 1. In many applications, like the evaluation of Asian options with a given distribution of the stock prices, all dimensions share the same one-dimensional distribution, and thus the distribution function can be factored into a product 2s of s identical factors: H(x) = i=1 H (1) (x(i) ), where H (1) (x) denotes the onedimensional distribution function. In that case, the distribution function has to be evaluated only N times, instead of sN time as in other methods. Since the sequence to approximate the distribution function and the sequence used for inversion are now decoupled, we can lower the generation effort even more by pre-sorting the support points H (1) (zk ) 1≤k≤N : 2s Lemma 3. Let H(x) = i=1 H (1) (x(i) ). The numerical cost of generating an N -element, H-distributed low-discrepancy sequence as defined in Theorem 3 has a numerical cost of O(sN log N ). Proof. The generation of the H-distributed sequence consists of several steps:

172

J. Hartinger and R. Kainhofer Hlawka Mück method

1

Inversion method with interpolation

1

0.8

0.8 xk

xk

0.6

0.6

0.4

0.4

0.2

0.2

x2

x4 0.2

x1 x5 0.4

yk yk x3 0.6 0.8

x2 1

x4 0.2

x1 x5 0.4

yk 0.6

yk

x3 0.8

1

Fig. 1. Hlawka-M¨ uck and our interpolation construction.

(1) (2) (3) (4) (5)

Generation of the uniformly distributed sequence (ˆ ωN ), Generation of the uniformly distributed sequence (ωN ), ˆ N = H (1) (ˆ ωN ), Calculation of the distribution function H ˆN , Pre-sorting the support points H For each 1 ≤ n ≤ N + a) finding the corresponding values z − k and z k , and b) calculating the resulting point y k .

Clearly, (1) and (3) are of order O(N ), and (2) is of order O(sN ). Sorting an N -element set of numerical values is of order O(N log N ) using Merge Sort or Heap Sort (see [9]). Finally, for each of the s dimensions of the N elements, (l)− (l)+ ˆ N are finding the values of z k and z k is of order O(log N ) since the H (l) (l)± already sorted. The actual calculation of y k from the z k is of constant order for each of the sN elements. Thus we obtain an asymptotic order of 2O(N ) + O(sN ) + O(N log N ) + O(sN log N ) + O(sN ) = O(sN log N ).  Remark 5. In many cases by far the most expensive operation will be the evaluation of H (1) , so for moderate values of N the computational cost will be roughly proportional to N . Asymptotically, however, the order will still become O(N log N ). ˜ = Remark 6. If one does not use an N -element sequence as ω ˆ N , but an N logp N  -element sequence, one can always add new points to the sequence in p ˜ . Only for these linear effort, until the number of elements gets larger then N logarithmically many points all N points created so far need to be readjusted. For all other cases the already existing points do not need to be touched. This is of advantage and will also lower the total simulation effort if one does not know the exact number of required points a priori.

Non-Uniform Low-Discrepancy Sequences and Singular Integration

173

4.2 Comparing the Actual Discrepancy As a quick check of our results we compared the discrepancy of the sequences generated by the Hlawka-M¨ uck and our interpolated transformation method with the discrepancy of the original sequence. Unfortunately, the discrepancy ωN ) cannot be calculated explicitly in dimensions higher than 2, so we DN (¯ compared the L2 -discrepancy, which describes the mean error instead of the maximum error over all intervals [0, a) ⊆ U s . As one can expect from the discrepancy bounds proved above, for both transforms we do not see any effect in the L2 -discrepancy compared to the untransformed sequences.

5 Non-Uniform Integration of Singular Functions Using the results from the previous sections, one can bound the QMC integration error for functions of finite variation. However, for functions with a singularity - which appear for example in many problems from finance - these bounds are infinite. Sobol’ [17] proved a convergence theorem for singular integration using uniformly distributed low-discrepancy sequences, Owen [15] proved the corresponding error orders under some growth conditions on the function. Hartinger, Kainhofer, and Tichy [4] proved a similar convergence theorem for non-uniformly distributed sequences, albeit using an L-shaped region for cutting off the integral. Sobol’ and Owen, in contrast, mainly looked at hyperbolic regions, which require more sophisticated proof techniques, but can give better error orders in general. Both their proofs make use of the so-called low-variation extension of a function (see [14], who credits Sobol’ with the idea). In the following we will use Owen’s notations, where (a : b) denotes the set of integers a through b, while for u ⊂ (1 : s) and x, y ∈ U s we denote by xu : y −u the point where the coordinates u are taken from x, and the coordinates (1 : s) \ u are taken from y. Also, we will use two special types of regions that exclude a certain volume around the origin or all corners: 8 9 orig (ε) = x ∈ U s min x(j) > ε , (8) Kmin 1≤j≤s

s  8 9 orig s x(j) > ε , Kprod (ε) = x ∈ U

(9)

j=1

8 9 corner (ε) = x ∈ U s min min(x(j) , 1 − x(j) ) > ε , Kmin

(10)

8 corner (ε) = x ∈ U s Kmin

(11)

1≤j≤s s 

9 min(x(j) , 1 − x(j) ) > ε .

j=1 orig avoids the origin and the lower boundaries via an L-shaped region, while Kmin orig corner corner Kprod avoids it via a hyperbolic region. Kmin and Kprod have similar avoidance patterns, but for all corners at the same time.

174

J. Hartinger and R. Kainhofer

Definition 4 (low-variation extension). Let f : U s → R be an s-times differentiable function (possibly unbounded at the definition boundaries, but bounded inside). Furthermore, let K ⊆ U s be a region with anchor c ∈ U s . That is, for each x ∈ K we have [x, c] ⊆ K. Then the low-variation extension f˜ of f from K to U s is defined by    ( |u| ˜ f (x) = f (c)+ 1 z(u) :c(−u) ∈K ∂ u f z (u) : c(−u) dz (u) . (−1) [x(u) ,c(u) ] ∅ =u⊆(1:s) (12) Owen [14] showed that its Vitali and Hardy-Krause variation are bounded by  (1:d) V f (x) dx , ∂ K   ( u VHK (f˜) ≤ ∂ f x(u) : 1(−u) dx(u) , Us

(f˜) ≤

u =∅

(13) (14)

Ku (1(−u) )

  using the definition Ku (b(−u) ) = x(u) ∈ U |u| |x(u) : b(−u) ∈ K . We will in the sequel only consider singular functions that fulfill one of the growth conditions for some Aj > 0, B < ∞, and all u ⊆ (1 : s): |∂ u f (x)| ≤ B |∂ u f (x)| ≤ B

s   j=1 s 

x(j)

−Aj −1 j∈u

, or

−Aj −1 j∈u  min x(j) , 1 − x(j) .

(15)

(16)

j=1

5.1 L-shaped Regions orig As a first case we will consider sequences that lie in Kmin (ε), and thus avoid the origin in an L-shaped region. This property can easily be seen for the Halton and general (0, s)-sequences, and also for non-uniform low-discrepancy sequences that are generated by the Hlawka-M¨ uck transformation (with shift, as shown in [4]). This case was already investigated by the authors in [4], but no explicit error bounds were given. The error bounds given by Owen [14] for the uniform distribution are easily generalized.

Theorem 4. Let f : U s → R, and ωN = {x1 , . . . , xN } be a sequence with orig xj ∈ Kmin (εN ) for 1 ≤ j ≤ N . Let furthermore H(x) be a distribution on U s with density h(x) and Mε = supx∈U s \K orig (ε) h(x) ≤ ∞. If f fulfills growth min condition (15), and 0 < εN = CN −r < 1, then

Non-Uniform Low-Discrepancy Sequences and Singular Integration

175

 N 1s 1 ( f (x)dH(x) − f (xn ) ≤ C1 DN,H N r j=1 Aj +C2 N r(max Aj −1) MεN Us N n=1 (17) with some explicitly computable, finite constants C1 and C2 . corner (εN ) for all j with 0 < εN = CN −r < 1/2, and f is Also, if xj ∈ Kmin s a real-valued function on (0, 1) that fulfills growth condition (16), then (17) corner (ε) in that case. holds. Mε has to be taken as the supremum over U s \ Kmin The proof is obtained by replacing the Koksma-Hlawka bound in [15, Proof of Theorem 5.2] by Chelson’s non-uniform bound (1) and factoring out the supremum of the density Mε when necessary. Proof. Using a 3 ε-argument, we have   N   1 ( ˜ f (x)dH(x) − f (xn ) ≤ f (x) − f (x) dH(x) + Us N n=1 Us  N N N 1 ( ( ( 1 1 f (xn ) . f˜(x)dH(x) − f˜ (xn ) + f˜ (xn ) − Us N N N n=1

n=1

(18)

n=1

orig The last term vanishes, since f (x) = f˜(x) on K = Kmin (εN ). The second term can be bounded by VHK (f˜)DN,H (ω) using the nonuniform Koksma-Hlawka inequality (1), and using Owen’s inequality (14) for 1s 2s VHK (f˜) even further by C1 N r j=1 Aj DN,H (ω) with C1 = B j=1 C −Aj A−1 j . Finally, for the first term we use Lemma 5.1 of [14]. If K ⊆ U s with anchor c = 1, and f fulfills growth condition (15), then for all x ∈U K = Us − K we  (j) −Aj 1 ˜ 2s ˜ = B 2s have f (x) − f˜(x) ≤ B with B j=1 x j=1 1 + Aj . Thus, the first term can be bounded by



Us



 ˜ MεN B

  ˜ f (x) − f (x) dH(x) ≤ s  

U K j=1

f (x) − f˜(x) h(x)dx ≤

UK

s  −Aj  (j) ˜ x dx ≤ MεN B j=1

1 1 − Aj



s C 1−min Ak N r(max Ak −1) .

The last inequality follows from direct  similar to [15, Proof of  integration, 1 ˜ 2s sC 1−min Aj . Theorem 5.2]. Thus we have C2 = B j=1 1−Aj

For the corner-case, we note that  the unit cube can be partitioned into 2s cubes with anchor 12 = 12 , . . . , 12 , and each of them can be bounded like above. Furthermore, the variation on each of them sums up to the variation on the whole unit interval, thus we get the same bound with an additional factor 2s in the constants. 

176

J. Hartinger and R. Kainhofer

Remark 7. Suppose that one uses some classical low discrepancy construction (e.g. Sobol’, Faure, or Halton sequences) in combination with (shifted) Hlawka-M¨ uck or the (shifted) interpolation method. Then r = 1 and DN,H ≤ CN −1+ε , so the corresponding error will be of the order   1s O N −1+ε+ j=1 Aj . When using importance sampling with a distribution that has different tail behavior than the original distribution, one often ends up with a singular integral, where the density also has a singularity at the boundary. In this case, Mε is not finite, and the bound from above does not give any sensible result. On the other hand, if the density tends to zero, one has to expect that the effect of the singularity of the functions should be somehow lightened. Thus we will now look at densities that fulfill another ”growth condition” in a region U s \ K around the origin: ∀x ∈ U s \ K : h(x) ≤ Ch

s  

x(j)

−A˜j

,

for some A˜j < 1, Ch ∈ R . (19)

j=1

If A˜j = 1, the bound is not integrable any more. However, since h is a distribution density function and thus integrable, one should be able to find an integrable bound. We will also assume that Aj + A˜j < 1, as otherwise the bound for the whole integral would be infinite. Using this growth condition, one can now prove a version of the theorem that takes into account the behavior of h(x) near the origin (or all corners): Theorem 5. Let ωN , H, and f be the sequence, distribution, and integrand from Theorem 4. If furthermore the density h(x) satisfies the growth condition (19), then  N 1s 1 ( ˜ f (x)dH(x) − f (xn ) ≤ C1 DN,H N r j=1 Aj +C˜2 N r(max(Aj +Aj )−1) Us N n=1 ˜ Ch 2s with C1 from Theorem 4, and C˜2 = B j=1

˜j ) 1 1−min(Aj +A . ˜j s C 1−Aj −A

Proof. The proof follows along the lines of Theorem 4, the major difference being in the bound for the first term:  U s \K

 ˜ h f (x) − f˜(x) h(x)dx ≤ BC

s  

x(j)

−(Aj +A˜j )

dx

U s \K j=1

˜ h ≤ BC

s 

1 ˜ ˜ C 1−min(Aj +Aj ) sN r(max(Aj +Aj )−1) . ˜ 1 − (Aj + Aj ) j=1



Non-Uniform Low-Discrepancy Sequences and Singular Integration

177

5.2 Hyperbolic Regions A serious improvement in the bound for the error order can be obtained by choosing sequences that avoid the origin in a hyperbolic sense (i.e. sequences orig that lie in Kprod (ε)) and thus more strongly, as Owen [15] showed for the 1 uniform distribution. In that case, the Aj in the bound can be replaced by max Aj . We will now state a similar theorem for arbitrary distributions H: Theorem 6. Let f (x) be a real-valued function on U s (possibly unbounded at the lower boundary) which satisfies growth condition (15). Let furthermore orig (εN ) and 0 < εN = CN −r < 1 ωN = (xi )1≤i≤N be a point set with xi ∈ Kprod for some constants C, r > 0. Finally, let H(x) be a distribution on U s with density h(x) that satisfies growth condition (19). Then for all η, η˜ > 0 we have  N 1 ( f (x)dH(x) − f (xn ) ≤ Cη(1) DN,H (ω) N η+r maxj Aj Us N n=1

˜

(2)

+ Cη˜ N η˜+r maxj (Aj +Aj )−r (1)

(20)

(2)

for constants Cη and Cη˜ . A similar bound holds for the corner case when εN < 2−s . The bound holds with η = 0 if the maximum among the Aj is unique, and with η˜ = 0 if the maximum among the Aj + A˜j is unique. orig Proof. We again denote by f˜ the low-variation extension of f from Kprod (εN ) s to U with anchor 1. Again (18) holds, and the first term can be bounded by

 U s \K

 ˜ ˜ h(x)dx ≤ BC f (x) − f (x) h

s  

x(j)

−(Aj +A˜j )

dx

U s \K j=1

  ˜ = O ε1−maxj (Aj +Aj )

(21)

using a lemma of Sobol’ ( [17, Lemma 3] or [14, Lemma 5.4]) if all Aj + A˜j are distinct. If any two of the Aj + A˜j are equal, and they are not the maximum, one can increase one of them by a small value without affecting the max. If the maximum is not distinct, one has to increase some of them and thus the maximum by no more than η˜/r. The variation of f˜ was already proved by Owen to be bounded by VHK (f˜) ≤ C1 N r maxj Aj if the maximum among the Aj is distinct. If this is not the case a similar argument like before brings in the η in the bound. Combining these two bounds, we arrive at (20). The corner case can be argued similarly (see [14, Proof of Theorem 5.5]) by splitting the unit cube into 2s sub-cubes and investigating each of them separately. 

178

J. Hartinger and R. Kainhofer

Remark 8. Determining the asymptotics of εN for hyperbolic regions is more delicate than for the L-shaped regions, even for the classical sequences and the uniform distribution. Some results for Halton sequences were obtained by Owen [15] and for more general sequences by Hartinger, Kainhofer and Ziegler [5]. Nevertheless, it is obvious that sequences obtained by Hlawka-M¨ uck’s construction or using interpolation do not result in better asymptotics than N −s . Thus, in combination with classical low discrepancy sequences one will get error estimates of the order   O N −1+ε+s maxj=1,...,s Aj . Acknowledgments The authors want to thank Art Owen for several inspiring discussions and suggestions, and the anonymous referee for pointing out the improved Lemma 2 and several other improvements to our results. This research was supported in part by the Austrian Science Fund Project S-8308-MAT.

References 1. P. Chelson. Quasi-Random Techniques for Monte Carlo Methods. PhD. Dissertation, The Claremont Graduate School, 1976. 2. L. Devroye. Non-Uniform Random Variate Generation. Springer-Verlag, New York, 1986. 3. M. Drmota and R. F. Tichy. Sequences, Discrepancies and Applications, volume 1651 of Lecture Notes in Mathematics. Springer-Verlag, Berlin, 1997. 4. J. Hartinger, R. Kainhofer, and R. Tichy. Quasi-Monte Carlo algorithms for unbounded, weighted integration problems. Journal of Complexity, 20(5):654– 668, 2004. 5. J. Hartinger, R. Kainhofer, and V. Ziegler. On the corner avoidance properties of various low-discrepancy sequences. Submitted, 2005. ¨ 6. E. Hlawka. Gleichverteilung und Simulation. Osterreich. Akad. Wiss. Math.Natur. Kl. Sitzungsber. II, 206:183–216, 1997. 7. E. Hlawka and R. M¨ uck. A transformation of equidistributed sequences. In Applications of Number Theory to Numerical Analysis, pages 371–388. Academic Press, New York, 1972. ¨ 8. E. Hlawka and R. M¨ uck. Uber eine Transformation von gleichverteilten Folgen. II. Computing, 9:127–138, 1972. 9. D. E. Knuth. The Art of Computer Programming. Volume 3. Sorting and Searching. Addison-Wesley, Reading, Massachusetts, 1973. 10. T. Kollig and A. Keller. Efficient multidimensional sampling. Computer Graphics Forum, 21(3):557–563, 2002. 11. W. J. Morokoff and R. E. Caflisch. Quasi-Monte Carlo integration. J. Comput. Phys., 122(2):218–230, 1995. 12. R. B. Nelsen. An Introduction to Copulas, volume 139 of Lecture Notes in Statistics. Springer-Verlag, New York, 1999.

Non-Uniform Low-Discrepancy Sequences and Singular Integration

179

13. H. Niederreiter. Random Number Generation and Quasi-Monte Carlo Methods, volume 63 of SIAM Conf. Ser. Appl. Math. SIAM, Philadelphia, 1992. 14. A. B. Owen. Multidimensional variation for Quasi-Monte Carlo. In J. Fan and G. Li, editors, International Conference on Statistics in Honour of Professor Kai-Tai Fang’s 65th Birthday, 2005. 15. A. B. Owen. Halton sequences avoid the origin. SIAM Review, 48, 2006. To appear. 16. S. H. Paskov and J. Traub. Faster valuation of financial derivatives. Journal of Portfolio Management, pages 113–120, 1995. 17. I. M. Sobol’. Calculation of improper integrals using uniformly distributed sequences. Soviet Math. Dokl., 14(3):734–738, 1973. 18. X. Wang. Improving the rejection sampling method in Quasi-Monte Carlo methods. J. Comput. Appl. Math., 114(2):231–246, 2000.

Construction of Good Rank-1 Lattice Rules Based on the Weighted Star Discrepancy Stephen Joe Department of Mathematics, University of Waikato, Private Bag 3105, Hamilton, New Zealand [email protected] Summary. The ‘goodness’ of a set of quadrature points in [0, 1]d may be measured by the weighted star discrepancy. If the weights for the weighted star discrepancy are summable, then we show that for n prime there  exist n-point rank-1 lattice rules  whose weighted star discrepancy is O n−1+δ for any δ > 0, where the implied constant depends on δ and the weights, but is independent of d and n. Further, we show that the generating vector z for such lattice rules may be obtained using a component-by-component construction. The results given here for the weighted star discrepancy are used to derive corresponding results for a weighted Lp discrepancy.

1 Introduction Integrals over the d-dimensional unit cube given by  f (x) dx Id (f ) = [0,1]d

may be approximated using n-point rank-1 lattice rules. These are quadrature rules of the form ' n−1  kz 1( Qn,d (f ) = f , n n k=0

where z ∈ Z is the ‘generating vector’ with no factor in common with n, and the braces around a vector indicate that we take the fractional part of each component of the vector. For our purposes, it is convenient to assume that gcd(zj , n) = 1 for 1 ≤ j ≤ d, where zj is the j-th component of z. The star discrepancy of the point set Pn (z) := {{kz/n}, 0 ≤ k ≤ n − 1} is defined by d

D∗ (Pn (z)) = Dn∗ (z) :=

sup |discr(x, Pn )| , x∈[0,1)d

where discr(x, Pn ) is the ‘local discrepancy’ defined by

182

S. Joe

|Pn (z) ∩ [0, x)| − Vol([0, x)) . n

discr(x, Pn ) :=

(1)

The star discrepancy occurs in the well-known Koksma-Hlawka inequality. Further details may be found in [3] and [19] or in more general works such as [11]. It is known (see [10] or [11]) that there exist d-dimensional rank-1 lattice rules whose star discrepancy is O(n−1 (ln(n))d ) with the implied constant depending on only d. For n prime it was shown in [4] that such rules may be obtained by constructing their generating vectors component-by-component. In this paper we extend these results to the case of a weighted star discrepancy. Such component-by-component constructions first appeared in [17], but there the integrands were in a periodic setting. Since then, there has been much work done in the L2 case both in the periodic setting of weighted Korobov spaces and in the non-periodic setting of weighted Sobolev spaces (for example, see [8–10, 14], and [15]). Here we consider the weighted star discrepancy, since, as we shall see later, we are able to derive corresponding results for the weighted Lp discrepancy. In order to introduce the weighted star discrepancy, let u be any subset of D := {1, 2, . . . , d − 1, d} with cardinality |u|. For the vector x ∈ [0, 1]d , let xu denote the vector from [0, 1]|u| containing the components of x whose indices belong to u. By (xu , 1) we mean the vector from [0, 1]d whose j-th component is xj if j ∈ u and 1 if j ∈ u. From Zaremba’s identity (see [18] or [19]) we have Qn,d (f ) − Id (f ) =

(



(−1)|u|

[0,1]|u|

∅ =u⊆D

discr ((xu , 1), Pn )

∂ |u| f (xu , 1) dxu . ∂xu

Now let us introduce a sequence of positive weights {γj }∞ j=1 and set γu =



γj

with

γ ∅ := 1 .

(3)

j∈u

Then we can write Qn,d (f ) − Id (f )  ( = (−1)|u| γ u ∅ =u⊆D

[0,1]|u|

discr ((xu , 1), Pn ) γ −1 u

∂ |u| f (xu , 1) dxu . ∂xu

Applying H¨ older’s inequality for integrals and sums we obtain  |Qn,d (f ) − Id (f )| ≤

sup

sup

∅ =u⊆D xu ∈[0,1]|u|

⎛

×⎝

( u⊆D

γ −1 u



γ u |discr ((xu , 1), Pn )|

⎞ |u| ∂ f (xu , 1) dxu ⎠ . [0,1]|u| ∂xu



(2)

Construction of Lattice Rules Using the Weighted Star Discrepancy

183

∗ Then we can define a weighted star discrepancy Dn,γ (z) by ∗ Dn,γ (z) := sup γ u ∅ =u⊆D

sup xu ∈[0,1]|u|

|discr ((xu , 1), Pn )| .

(4)

In Sect. 2 we use an averaging argument to show that if the weights γj are summable, there exist rank-1 lattice rules whose weighted star discrepancy is O(n−1+δ ) for any δ > 0, where the implied constant depends on δ and the weights. A more specific averaging argument is applied to lattice rules of the Korobov form, namely those for which z = (1, a, . . . , ad−1 ) (mod n), 1 ≤ a ≤ n − 1, to show there exist lattice rules of the Korobov form having O(n−1+δ ) weighted star discrepancy. Besides existence results we are interested in how to find such lattice rules. One way, of course, is to find an appropriate a in the Korobov form. However, such rules are not extensible in dimension; a value of a that is good for one value of the dimension d may not be good for a different value of the dimension. In Sect. 3 we present results showing that, alternatively, the generating vectors z for such lattice rules may be constructed a component at a time resulting in a z which is extensible in dimension. The cost of this componentby-component construction is O(n2 d2 ) operations, but it may be reduced to O(n2 d) operations at the extra cost of O(n) storage. It may be reduced even further to O(n ln(n)d) operations by making use of the approach proposed by Nuyens and Cools in [12]. We remark that constructions for polynomial lattice rules having small weighted star discrepancy have recently been proposed in [1]. As here, they consider a Korobov construction and a component-bycomponent construction. The weighted star discrepancy considered here may be viewed as the L∞ version of a weighted Lp discrepancy for p ≥ 1. Weighted Lp discrepancies have been considered in works such as [2] and [18]. In Sect. 4 we use the results obtained in Sections 2 and 3 for the weighted star discrepancy to derive corresponding results for the weighted Lp discrepancy. Unlike the earlier results in the L2 setting, the results presented here do not require the lattice points to be shifted.

2 Rank-1 Lattice Rules Having Certain Weighted Star Discrepancy Bounds It follows from (4) that the weighted star discrepancy satisfies ( ∗ (z) ≤ γ u sup |discr ((xu , 1), Pn )| . Dn,γ u⊆D

(5)

xu ∈[0,1]|u|

Moreover, it follows from [11, Theorem 3.10 and Theorem 5.6] (see also [2]) that

184

S. Joe

sup xu

∈[0,1]|u|

|discr ((xu , 1), Pn )| ≤ 1 − (1 − 1/n)|u| +

where

|u| 

(

Rn (z, u) =

Rn (z, u) , 2

h·z u ≡0 ( mod n) ∗ h∈Cn,|u|

1 . max(1, |hj |) j=1

Here z u is the vector consisting of the components of z whose indices belong to u and ∗ Cn,|u| = {h ∈ Z|u| , h = 0 : −n/2 < hj ≤ n/2, 1 ≤ j ≤ |u|} .

We then obtain ∗ Dn,γ (z) ≤

( u⊆D

  Rn (z, u) γ u 1 − (1 − 1/n)|u| + . 2

(6)

Under the assumption that gcd(zj , n) = 1 for 1 ≤ j ≤ d, then z u is the generating vector for a |u|-dimensional rank-1 lattice rule having n points. It then follows from the error theory of lattice rules (for example, see [11, Chapter 5] or [16, Chapter 4]) that we may write Rn (z, u) as ⎞ ⎛ n−1 ( e2πihkzj /n ⎠ 1 (⎝ Rn (z, u) = −1 , (7) 1+ n |h| j∈u −n/2
k=0

where the  on the sum indicates that we omit the h = 0 term. ∗ (z) may be obtained by Bounds on the weighted star discrepancy Dn,γ   1 making use of (6). We first consider γ u 1 − (1 − 1/n)|u| . u⊆D

Lemma 1. Suppose the weights γj are summable, that is,

∞ 1

γj < ∞. Then

j=1

(



|u|

γ u 1 − (1 − 1/n)

u⊆D

where Γ :=

∞ 1



1∞

∞ max (1, Γ )  max (1, Γ ) e ≤ (1 + γj ) ≤ n n j=1

j=1

γj

γj /(1 + γj ) < ∞.

j=1

Proof. We may write (

d d     γ u 1 − (1 − 1/n)|u| = (1 + γj ) − (1 + γj (1 − 1/n))

u⊆D

j=1

=

d  j=1

⎡

j=1

(1 + γj ) ⎣1 −

d   j=1

⎤  γj ⎦ . 1− n(1 + γj )

,

Construction of Lattice Rules Using the Weighted Star Discrepancy

According to [2] we have ⎛ d   ⎝ ln 1− j=1

185

⎞  d ( γj γj ⎠ ≥ ln(1 − 1/n) , n(1 + γj ) 1 + γj j=1

which leads to (



γ u 1 − (1 − 1/n)|u|



u⊆D

 1 dj=1 γj /(1+γj ) & 1 . (8) (1 + γj ) 1 − 1 − ≤ n j=1 "

d 

Since 0 < γj /(1 + γj ) < γj , we see that since the γj are summable, then so are the γj /(1 + γj ), that is, Γ < ∞. If Γ ≤ 1, then we have (1 − 1/n)Γ ≥ 1 − 1/n and hence 

1 1− 1− n

Γ ≤

1 . n

Now suppose Γ > 1 and set v(x) = (1 + x)Γ − Γ x − 1 for x > −1. Then it is easily verified that v  (0) = 0. Moreover, v  (0) = Γ 2 − Γ which is positive for Γ > 1. Since v  (x) < 0 for −1 < x < 0 and v  (x) > 0 for x > 0, we deduce that if Γ > 1, then v(x) ≥ v(0) = 0 or (1 + x)Γ ≥ Γ x + 1 for x > −1. With x = −1/n we thus obtain  1−

1 n

Γ ≥−

 Γ 1 Γ Γ + 1 and so 1 − 1 − . ≤ n n n

It then follows from (8) that (



|u|

γ u 1 − (1 − 1/n)

u⊆D



d 

"



1 ≤ (1 + γj ) 1 − 1 − n j=1

Γ &

d ∞ max (1, Γ )  max (1, Γ )  (1 + γj ) ≤ (1 + γj ) ≤ n n j=1 j=1 1∞

max (1, Γ ) e max (1, Γ ) 1 ∞ e j=1 ln(1+γj ) ≤ = n n

j=1

where we have used ln(1 + x) ≤ x for x ≥ 0.

=

( u⊆D

⎡

⎛ n−1 ( 1 ⎝1 + γu ⎣ n j∈u k=0

, 

With γ ∅ = 1, we make use of (3) and (7) to next consider ( Rn,γ (z) := γ u Rn (z, u) u⊆D

γj

( −n/2
⎞ ⎤ e2πihkzj /n ⎠ − 1⎦ |h|

186

S. Joe

⎞ ⎛ ⎤ n−1 2πihkzj /n (  ( e 1 ⎠− ⎣ = γj ⎝1 + γj ⎦ n |h| j∈u u⊆D k=0 j∈u −n/2
⎡

u⊆D

−n/2
k=0

By interchanging the first two sums, we obtain ⎞ ⎛ n−1 d ( e2πihkzj /n ⎠  1(( ⎝ − Rn,γ (z) = γj 1 + (1 + γj ) n |h| j=1 k=0 u⊆D j∈u −n/2
k=0

Setting βj = 1 + γj , we then see that ⎛ n−1 d (  ( 1 ⎝βj + γj Rn,γ (z) = n j=1

⎞ 2πihkzj /n

e

−n/2
k=0

|h|

⎠−

d 

βj .

(9)

j=1

In the case d = 1, it is not hard to verify that Rn,γ (z) = 0. We also see from this expression that for given dimension d, calculation of Rn,γ (z) would require O(n2 d) operations. However, the asymptotic expansion techniques found in [12] may be used to reduce this to O(nd) operations. Further details may be found in Appendix A. We shall obtain bounds on Rn,γ (z) for the case in which n is prime by obtaining an expression for the mean value of Rn,γ (z) taken over all integer vectors z ∈ Znd , where Zn = {1, 2, . . . , n−1}. Thus the mean Mn,d,γ is defined by ( 1 Mn,d,γ := Rn,γ (z) . (n − 1)d d z∈Zn

Theorem 1. Let n be a prime number. Then Mn,d,γ

  d d  d Sn 1 n−1  = (βj + γj Sn ) + βj , βj − γj − n j=1 n j=1 n−1 j=1

where Sn =

( −n/2
1 . |h|

Proof. In (9) we can take out the k = 0 term which is independent of z to obtain

Construction of Lattice Rules Using the Weighted Star Discrepancy

187

Mn,d,γ d 1 = (βj + γj Sn ) n j=1 ⎡ ⎛ n−1 n−1 d (  ( 1 ⎣ 1 ⎝βj + γj + n n − 1 z =1 j=1 k=1

j

(

⎞⎤ 2πihkzj /n

e

|h|

−n/2
⎠⎦ −

d 

βj .

j=1

Now define Tn (k) :=

n−1 1 ( n − 1 z=1

( −n/2
e2πihkz/n , |h|

0≤k ≤n−1.

(10)

When k = 0, Tn (0) is simply Sn . For n prime and 1 ≤ k ≤ n − 1 we see that k cannot be a multiple of n, and nor can h in the situation when −n/2 < h ≤ n/2 with h = 0. Hence hk ≡ 0 (mod n) and we have n−1 (

(

e2πihkz/n |h| −n/2
Tn (k) =

1 n−1

(11)

−n/2
which we note is independent of k. It then follows that Mn,d,γ =

  d n−1 d  d −Sn 1 1 ( (βj + γj Sn ) + βj , βj + γj − n j=1 n n−1 j=1 j=1 k=1

which leads to the desired result.



In the case d = 1, the expression for Mn,1,γ1 simplifies to 0, which is as expected, since for d = 1 the values of Rn,γ1 (z1 ) are all zero. Since βj = 1 + γj > γj and Sn ≤ n − 1, we have βj > βj − γj Sn /(n − 1) ≥ 1 and so   d  d Sn n−1  βj < 0 . βj − γj − n j=1 n−1 j=1 Moreover, we have from [10, Lemmas 1 and 2] that Sn < 2 ln(n) + 1/n2 − 0.2319. So for n ≥ 3 we have Sn < 2 ln(n)

(12)

and direct calculation shows this holds for n = 2 also. We then obtain the following corollary.

188

S. Joe

Corollary 1. Let n be a prime number. Then there exists a generating vector z such that Rn,γ (z) ≤

d d 1 1 (1 + γj + γj Sn ) ≤ (1 + γj + 2γj ln(n)) . n j=1 n j=1

Now recall from (6) and the definition of Rn,γ (z) that ∗ Dn,γ (z) ≤

( u⊆D

  R (z) n,γ . γ u 1 − (1 − 1/n)|u| + 2

(13)

This equation together with Lemma 1 and Corollary 1 show that if the γj are summable, then there exists a generating vector z such that d   1  ∗ (z) ≤ O n−1 + (1 + γj + 2γj ln(n)) , Dn,γ 2n j=1

where the implied constant depends on the weights, but is independent of d. ∗ (z) has a ln(n) dependency. In order to obtain a bound This bound for Dn,γ without this ln(n) dependency, we can make use of the next lemma (stated and proved in [2]) and conclude that there exists a generating vector z such that   ∗ (z) = O n−1+δ , Dn,γ for any δ > 0, where the implied constant depends on δ and the weights, but is independent of d and n. Lemma 2. Let γ˜j = 2γj /(1 + γj ) and suppose that the γj are summable so that ∞ ( γ˜j < ∞ . j=1

Then for any δ > 0, there exists C(˜ γ , δ), independent of d and n, such that d 

(1 + γj + 2γj ln(n)) ≤ C(˜ γ , δ)nδ

j=1

∞ 

1∞

(1 + γj ) ≤ C(˜ γ , δ)nδ e

j=1

γj

.

j=1

We recall from Sect. 1 that lattice rules of the Korobov form are those for which z = (1, a, . . . , ad−1 ) (mod n) for some a satisfying 1 ≤ a ≤ n − 1. Writing such generating vectors as z(a), we now define the mean µn,d,γ :=

n−1 1 ( Rn,γ (z(a)) . n − 1 a=1

The next result shows that µn,d,γ satisfies a bound of the same order as the one given in Corollary  1. Hence there exist lattice rules of the Korobov form  which have O n−1+δ weighted star discrepancy.

Construction of Lattice Rules Using the Weighted Star Discrepancy

189

Theorem 2. Let n be a prime number. Then µn,d,γ ≤

d d  (1 + γj + γj Sn ) . n − 1 j=1

Proof. The proof we present is similar to the proof of Theorem 1 in [18]. We see from (9) that Rn,γ (z(a)) is the error from applying the lattice rule to the function ( e2πih·x , f (x) = 2d j=1 r(γj , hj ) h∈C ∗ ∪{0} n,d

where r(γ, h) =

1 + γ,

h=0 ,

|h|/γ,

h = 0 .

It then follows from the theory of lattice rules that we may write (

Rn,γ (z(a)) =

∗ h∈Cn,d

δn (h · z(a)) , 2d j=1 r(γj , hj )

where δn (m) denotes one or zero depending on whether m ≡ 0 (mod n) or not. From the definition of µn,d,γ , it follows that we have µn,d,γ

d n−1 ( (  1 1 = δn (h · z(a)) . n−1 r(γj , hj ) a=1 ∗ j=1

(14)

h∈Cn,d

Since h · z(a) = h1 + h2 a + · · · + hd ad−1 , we see this last sum is just the number of solutions of the congruence h1 + h2 a + · · · + hd ad−1 ≡ 0 (mod n). ∗ , then the greatest common divisor of Now because n is prime and h ∈ Cn,d the numbers h1 , h2 , . . . , hd cannot be a multiple of n. It then follows from a well-known result in number theory (for example, see [6]) that the last sum in (14) is bounded by d − 1. We then have d (  1 d n−1 r(γ j , hj ) ∗ j=1 h∈Cn,d ⎛ d ( d ⎝ < 1 + γj + γj n − 1 j=1

µn,d,γ ≤

−n/2
which leads to the desired bound.

⎞ 1 ⎠ , |h| 

190

S. Joe

3 A Component-by-Component Construction We shall now prove that for n prime we can construct z component-bycomponent such that Rn,γ (z) ≤

d 1  (βj + γj Sn ) , n − 1 j=1

where we recall that βj = 1 + γj . Theorem 3. Let n be a prime number. Suppose there exists a z ∈ Znd such that d 1  (βj + γj Sn ) , Rn,γ (z) ≤ n − 1 j=1

where

(

Sn =

−n/2
1 . |h|

Then there exists zd+1 ∈ Zn such that d+1 1  (βj + γj Sn ) . n − 1 j=1

Rn,γ (z, zd+1 ) ≤

Such a zd+1 can be found by minimizing Rn,γ (z, zd+1 ) over the set Zn . Proof. For any zd+1 ∈ Zn we have from (9) that Rn,γ (z, zd+1 ) = βd+1 Rn,γ (z) +

γd+1 n ⎡

n−1 (

× ⎣

⎡ ⎣

k=0

d 

⎛ ⎝βj + γj

−n/2
j=1

(

(

e

|h|

⎠⎦

⎤

2πihkzd+1 /n

e

−n/2
⎞⎤ 2πihkzj /n

|h|

⎦ .

(15)

Next we average over the possible n − 1 values of zd+1 in the last term to form ( 1 n − 1 z =1 n−1

d+1

( −n/2
e2πihkzd+1 /n , |h|

0≤k ≤n−1.

However, this is just the quantity Tn (k) defined previously in (10). It then follows from (15) by separating out the k = 0 term that there exists a zd+1 ∈ Zn such that

Construction of Lattice Rules Using the Weighted Star Discrepancy

191

d γd+1  (βj + γj Sn ) Sn n j=1 ⎞ ⎛ n−1 d ( e2πihkzj /n ⎠ γd+1 (  ⎝ Tn (k) βj + γj + n |h| j=1

Rn,γ (z, zd+1 ) ≤ βd+1 Rn,γ (z) +

−n/2
k=1

= βd+1 Rn,γ (z) +

γd+1 n ⎛

d 

n−1 d γd+1 (  ⎝ βj + γj + n j=1 k=1

= βd+1 Rn,γ (z) +

γd+1 n

(βj + γj Sn ) Sn

j=1

d 

( −n/2
⎞   e2πihkzj /n ⎠ −Sn |h| n−1

(βj + γj Sn ) Sn

j=1

⎛ ⎛ n−1 d γd+1 Sn ⎝ 1 (  ⎝ − βj + γj + n−1 n j=1 k=0

( −n/2
⎞ e2πihkzj /n ⎠ |h|

⎞ d 1 + (βj + γj Sn )⎠ , n j=1

where we have made use of (11) and in the last equation, subtracted and added in the k = 0 term. By using (9) we find that for this zd+1 we have Rn,γ (z, zd+1 ) ≤ βd+1 Rn,γ (z) +

d γd+1  (βj + γj Sn ) Sn n j=1

⎛ ⎞ d d  γd+1 Sn ⎝ 1 −Rn,γ (z) − + βj + (βj + γj Sn )⎠ n−1 n j=1 j=1 ⎛ ⎞   d γd+1 Sn ⎝  1 ≤ βd+1 Rn,γ (z) + (βj + γj Sn )⎠ 1 + n n−1 j=1 d γd+1 Sn  (βj + γj Sn ) = βd+1 Rn,γ (z) + n − 1 j=1 ⎛ ⎞ d 1 ⎝ ≤ (βj + γj Sn )⎠ (βd+1 + γd+1 Sn ) n − 1 j=1 d+1 1  = (βj + γj Sn ) , n − 1 j=1

192

S. Joe

where we have made use of the fact that Rn,γ (z) satisfies the assumed bound. This completes the proof.  Recalling that for d = 1 we have Rn,γ1 (z1 ) = 0, the previous theorem leads to the following corollary. Corollary 2. Let n be a prime number. We can construct z ∈ Znd componentby-component such that for all s = 1, . . . , d, Rn,γ (z1 , . . . , zs ) ≤

s 1  (βj + γj Sn ) . n − 1 j=1

We can set z1 = 1, and for 2 ≤ s ≤ d, each zs can be found by minimizing Rn,γ (z1 , . . . , zs ) over the set Zn . Since 1/(n − 1) ≤ 2/n for n ≥ 2, this corollary together with (12) and (13) show that for n a prime number, we can construct z component-by-component such that ∗ Dn,γ (z) ≤

( u⊆D

d   1 γ u 1 − (1 − 1/n)|u| + (1 + γj + 2γj ln(n)) . n j=1

If the γj are summable we then see from Lemma 1 and Lemma 2 that the rank-1 lattice rule constructed in this manner is such that ∗ Dn,γ (z) = O(n−1+δ ) ,

δ>0 ,

where the implied constant depends on δ and the weights, but is independent of d and n. Appendix A shows that Rn,γ (z) may be calculated using asymptotic expansion techniques in O(nd) operations. This together with Corollary 2 then shows that the cost of constructing the integer vector z up to dimension d is O(n2 d2 ) operations. This can be reduced to O(n2 d) operations if we store the products during the construction, but this would be at the expense of O(n) storage. We remark that in [12], Nuyens and Cools proposed a more efficient implementation of the component-by-component construction. Their construction of z was based on minimizing a function of the form  ' n−1 d  kzj 1 ( 1 + γj ω −1 , n n j=1 k=0

where ω was a certain function. Now we see from (9) that Rn,γ (z) may be written in a similar form since Rn,γ (z) =

 '  n−1 d  d kzj 1 ( βj , βj + γj ω − n n j=1 j=1 k=0

Construction of Lattice Rules Using the Weighted Star Discrepancy

where

(

ω(x) =

−n/2
e2πihx , |h|

193

x ∈ [0, 1] .

With some minor modifications, the approach of Nuyens and Cools may then be used to similarly speed up the component-by-component construction proposed here so that only O(n ln(n)d) operations are required.

4 Results for the Weighted Lp Discrepancy In this section we apply the results of the previous two sections to obtain corresponding results for the weighted Lp discrepancy which we define below. From Zaremba’s identity given in (2) one can apply H¨ older’s inequality for integrals and sums to obtain ⎛ |Qn,d (f ) − Id (f )| ≤ Dn,γ,p (z) ⎝

(

γ −q u

⎞1/q |u| q ∂ f (xu , 1) dxu ⎠ , [0,1]|u| ∂xu



u⊆D

where Dn,γ,p (z), the weighted Lp discrepancy, is defined by ⎛ Dn,γ,p (z) := ⎝

⎞1/p



(

|discr ((xu , 1), Pn )| dxu ⎠ p

γ pu

[0,1]|u|

∅ =u⊆D

,

with the local discrepancy discr (x, Pn ) for any x ∈ [0, 1]d being defined in (1) and 1/p + 1/q = 1, p, q ≥ 1. Then we see that we have ⎡ Dn,γ,p (z) ≤ ⎣

(

 γu

u⊆D

sup xu ∈[0,1]|u|

p ⎤1/p |discr ((xu , 1), Pn )| ⎦ .

Now Jensen’s inequality shows that for λ ≥ 1, (

aλi

1/λ

≤

(

ai ,

where the ai are arbitrary non-negative numbers. So for p ≥ 1 we can take λ = p and hence obtain ( Dn,γ,p (z) ≤ γ u sup |discr ((xu , 1), Pn )| . u⊆D

x∈[0,1]|u|

The bound on the right-hand side of this expression is the bound analyzed in Sect. 2 (for example, see (5) and (6)). Hence the results from that section and Sect. 3 hold. Suppose we apply the component-by-component algorithm

194

S. Joe

implied in Corollary 2. Then, under the assumption that the weights are summable, the generating vector z constructed yields a point set that not only has a weighted star discrepancy of O(n−1+δ ), δ > 0, but also has a weighted Lp discrepancy of the same order. In the case p = 2, Kuo [8] showed that the component-by-component algorithm achieves the optimal O(n−1+δ ) rate for the weighted L2 discrepancy if the sum of the square roots of the weights is finite. Since the weights used in [8] are the squares of the weights considered in this paper, the condition in [8] is equivalent to the condition here that the weights are summable. Moreover, the proof of the result in [8] was in a randomized setting, that is, it applied only to randomly shifted lattice rules. In contrast, the previous paragraph indicates that under the same condition on the weights, the component-bycomponent contruction presented here yields a purely deterministic point set whose weighted L2 discrepancy is O(n−1+δ ).

A Calculation of Rn,γ (z) Here we provide details of how ⎛ n−1 d (  1 ⎝βj + γj Rn,γ (z) = n j=1 k=0

( −n/2
⎞ 2πihkzj /n

e

|h|

⎠−

d 

βj

j=1

may be calculated in O(nd) operations. We see that because {kzj /n} = m/n for some m satisfying 0 ≤ m ≤ n − 1, then to calculate Rn,γ (z) we need to have the values of ( e2πihm/n , 0≤m≤n−1. βj + γj |h| −n/2
However, if

(

fn (x) = 1 +

−n/2
then βj + γj

( −n/2
e2πihx , |h|

x ∈ [0, 1] ,

(16)

e2πihm/n = βj + γj (fn (m/n) − 1) . |h|

Since fn (1 − x) = fn (x) for x ∈ [0, 1], then to calculate Rn,γ (z) we need to have the values of fn (m/n) for 0 ≤ m ≤ #n/2$. These #n/2$ + 1 values may be calculated once and then stored. Suppose we wish to calculate fn (m/n) with an absolute error of at most ε. Then the results in [12] show that if  and L are positive integers satisfying  2 1/3 6n 4(L + 1)! and ≤ε , (17) 2≤≤ π2 ( − 1)L+2 π L+2

Construction of Lattice Rules Using the Weighted Star Discrepancy

195

then we should calculate fn (m/n) directly using (16) for 0 ≤ m < . For  ≤ m ≤ #n/2$ we use the approximation G(m/n), where for n odd, G(x) = 1 − 2 ln(2| sin(πx)|) − 2

L (

bi (x) cos(π[(n + i)x + (i + 1)/2]) .

i=0

In this expression, b0 (x) = 1/ [(n + 1)| sin(πx)|] and bi+1 (x) =

−(i + 1) bi (x) . (n + 2i + 3)| sin(πx)|

Similar expressions for G and the bi are available when n is an even number. When ε = 2.0 × 10−16 , then for n ≥ 115, (17) is satisfied with the choices  = 20 and L = 14. As another example, if ε = 1.0 × 10−18 , then for n ≥ 161, we can take  = 25 and L = 15. So we see that the #n/2$ + 1 values of fn (m/n) required may be obtained with an absolute error of at most ε in O(n) + O(L) × (#n/2$ + 1 − ) = O(n) operations which means that even if n is large, Rn,γ (z) may be calculated in O(nd) operations.

Acknowledgments The author would like to thank Henryk Wo´zniakowski for the suggestion to consider the weighted Lp discrepancy case and for his detailed and constructive comments on an earlier version of this manuscript. These comments included the derivation of the bound given in Lemma 1. The author would also like to thank Greg Wasilkowski for bringing to his attention the result given in Lemma 2.

References 1. J. Dick, G. Leobacher, F. Pillichshammer, Construction algorithms for digital nets with small weighted star discrepancy, SIAM J. Numer. Anal., to appear. 2. F.J. Hickernell, H. Niederreiter, The existence of good extensible rank-1 lattices, J. Complexity 19 (2003) 286–300. 3. E. Hlawka, Funktionen von beschr¨ ankter Variation in der Theorie der Gleichverteilung, Ann. Mat. Pura ed Appl. 54 (1961) 325–334. 4. S. Joe, Component by component construction of rank-1 lattice rules having O(n−1 (ln(n))d ) star discrepancy, in: H. Niederreiter (Ed.), Monte Carlo and Quasi-Monte Carlo Methods 2002, Springer, Berlin, 2004, pp. 293–298. 5. S. Joe, I.H. Sloan, On computing the lattice rule criterion R, Math. Comp. 59 (1992) 557–568. 6. N.M. Korobov, Properties and calculation of optimal coefficients, Soviet Math. Dokl. 1 (1960) 696–700.

196

S. Joe

7. F.Y. Kuo, Component-by-component constructions achieve the optimal rate of convergence for multivariate integration in weighted Korobov and Sobolev spaces, J. Complexity 19 (2003) 301–320. 8. F.Y. Kuo, S. Joe, Component-by-component construction of good lattice rules with a composite number of points, J. Complexity 18 (2002) 943–976. 9. F.Y. Kuo, S. Joe, Component-by-component construction of good intermediaterank lattice rules, SIAM J. Numer. Anal. 41 (2003) 1465–1486. 10. H. Niederreiter, Existence of good lattice points in the sense of Hlawka, Monatsh. Math. 86 (1978) 203–219. 11. H. Niederreiter, Random Number Generation and Quasi-Monte Carlo Methods, SIAM, Philadelphia, 1992. 12. D. Nuyens, R. Cools, Fast algorithms for component-by-component construction of rank-1 lattice rules in shift-invariant reproducing kernel Hilbert spaces, Math. Comp., to appear. 13. I.H. Sloan, S. Joe, Lattice Methods for Multiple Integration, Clarendon Press, Oxford, 1994. 14. I.H. Sloan, F.Y. Kuo, S. Joe, On the step-by-step construction of quasi-Monte Carlo integration rules that achieve strong tractability error bounds in weighted Sobolev spaces, Math. Comp. 71 (2002) 1609–1640. 15. I.H. Sloan, F.Y. Kuo, S. Joe, Constructing randomly shifted lattice rules in weighted Sobolev spaces, SIAM J. Numer. Anal. 40 (2002) 1650–1665. 16. I.H. Sloan, A.V. Reztsov, Component-by-component construction of good lattice rules, Math. Comp. 71 (2002) 263–273. 17. I.H. Sloan, H. Wo´zniakowski, When are quasi-Monte Carlo algorithms efficient for high dimensional integrals?, J. Complexity 14 (1998) 1–33. 18. X. Wang, I.H. Sloan, J. Dick, On Korobov lattice rules in weighted spaces, SIAM J. Numer. Anal. 42 (2004) 1760–1779. 19. S.K. Zaremba, Some applications of multidimensional integration by parts, Ann. Polon. Math. 21 (1968) 85–96.

Probabilistic Approximation via Spatial Derivation of Some Nonlinear Parabolic Evolution Equations B. Jourdain ENPC-CERMICS, 6-8 av Blaise Pascal, Cit´e Descartes, Champs sur Marne, 77455 Marne-la-Vall´ee Cedex 2, France [email protected] Summary. For some parabolic equations with a local nonlinearity, a suitable spatial derivation leads to a Fokker-Planck equation with a nonlocal nonlinearity. In this paper we present a review of the particle methods obtained by replacing the nonlinearity in this Fokker-Planck equation by interaction. We are interested in the convergence results for the particle approximations of the original equations and give the milestones of their proofs.

Introduction Let us justify the key idea leading to the probabilistic interpretation of the parabolic evolution equations treated in this paper on the simple example of the viscous Burgers equation ∂t u(t, x) = ν∂xx u(t, x) − ∂x (u(t, x))2 , (t, x) ∈ R+ × R . Following [10,26,27,31], this equation can be seen as the Fokker-Planck equation associated with a diffusion process with diffusion coefficient equal to ν and with drift coefficient given by the solution u itself. This leads to the following stochastic differential equation nonlinear in the sense of McKean since the drift coefficient at time t is the density of the solution at the same time : dXt =

√ 2ν dWt + u(t, Xt )dt where ∀t ≥ 0, u(t, x) is the density of Xt .

Here (Wt )t≥0 is a standard Brownian motion. Of course, because of the nonlinearity u in the drift term, it is not possible to simulate this stochastic differential equation. But one can simulate systems of n particles where the nonlinearity is replaced by interaction. More precisely, the particles are driven by independent Brownian motions. And the drift coefficient of the i-th particle (1 ≤ i ≤ n) is obtained as the empirical density of the system estimated

198

B. Jourdain

at the position of the i-th particle itself. From a numerical point of view, this leads to a difficulty, since the numerical approximation of a density function is unstable. The approach proposed by Bossy and Talay [7, 8] for the viscous Burgers equation enables to avoid this difficulty. It consists in giving a probabilistic interpretation to the equation satisfied by v = ∂x u : ∂t u = ν∂xx u − 2∂x (uv) . Since v is the derivative of u, the latter function is the integral of the former. This leads to the following nonlinear stochastic differential equation : √ dXt = 2ν dWt + 2u(t, Xt )dt where ∀(t, x) ∈ R+ × R, u(t, x) = P (Xt ≤ x) . In the associated system with n particles, the drift coefficient of the i-th particle is obtained as twice the empirical cumulative distribution fonction of the system computed at the position of this particle : dXti,n =

n √ 2( 2ν dWti + 1 j,n i,n dt, 1 ≤ i ≤ n n j=1 {Xt ≤Xt }

where (W 1 , . . . , W n ) is a n-dimensional Brownian motion. Notice that ordering the particles (with computational cost O(n log n)) is enough to compute the drift coefficients of all particles. 1nThe solution u(t, x) of the Burgers equation is then approximated by n1 i=1 1{X i,n ≤x} . This provides a numerical t method which may be more efficient than classical ones, when e.g. the viscosity coefficient ν is small (see [5]). In the first section of the paper, we show how this approach can be adapted for general one-dimensional parabolic equations. In the second section, we consider spatial domains equal to Rd , d ≥ 2. The last section adresses bounded spatial domains.

1 One Dimensional Equations We are interested in the initial value problem for the following parabolic evolution equation with local nonlinearity ∂t u(t, x) = ∂xx [α(u(t, x))] − ∂x [β(u(t, x))] , (t, x) ∈ R+ × R u(0, x) = u0 (x), x ∈ R

(1)

where α, β : R → R are C 1 functions with α non-decreasing (at least on the interval (inf R u0 (x), supR u0 (x))) and u0
: R → R is a non-constant function . with bounded variation i.e. u0 (.) = c + −∞ m(dy) with m a bounded signed measure on the real line. By modifying the functions α and β without changing their smoothness, we may assume without restriction that c is equal to zero and that m is a probability measure when the function u0 is monotonic.

Probabilistic Approximation of Some Nonlinear Parabolic Equations

199

The following equations of physical interest may be written in the form (1) : • the viscous scalar conservation law ∂t u = ν∂xx u − ∂x β(u) where ν > 0 obtained in the case α(u) = νu. • the inviscid scalar conservation law ∂t u + ∂x β(u) = 0 obtained in the case α ≡ 0. • the porous medium equation ∂t u = ∂xx uq with q > 1 obtained in the case α(u) = uq and β ≡ 0. Then the initial condition u0 has to be non-negative. Before dealing with general initial conditions, we consider monotonic functions u0 . 1.1 Monotonic Initial Conditions We assume that u0 (x) = m((−∞, x]) with m a probability measure on R. By spatial derivation of (1), one obtains that v = ∂x u formally solves ∂t v = ∂xx [α (u)v] − ∂x [β  (u)v] , v(0, .) = m . To obtain a closed equation, one has to express u which appears in α and β  in terms of v. Since v is the
spatial derivative of u (and limx→−∞ u0 (x) = 0), x one expects that u(t, x) = −∞ v(t, y) = H ∗ v(t, .)(x) where H(x) = 1{x≥0} denotes the Heaviside function. Hence v formally solves the following parabolic equation with nonlocal nonlinearity ∂t v = ∂xx [α (H ∗ v)v] − ∂x [β  (H ∗ v)v] , v(0, .) = m .

(2)

We are now going to distinguish the non-degenerate case obtained when α is a positive function on the real line (assumption satisfied for the viscous scalar conservation law) and the case of inviscid scalar conservation laws (α ≡ 0). Non-Degenerate Case We assume that ∀u ∈ R, α (u) > 0. Let (Xt )t≥0 and P(C([0, +∞), R)) denote respectively the canonical process and the space of probability measures on C([0, +∞), R). Interpreting (2) as a nonlinear Fokker-Planck equation, we associate the following nonlinear martingale problem with it Definition 1. A probability measure P ∈ P(C([0, +∞), R)) with timemarginals (Pt )t≥0 solves problem (MP) if 1. P0 = m
t 2. Mtϕ = ϕ(Xt )−ϕ(X0 )− 0 α (H ∗Ps (Xs ))ϕ (Xs )+β  (H ∗Ps (Xs ))ϕ (Xs )ds is a P -martingale for any function ϕ ∈ Cb2 (R) .

200

B. Jourdain

If P solves (MP), then the function t → E P (Mtϕ ) is constant. One deduces that t → Pt is a weak solution of (2). Since α is bounded from below by a positive constant on the interval [0, 1], according to Theorem 2 [21], dt a.e. Pt does not weight points. This ensures that α (H ∗ Pt )Pt (resp. β  (H ∗ Pt )Pt ) is the spatial derivative of α(H ∗ Pt ) (resp. β(H ∗ Pt )) in the distribution sense ∗ × R. Therefore if P solves (MP), then H ∗ Pt (x) is a weak solution on R+ of (1). Consequently when uniqueness holds for weak solutions of (1), uniqueness for problem (MP) is equivalent to uniqueness for a standard martingale problem which follows from exercise 7.3.3 [30]. Uniqueness for weak solutions of the viscous scalar conservation law (α(u) = νu) can be checked easily [15]. When β ≡ 0, the uniqueness result of [9] ensures that (1) has no more than one weak solution u such that u(t, .) − u0 L1 (R) is locally integrable with respect to t ∈ R+ and tends to 0 as t tends to 0. Lemma 1 stated in the Appendix in the framework of Sect. 1.2 which generalizes the present one, allows to deduce that if P and Q both solve problem (MP), then ∀(t, x) ∈ R+ × R, H ∗ Pt (x) = H ∗ Qt (x). Apart from both these cases, to our knowledge, no general result ensures uniqueness of the weak solution of (1). Because of the presence of the marginal Ps in the coefficients at time s, it is not possible to simulate the diffusion dynamics given by (MP). But one obtains a simulable system of n particles by replacing Ps by the empirical measure of the system at time s in the coefficients :  t√  t Xti,n = X0i + 2α (H ∗ µns (Xsi,n ))dWsi + β  (H ∗ µns (Xsi,n ))ds, i ≤ n (3) 0 1 n

1n

0

1 n where µ = j=1 δX j,n and the initial variables X0 , . . . , X0 are i.i.d. according to m and independent from the n-dimensional Brownian motion (W 1 , . . . , W n ). Existence of a weak solution is ensured by [30] exercise 7.3.2. Weak uniqueness follows from [2] since the diffusion matrix is constant on the polyhedrons {x = (x1 , . . . , xn ) ∈ Rn , xσ(1) < xσ(2) < . . . < xσ(n) } indexed by the permutations σ of {1, . . . , n}. Let π n denote the law of the empirical measure µn considered as a random variable with values in P(C([0, +∞), R)). Since in view of weak uniqueness, the processes X i,n , 1 ≤ i ≤ n are exchangeable, the tightness of the sequence (π n )n is equivalent to the tightness of the laws of X 1,n , n ≥ 1 (see [32]) which follows from the boundedness of α and β  on the interval [0, 1]. It is not difficult [16] to prove that the weak limit of any converging subsequence gives full weight to the set of probability measures Q ∈ P(C([0, +∞), R)) such that H ∗ Qt (x) is a weak solution of (1). In case this equation admits a unique weak solution u(t, x), one deduces [18] :  |u(t, x) − H ∗ µnt (x)| dx = 0 . (4) ∀T > 0, lim E sup n→+∞ 1 + x2 t∈[0,T ] R n

Hence the empirical cumulative distribution functions H ∗µnt (x) of the particle systems converge to u(t, x) as n tend to infinity.

Probabilistic Approximation of Some Nonlinear Parabolic Equations

201

The next step consists in proving that the limit π ∞ of any converging subsequence of (π n )n gives full weight to solutions of problem (MP). In case uniqueness holds for this problem, one deduces that when n tends to +∞, the empirical measures µn converge in probability to the constant P where P denotes the unique solution of problem (MP). The choice of the initial distribution of particles ensures that π ∞ a.s. condition 1 in definition 1 is satisfied. According to [16], the mappings on P(C([0, +∞), R)) naturally introduced in order to check condition 2 are continuous at any probability measure Q such that dt a.e. Qt does not weight points. Therefore it is enough to check that π ∞ gives full weight to such probability measures to conclude. For the viscous scalar conservation law , one may deduce this property from estimates of the density of (Xt1,n , Xt2,n ) obtained by Girsanov theorem [14,15]. More generally, it is a consequence of the estimate  t  E |f (Xs1,n , Xs2,n )|ds ≤ Cf L3 (R2 ) with C not depending on n, (5) 0

which follows from Theorem 2 [21] since α is bounded from below by a positive constant on the interval [0, 1] and the coefficients of the stochastic differential equation (3) are globally bounded. One may also be interested in the asymptotic behaviour of the reordered system (Y 1,n , . . . , Y n,n ) defined in the following way : for any t ≥ 0, Yt1,n ≤ Yt2,n ≤ . . . ≤ Ytn,n is the increasing reordering of (Xt1,n , . . . , Xtn,n ). According to [16], (Y 1,n , . . . , Y n,n ) is a diffusion process with constant diffusion matrix diag(α (1/n), α (2/n), . . . , α (n/n)) and with drift coefficient (β  (1/n), β  (2/n), . . . , β  (n/n)) normaly reflected at the boundary of the convex subset {(y1 , . . . , yn ) : y1 ≤ y2 ≤ . . . ≤ yn } of 1 Rn . n 1 n n ¯ = n i=1 δY i,n are easily proved The laws π ¯ of the empirical measures µ n to be tight [16]. In general µ ¯ is different from µn , but of course for any n non-negative t, one has µ ¯t = µnt . Hence the limit π ¯ ∞ of any converging n subsequence of (¯ π )n also gives full weight to the set of probability measures Q ∈ P(C([0, +∞), R)) such that H ∗ Qt (x) is a weak solution of (1). The order between particles enables to check that moreover, π ¯ ∞ gives full weight to probability measures Q with finite dimensional marginals Qt1 ,...,tk equal to the image of the Lebesgue measure on the interval [0, 1] by y → ((H ∗ Qt1 )−1 (y), . . . , (H ∗ Qtk )−1 (y)) where for any non-negative t, (H ∗ Qt )−1 (y) = inf{x : H ∗ Qt (x) ≥ y} (see [18]). When uniqueness of weak solutions holds for (1), there is a unique probability measure P¯ ∈ P(C([0, +∞), R)) with both these properties and the sequence (¯ µn )n converges in probability to P¯ . If in addition, the unique weak solution of (1) is continuous on ]0, +∞[×R, then according to [16], the second property characterizing P¯ is equivalent to the fact that P¯ gives full weight to sample-paths X such that t → u(t, Xt ) is constant on (0, +∞). The following Theorem summarizes the results obtained above for the viscous scalar conservation law and the equation with no first order term:

202

B. Jourdain

Theorem 1. Assume that α(u) = νu with ν > 0 (resp. that α is a positive function and that β ≡ 0). Then (1) admits a unique solution u and this solution is continuous on (0, +∞) × R (resp. (1) admits a unique solution u such that u(t, .) − u0 L1 (R) is locally integrable with respect to t ∈ R+ and tends to 0 as t tends to 0). As n tend to infinity, the empirical cumulative distribution functions H ∗ µnt (x) of the particle systems converge to u(t, x) in the sense given by (4). The empirical measures of the original particles converge in probability to the unique solution P of problem (MP). The empirical measures of the reordered particles converge in probability to the unique P¯ ∈ P(C([0, +∞), R)) such that H ∗ P¯t (x) = u(t, x) and that P¯ a.s. t → u(t, Xt ) is constant on (0, +∞) (resp. and that the finite dimensional marginals P¯t1 ,...,tk are equal to the image of the Lebesgue measure on the interval [0, 1] by y → ((H ∗ P¯t1 )−1 (y), . . . , (H ∗ P¯tk )−1 (y))). Remark 1. (see [16]) In case of the porous medium equation, α(u) = uq with q > 1 and α vanishes for u = 0. By combining weak uniqueness for this equation and existence of a solution u with sufficient spatial regularity, one obtains that problem (MP) has no more than one solution P such that dt a.e., Pt does not weight points. As n tends to infinity, the empirical cumulative distribution functions H ∗ µnt (x) of the particle systems converge to u(t, x) in the sense given by (4) . Since u is continuous on (0, +∞) × R, one deduces that any limit point of the sequence (π n )n gives full weight to probability measures Q such that dt a.e., Qt does not weight points, and then concludes that the empirical measures µn converge in probability to the unique solution P of (MP) such that dt a.e., Pt does not weight points. Last, the empirical measures µ ¯n of the reordered particles converge in probability to the unique P¯ ∈ P(C([0, +∞), R)) such that H ∗ P¯t (x) = u(t, x) and that P¯ a.s. t → u(t, Xt ) is constant on (0, +∞). The Inviscid Scalar Conservation Law We suppose that α ≡ 0. Then uniqueness fails to hold for weak solutions of (1). However, according to Kruzkhov’s theorem, there is a unique entropic solution characterized by the following inequalities   ∂t |u(t, x) − c| + ∂x sign(u(t, x) − c)(β(u(t, x)) − β(c)) ≤ 0 (6) holding in the distribution sense on R+ × R for each real constant c ∈ [0, 1]. Here sign(y) = 1{y>0} − 1{y≤0} . The entropic solution is the limit of the solution of the viscous scalar conservation law as the viscosity coefficient ν vanishes. This suggests to approximate u(t, x) by H ∗ µnt (x) where µn is the empirical measure of the system  t √ β  (H ∗ µns (Xsi,n ))ds, 1 ≤ i ≤ n Xti,n = X0i + 2νn Wti + 0

Probabilistic Approximation of Some Nonlinear Parabolic Equations

203

with small viscosity coefficient νn > 0. Numerical experiments by Bossy and Talay [8] and Bossy, Piperno and Fezoui [5] show the good behaviour of this approximation. The reordered particle system (Y 1,n , . . . , Y n,n ) turns out to be a useful1tool for the theoretical study of the limit behaviour of H ∗ µnt (x) = n i,n 1 i=1 H(x − Yt ) when νn tends to 0 as n tends to infinity. Indeed, denotn ing by [y] the integer part of any real y, one remarks that the approximations |H ∗ µnt (x) − [nc]/n| and sign(H ∗ µnt (x) − [nc]/n)[β(H ∗ µnt (x)) − β([nc]/n)] of the functions which appear 1 in (6) have spatial distribution derivatives n respectively equal to ηtn = n1 i=1 sign(i − [cn])δY i,n and close to ξtn = t 1 n 1  i=1 sign(i−[cn])β (i/n)δYti,n . For g a non-negative test function on R+ ×R n
x o’s formula and sumand ψ(t, x) = −∞ g(t, y)dy, computing ψ(t, Yti,n ) by Itˆ ming over i the result multiplied by sign(i − [cn])/n, one obtains that 

t

< ηtn , ψ(t, .) > − < η0n , ψ(0, .) > −

< ηsn , ∂s ψ(s, .) > + < ξsn , ∂x ψ(s, .) > ds 0

is equal to the sum of the contribution of the reflection local time term which is non-negative and of the contribution of the Brownian terms which vanishes as n → +∞. By making spatial integration by parts, one concludes that the entropic inequality tested against the function g approximately holds. This intuitive reasoning can be made rigourous [18] to prove the following result : Theorem 2. Assume that (νn )n is a sequence of positive numbers converging to 0 as n tends to infinity. Then the empirical cumulative distribution functions H ∗ µnt (x) of the particle systems converge to the unique entropic solution u(t, x) of the inviscid scalar conservation law in the sense given by (4). In addition, the empirical measures µ ¯n of the reordered systems converge in probability to the unique P¯ ∈ P(C([0, +∞), R)) such that H ∗ P¯t (x) = u(t, x) and that the finite dimensional marginals P¯t1 ,...,tk are equal to the image of the Lebesgue measure on the interval [0, 1] by y → ((H ∗ P¯t1 )−1 (y), . . . , (H ∗ P¯tk )−1 (y)). Remark 2. Even if this is not rigorous because of the possible discontinuities of the function u, the second condition which characterizes P¯ in the previous theorem intuitively means that P¯ gives full weight to the characteristic curves X along which t → u(t, Xt ) is constant . Remark 3. Paper [20] investigates the limit behaviour of the particles  Xti,n = X0i + νn1/α Sti +

t

β  (H ∗ µns (Xsi,n ))ds, 1 ≤ i ≤ n

0

where S i , 1 ≤ i ≤ n are independent symmetric stable processes with index α ∈ (1, 2) independent from the initial positions (X01 , . . . , X0n ) and νn > 0. When the sequence νn is constant and equal to ν (resp. converges to 0), the

204

B. Jourdain

empirical cumulative distribution functions H ∗ µnt (x) converge to the unique weak solution u of ∂t u = −ν(−∂xx )α/2 u − ∂x β(u) (resp. the unique entropic solution u of the inviscid scalar conservation law). 1.2 General Initial Conditions The initial condition u0 is now supposed to be the cumulative distribution function of a signed measure m on R with total mass m ∈ (0, +∞) and total variation measure |m|. Let us first deal with the non-degenerate case : ∀u ∈ R, α (u) > 0. As explained in [15], to associate a single martingale problem with (2), one may introduce h : R → {−m, m} a density of m with respect to the probability measure |m|/m. With any Q ∈ P(C([0, +∞), R)), one ˜ with density h(X0 ) with respect to Q. then associates the signed measure Q ˜ ˜ t )t≥0 . Notice that if Q0 = |m|/m The time-marginals of Q are denoted by (Q ˜ 0 (dx) = h(x)|m|(dx)/m = m(dx). then Q Definition 2. A probability measure P ∈ P(C([0, +∞), R)) solves problem (MP) if 1. P0 = |m|/m
t 2. Mtϕ = ϕ(Xt )−ϕ(X0 )− 0 α (H ∗ P˜s (Xs ))ϕ (Xs )+β  (H ∗ P˜s (Xs ))ϕ (Xs )ds is a P -martingale for any function ϕ ∈ Cb2 (R) . Since P˜ = P when m is a probability measure, this definition of problem (MP) generalizes the previous one. If P solves (MP), then the function t → E P (h(X0 )Mtϕ ) is constant. This implies that t → P˜t is a weak solution of (2). With [21], one deduces that H ∗ P˜t (x) is a weak solution of (1). Like for monotonic initial conditions u0 , uniqueness for problem (MP) follows in case α(u) = νu with ν > 0 [15] and in case β ≡ 0 (see [9] and Lemma 1 in the appendix). The particle dynamics generalizing (3) is Xti,n

=

X0i +

 t√  t n i,n i  2α (H ∗ µ ˜s (Xs )dWs + β  (H ∗ µ ˜ns (Xsi,n ))ds, i ≤ n (7) 0

0

where the 1ncontribution of the i-th particle to the signed empirical measure µ ˜n = n1 j=1 h(X0j )δX j,n has the weight h(X0i ). Because of the possible irregularity of the function h, the choice of initial variables X0i , 1 ≤ i ≤ n i.i.d. according to |m|/m is crucial for the validity of the convergence results below. The stochastic differential equation (7) still admits 1an unique weak solution and the laws π n of the empirical measures µn = n1 i=1 δX i,n form a tight sequence. By an easy adaptation of Proposition 2.4 [18], the limit π ∞ of any convergent subsequence of (π n )n gives full weight to the probability ˜ t (x) is a weak solution of (1). measures Q ∈ P(C([0, +∞), R)) such that H ∗ Q When this equation admits a unique weak solution u(t, x), one deduces :

Probabilistic Approximation of Some Nonlinear Parabolic Equations

 ∀T > 0,

lim E sup

n→+∞

t∈[0,T ]

R

|u(t, x) − H ∗ µ ˜nt (x)| dx = 0 . 2 1+x

205

(8)

Since (5) still holds, one obtains that π ∞ a.s., dt a.e., Qt does not weight points and concludes that π ∞ gives full weight to solutions of problem (MP). The next theorem puts together the results obtained above and a fluctuation result adapted from [29] √ in [15] which ensures that the rate of convergence of H ∗µ ˜nt (x) to u(t, x) is 1/ n in the case of the viscous scalar conservation law. Theorem 3. Assume that α(u) = νu with ν > 0 (resp. that α is a positive function and that β ≡ 0). Then (1) admits a unique solution u (resp. a unique solution u such that u(t, .) − u0 L1 (R) is locally integrable with respect to t ∈ R+ and tends to 0 as t tends to 0). As n tend to infinity, the weighted empirical cumulative distribution functions H ∗ µ ˜nt (x) of the particle systems converge to u(t, x) in the sense given by (8). The empirical measures of the particles converge in probability to the unique solution P of problem (MP). If α(u) = νu with ν > 0 and the function1β is C 3 , then the finiten dimensional marginals of the fluctuation field { √1n i=1 (ϕ(X i,n )− < P, ϕ(.) > ), ϕ ∈ L2 (P )} converge weakly to those of a centered Gaussian field. Remark 4. In practice, the particles positions Xti,n cannot be computed exactly. The stochastic differential equation giving their evolution has to be discretized with respect to time. Let us denote by Un (k∆t, x) the approximate solution of (1) obtained with the standard Euler scheme with time-step √ ∆t. The convergence estimate supk≤T /∆t EUn (k∆t, x) − u(t, .)L1 (R) ≤ C( ∆t + √1n ) obtained by Bossy and Talay [7,8] for the viscous Burgers equation (α(u) = νu with ν > 0 and β(u) = u2 ) was extended to general viscous scalar conservation laws and improved into     1 sup E |Un (k∆t, x) − u(t, x)| + Un (k∆t, x) − u(t, .)L1 (R) ≤ C ∆t+ √ n k≤T /∆t x∈R

by Bossy [4]. Because of the possibility for the paths of particles with opposite weights to intersect, the reordered system no longer evolves according to a diffusion dynamics. In addition, when trying to approximate the entropic inequalities in the vanishing viscosity limit, the contribution of the reflection local time corresponding to such crossings has the wrong sign. To overcome these difficulties, it is possible to modify the particles dynamics by killing the couples of particles with opposite weights that merge [18]. From the point of view of approximation of the equation (1), this idea turns out to be natural. Indeed, if It ⊂ {1, . . . , n} denotes the set of indices 1 of particles still alive at time t, the approximate solution Un (t, x) = n1 i∈It h(X0i )H(x − Xti,n ) is such that maxx∈R Un (t, x) (resp. minx∈R Un (t, x)) is non-increasing (resp. non-decreasing) with t, property which is a discrete level translation of the

206

B. Jourdain

maximum principle satisfied by the solutions of (1). In addition, the total variation of x → Un (t, x) is non-increasing with t. In [18], existence for the particle system with killing √  dXti,n = 1It (i) 2νn dWti + β  (Un (s, Xsi,n ))dt , i ≤ n associated with the scalar conservation law is checked and the following convergence result is proved for the approximate solution Un : Theorem 4. Assume that the sequence (νn )n converges to ν ≥ 0. Then  |u(t, x) − Un (t, x)| dx = 0 , ∀T > 0, lim E sup n→+∞ 1 + x2 t∈[0,T ] R where u denotes the solution of the viscous scalar conservation law if ν > 0 and the entropic solution of the inviscid conservation law otherwise.

2 Multidimensional Equations We are first going to deal with a viscous scalar conservation law in arbitrary space dimension. Then we will consider the incompressible Navier-Stokes equation in space dimension 2. 2.1 Viscous Scalar Conservation Law Let us consider the following viscous scalar conservation law in space dimension d. ∂t u(t, x) = ν∆u(t, x) − ∇.β(u(t, x)), (t, x) ∈ R+ × Rd u(0, x) = u0 (x), x ∈ Rd

(9)

where ν > 0, β : R → Rd is a C 2 function bounded together with its first and second order derivatives. By spatial derivation of (9), one obtains formally that for 1 ≤ i ≤ d, vi = ∂xi u solves ∂t vi = ν∆vi − ∇.(β  (u)vi ) and vi (0, x) = ∂xi u0 (x) . To obtain a closed system for (v1 , . . . , vd ), one has to express u in terms of its spatial gradient. In space dimension d ≥ 2, this question is less obvious than in the previous one-dimensional setting. Anderson [1] first proposed to use the fundamental solution of the Laplacian γ(x) for this purpose in a particle method context (see also [13] [28]). One has γ(x) = g(|x|) where g(r) = log(r)/S2 if d = 2 and g(r) = −1/Sd rd−2 if d ≥ 3, with Sd denoting the unit sphere area in Rd . When f : Rd → R is a C ∞ function equal to

Probabilistic Approximation of Some Nonlinear Parabolic Equations

207

a constant c outside of a compact set f = c + γ ∗ ∆f . By integration 1then d be parts, one deduces that f = c + j=1 ∂xj γ ∗ ∂xj f = c + ∇γ ∗ ∇f . This formula is in fact a generalization of the one used in dimension one, since then the derivative of the fundamental solution of the Laplacian |x|/2 is equal to H − 1/2. We assume from now on that u0 is bounded with first order distribution derivatives belonging to L1 ∩ L∞ (Rd ). Then the representation formula u0 = c + ∇γ ∗ ∇u0 still holds for some real constant c. In addition, (9) has a unique classical solution u(t, x) and this solution is such that for any non-negative t, u(t, .) = c + ∇γ ∗ ∇u(t, .) [17]. Hence the spatial derivatives (∂x1 u, . . . , ∂xd u) of u solve weakly the following system : ⎞ ⎤ ⎡ ⎛ d ( ∂t vi = ν∆vi − ∇. ⎣β  ⎝c + ∂xj γ ∗ vj ⎠ vi ⎦ , vi (0, .) = ∂xi u0 . (10) j=1

As in space dimension one, by spatial derivation we have transformed the equation (9) with local nonlinearity into the previous system with nonlocal nonlinearity. In (9), in the function β, the unknown u is convoluted with the Dirac mass at the origin, whereas in (10), in the function β  , the unknown (v1 , . . . , vd ) is convoluted with ∇γ which can be seen as the Dirac mass integrated once. As the solution of (9) can be deduced from the solution of a similar equation where all the spatial coordinates i ≤ d such that ∂xi u0 L1 (Rd ) = 0 have been removed, we suppose without restriction that for i ≤ d, ∂xi u0 L1 (R) > 0. We are now going to give a probabilistic interpretation to the system (10). Since the functions ∂xi u0 are not in general probability densities, we associate with any (P 1 , . . . , P d ) ∈ P(C([0, +∞), Rd ))d , the measures P˜ i with density hi (X0 ) with respect to P i where hi (x) = ∂xi u0 L1 (Rd ) sign(∂xi u0 (x)) and (Xt )t≥0 denotes the canonical process on C([0, +∞), Rd ). Let us denote respectively by (Pti )t≥0 and (P˜ti )t≥0 the time marginals of the measures P i and P˜ i . Definition 3. (P 1 , . . . , P d ) solves problem (MP) if for 1 ≤ i ≤ d, |∂xi u0 (x)| ∂xi u0 L1 (Rd ) dx. any t ≥ 0, Pti has

1. P0i =

a bounded density w.r.t. the Lebesgue measure on 2. For Rd .
t 3. Mtϕ = ϕ(Xt )−ϕ(X0 )− 0 ν∆ϕ(Xs )+β  (c+∇γ∗P˜s (Xs )).∇ϕ(Xs )ds is a P i 1d martingale for any function ϕ ∈ Cb2 (Rd ) (here ∇γ ∗ P˜s = j=1 ∂xj γ ∗ P˜sj ). If (P 1 , . . . , P d ) solves this problem then t → (P˜t1 , . . . , P˜td ) is a weak solution of (10). Because of the explosion of the kernel ∇γ at the origin, it is necessary to introduce a cutoff to construct systems of particles associated with (MP). For ε > 0 let γ ε (x) = g ε (|x|) where

208

B. Jourdain

 ε

g (r) = g(r)1{r≥ε} +

g  (ε)r2 g  (ε)ε + g(ε) − 2ε 2

 1{0≤r<ε} .

Let εn > 0 denote the cutoff parameter ruling the interaction for the system with n particles. Each particle (X1i,n (t), . . . , Xdi,n (t)) has d coordinates Xji,n (t), j ≤ d which evolve in Rd according to √ Xji,n (t) = Xji (0) + 2ν Wti ⎛ ⎞  t d ( ( 1 + β  ⎝c + ∂xl γ n (Xji,n (s) − Xlk,n (s))hl (Xlk (0))⎠ ds, n − 1 0 k =i

l=1

where W 1 , . . . , W n are independent d-dimensional Brownian motions which are independent from the Rd×d -valued initial variables (X1i (0), . . . , Xdi (0)), 1 ≤ i ≤ n. These initial varibles are supposed to be i.i.d. with Xj1 (0) distributed according to |∂xj u0 (x)|dx/∂xj u0 L1 (Rd ) for 1 ≤ j ≤ d. The next theorem summarizes the results proved in [17] : Theorem 5. Problem (MP) has a unique solution (P 1 , . . . , P d ). In addition, the unique classical solution u of (9) is such that ∀(t, x) ∈ [0, +∞) × ˜ (εn )n converges to 0 as n Rd , u(t, x) = c + ∇γ 1n∗ Pt (x). If the 1sequence n 1 tends to ∞, then ( n i=1 δX i,n , . . . , n1 i=1 δX i,n ) converges in probability to (P 1 , . . . , P d ).

1

d

As a consequence of this convergence result, n ( d ( 1 sup E u(t, x) − c − ∂xj γ n (x − Xji,n (t))hj (Xji (0)) n i=1 j=1 (t,x)∈[0,T ]×Rd converges to 0 as n tends to ∞ which ensures that the approximate solution of (9) converges to the exact solution. 2.2 Incompressible Navier-Stokes Equation in R2 In absence of external forces, the velocity field u = (u1 , u2 ) and the pressure field p of an incompressible Newtonian fluid in R2 solve the following NavierStokes equation ∂t u = ν∆u − (u.∇)u − ∇p, t ≥ 0, x = (x1 , x2 ) ∈ R2

(11)

∇.u = 0, t ≥ 0, x ∈ R u(t, x) → 0 as |x| → +∞ and u(0, x) = u0 (x), x ∈ R2 2

It is well known that this equation can be reformulated in terms of the vorticity def field : w = curlu = ∂x1 u2 − ∂x2 u1 . By derivation of (11), one obtains

Probabilistic Approximation of Some Nonlinear Parabolic Equations

209

∂t w = ν∆w − ∇.(uw) . To obtain a closed equation for w, one has to express the velocity in terms of the vorticity. Since ∇.u = 0, there exists a function ψ such that u = ∇⊥ ψ = (∂x2 ψ, −∂x1 ψ). On deduces that w = ∂x1 (−∂x1 ψ) − ∂x2 (∂x2 ψ) = −∆ψ. Hence ψ = −γ ∗ w where γ denotes the fundamental solution of the Laplacian on R2 and u = −∇⊥ γ ∗ w is the spatial convolution of the kernel of Biot and Savart −∇⊥ γ with the vorticity field w. Finally, the vorticity field solves ∂t w = ν∆w + ∇.((∇⊥ γ ∗ w)w) and w(0, .) = curlu0 (.) .

(12)

From now on, we follow M´el´eard [25] and assume that m = curlu0 is a bounded signed measure on R2 with total mass m > 0 (see also [23] and [24] for similar results under more restrictive assumptions). For h : R2 → {−m, m} a density of m with respect to the probability measure |m|/m, one as˜ with densociates with any Q ∈ P(C([0, +∞), R2 )) the signed measure Q sity h(X0 ) with respect to Q (here (Xt )t≥0 denotes the canonical process on ˜ are denoted by (Q ˜ t )t≥0 . InterpretC([0, +∞), R2 ))). The time-marginals of Q ing (12) as a Fokker-Planck equation, one associates the following nonlinear martingale problem with this equation Definition 4. A probability measure P ∈ P(C([0, +∞), R2 )) solves problem (MP) if 1. P0 = |m|/m
t 2. Mtϕ = ϕ(Xt ) − ϕ(X0 ) − 0 ν∆ϕ(Xs ) − ∇⊥ γ ∗ P˜s (Xs ).∇ϕ(Xs )ds is a P martingale for any function ϕ ∈ Cb2 (R2 ). Because of the singularity of ∇⊥ γ at the origin, it is necessary to cutoff this kernel in order to construct associated interacting particle systems. To keep important features of ∇⊥ γ such as the divergence-free property, one defines kε = −∇⊥ γ ∗ φε where φε (x) = φ(x/ε)/ε2 with φ a smooth function with radial symmetry and integral equal to 1. This way, kε is bounded by Mε and Lipschitz continuous with constant Lε . One may choose Mε = O(1/ε2 ) and Lε = O(1/ε3 ) as ε tends to 0. For W 1 , . . . , W n independent 2-dimensional Brownian motions independent from the initial variables X01 , . . . , X0n i.i.d. according to |m|/m, the system with n particles is defined by  t √ Xti,n = X0i + 2ν Wti + kεn ∗ µ ˜ns (Xsi,n )ds, 1 ≤ i ≤ n 0

where µ ˜ns =

1 n

1n j=1

h(X0j )δXsj,n and εn > 0. According to [25],

Theorem 6. Existence holds for problem (MP). The solution P is unique when either m does not weight points or m is small enough. When uniqueness holds, for T > 0, if

210

B. Jourdain

lim εn = 0 and

n→+∞

√ lim Mεn emLεn T / nLεn = 0 ,

n→+∞

(13)

1n then the empirical measures µn = n1 i=1 δX i,n considered as random variables with values in P(C([0, T ], R2 )) converge in probability to P ◦ ((Xt )t∈[0,T ] )−1 . When u0 is such that supy>0 y(mes({x : |u0 (x)| > y}))1/2 < +∞, ∇.u0 = 0 and m = curlu0 either does not weight points or has a small enough total mass, then the Navier-Stokes equation (11) admits a unique solution (u, p). And it is possible to approximate the velocity field 1nthanks to the particle system since under (13), for (t, x) ∈ [0, T ] × R2 , n1 i=1 h(X0i )kεn (x − Xti,n ) converges in probability to u(t, x).

3 Bounded Spatial Domains We are first going to deal with a viscous scalar conservation law posed on the spatial interval [0, 1] before considering the incompressible Navier-Stokes equation in a bounded domain of R2 . 3.1 Viscous Scalar Conservation Law in the Spatial Interval [0, 1] We are now interested in the following viscous scalar conservation law posed in the spatial interval [0, 1] with non-homogeneous Dirichlet boundary conditions 2 u(t, x) − ∂x β(u(t, x)), (t, x) ∈ R+ × (0, 1) ∂t u(t, x) = ν∂xx ∀x ∈ [0, 1], u(0, x) = u0 (x) and ∀t ≥ 0, u(t, 0) = 0 and u(t, 1) = 1 (14)

where u0 is the cumulative distribution function of a probability measure m on the interval [0, 1]. The probabilistic interpretation of this equation involves a diffusion process with normal reflection at the boundary of the interval [0, 1]. That is why we introduce (Xt , Kt ) the canonical process on C = C([0, +∞), [0, 1]) × C([0, +∞), R). For P a probability measure on C, we set Pˆt = P ◦ Xt−1 . Definition 5. A probability measure P ∈ P(C) solves problem (MP) if 1. P ◦ (X0 , K0 )−1 = m ⊗ δ0
t 2. ϕ(Xt −Kt )−ϕ(X0 −K0 )− 0 νϕ (Xs −Ks )+β  (H ∗ Pˆs (Xs ))ϕ (Xs −Ks )ds is a P martingale for any ϕ ∈ Cb2 (R).
t
t 3. P a.s. ∀t ≥ 0, |K|t = 0 1{0,1} (Xs )d|K|s < +∞ and Kt = 0 1{Xs =0} − 1{Xs =1} d|K|s . The process Kt with finite variation which increases when Xt is equal to 0 and decreases when Xt is equal to 1 accounts for reflection and prevents Xt from leaving the interval [0, 1].

Probabilistic Approximation of Some Nonlinear Parabolic Equations

211

The associated particles are also reflected at the boundary of interval [0, 1]: √
t Xti,n = X0i + 2νWti + 0 β  (H ∗ µ ˆns (Xsi,n ))ds + Kti,n , 1 ≤ i ≤ n

t t i,n |K i,n |t = 0 1{0,1} (Xsi,n )d|K|i,n = 0 1{Xsi,n =0} − 1{Xsi,n =1} d|K i,n |s s and Kt 1n 1n µn = n1 j=1 δ(X j,n ,K j,n ) and µ ˆns = n1 j=1 δXsj,n . The next theorem states some of the results proved in [6] Theorem 7. Problem (MP) has a unique solution P . In addition, u(t, x) = H ∗ Pˆt (x) is a weak solution of (14). As n tends to infinity, the empirical measures µn converge in probability to P and ∀(t, x) ∈ R+ × [0, 1],

lim E|u(t, x) − H ∗ µ ˆnt (x)| = 0 .

n→+∞

To obtain a practical algorithm, the stochastic differential equation giving the evolution of the positions Xti,n has to be discretized with respect to time. The main part of [6] is dedicated to the numerical analysis of the algorithm obtained by using for this purpose the version of the Euler scheme with step ∆t proposed by L´epingle [22]. To precise this scheme, one needs two constant α0 < α1 in (0, 1). As in a standard Euler scheme, the value of the drift coefficient of each particle is frozen on each time-step to a value depending on the positions of all particles at the beginning of the time-step. If the position of the ith particle at the beginning of a time-step is in [α0 , α1 ], then one computes the value of the process with frozen drift coefficient and without reflection at the end of the time-step. If the position is in [0, α0 ) (resp. (α1 , 1]) then one computes the value of the process with frozen drift coefficient normally reflected at 0 (resp. 1). In each case, the result is projected on [0, 1] to obtain the position of the i-th particle at the beginning of the next time-step. ¯ i,n , 1 ≤ i ≤ n, 0 ≤ k ≤ T /∆t the discretized positions Let us denote by X k∆t of the particles obtained thanks to this scheme. The following convergence rate is proved in [6] : n   1 ( 1 i,n ¯ √ ≤ C ∆t + H(x − X ) − u(k∆t, x) sup E . k∆t n n x∈[0,1] i=1 k≤T /∆t

This rate is the same as the one obtained before by Bossy [4] when the spatial domain is the whole real line i.e. in the absence of reflection of particles . On the present example, because of the appropriate choice of the boundary conditions at positions 0 and 1, everything works as if the spatial domain was the whole real line. Let us now turn to an example in which the boundary conditions come from the physics.

212

B. Jourdain

3.2 Incompressible Navier-Stokes Equation in a Bounded Domain We are interested in the incompressible Navier-Stokes equation in a bounded domain Θ of R2 with no-slip boundary conditions on ∂Θ : ∂t u = ν∆u − (u.∇)u − ∇p, t ≥ 0, x ∈ Θ ∇.u = 0, t ≥ 0, x ∈ Θ u(0, x) = u0 (x), x ∈ Θ and u(t, x) = 0, t ≥ 0, x ∈ ∂Θ .

(15)

As when the spatial domain is the whole plane, the vorticity field w = curlu solves ∂t w = ν∆w − ∇.(uw) and w(0, .) = curlu0 To obtain a closed equation for w, one faces two difficulties. First, as usual, one has to express u in terms of w. Second, one has to translate the no-slip boundary condition for the velocity field in terms of a boundary condition for the vorticity field. Of course, both issues are closely related. ⊥
Since ∇.u = 0, one has u = ∇ ψ and w = −∆ψ. One may choose ψ(t, x) = − Θ γ(x, y)w(t, y)dy where γ is the Green function of the Laplacian on Θ with homogeneous Dirichlet boundary conditions. One deduces  u = −[∇⊥ γw] where − [∇⊥ γw](t, x) stands for − ∇⊥ x γ(x, y)w(t, y)dy . Θ

Because of the choice of homogeneous Dirichlet boundary conditions, ψ vanishes on ∂Θ. As a consequence, the tangential derivative of ψ is zero. Since u = ∇⊥ ψ, the normal component of the velocity vanishes on the boundary. As pointed out by Chorin [11], to ensure that the tangential component of the velocity is also zero, vorticity has to be created on the boundary. The following Neumann’s boundary condition involving a nonlocal right-hand-side proposed by Cottet [12] ensures the correct creation of vorticity

∂n w = ∂n curlγ(−∂2 w, ∂1 w) −

1 |∂Θ|

 ∂n curlγ(−∂2 w, ∂1 w), t ≥ 0, x ∈ ∂θ . ∂Θ

Here ∂n denotes the normal component of the gradient and     γ(x, y)∂2 w(t, y)dy, γ(x, y)∂1 w(t, y)dy . γ(−∂2 w, ∂1 w)(t, x) = − Θ

Θ

This condition of Neumann’s type seems well-suited for the probabilistic interpretation since it can be translated into mass creation on the boundary. In [19], as a first step towards this probabilistic interpretation we deal with the vortex equation ∂t w = ν∆w + ∇.([∇⊥ γw]w) and w(0, .) = w0 (.)

Probabilistic Approximation of Some Nonlinear Parabolic Equations

213

with initial condition w0 ∈ L2 (Θ) supplemented by the Neumann’s boundary condition ∂n w(t, x) = g(t, x), t ≥ 0, x ∈ ∂Θ , where g is a given function belonging to L2 ([0, T ] × ∂Θ) with T > 0 a finite time horizon. We associate with this equation a nonlinear martingale problem on a space of reflected paths either starting initially from Θ or starting from ∂Θ after that. We show uniqueness for the martingale problem and prove the convergence of the empirical measures of well-chosen particle systems to the solution of the martingale problem. Finally, we show that the corresponding velocity field −[∇⊥ γw] can be approximated thanks to the particles (see [19] for details). Next, we have tried to deal with the nonlocal boundary condition proposed by Cottet. Since, to our knowledge, no energy estimate is available for the vortex equation supplemented by this boundary condition, the theoretical study appears really difficult. And so far, we have not been able to obtain convincing numerical results. Notice that a different probabilistic interpretation based on branching processes was developped by Benachour, Roynette and Vallois [3] for (15). But even if the authors propose some particle approximations, the convergence of the method is not shown and the particle systems are not for use in practice.

Conclusion For all the parabolic evolution equations with local nonlinearity treated in the present paper, we have shown that a suitable derivation of the solution with respect to spatial variables enables to obtain a closed equation (or a closed system of equations) of Fokker-Planck type with nonlocal nonlinearity : the unknown function is convoluted with a kernel obtained by some spatial integration of the Dirac mass at the origin. In space dimension one, this kernel is equal to the Heaviside function : it is discontinuous at the origin but since it is bounded, its spatial convolution with any bounded signed measure makes sense. This makes the probabilistic interpretation of the equation obtained by derivation easy and enables to obtain rather general approximation results for the solution of the original equation. In space dimension d ≥ 2, the kernel is singular at the origin. Even if, from a probabilistic point of view, such a kernel is much easier to take into account than the Dirac mass at the origin, one has to be more cautious than in space dimension one. We have also explained how the use of signed weights depending on the initial positions both in the nonlinear martingale problem and in the particle dynamics allows to give a probabilistic interpretation and design particle approximations for an enlarged class of initial conditions. Last, we have seen that when the original equation is posed in a bounded spatial domain, the treatment of its boundary conditions after spatial derivation can be rather delicate.

214

B. Jourdain

Appendix Lemma 1. Let P solve problem (MP) given in Definition 2. Then,   0   ˜ ∀t ≥ 0, H∗Pt (.)−u0 L1 (R) ≤ m t sup |β (u)| + 2t sup |α (u)| . |u|≤m

|u|≤m

Proof. Let (P y )y∈R be a regular conditional probability distribution of P given X0 = y. One has   y ˜ H ∗ Pt (.) − u0 L1 (R) = (P (Xt ≤ x) − 1{y≤x} )m(dy) dx R R   ≤ 1{y>x} P y (Xt − y ≤ x − y) + 1{y≤x} P y (Xt − y > x − y)dx|m|(dy) R R < P y , |Xt − y| > |m|(dy) . = R

One concludes by remarking that |m|(dy) a.e., under P y , the canonical process (Xt )t≥0 solves weakly Xt = y +

 t√



t

2α (H ∗ P˜s (Xs ))dWs +

0

β  (H ∗ P˜s (Xs ))ds ,

0

which ensures that  < P , |Xt − y| >≤ y



t sup |β (u)| + |u|≤m

0

 2t sup |u|≤m

|α (u)|

.

References 1. C.R. Anderson. A vortex method for flows with slight density variations. J. Comp. Physics, 61:417–444, 1985. 2. R.F. Bass and E. Pardoux. Uniqueness for Diffusions with Piecewise Constant Coefficients. Probab. Theory and Related Fields, 76:557–572, 1987. 3. S. Benachour, B. Roynette, and P. Vallois. Branching process associated with 2d Navier-Stokes equation. Rev. Mat. Iberoamericana, 17(2):331–373, 2001. 4. M. Bossy. Optimal rate of convergence of a stochastic particle method for the solution of a 1d viscous scalar conservation law. Math. Comput., 73(246):777– 812, 2004. 5. M. Bossy, L. Fezoui, and S. Piperno. Comparison of a stochastic particle method and a finite volume deterministic method applied to Burgers equation. Monte Carlo Methods Appl., 3(2):113–140, 1997. 6. M. Bossy and B. Jourdain. Rate of convergence of a particle method for the solution of a 1d viscous scalar conservation law in a bounded interval. Ann. Probab., 30(4):1797–1832, 2002.

Probabilistic Approximation of Some Nonlinear Parabolic Equations

215

7. M. Bossy and D. Talay. Convergence rate for the approximation of the limit law of weakly interacting particles: application to the Burgers equation. Ann. Appl. Probab., 6(3):818–861, 1996. 8. M. Bossy and D. Talay. A stochastic particle method for the McKean-Vlasov and the Burgers equation. Math. Comp., 66(217):157–192, 1997. 9. H. Brezis and M.G. Crandall. Uniqueness of solutions of the initial-value problem for ut − ∆φ(u) = 0. J. Math. pures et appl., 58:153–163, 1979. 10. P. Calderoni and M. Pulvirenti. Propagation of chaos for Burgers’ equation. Ann. Inst. Henri Poincar´ e Section A, 39(1):85–97, 1983. 11. A.J. Chorin. Numerical study of slightly viscous flows J. Fluid Mech. 57:785– 793, 1973. 12. G.H. Cottet. A vorticity creation algorithm, Mathematical aspects of vortex dynamics, SIAM, Philadelphia PA, 1989. 13. A.L. Fogelson. Particle-method solution of two-dimensional convection-diffusion equations. J. Comp. Physics, 100:1–16, 1992. 14. B. Jourdain. Diffusions with a nonlinear irregular drift coefficient and probabilistic interpretation of generalized Burgers’ equations. ESAIM, Probab. Stat. (http://www.emath.fr/ps/), 1:339–355, 1997. 15. B. Jourdain. Diffusion processes associated with nonlinear evolution equations for signed measures. Methodol. Comput. Appl. Probab., 2(1):69–91, 2000. 16. B. Jourdain. Probabilistic approximation for a porous medium equation. Stochastic Process. Appl., 89(1):81–99, 2000. 17. B. Jourdain. Probabilistic gradient approximation for a viscous scalar conservation law in space dimension d ≥ 2. Stochastics Stochastics Rep., 71:243–268, 2001. 18. B. Jourdain. Probabilistic characteristics method for a 1d scalar conservation law. Ann. Appl. Probab., 12(1):334–360, 2002. 19. B. Jourdain and S. M´el´eard. Probabilistic interpretation and particle method for vortex equations with Neumann’s boundary condition. Proc. Edinburgh Math. Soc., 47(3), pp 597-624, 2004. 20. B. Jourdain, S. M´el´eard, and W. Woyczynski. Probabilistic approximation and inviscid limits for 1-d fractional conservation laws. Bernoulli, to appear. 21. N.V. Krylov. Some estimates of the probability density of a stochastic integral. Math. USSR Izvestija, 8(1):233–254, 1974. 22. D. L´epingle. Euler scheme for reflected stochastic differential equations. Math. Comput. Simul., 38(1-3):119–126, 1995. 23. C. Marchioro and M. Pulvirenti. Hydrodynamics in two dimensions and vortex theory. Comm. Math. Phys., 84:483–503, 1982. 24. S. M´el´eard. A trajectorial proof of the vortex method for the two-dimensional Navier Stokes equations. Ann. Appl. Probab., 10(4):1197–1211, 2000. 25. S. M´el´eard. Monte-Carlo approximations for 2d Navier-Stokes equation with measure initial data. Probab. Theory Relat. Fields, 121:367–388, 2001. 26. K. Oelschl¨ ager. A law of large numbers for moderately interacting diffusion processes. Z. Warsch. Verw. Geb., 69:279–322, 1985. 27. B. Roynette and P. Vallois. Instabilit´e de certaines equations diff´erentielles stochastiques non lin´eaires. J. Funct. Anal., 130(2):477–523, 1995. 28. A. Sherman and M. Mascagni. A gradient random walk method for twodimensional reaction-diffusion equations. SIAM J. Sci. Comput., 15(6):1280– 1293, november 1994.

216

B. Jourdain

29. T. Shiga and H. Tanaka. Central Limit Theorem for a System of Markovian Particles with Mean Field Interactions. Z. Warsch. Verw. Geb., 69:439–459, 1985. 30. D.W. Stroock and S.R.S. Varadhan. Multidimensional Diffusion Processes. Springer, 1997. 31. A.S. Sznitman. A propagation of chaos result for Burgers’ Equation. Probab. Theory Relat. Fields, 71:581–613, 1986. 32. A.S. Sznitman. Topics in propagation of chaos. In Ecole d’´et´e de probabilit´es de Saint-Flour XIX - 1989, Lect. Notes in Math. 1464. Springer-Verlag, 1991.

Myths of Computer Graphics Alexander Keller Abt. Medieninformatik, University of Ulm, 89069 Ulm, Germany [email protected]

Summary. Computer graphics textbooks teach that sampling by deterministic patterns or even lattices causes aliasing, which only can be avoided by random, i.e. independent sampling. They recommend random samples with blue noise characteristic, which however are highly correlated due to their maximized minimum mutual distance. On the other hand the rendering software mental ray, which is used to generate the majority of visual effects in movies, entirely is based on parametric integration by quasi-Monte Carlo methods and consequently is strictly deterministic. For its superior quality the software even received a Technical Achievement Award (Oscar) by the American Academy of Motion Picture Arts and Sciences in 2003. Along the milestones of more than ten years of development of quasi-Monte Carlo methods in computer graphics, we point out that the two previous statements are not contradictory.

1 Introduction Image synthesis is the most visible part of computer graphics. On the one hand it is concerned with physically correct image synthesis, which intends to identify light paths that connect light sources and cameras and to sum up their contributions. On the other hand it also comprises non-photorealistic rendering, like e.g. the simulation of pen strokes or watercolor. Image synthesis is an integro-approximation problem for which analytical solutions are available in exceptional cases only. Therefore numerical techniques have to be applied. While standard graphics text books still recommend elements of classical Monte Carlo integration, the majority of visual effects in movie industry is produced by using more efficient quasi-Monte Carlo techniques. The Mathematical Problem The underlying mathematical task is to determine the intensity I(k, l, t, λ), where (k, l) is the location of a pixel on the display medium. For the sake of

218

A. Keller

clarity, we will omit the dependency on the time t and the wavelength λ of a color component of a pixel in the sequel. Determining the intensity of a single pixel I(k, l), i.e. measuring the light flux through a pixel, requires to compute a functional of the solution of the radiance transport integral equation  L(h(x, ωi ), −ωi )f (ωi , x, ω)| cos θi |dωi . L(x, ω) = Le (x, ω) + 2 S

 =:(Tf L)(x,ω)

As a Fredholm integral equation of the second kind, the radiance L in the point x into the direction ω is the sum of the source radiance Le and the reflected and transmitted radiance Tf L, which is an integral over the unit sphere S 2 . The cosine of the angle θi between the surface normal in x and the direction of incidence ωi accounts for the perpendicular incident radiance only, which is colored by the surface interface properties given by f . Finally h determines the closest point of intersection of a ray from x into the direction ωi . The extension to participating media, which we omit here for lack of space, exposes the same structure. Simultaneously computing all pixels   Rα (L(x, ω), k, l, x, ω)dωdx I(k, l) := ∂V

S2

of an image is an integro-approximation problem. The mapping Rα represents the mathematical description of the camera and its response to the radiance L. Rα often is non-linear in order to be able to compensate for the limited dynamic range of most display media. Function Classes In a physically correct setting the norm Tf  must be bounded by 1 in order to guarantee energy conservation. Then the Neumann-series converges and the computation of the radiance L = SLe :=

∞ (

Tfi Le

i=0

can be reduced to an infinite sum of integrals with increasing dimension. The single integrals Tfi Le have a repetitive low dimensional structure inherited from stacking transport operators. Obviously lower powers of the transport operator are likely to be more important. Real world light sources Le are bounded and consequently the radiance L uniformly can be bounded by some b > 0. In addition real world radiance L(y, ω, t, λ) ∈ L2b is a signal of finite energy and thus must be square integrable. However, often singular surface properties, as for example specular reflection, are modeled by

Myths of Computer Graphics

219

(Tδω L)(x, ω) := L(h(x, ω  ), −ω  ) using Dirac’s δ distribution, where ω  ≡ ω  (ω) is the direction of specular reflection. Then the operator norm of the solution operator can even reach 1 and the Neumann series can diverge. The additional problem of insufficient techniques [KK02a] is caused by δ ∈ L2b , because some transport paths cannot be efficiently sampled and force the need of biased approximations like e.g. the photon mapping algorithm [Jen01] for rendering caustics. Both the radiance L and the intensity I are non-negative and piecewise continuous, where the discontinuities cannot be efficiently predicted. The actual basis of the function class to represent and approximate the intensity I(k, l, t, λ) in fact is determined by the display medium or image storage format, e.g. an interleaved box basis for the color components of TFT displays [BBD+ 00], cosines for JPEG compressed images, etc.

2 Quasi-Monte Carlo Methods Due to the lack of efficient analytical solutions, rendering algorithms reduce image synthesis to numerical integro-approximation. Simulating a camera with anti-aliasing, motion blur, and depth of field already contributes 5 dimensions to the integration domain of the intensity I. Area light sources and each level of reflection contribute another 2 dimensions. Consequently the mathematical problem is high-dimensional, discontinuous, and in L2b . Since tensor product techniques will fail due to dimensionality and a lack of continuity, Monte Carlo and quasi-Monte Carlo methods are the obvious choice. Monte Carlo methods use random sampling for estimating integrals by means. Quasi-Monte Carlo methods look like Monte Carlo methods, however, they use deterministic points for sampling an integrand. In contrast to random samples, the specifically designed deterministic point sets are highly correlated, which allows for a much higher uniformity and results in a faster convergence. 2.1 Quasi-Monte Carlo Points Real random numbers on the unit interval are characterized by independence, unpredictability, and uniformity. For Monte Carlo integration the independence is required to prove error bounds and the uniformity is required to prove the order of convergence. Since real random numbers are expensive to generate, usually efficient deterministic algorithms are used to simulate pseudorandom numbers [Nie92b], which then of course are perfectly predictable but seemingly independent. However, the independence cannot be observed any longer after averaging the samples. Quasi-Monte Carlo integration is based on these observations. By neglecting independence and unpredictability it is possible to construct deterministic points, which are much more uniform than random number samples

220

A. Keller

can be. There exist a lot of constructions for such deterministic point sets Pn = {x0 , . . . , xn−1 } ⊂ [0, 1)s , which are based on only two basic principles: Radical inversion based point sets determine samples by   i xi = , Φb1 (i), . . . , Φbs−1 (i) , n where Φb : N0 → Q ∩ [0, 1) ∞ ∞ ( ( i= al (i)bl → al (i)b−l−1 l=0

(1)

l=0

is the radical inverse [Nie92b] in an integer base b. The digit aj (i) is the j-th digit of the index i represented in base b. The Hammersley point set is obtained by choosing bc as the c-th prime number. The uniformity of these points has been improved by applying permutations to the aj (i) before computing the inverse. Zaremba [Zar70] used the simple permutation πb (aj (i)) = aj (i) + j mod b and later Faure [Fau92] developed a set of permutations even generalizing and improving Zaremba’s results. Choosing all bc = b along with an appropriate set of mappings applied to the digits aj (i) yields the construction and theory of (t, m, s)-nets [Nie92b]. There has been a lot of research in order to efficiently compute radical inverses. A simple and very efficient method is to tabulate the sum of the least significant T digits and to reuse them while generating the points ∞ (

πb (aj (i))b−j−1 =

j=0

∞ ( j=T



πb (aj (i))b−j−1 +



only every bT -th time

T −1 ( j=0



πb (aj (i))b−j−1 .



Table of size bT

This method has been developed in the context of scrambled radical inversion [FK02]. Rather than using Gray-codes, this method generates the points in their natural order at comparable speed. Rank-1 lattice points xi =

i (1, g1 , . . . , gs−1 ) mod 1 n

are faster to generate than radical inversion based points. Their quality depends on the integer generator vector (1, g1 , . . . , gs−1 ) ∈ Ns , however, the construction of good generator vectors is not obvious. In order to reduce the search space, Korobov determined the generator vectors by only one parameter a with gi = ai . Higher rank lattices can be constructed by linear combinations of rank-1 lattices.

Myths of Computer Graphics

221

Both principles can be generalized to yield sequences of points, which allow for adaptive sampling without discarding previously taken samples, however, at the price of a slight loss of uniformity: The Halton sequence and its variations corresponding to the Hammersley points, (t, s)-sequences containing (t, m, s)nets, and extensible lattice rules containing lattices. The above constructions yield rational numbers in the unit interval. It is especially interesting to use the base b = 2 and n = 2m points, because then the points can be represented exactly in the actual machine numbers M ⊂ Q as defined by the ANSI/IEEE 754-1985 standard for binary floating point arithmetic. 2.2 Uniformity The different constructions of the previous section in fact have one common feature: They induce uniform partitions of the unit cube. Niederreiter [Nie03] characterized this kind of uniformity by the Definition 1. Let (X, B, µ) be an arbitrary probability space and let M be a nonempty subset of B. A point set Pn of n elements of X is called (M, µ)uniform if n−1 ( χM (xi ) = µ(M ) · n for all M ∈ M , i=0

where χM (xi ) = 1 if xi ∈ M , zero otherwise. Examples of (M, µ)-uniform point sets mentioned in [Nie03] are samples from the cartesian product midpoint rule and radical inversion based points. In addition, rank-1 lattices are (M, µ)-uniform, too: The Voronoi-diagram of a lattice partitions the unit cube into n sets of identical shape and volume n1 (see Fig. 7 and [Kel04]). This underlines that for (M, µ)-uniformity all µ(M ) must have the same denominator n. The function classes of computer graphics imply to use the probability space ([0, 1)s , B, λs ) with the Borel-sets B and the s-dimensional Lebesguemeasure λs . What’s wrong with Discrepancy? A sequence of point sets is uniformly distributed if and only if its discrepancy [Nie92b] vanishes in the limit. The deterministic constructions sketched in previous section can obtain so-called low discrepancy, which vanishes with roughly speaking n1 , while independent random points only can obtain roughly 1 √1 and points from the cartesian product midpoint rule even only acquire √ s n. n There are some facts about discrepancy that make it problematic. Discrepancy is an anisotropic measure, because its concept is based on axis-aligned

222

A. Keller

boxes and consequently rotating point sets influences discrepancy. While samples from the cartesian product midpoint rule result in bad discrepancy, lattice points from the Fibonacci lattices [SJ94] have low discrepancy, although some of them are just rotated rectangular grids. Discrepancy is even not shiftinvariant since shifting a point set on the unit torus also changes discrepancy. Maximized Minimum Distance Definition 1 supports partitions which are not axis-aligned, as for example the Voronoi-diagram of a rank-1 lattice. Maximum uniformity in this sense can be obtained by selecting the points such that the regions of the Voronoidiagram approximate spheres as much as possible, i.e. by maximizing the mutual minimum distance dmin (Pn ) := min min xj − xi T 0≤i
among all sample points in Pn . By  · T we denote the Euclidean distance on the unit torus. The minimum distance measure is isotropic and shift-invariant thus overcoming these disadvantages of discrepancy. In Fig. 1 we investigated the minimum distance of known quasi-Monte Carlo points. It is interesting to observe that the constructions with better discrepancy have larger minimum distance, too, as can be seen for the Hammersley points, the Sobol’ sequence, and the Larcher-Pillichshammer points [LP01]. Hammersley

Sobol‘ sequence

Larcher-P’hammer

Rank-1 lattice

dmin = 0.0883883

dmin = 0.0883883

dmin = 0.176777

dmin = 0.25

dmin = 0.0220971

dmin = 0.0441942

dmin = 0.0883883

dmin = 0.125973

Fig. 1. Classical quasi-Monte Carlo points for n = 16 (top row ) and n = 64 (bottom row ) along with their mutual minimum distance dmin . The rank-1 lattice has been selected such that its minimum distance is maximal.

Myths of Computer Graphics

223

It also can be observed that the minimum distance of the Halton sequence with the permutations by Faure [Fau92] is maximized as compared to the original Halton sequence. The rank-1 lattices in Fig. 1 are Korobov lattices, where the parameter a has been chosen to maximize the minimum distance. The rank-1 lattice at n = 16 points in Fig. 1 in fact can be generated as a (t, 4, 2)-net in base b = 2. Obviously this is not possible for the quality parameter t = 0, because at least two points lie on one line parallel to the x-axis. In this case the best quality parameter prevents the points from reaching maximum minimum distance. Similarly, postulating gcd(gi , n) = 1 restricts the search space for generator vectors of lattices in such a way that the lattices with maximum minimum distance cannot be found. Forcing Latin hypercube properties by t = 0 or gcd(gi , n) = 1, respectively, may be useful in some situations, however, prevents the sample points to optimally cover the unit torus in the minimum distance sense. Maximizing the minimum distance for generating highly uniform point sets is a principle of nature. For example the distribution of the receptors in the retina is grown that way [Yel83]. Algorithmically this principle is known as Lloyd’s relaxation scheme, where samples with identical charges are placed on the unit torus and then the points are let repel themselves until they reach some equilibrium. From a mathematical point of view the points are moved to the center of gravity of their Voronoi cells during relaxation. The convergence of this scheme has been improved quadratically in [KK03]. Note that rank-1 lattices are invariant under this kind of relaxation scheme. Point sets selected by maximizing the mutual Euclidean distance on the unit torus induced by  · T have the advantage that they seamlessly can be tiled in order to fill s-dimensional space. This is obvious and intrinsic for lattices. However, it is not true for radical inversion based points in general: While the points by Larcher and Pillichshammer tile seamlessly, the Hammersley points don’t. The norm  · T in fact should be a weighted norm, which includes the size of the integration domain. In Fig. 2 we illustrated that maximizing the minimum distance on the unit cube does not imply nice uniformity when scaling the domain of integration. What’s wrong with the classical Error Bounds? It is common knowledge that quasi-Monte Carlo integration outperforms Monte Carlo integration in many applications, however, the classical error bound theorems often do not fit the function classes of the application. The Koksma-Hlawka inequality [Nie92b, Sect. 2.2] deterministically bounds the integration error by the product of the discrepancy of the sample points used to determine the sample mean and the variation of the integrand in the sense of Hardy and Krause. The variation in the sense of Hardy and Krause can be considered as the remainder of trying to obtain discrepancy as a factor in the error bound.

224

A. Keller

(a)

(b)

(c)

Fig. 2. Selection of lattices by maximum minimum distance (a) in the unit cube, (b) in the unit cube scaled to the integration domain, and (c) in the integration domain. Obviously, considering the scale of the integration domain yields more uniform points.

The class of functions of bounded variation in the sense of Hardy and Krause is very unhandy as for example discontinuities that are not aligned with the coordinate axes already yield infinite variation. The error bound thus becomes useless in already simple settings of computer graphics such as an edge

1 y>x f (x, y) = (2) 0 else in the unit square, for which the variation is unbounded. Using other kinds, like e.g. isotropic discrepancy, it is possible to find an error bound [HK94a], which works for this case. However, it becomes far too pessimistic in higher dimensions. Similar to the Koskma-Hlawka inequality there exist deterministic error bounds for the integration by lattices [SJ94]. The used function class requires periodic functions and imposes certain constraints on the Fourier coefficients of the integrand, which do not apply for discontinuous functions as used in computer graphics. In a vast number of publications experiments give numerical evidence that quasi-Monte Carlo methods outperform Monte Carlo methods in practice, however, in the majority of the cases classical theorems cannot explain the observed results. The main reason is that general discontinuities cannot be accounted for by the classical error bounds. 2.3 Quasi-Monte Carlo Integro-Approximation As stated in the introduction image synthesis is an integro-approximation problem in L2b and quasi-Monte Carlo integro-approximation in fact successfully has been used in computer graphics [Kel01a, Kel03]. We therefore generalize Niederreiter’s theorem 2 from [Nie03] in the sense of parametric integration [FC62]:

Myths of Computer Graphics

225

Theorem 1. Let (X, B, µ) be an arbitrary probability space and let M = {M1 , . . . , Mk } be a partition of X with Mj ∈ B for 1 ≤ j ≤ k. Then for any (M, µ)-uniform point set P = {x1 , . . . , xn } and any bounded function f , which restricted to X is µ-integrable, we have ; n−1 ;  ;1 ( ; ; ; f (xi , y) − f (x, y)dµ(x); ; ;n ; X i=0 ; ; k ; ; ( ; ; ≤ µ(Mj ) ; sup f (x, y) − inf f (x, y); x∈Mj ;x∈Mj ; j=1

for any suitable norm  · . Proof. For all y ∈ Y we now consider an arbitrary stratum Mj ∈ M. Then we have n−1 (

χMj (xi ) inf f (x, y) ≤ x∈Mj

i=0

n−1 (

f (xi , y) ≤

i=0 xi ∈Mj

n−1 (

χMj (xi ) sup f (x, y) , x∈Mj

i=0

which implies µ(Mj ) inf f (x, y) ≤ x∈Mj

n−1 1 ( f (xi , y) ≤ µ(Mj ) sup f (x, y) , n i=0 x∈Mj xi ∈Mj

because P is an (M, µ)-uniform point set. Similarly we obtain  µ(Mj ) inf f (x, y) ≤ f (x, y)dµ(x) ≤ µ(Mj ) sup f (x, y) . x∈Mj

x∈Mj

Mj

From the latter two sets of inequalities it follows that  1 n−1 ( f (xi , y) − f (x, y)dµ(x) n Mj i=0 xi ∈Mj   ≤ µ(Mj )

sup f (x, y) − inf f (x, y) x∈Mj

Since M is a partition of X, we can write

x∈Mj

226

A. Keller

 n−1 1( f (xi , y) − f (x, y)dµ(x) n i=0 X k k  n−1 ( 1( ( = f (xi , y) − f (x, y)dµ(x) n j=1 i=0 j=1 Mj ⎛ =

xi ∈Mj

k (

⎜1 ⎝ n j=1

n−1 (

 f (xi , y) − Mj

i=0 xi ∈Mj

⎞ ⎟ f (x, y)dµ(x)⎠ .

Using the previous inequality and applying the norm  ·  to both sides of the resulting inequality yields the desired bound.  By the omission of y the norm reduces to the absolute value and it remains the original theorem and proof of [Nie03, Theorem 2]: Theorem 2. Let (X, B, µ) be an arbitrary probability space and let M = {M1 , . . . , Mk } be a partition of X with Mj ∈ B for 1 ≤ j ≤ k. Then for any (M, µ)-uniform point set P = {x0 , . . . , xn−1 } and any bounded µ-integrable function f on X we have    n−1 k ( 1( f (xi ) ≤ µ(Mj ) sup f (x) − inf f (x) . f (x)dµ(x) − x∈Mj X n x∈Mj i=0

j=1

The latter theorem resembles the results of Hlawka [Hla71] in the setting of Riemann integration, where the upper bound has been named mean oscillation. However, by using the concept of (M, µ)-uniform point sets instead of discrepancy, proofs become simpler and results are more general. With (X, B, µ) = ([0, 1)s , B, λs ) both theorems are applicable in the setting of computer graphics. For the example function (2) the deterministic error O(n−1/2 ) bound can be obtained by selecting an (M, µ)-uniform point set with k = n. The difference of the supremum and the infimum can only be one in the O(n1/2 ) sets of the partition, which are crossed by the discontinuity otherwise it must be zero. With µ(Mj ) = n1 we obtain the bound. This argument is similar to the argument used in the Numerical Recipes book [PTVF92], but does not use probabilistic arguments for a deterministic algorithm. 2.4 Bias Elimination by Randomization Quasi-Monte Carlo methods are biased, because they are deterministic, but consistent, because they asymptotically converge to the right solution. Randomizing these algorithms allows for unbiased estimators and for unbiased error estimators.

Myths of Computer Graphics

227

It is very useful to consider the method of dependent tests 

 f (x, y)dx =

[0,1)s

n−1 (

wi (x, y)f (Ri (x), y)dx

(3)

[0,1)s i=0

≈

n−1 (

wi (ω, y)f (Ri (ω), y)

i=0

by first applying an equivalence transformation, which does not change the result, and then using one random sample ω in order to obtain an unbiased estimate. This formulation is able to cover a wide range of techniques that increase the efficiency of the method of dependent tests. For example random scrambling of arbitrary points [Owe95], the random translation on the unit torus [CP76], random padding of arbitrary point sets [KK02a], stratification and stratification induced by rank-1 lattices [Lem00, Kel04], trajectory splitting [Kel03, Kel01b], and many more techniques easily can be formulated by a tuple (wi , Ri )n−1 i=0 of replications Ri with associated weights wi . Omitting y, the special case with equal weights wi (x) = n1 , where for fixed ω the set of points (Ri (ω))n−1 i=0 is of low discrepancy, is defined to be randomized quasi-Monte Carlo integration. Repeating independent realizations allows one to estimate the error of the approximation in an unbiased way. However, some convergence is sacrificed, since independent random samples cannot be as uniformly distributed as correlated samples can be. This is especially noticeable in the setting of computer graphics, where small numbers of samples are used.

3 Quasi-Monte Carlo in Computer Graphics Although the generalizations of the previous sections apply to many other transport problems, they were developed for computer graphics. In the sequel we briefly summarize the most important quasi-Monte Carlo rendering algorithms that have been developed during the last 10 years in order to present new techniques for adaptive sampling. In addition we try to rectify some seemingly persistent myths on quasi-Monte Carlo methods in computer graphics. 3.1 Myths In principle, developments of quasi-Monte Carlo methods in computer graphics already had been anticipated by Niederreiter in [Nie92a] after Shirley [Shi91] performed first successful experiments on quasi-Monte Carlo methods in computer graphics. A lot of subsequent numerical experiments clearly showed the superiority of quasi-Monte Carlo methods in computer graphics,

228

A. Keller

however, without any useful proofs. The two major reasons why quasi-Monte Carlo methods had a tough way in computer graphics are that the KoksmaHlawka inequality is useless in the setting of computer graphics and that randomized quasi-Monte Carlo methods were not known in the field. With the availability of simple, unbiased, randomized quasi-Monte Carlo methods for efficient multidimensional sampling in computer graphics [KK02b], efficient quasi-Monte Carlo methods slowly were entering graphics text books first by reference and then by full text [PH04]. Besides the lack of a solid representation in textbooks, the subsequent citations from famous publications caused misinterpretations and myths about quasi-Monte Carlo methods in computer graphics: “One-dimensional jittered sampling” has been used in the famous teddy bear paper [KK89] for line integration along a ray through a volume. The sampling scheme has been continuously used, although more efficient quasi-Monte Carlo techniques are available (see [PKK00, Kel01b] and Sect. 3.5). “In particular the results are not consistent with image quality.” is cited from the conclusions of [Mit92]: Although faster convergence was determined, aliasing artifacts were misinterpreted as a disadvantage of quasi-Monte Carlo methods. This terminated the broad investigation of quasi-Monte Carlo methods in the context of computer graphics for a long time (see [Kel03] and Sect. 3.2). “So far these methods have not found significant uses in graphics, so our discussion will be brief.” is a comment from Veach’s excellent PhD thesis [Vea97] on correlated estimators. At that time randomized quasi-Monte Carlo methods, which can be considered as an instance of correlated sampling, had not yet been investigated in the context of com√ puter graphics (see [Kel03, KK02a, KK02b] and Sect. 3.3). “ 5-sampling” has been used for√adaptively sampling surfaces for point based rendering [SD01]. In fact 5-sampling is one instance of the more general construction of Fibonacci-lattices and as such of rank-1 lattices (see [Kel04] and Sects. 3.2 and 3.5). “The periodicity of the Halton sequence” [DBMS02] has been used for adaptively sampling path space seemingly independent of dimension. In fact the method is a stratification method that suffers the curse of dimension (see Sect. 3.4). Besides their excellent contributions, the above publications missed important facts about quasi-Monte Carlo integration. On the other hand quasi-Monte Carlo research did not recognize minimum distance as a versatile concept for the uniformity of sample points.

Myths of Computer Graphics

229

3.2 Anti-Aliasing Anti-aliasing is a central and tough problem of computer graphics [Gla95]. Various difficulties already can be experienced by rendering a simple checker board as displayed in Fig. 3: While it is simple to compute the color of a pixel as an average as long as the checker board cells are clearly distinguishable, this is no longer possible at the horizon, where through one pixel infinitely many cells can be seen. By common sense, we expect the pixels at the horizon to be gray, i.e. the average of black and white, however, zooming into a pixel and looking at what we see there, we will be surprised: The areas of black and white tiles are not equal in general. This means that no matter what quadrature we use, the horizon will not appear gray, but somehow patterned. The lens of the eye is not perfectly transparent and thus slightly blurs the light before it reaches the retina. The amount of blur perfectly fits the

(a)

(b)

(c)

(d)

Fig. 3. The image shows an infinite plane with a checker board texture seen in perspective. While in the front the fields clearly can be distinguished, fancy patterns appear towards the horizon. Avoiding these so-called aliases is a core problem of computer graphics. It partially has been ameliorated by the choice of classical sampling patterns in graphics: (a) Stratified sampling, (b) Latin hypercube sampling, and (c) Blue noise sampling. Quasi-Monte Carlo points in d) can unite major properties of the point sets in (a), (b), and (c).

230

A. Keller

resolution of the receptors in the retina. A similar trick in computer graphics is to blur the texture before integration, where the strength of the blur depends on the distance from the eye. However, this does not help, if the mathematical problems are not caused by textures. In fact situations, where filtering fails, are common. Another compromise is to filter the resulting image. Then, however, not only aliases but also previously sharp details become blurred. Aliasing only can be hidden by random sampling. Taking one independent random sample inside each pixel, the horizon will appear as uncorrelated black and white pixels. The structured aliases thus are mapped to noise, which is less disturbing to the eye. However, taking more random samples per pixel, the quadrature will eventually converge and aliases will appear instead of the noise. Assuming that situations like the ones mentioned before can be managed by suitable filtering, we now take a look at the most popular sampling patterns of computer graphics, which are depicted in Fig. 3: Stratified sampling is at least as good as random sampling, however, suffers the curse of dimension, since each axis of the integration domain has to be divided at least once. Latin hypercube sampling can never be much worse than random sampling and is available for any dimension and number of samples. The average observed performance is good, although it cannot be guaranteed. Poisson disk sampling simulates the distribution of receptors in our eyes. The mutual minimum distance of the points is maximized, which results in a reduced noise level in the rendered image. Although restricted by a guaranteed minimum distance, i.e. an empty disk around each sample, the samples points are randomly placed. Thus representable details remain sharp in the final image and aliases are efficiently mapped to noise. However, Poisson disk sampling patterns are expensive to generate. The properties of the sampling patterns seem disjoint, however, there exist efficient-to-generate quasi-Monte Carlo points which unite the above properties. Anti-Aliasing by (0, 2m, 2)-Nets in Base b = 2 In [HK94a] we investigated the use of Hammersley points in the accumulation buffer algorithm [HA90]. Despite considerable improvements, each pixel had to use the same samples, which required higher sampling rates in order to avoid aliases. This was also criticized in [Mit92]. A solution to that problem has been found in [Kel03] by exploiting the structure of (0, 2m, 2)-nets in base b = 2. In Fig. 3d) we illustrate that (0, 2m, 2)-nets in base b = 2 unite the properties of classical sampling patterns in computer graphics. The first two dimensions   i , Φ (i) xi = 2 (2m )2

Myths of Computer Graphics

231

Fig. 4. Left image: Anti-aliasing of a test pattern by 4 × 4 stratified random sampling. Center image: Improved anti-aliasing by using the tiled Hammersley point set at 16 samples per pixel. Right image: Illustration of how the samples in a pixel are determined by tiled instances of a Hammersley point set. The solid lines indicate the unit square including one set of 16 Hammersley points and the dashed lines indicate screen pixels’ boundaries. For a lack of space the illustration only uses 4 samples per pixel.

of the Hammersley points are an example of such a point set, which is stratified, a Latin hypercube sample, and has a guaranteed minimum distance of at least (2m1 )2 . The method of dependent tests is realized by tiling the image plane with copies of the (0, 2m, 2)-net as illustrated in Fig. 4. In order to reduce aliasing artifacts, neighboring pixels should have different samples. This is achieved by letting one Hammersley point set cover multiple pixels. The improvement in convergence is visible in the same figure, where we compare stratified random sampling and anti-aliasing by using the Hammersley point set. The improvements directly transfer to other rendering algorithms like e.g. the REYES architecture [CCC87] as used in PIXAR’s RenderMan software. In [LP01] it has been shown that the Hammersley point set actually is among the worst (0, m, 2)-nets in base b = 2. The point set introduced in the same article has much better uniformity properties in the sense of discrepancy. As shown in Fig. 1 its maximum minimum distance is larger, too. As mentioned before, this point set along with an efficient randomization scheme [KK02b] is slowly becoming standard in computer graphics [PH04] due to its superior performance. Anti-Aliasing by Lattices Selecting rank-1 lattices by maximum minimum distance takes the principle of Lloyd relaxation and Poisson disk sampling to the limit. Since rank-1 lattices are available for any number n of points, the sampling rate can be chosen freely. A factorization of n as required for axis-aligned stratified sampling is not needed. In order to attenuate aliasing in each pixel a different random shift is added to the lattice points resulting in an unbiased estimate. The images obtained that way expose minimal noise and aliasing as demonstrated in [Kel04]. The samples of a randomly shifted lattice can be generated fast and

232

A. Keller

the observed convergence is even faster as compared to the radical inversion based methods from the previous section. The method can be derandomized by extracting the shift from e.g. a Hammersley point set using the stratification properties pointed out in the previous section. 3.3 Path Tracing The principles of the previous sections can be extended to approximate the full integro-approximation problem. Using the Neumann series the computation of L is reduced to a sum of integrals as sketched in the introduction. We carried out extensive investigations [HK94b, Kel95, Kel96a, Kel96b, Kel97, Kel98, Kel98b, Kel03, KK02a, KK02b] on path tracing algorithms in connection with quasi-Monte Carlo methods and randomized versions of the latter methods. All the experiments resulted in improvements when using quasi-Monte Carlo methods as compared to classical algorithms of computer graphics. Deterministic Scrambling As mentioned before randomizing quasi-Monte Carlo methods allows for unbiased estimators and unbiased error estimates. The latter is not very interesting in computer graphics, as there exist better adaptive methods (see Sect. 3.4). Furthermore the resulting images can be generated in the same time and quality no matter whether a quasi-Monte Carlo or its randomized counterpart is used. However, it is interesting to look at the effect of randomization schemes on the minimum distance. While randomizing point sets by random shifts [CP76] does not change their maximum minimum distance on the torus, random scrambling [Owe95, Mat98] does. Applying random scrambling to the classical low discrepancy points sets it can be observed that often the uniformity is improved and rarely decreased. This is true for both discrepancy and minimum distance. Our experiments show that random scrambling only marginally changes the uniformity of the scrambled Halton sequence, indicating that the deterministic permutations by Faure [Fau92] already are a very good choice. The permutations for the Halton sequence by Zaremba [Zar70] and Faure [Fau92] are realizations of random scrambling and can be considered as deterministic scrambling long before random scrambling was introduced. Although the permutations have been introduced to improve on discrepancy, they also increase the maximum minimum distance of the Halton points. Implementing the scrambled Halton sequence is much simpler than random scrambling [FK02] and avoids the risk that uniformity if influenced negatively.

Myths of Computer Graphics

233

3.4 Adaptive Integro-Approximation In computer graphics the difficulty of the integro-approximation problem differs from pixel to pixel and adaptive methods pay off. A possible criterion for refinement is to compare the image gradient ∇I(k, l)2 > T · n−α k,l

(4)

against a threshold T . By the exponent α ∈ [0.5, 1.5] we can adapt to the speed of convergence and nk,l is the number of samples used for pixel (k, l). Alternatively refinement can be indicated by the routines that evaluate the radiance L, since these routines can access more information of the scene to be rendered. In the sequel we develop a method for adaptive anti-aliasing by elements of the Halton sequence. The method is superior to random sampling, since the points of the Halton sequence are more uniform and spoken visually exactly fall into the largest unsampled gaps of the sampling domain resulting in a smooth convergence. Stratification by Radical Inversion The radical inverse (1) mirrors the representation of the index i by the digits al (i) ∈ {0, . . . , b − 1} in the integer base b ∈ N at the decimal point. In base b = 2 this means that

[0, 1/2) for i even, (5) Φ2 (i) ∈ [1/2, 1) else. This observation has been generalized in [DBMS02] and was named periodicity. In fact, however, this property relates much more to stratification as formalized by the definition of (0, 1)-sequences [Nie92b]. Therefore a different derivation is presented that stresses the actual stratification property. The index i is chosen as i = j · bn + k for k ∈ {0, . . . , bn − 1} and inserted into (1) yielding Φb (i) = Φb (j · bn + k) ∞ ( al (j · bn + k)b−l−1 = l=0

=

∞ (

al (j · bn )b−l−1 +

l=0

=

al (k)b−l−1

l=0

∞ (

al (j)b−n−l−1 + Φb (k)

l=0 −n

=b 

∞ (

Φb (j) +Φb (k) 

∈[0,b−n )

(6)

234

A. Keller

by exploiting the additivity of the radical inverses that belong to the class of (0, 1)-sequences. The first term of the result depends on j and obviously is bounded by b−n , while the second term is a constant offset by the radical inverse of k. Since k ∈ {0, . . . , bn − 1} it follows that Φk ∈ {0, b−n , 2b−n , . . . , (bn − 1)b−n }. Fixing the n first digits by k then for j ∈ N0 yields radical inverses only in the interval [Φb (k), Φb (k) + b−n ). Stratified Samples from the Halton Sequence The one-dimensional observations from the previous section generalize to higher dimensions: The findings in [DBMS02] implicitly embody stratified sampling. For the Halton sequence [HW64] xi = (Φb1 (i), . . . , Φbs (i)) ∈ [0, 1)s , where for the c-th component bc is the c-th prime number, this is seen by choosing the index i=j·

s 

bnd d + k with 0 ≤ k <

d=1

Φbc (i) = Φbc

j·

s 

bnd d

(7)

d=1

yielding 

s 

 bnd d + k

d=1

⎛ ⎜ c = b−n Φbc ⎝j · c

⎞ s 

⎟ bnd d ⎠ + Φbc (k)

d=1 d =c

analogue to (6) and consequently xi ∈

s    c Φbc (k), Φbc (k) + b−n c c=1

for fixed 2s k and j ∈ N0 with the choice of the index i according to (7). Obviously the d=1 bnd d disjoint strata selected by k are disjoint and form a partition of the unit cube [0, 1)s . However the scheme suffers from the curse of dimension, since stratifying all s dimensions results in an exponential growth of the number of strata. This may be the reason, 2s why [DBMS02] only stratified the first four dimensions. The increment d=1 bnd d even grows much faster than 2s since except the first, all bd > 2. In Fig. 5 we compared a tensor product approach to using the Halton sequence. For 2s a stratification as fine as the tensor product approach large increments d=1 bnd d are required that hardly fit the integer representation of a computer. In addition the assumption that close points in the unit cube result in close paths in path space is not valid for more complex scenes. Depending on what part of the geometry is hit, the photons can be scattered in completely different parts of the scene, although their generating points had been close in the unit cube. The longer the trajectories the more diverging they will be. The above observations easily can be transferred to (t, s)-sequences in base b.

Myths of Computer Graphics

Voxel size of

1 1288

235

Increment of 23 33 53 73 113 133 17 · 19

Fig. 5. Illustration of the stratification of path space for photons that are emitted on the light source and traced for 2 reflections. The problem requires samples from the 8-dimensional unit cube. On the left the sampling domain has been stratified into 1288 strata of identical measure. Then one stratum has been selected and 8 random samples have been drawn from it to determine the photon trajectories. On the right the idea of stratification by the Halton sequence has been used to determine the 8 paths. In spite of an enormously large increment that hardly fits into the integer representation of a computer, the trajectories start to diverge after the 2nd reflection.

Adaptive Anti-Aliasing While stratification by the Halton sequence is not useful in high dimensions, it can be very useful in small dimensions as for example pixel anti-aliasing. The properties in two dimensions are illustrated in Fig. 6, where the first two components (Φ2 (i), Φ3 (i)) of the Halton sequence are plotted for 0 ≤ i ≡ ik (j) = j · 6 + k < 23 · 33 = 216. The stratum with the emphasized points contains all indices i ≡ ik (j) with k = 1. In  order to match the  square pixels the coordinates are scaled, i.e. xi → xi = 21 · Φ2 (i), 31 · Φ3 (i) . (1) (2) In general the first component xi is scaled by bn1 1 and the second xi by n2 n1 n2 b2 . Thus a b1 × b2 stratified sample pattern is obtained that can be periodically tiled over the image plane (according to Sect. 3.2). Identifying each stratum with a pixel, the identification k easily is determined (for example by a table lookup) by the pixel coordinates and a Halton sequence restricted to that pixel is obtained from i ≡ ik (j) = j · bn1 1 · bn2 2 + k for j ∈ N0 . The image in Fig. 6 has been computed using a path tracer (see Sect. 3.3) with the scrambled Halton sequence by Faure [Fau92]. Refinement was triggered by the gradient (4). Note that Faure’s scrambling does not change Φ2

236

A. Keller

↓

Fig. 6. The image on the left has been rendered by adaptive quasi-Monte Carlo integro-approximation. The thermograph in the center shows the effort spent for sampling. Darker regions required only a basic sampling rate, while brighter regions were classified hard and had to be sampled more. On the right we illustrate how the samples from the Halton sequence in the unit square were scaled to fit the pixel raster. The plots show the first two components xi = (Φ2 (i), Φ3 (i)) of the Halton sequence for 0 ≤ i < 23 · 33 = 216. The solid points have the indices i ≡ ik (j) = 21 · 31 · j + k selected by k = 1.

and Φ3 . Consequently the above algorithm can be applied directly and benefits from the improved uniformity of the scrambled sequence. 3.5 Trajectory Splitting Trajectory splitting can increase efficiency in rendering algorithms. A typical example is volume rendering as used in [KK89]: While tracing one ray through a pixel it is useful to send multiple rays to the light sources along that ray. In [PKK00,KH01] we showed that taking equidistant samples on the ray that all have been randomly shifted by the same amount is much more efficient than the original method that used jittered sampling. A general approach to trajectory splitting is to restrict the replications in (3) to some dimensions of the integrand. For quasi-Monte Carlo methods this was first experienced in [Kel03], where a strictly deterministic version of distribution ray tracing [CPC84] was developed. A systematic approach has been taken in [Kel01b], where randomization techniques from the field of randomized quasi-Monte Carlo methods have been parameterized. Instead of using random parameters, deterministic quasi-Monte Carlo points have been applied. Seen from a practical point of view, trajectory splitting can be considered as low-pass filtering of the integrand with respect to the splitting dimensions. Replications by Rank-1 Lattices The most powerful method is to split trajectories using domain stratification induced by rank-1 lattices [Kel04]. For the interesting s dimensions of the

Myths of Computer Graphics

237

problem domain a rank-1 lattice is selected. The matrix B contains the vectors spanning the unit cell as identified by the Voronoi-diagram of the rank-1 lattice. Then Ri : [0, 1)s → Ai   i · (1, g1 , . . . , gs−1 ) + Bx mod [0, 1)s x → n maps points from the unit cube to the i-th stratum Ai of the rank-1 lattice as depicted in Fig. 7b). In contrast to Cranley-Patterson rotations [CP76] this scheme can be applied recursively yielding recursive Korobov filters.

x4

x2 x0

b1

xi Ai b2

x3 x1

(a)

(b)

(c)

Fig. 7. The Voronoi-diagram of a rank-1 lattice induces a stratification a). All cells Ai are of identical measure and in fact rank-1 lattices are (M, µ)-uniform. A cell Ai is anchored at the i-th lattice point xi and is spanned by the basis vectors (b1 , b2 ). This can be used for recursive Korobov-filters b), where the points inside a lattice cell are determined by another set of lattice points transformed into that lattice √ cell. In computer graphics one special case c) of this principle has been named 5√ sampling, because the length of the dashed lines is 1/ 5. It is in fact a recursive Korobov filter with points from the Fibonacci lattice at n = 5 points.

For the special case of the Fibonacci lattice at n = 5 points the recursive procedure has been used for adaptive sampling in computer graphics [SD01]. Starting with the lattice Z2 the next√refinement level was found by rotating Z2 by arctan(1/2) and scaling it by 1/ 5 as indicated in Fig. 7c). The resulting lattice again is a rectangular lattice and the procedure recursively can be continued. Thus the construction was completely unrelated to rank-1 lattices. Taking it to the Maximum In [Kel04, Fig. 8] we showed results from a distribution ray tracer that used randomly shifted rank-1 lattices with maximized minimum distance. Trajectory splitting was realized using rank-1 lattices with maximized minimum distance, too.

238

A. Keller

Fig. 8. Images from movies that have been rendered with the software mental ray using strictly deterministic quasi-Monte Carlo integro-approximation: Top row : Fight Club, 20th Century Fox International, 1999. 3D Animation and Visual Efc fects by BUF Compagnie, Paris. Bottom row : The Cell, New Line Cinema, 2000, Director: Tarsem. Visual Effects by BUF Compagnie, Paris.

Per pixel one random vector was used to shift the lattice points in order to obtain an unbiased estimator and to decorrelate neighboring pixels. The resulting images exposed minimal noise and while aliasing artifacts are pushed to noise. Compared to previous sampling methods the convergence was superior, which is due to the high uniformity of the lattice points. As lattice points are maximally correlated this is a good example for correlated sampling in computer graphics. In this context quasi-Monte Carlo integro-approximation by lattice points can be considered as Korobov filtering.

4 Quasi-Monte Carlo in Movie Industry While applications of quasi-Monte Carlo integration in finance attracted a lot of attention instantly, developments in computer graphics were not that spectacular. Today, however, about half of the rendered images in movie industry are synthesized using strictly deterministic quasi-Monte Carlo integroapproximation (for popular examples see Fig. 8). In 2003 these techniques [Kel03] even have been awarded a Technical Achievement Award (Oscar) by the American Academy of Motion Picture Arts and Sciences. In contrast to academia, graphics hardware and software industry early recognized the benefits of quasi-Monte Carlo methods.

Myths of Computer Graphics

239

Deterministic quasi-Monte Carlo methods have the advantage that they can be parallelized without having to consider correlation as encountered when using pseudo-random number generators. By their deterministic nature the results are exactly reproducible even in a parallel computing environment. Compared to classical algorithms of computer graphics the algorithms are smaller and more efficient, since high uniformity is intrinsic to the sample points. A good example is trajectory splitting by rank-1 lattice [Kel04] that have maximized minimum distance. In computer graphics it is known that maximizing the minimum distance of point sets increases convergence speed. However, algorithms to create such points, like e.g. Lloyd’s relaxation method, are expensive. With quasi-Monte Carlo methods selected by maximized minimum distance, efficient algorithms are available and savings up to 30% of the computation time for images of the same quality as compared to random sampling methods can be observed. In the setting of computer graphics quasi-Monte Carlo methods benefit from the piecewise continuity of the integrands in L2b . Around the lines of discontinuity the methods are observed to perform no worse than random sampling, while in the regions of continuity the better uniformity guarantees for faster convergence. The observed convergence rate is between O(n−1 ) and O(n−1/2 ). It depends on the ratio of the number of sets in the partition induced by (M, µ)-uniform points and the number of these sets containing discontinuities. Since with increasing number of dimensions the integrands tend to contain more discontinuities the largest improvements are observed for smaller dimensions.

5 Conclusion Since photorealistic image generation comprises the simulation of light transport by computing functionals of the solution of a Fredholm integral equation of the second kind, the quasi-Monte Carlo methods developed for computer graphics apply to other problems of transport theory as well. The concept of maximized minimum distance as used in computer graphics nicely fits the concept of (M, µ)-uniformity as used in quasi-Monte Carlo theory. Rank-1 lattices selected by maximized minimum distance ideally fit both requirements and yield superior results in computer graphics. Open Issues Although physically correct illumination is state of the art in movie production, there are still unresolved issues. • There is no algorithm to robustly handle aliasing as caused by situations similar to the checker board problem in Fig. 3. • Large numbers of light sources, strong indirect caustics, and difficult lighting situations like rooms lit through a door slit cannot be handled sufficiently efficiently.

240

A. Keller

• Even if these issues can be resolved, the mathematical problem can be made arbitrarily difficult by considering motion blur from fast motion and wavelength dependent effects. Only very little successful research has been carried out in this direction. • In order to compare rendering algorithms images have to be compared. The squared distance of two images is sufficient to verify convergence, however, the L2 -norm is not very sensitive to noise. Using the H 1 -Sobolevnorm detects the noise, however, edges also contribute to the norm. In fact there is no accepted visual error norm, which simulates how the human visual system distinguishes images.

Acknowledgments The author is very thankful to Stefan Heinrich, Harald Niederreiter, Peter Schr¨ oder, Markus Gross, Rolf Herken, Fred Hickernell, Yan Chen, Pierre L’Ecuyer, Jerome Spanier, and especially the information based complexity community for their continuous support over many years. Parts of this research were supported by the Stiftung Rheinland-Pfalz f¨ ur Innovation.

References [BBD+ 00] C. Betrisey, J. Blinn, B. Dresevic, B. Hill, G. Hitchcock, B. Keely, D. Mitchell, J. Platt, and T. Whitted, Displaced Filtering for Patterned Displays, Proc. Society for Information Display Symposium (2000), 296– 299. [CCC87] R. Cook, L. Carpenter, and E. Catmull, The REYES Image Rendering Architecture, Computer Graphics (SIGGRAPH ’87 Proceedings), July 1987, pp. 95–102. [CP76] R. Cranley and T. Patterson, Randomization of number theoretic methods for multiple integration, SIAM Journal on Numerical Analysis 13 (1976), 904–914. [CPC84] R. Cook, T. Porter, and L. Carpenter, Distributed Ray Tracing, Computer Graphics (SIGGRAPH ’84 Conference Proceedings), 1984, pp. 137–145. [DBMS02] K. Dimitriev, S. Brabec, K. Myszkowski, and H.-P. Seidel, Interactive Global Illumination using Selective Photon Tracing, Rendering Techniques 2002 (Proc. 13th Eurographics Workshop on Rendering) (P. Debevec and S. Gibson, eds.), Springer, 2002, pp. 25–36. [Fau92] H. Faure, Good Permutations for Extreme Discrepancy, J. Number Theory 42 (1992), 47–56. [FC62] A. Frolov and N. Chentsov, On the calculation of certain integrals dependent on a parameter by the Monte Carlo method, Zh. Vychisl. Mat. Fiz. 2 (1962), no. 4, 714 – 717, (in Russian).

Myths of Computer Graphics [FK02]

[Gla95] [HA90]

[HK94a]

[HK94b]

[Hla71] [HW64] [Jen01] [Kel95]

[Kel96a] [Kel96b]

[Kel97] [Kel98a] [Kel98b]

[Kel01a] [Kel01b] [Kel03]

[Kel04] [KH01]

241

I. Friedel and A. Keller, Fast Generation of Randomized LowDiscrepancy Point Sets, Monte Carlo and Quasi-Monte Carlo Methods 2000 (H. Niederreiter, K. Fang, and F. Hickernell, eds.), Springer, 2002, pp. 257–273. A. Glassner, Principles of Digital Image Synthesis, Morgan Kaufmann, 1995. P. Haeberli and K. Akeley, The Accumulation Buffer: Hardware Support for High-Quality Rendering, Computer Graphics (SIGGRAPH 90 Conference Proceedings), 1990, pp. 309–318. S. Heinrich and A. Keller, Quasi-Monte Carlo Methods in Computer Graphics, Part I: The QMC-Buffer, Interner Bericht 242/94, University of Kaiserslautern, 1994. , Quasi-Monte Carlo Methods in Computer Graphics, Part II: The Radiance Equation, Interner Bericht 243/94, University of Kaiserslautern, 1994. E. Hlawka, Discrepancy and Riemann Integration, Studies in Pure Mathematics (L. Mirsky, ed.), Academic Press, New York, 1971, pp. 121–129. J. Halton and G. Weller, Algorithm 247: Radical-inverse quasi-random point sequence, Comm. ACM 7 (1964), no. 12, 701–702. H. Jensen, Realistic Image Synthesis Using Photon Mapping, AK Peters, 2001. A. Keller, A Quasi-Monte Carlo Algorithm for the Global Illumination Problem in the Radiosity Setting, Monte Carlo and Quasi-Monte Carlo Methods in Scientific Computing (H. Niederreiter and P. Shiue, eds.), Lecture Notes in Statistics, vol. 106, Springer, 1995, pp. 239–251. , Quasi-Monte Carlo Methods in Computer Graphics: The Global Illumination Problem, Lectures in App. Math. 32 (1996), 455–469. A. Keller, Quasi-Monte Carlo Radiosity, Rendering Techniques ’96 (Proc. 7th Eurographics Workshop on Rendering) (X. Pueyo and P. Schr¨ oder, eds.), Springer, 1996, pp. 101–110. , Instant Radiosity, SIGGRAPH 97 Conference Proceedings, Annual Conference Series, 1997, pp. 49–56. , Quasi-Monte Carlo Methods for Photorealistic Image Synthesis, Ph.D. thesis, Shaker Verlag Aachen, 1998. , The Quasi-Random Walk, Monte Carlo and Quasi-Monte Carlo Methods in Scientific Computing 1996 (H. Niederreiter, P. Hellekalek, G. Larcher, and P. Zinterhof, eds.), vol. 127, Springer, 1998, pp. 277–291. , Hierarchical Monte Carlo Image Synthesis, Mathematics and Computers in Simulation 55 (2001), no. 1-3, 79–92. , Trajectory Splitting by Restricted Replication, Interner Bericht 316/01, Universit¨ at Kaiserslautern, 2001. , Strictly Deterministic Sampling Methods in Computer Graphics, SIGGRAPH 2003 Course Notes, Course #44: Monte Carlo Ray Tracing (2003). , Stratification by Rank-1 Lattices, Monte Carlo and Quasi-Monte Carlo Methods 2002 (H. Niederreiter, ed.), Springer, 2004, pp. 299–313. A. Keller and W. Heidrich, Interleaved Sampling, Rendering Techniques 2001 (Proc. 12th Eurographics Workshop on Rendering) (K. Myszkowski and S. Gortler, eds.), Springer, 2001, pp. 269–276.

242 [KK89]

A. Keller

J. Kajiya and T. Kay, Rendering Fur with Three Dimensional Textures, Computer Graphics (Proceedings of SIGGRAPH 89) 23 (1989), no. 3, 271–280. [KK02a] T. Kollig and A. Keller, Efficient Bidirectional Path Tracing by Randomized Quasi-Monte Carlo Integration, Monte Carlo and Quasi-Monte Carlo Methods 2000 (H. Niederreiter, K. Fang, and F. Hickernell, eds.), Springer, 2002, pp. 290–305. , Efficient Multidimensional Sampling, Computer Graphics Fo[KK02b] rum 21 (2002), no. 3, 557–563. , Efficient Illumination by High Dynamic Range Images, Render[KK03] ing Techniques 2003 (Proc. 14th Eurographics Symposium on Rendering) (P. Christensen and D. Cohen-Or, eds.), Springer, 2003, pp. 45–51. [Lem00] C. Lemieux, L’utilisation de r` egles de r´eseau en simulation comme technique de r´eduction de la variance, Ph.d. thesis, Universit´e de Montr´eal, d´epartement d’informatique et de recherche op´erationnelle, May 2000. [LP01] G. Larcher and F. Pillichshammer, Walsh Series Analysis of the L2 Discrepancy of Symmetrisized Point Sets, Monatsh. Math. 132 (2001), 1–18. [Mat98] J. Matouˇsek, On the L2 -discrepancy for anchored boxes, J. of Complexity 14 (1998), no. 4, 527–556. [Mit92] D. Mitchell, Ray Tracing and Irregularities of Distribution, Proc. 3rd Eurographics Workshop on Rendering (Bristol, UK), 1992, pp. 61–69. [Nie92a] H. Niederreiter, Quasirandom Sampling in Computer Graphics, Proc. 3rd Internat. Seminar on Digital Image Processing in Medicine, Remote Sensing and Visualization of Information (Riga, Latvia), 1992, pp. 29– 34. [Nie92b] H. Niederreiter, Random Number Generation and Quasi-Monte Carlo Methods, SIAM, Philadelphia, 1992. [Nie03] H. Niederreiter, Error bounds for quasi-Monte Carlo integration with uniform point sets, J. Comput. Appl. Math. 150 (2003), 283–292. [Owe95] A. Owen, Randomly Permuted (t, m, s)-Nets and (t, s)-Sequences, Monte Carlo and Quasi-Monte Carlo Methods in Scientific Computing (H. Niederreiter and P. Shiue, eds.), Lecture Notes in Statistics, vol. 106, Springer, 1995, pp. 299–315. [PH04] M. Pharr and G. Humphreys, Physically Based Rendering, Morgan Kaufmann, 2004. [PKK00] M. Pauly, T. Kollig, and A. Keller, Metropolis Light Transport for Participating Media, Rendering Techniques 2000 (Proc. 11th Eurographics Workshop on Rendering) (B. P´eroche and H. Rushmeier, eds.), Springer, 2000, pp. 11–22. [PTVF92] H. Press, S. Teukolsky, T. Vetterling, and B. Flannery, Numerical Recipes in C, Cambridge University Press, 1992. [SD01] M. Stamminger and G. Drettakis, Interactive Sampling and Rendering for Complex and Procedural Geometry, Rendering Techniques 2001 (Proc. 12th Eurographics Workshop on Rendering) (K. Myszkowski and S. Gortler, eds.), Springer, 2001, pp. 151–162. [Shi91] P. Shirley, Discrepancy as a Quality Measure for Sampling Distributions, Eurographics ’91 (Amsterdam, North-Holland), Elsevier Science Publishers, 1991, pp. 183–194.

Myths of Computer Graphics [SJ94] [Vea97] [Yel83] [Zar70]

243

I. Sloan and S. Joe, Lattice Methods for Multiple Integration, Clarendon Press, Oxford, 1994. E. Veach, Robust Monte Carlo Methods for Light Transport Simulation, Ph.D. thesis, Stanford University, 1997. J. Yellot, Spectral Consequences of Photoreceptor Sampling in the Rhesus Retina, Science 221 (1983), 382–385. S.K. Zaremba, La discr´ epance isotrope et l’int´egration num´erique, Ann. Mat. Pura Appl. 87 (1970), 125–136.

Illumination in the Presence of Weak Singularities Thomas Kollig1 and Alexander Keller2 1

2

Dept. of Computer Science, Kaiserslautern University of Technology, 67653 Kaiserslautern, Germany [email protected] Dept. of Computer Science, University of Ulm, 89069 Ulm, Germany [email protected]

Summary. Approximating illumination by point light sources, as done in many professional applications, allows for efficient algorithms, but suffers from the problem of the weak singularity: Besides avoiding numerical exceptions caused by the division by the squared distance between the point light source and the point to be illuminated, the estimator should be unbiased and of finite variance. We first illustrate that the common practice of clipping weak singularities to a reasonable value yields clearly visible bias. Then we present a new global illumination algorithm that is unbiased and as simple as a path tracer, but elegantly avoids the problem of the weak singularity. In order to demonstrate its performance, the algorithm has been integrated in an interactive global illumination system.

1 Introduction Simulating light transport in a physically correct way has become a mainstream feature in movie productions and interactive rendering systems. On the one hand many approximations are used to make the algorithms simpler, faster, and more numerically robust. On the other hand unbiased approaches like e.g. bidirectional path tracing or the Metropolis light transport algorithm are too complicated for use in professional productions and not sufficiently efficient. Based on the popular approximation instant radiosity [Kel97], we present an unbiased, robust, and very simple global illumination algorithm that is used for production as well as interactive rendering. The new algorithm is easily implemented in any ray tracing system and exposes many advantages over previous techniques, as we will illustrate in the sequel.

246

T. Kollig and A. Keller

2 Avoiding Bias Caused by Bounding Before we introduce our new global illumination algorithm, we need to explain an important observation for the example problem of direct illumination by an area light source. The direct illumination is the radiance  Lr (x, ωr ) = fr (ω, x, ωr )G(x, y)V (x, y)Le (y, −ω)dy A

reflected in x into direction ωr , which is the integral over the surface A of the light source. Its radiance Le from y towards x, i.e. into direction −ω, is attenuated by the geometry term G and the bidirectional reflectance distribution function fr , which represents the physical surface properties. The visibility is accounted for by V (x, y), which is 1 if x and y are mutually visible and zero otherwise. It is common practice to use a bounded geometry term G (x, y) := min{G(x, y), b} for some bound b > 0 instead of the correct expression G(x, y) :=

cos+ θx · cos+ θy , x − y22

where the positive cosine cos+ θx is the scalar product between the unit direction of y − x and the surface normal in x, which is set to zero, if the cosine is less than zero (analogous for cos+ θy ). The obvious reason for uniformly bounding G by some b > 0 is to avoid infinite variance from the weak singularity, which can be caused by an arbitrarily small Euclidean distance x − y2 , i.e. when the point x to be lit and the sampled point y on the light source are very close. 2.1 Consequences of Bounding the Integrand Almost any rendering software approximates the direct illumination Lr (x, ωr ) by  fr (ω, x, ωr )G (x, y)V (x, y)Le (y, −ω)dy Lr (x, ωr ) = A

≈

N −1 |A| ( fr (ωi , x, ωr )G (x, yi )V (x, yi )Le (yi , −ωi ) N i=0

(1)

using Monte Carlo integration or a variant of it. Here yi are uniformly distributed samples on the area A of the light source and ωi is the unit vector pointing from x to yi .

Illumination in the Presence of Weak Singularities

247

The consequence of uniformly bounding the integrand is an exponential decay of the error probability. In the case of pure Monte Carlo integration Hoeffding’s inequality yields the probability    N −1 2 1 1 ( ≤ 2e− 4τ 2 N  f (x)dx − f (xi ) ≥ Prob [0,1)s N i=0 of an integration error more than an arbitrary threshold > 0 if it is possible to uniformly bound |f (x) − I| < τ for almost all x ∈ [0, 1)s , where I is the integral of f . Since most Monte Carlo rendering algorithms bound the samples before averaging, the previous formula explains the observed fast convergence. By the fast convergence visible artifacts rapidly disappear and the images look nice. However, the estimator is biased, i.e. does not converge to the desired value Lr (x, ωr ), and important visible contributions of the illumination are missing, as can be seen from the differences between Figs. 1a) and e). Obviously the bias is especially high in the vicinity of concave geometry such as the curtains and the fine geometry of the chairs. In fact the bias introduced by bounding the geometry term cannot be ignored.

a) Classic biased approximation

b) Eye path length 2

e) Unbiased solution (see e.g. curtains)

c) Eye path length 3

d) Eye path length 4

Fig. 1. Looking at the curtains, the bias between the true global illumination in (e) and the classic approximation in (a) is clearly visible as a difference in brightness. The images (b) to (d) show the missing contributions according to the eye path length. For display the images (b) to (d) have been amplified by a factor of 3, 9, and 27, repectively. The bias of the classic approximation is clearly located in regions of concave geometry.

248

T. Kollig and A. Keller

2.2 Unbiased Robust Estimator It is favorable to preserve the fast convergence of the estimator (1), since it actually contributes most of the illumination and exposes low variance. The so-called bias, i.e. the difference between the desired and the computed integral is Lr (x, ωr ) − Lr (x, ωr )  = Le (y, −ω)fr (ω, x, ωr )V (x, y) max{G(x, y) − b, 0}dy A max{G(x, y) − b, 0} = G(x, y)dy Le (y, −ω)fr (ω, x, ωr )V (x, y) G(x, y) A  max{G(x, h(x, ω)) − b, 0} = fr (ω, x, ωr ) cos+ θx dω. (2) Le (h(x, ω), −ω) G(x, h(x, ω)) 2 S Changing the domain of integration to the unit sphere S 2 requires the ray tracing function h(x, ω), which returns the first surface point hit when shooting a ray from x into direction ω. In order to obtain an unbiased estimate of the direct illumination Lr (x, ωr ), we use the estimator (1) and add an estimate of the above equation (2). Applying importance sampling according to the density fr (ω, x, ωr ) · cos+ θx , the integrand becomes bounded [Shr66], too. Although the method seems simple, it never before has been used to single out the weak singularities contained in the geometry term G. There are several advantages to this approach: Bounding the integrand in (1) does not add new discontinuities and consequently variance is not increased. Since both integrands are bounded, the variance remains finite and the estimate is numerically robust. In the context of parametric integration, Heinrich [Hei00] proposed an optimal method for the Monte Carlo approximation of weakly singular operators: For smooth function classes his algorithm used a stratification idea to separate the weak singularity. This is related to our approach, however, introduces more discontinuities to the integrand as compared to bounding. 2.3 Choice of the Bound Obviously, the radiance Lr is estimated in an unbiased way for any choice of 0 ≤ b < ∞. However, most renderers implicitly are using a fixed bound without compensating the bias (2). Choosing b ≡ b(ω, x, ωr ) =

c fr (ω, x, ωr )

(3)

allows one to use the very efficient estimator (1) as long as G(x, y)fr (ω, x, ωr ) ≤ c. Using importance sampling as indicated in the previous section, the transport kernel in (2) then is bounded by 1.

Illumination in the Presence of Weak Singularities

249

By c we can adjust the efficiency, i.e. how much of the estimate is obtained by sampling the area of the light source and how much is contributed by importance sampling of the solid angle. Choosing c independent of the scene geometry, the contributions of the estimators for (1) and (2) depend on the scale of the geometry, which is hidden in the geometry term G. 1 no sample from either (1) or (2) can be larger than the source For c = |A| radiance Le , i.e. the radiance is never amplified but only attenuated. Consequently the contributions of the integration over the light source as well as the solid angle can contribute about the same noise level at maximum. In addition the bound becomes independent of the scale, since both G and A contain the scale of the scene. Since usually the radiance is vector valued, i.e. it comprises components for red, green, and blue, in fact the maximum norm fr (ω, x, ωr )∞ should be used in the denominator of b.

3 The New Global Illumination Algorithm Using the observation from the previous section, it is simple to construct a global illumination algorithm that is unbiased and numerically robust. We just need to compensate for the bias of popular approaches like instant radiosity [Kel97] or successor approaches to interactive global illumination [WKB+ 02, BWS03]. The procedure to compute a local solution from the radiance Fredholm integral equation of the second kind  L(x, ωr ) = Le (x, ωr ) + L(h(x, ω), −ω)fr (ω, x, ωr ) cos+ θx dω S2

is illustrated in Fig. 2: 1. Generation of point light sources: Identical to the preprocessing of the instant radiosity algorithm [Kel97] or a very sparse global photon map [2], −1 a set (yj , Lj )M j=0 of M point light sources is created. This corresponds to tracing paths starting at the lights and storing all the points yj ∈ R3 of incidence with their power Lj . 2. Shading: Similar to a path tracer, an eye path is started from the lens incident in point x0 from direction ω0 . Starting with i = 0, we sum up three contributions for the current point xi until the eye path is terminated: a) Light sources that are hit contribute their emission Le (xi , −ωi ). b) Illumination: The contribution of the j-th point light source is fr (ωxi ,yj , xi , ωi )G (xi , yj )V (xi , yj )Lj ,

(4)

where the direction ωxi ,yj points from xi to yj and ωi is the direction from where xi has been hit. Note that G is the bounded version of the geometry term G.

250

T. Kollig and A. Keller G(xi , yj ) < b (yj , Lj ) ωi xi G(xi , yj ) ≥ b

1.) Generation of point light sources G(xi+1 , xi+2 ) < b

2.b) Robust shadowing

ωi xi G(xi , xi+1 ) ≥ b

2.c) Bias compensation

3.) Average eye path length

Fig. 2. Principal steps of the unbiased robust global illumination algorithm: In −1 the first step a set (yj , Lj )M j=0 of point light sources is generated. Hitting xi from direction ωi in the second step, the highlighted areas show the domain, where the geometry term G(xi , ·) is below the bound b = 0.5. In step 2.b shadow rays towards the point lights are traced. This is robust, because numerical exceptions by the inverse squared distance in G cannot occur due to bounding. In order to be unbiased, step 2.c continues the eye path from xi by scattering a ray. While the ray hits the domain, we continue with step 2.a, otherwise the eye path is terminated. Image 3.) shows the average eye path length as a gray image, where the maximum path length considered was 5. Darker areas in the image indicate longer eye paths. The resulting image clearly resembles images obtained by ambient occlusion, i.e. concave corners are darker.

c) Bias compensation: Because the weak singularity was avoided by bounding the integrand, we have to account for the bias. Therefore we trace a ray into a random direction yielding the next vertex xi+1 along the eye path on the scene surface S. If G(xi , xi+1 ) < b then this contribution has already been accounted for in the previous step and the eye path is terminated. Otherwise i is incremented and we continue with step 2.a, whose result has to be attenuated by the product of

Illumination in the Presence of Weak Singularities

the bidirectional reflectance distribution function fr and because of the derivation in equation (2).

251

G(xi ,xi+1 )−b G(xi ,xi+1 )

The evolution of an image with the eye path length can be seen in Fig. 1, where Figs. 1a) – d) show the contribution of the eye path length i = 1, . . . , 4 and Fig. 1e) the sum of the contributions computed by our algorithm. Note that the contributions have been amplified by 3(i−1) for display, i.e. the bias decays exponentially over the eye path length in our new algorithm. 3.1 Numerical Comparison to Bidirectional Path Tracing Our new technique can be formulated as a heuristic for multiple importance sampling [VG95,Vea97] and consequently belongs to the class of bidirectional path tracing algorithms. Although this larger mathematical framework is not required for the derivation, it is interesting to compare the efficiency of our method to the classical techniques. The New Algorithm as a Heuristic for Bidirectional Path Tracing Using the notions as defined in [KK02a], our algorithm computes a path integral ∞  ∞ N −1 −1 ( ( 1 (( f (¯ x,i,j ) f (¯ x)dµ(¯ x) ≈ w,i (¯ x,i,j ) N j=0 i=0 p,i (¯ x,i,j ) P =1

=1

by multiple importance sampling. P is the path space containing all transport paths x ¯ = x0 x1 · · · x−1 of length  and f is the measurement contribution function, which contributes the radiance of the path x¯. For a fixed path length  there are  techniques to generate it by a corresponding probability density ¯,i,j has been generated by sampling the probability function p,i . The path x density p,i . With these definitions our new algorithm results in the weights  ' b w,−1 (¯ x) = min 1, G(x−2 , x−1 )  ' b w,−2 (¯ x) = min 1, x)) · (1 − w,−1 (¯ G(x−3 , x−2 ) .. .  ' b x) = min 1, x)) · . . . · (1 − w,−1 (¯ x)) w,1 (¯ · (1 − w,2 (¯ G(x0 , x1 ) w,0 (¯ x) = (1 − w,1 (¯ x)) · . . . · (1 − w,−1 (¯ x)) , (5) 1−1 x) = 1 for any path x ¯ of length  as required which obviously fulfill i=0 w,i (¯ for an unbiased estimator [KK02a].

252

T. Kollig and A. Keller

Numerical Evidence for the Increased Efficiency Since our method is unbiased, for a sufficient number of samples we obtain images without artifacts. At too low sampling rates noise, blossoming, and sharp shadow boundaries can be visible. One might think that this noise is caused by step 2.a of the algorithm, which however rarely happens. It is more likely that the point lighting after scattering in step 2.c contributes noise. Sharp shadow boundaries become visible, if one set of point light sources is used for the whole image in step 2.b. If furthermore point light sources are located in concave geometry, it can happen that the close-by geometry is brightly lit, which we call blossoming. Using a different set of point light sources for adjacent pixels (uncorrelated sampling, e.g. interleaved sampling [Kel03,KH01]) the latter two artifacts are turned into noise. All these artifacts, however, are bounded as proved in Sect. 2.3 and thus rapidly average out during Monte Carlo integration. The choice of c balances the artifacts at low sampling rates and thus controls the efficiency of the algorithm: The larger c, the more artifacts are caused by the point light sources, the smaller c the more noise from scattering becomes noticeable. Known heuristics from multiple importance sampling, as e.g. the power heuristic [VG95, Vea97], are ratios of probability densities and therefore independent of the scale of a given scene. For area light sources we obtain this 1 as previously mentioned. For global illumination, which property for c = |A| includes indirect illumination, however, this choice no longer is obvious. Therefore we numerically analyzed the efficiency, i.e. the reciprocal of the variance multiplied by the rendering time. In Fig. 3 the relative running time of the balance, power, and maximum heuristic is compared to our new algorithm at identical image quality. We used a bidirectional path tracer that

1 0.8 0.6

Conference Room Scene relative cost

relative cost

Office Room Scene 1.2

0

0.05 0.1 0.15 0.2 0.25 0.3 bounding parameter c balance heuristic power heuristic

1.2 1 0.8 0.6

0

0.05 0.1 0.15 0.2 0.25 0.3 bounding parameter c maximum heuristic new algorithm

Fig. 3. Comparison of the relative rendering time of the power heuristic with β = 2, the balance heuristic, the maximum heuristic, and our new algorithm at identical image quality. We used the more efficient interleaved sampling, i.e. the method of dependent tests. For a wide choice of the bounding parameter c our new algorithm reliably outperforms the classical techniques saving up to 20% of the rendering time.

Illumination in the Presence of Weak Singularities

253

has been improved by dramatically reducing the number of light paths by interleaved sampling [KH01], which decreases the realization cost by the same constant amount of time for all heuristics. Although our algorithm may not have optimal variance, it is more efficient than other heuristics for a broad range of the parameter c. Efficient Implementation Contrary to classical bidirectional path tracing techniques [VG95, Vea97], the algorithm from section 3 is easily implemented in the standard shader concept of industrial ray tracing software. A complicated implementation of the balance, power, or maximum heuristic is not required, because our algorithm just is a double for-loop over the eye path length and the number of point lights. This has been a big advantage for the acceptance in a production environment. The observed increased efficiency has several reasons: Cheap rays: Equating the geometry term G and the bound b allows one to bound the maximum length 0 cos+ θx r(x) ≤ b of the eye rays, where we used cos+ θyi ≤ 1. This distance often is much shorter than the obvious bound determined by the bounding box of the scene. Consequently the amount of geometry loaded into the caches remains much smaller and less voxels of the acceleration data structures have to be traversed. Short eye paths: Compared to previous bidirectional path tracing heuristics, the eye path length of our new method is shorter on the average. Thus less rays have to be traced and shaded as can be seen in Figs. 1 and 2. Less shadow rays: The short average eye path length directly results in a moderate number of shadow rays to be shot and consequently a higher data locality. One might argue that the maximum heuristic in bidirectional path tracing also allows one to avoid the shooting of shadow rays. This is true, however, most of the possible path weights have to be computed in order to determine, whether or not to omit a shadow ray. The second disadvantage of the maximum heuristic is that it introduces discontinuities in the integrands around the weak singularity, which in our approach we explicitly avoided in order to obtain a lower noise level. Intrinsic cache coherence: Only in the vicinity of concave corners the eye path length slightly increases as illustrated in Fig. 2. Then the ray length is short, which implies that most of the geometry already is in the processor cache. This corresponds to the idea of local illumination environments [FBG02], however, our method is unbiased and implicit, i.e. does not require an extra implementation for cache locality. Working with point

254

T. Kollig and A. Keller

light sources, the shadow rays can be traced as bundles originating from one point [WBWS01]. Because shadow rays only access the scene geometry and do not require shader calls, less cache memory is required for shader data. Compared to bidirectional path tracing, the cost of the light path generation remains the same for our new algorithm. However, since eye rays are cheaper, eye paths are shorter, and less shadow rays have to be shot, the new algorithm is more efficient and in addition it benefits much more from speedups in tracing rays. Because cache requirements are minimal, the efficient use of the processor cache is intrinsic to our algorithm and does not require extra care while coding. 3.2 Extensions Our approach is a very general mathematical concept and unifies many seemingly isolated techniques in a simple way: Russian roulette, ambient occlusion and local illumination environments, and final gathering and secondary final gathering are all intrinsic. In order to further increase efficiency, the algorithm can easily be complemented by the following, orthogonal techniques: Efficient multidimensional sampling: For the sake of clarity, we based all explanations on arguments using pure random sampling. It is straightforward to improve the efficiency by quasi-Monte Carlo and randomized quasi-Monte Carlo sampling methods [Kel02, KK02a, KK02b]. The big advantage of our approach is that additional discontinuities, which could have harmed the performance of stratified sampling patterns, are explicitly avoided. Shadow computation: The techniques of Ward [War91], Keller [Kel98], and Wald [WBS03] can be used for reducing the number of shadow rays. The shadows also could be computed using various algorithms on graphics hardware. Due to the rapidly decaying contribution of longer eye paths, it is also possible to reduce the number of point light sources used. Discontinuity buffer: It is straightforward to apply the discontinuity buffer [Kel98] for faster but biased anti-aliasing. Non-blocking parallelization: Our method is a Monte Carlo algorithm and as such trivial to parallelize. By the high coherency our algorithm in addition benefits from realtime ray tracing architectures as introduced in [WKB+ 02, BWS03] and improves their image quality. Finally, it is known that some caustic paths cannot be captured efficiently by any path tracing algorithm [KK02a], however, these are easily complemented by a caustic photon map [Jen01]. As shown in several papers [WKB+ 02, BWS03], approximations to global illumination can be computed at interactive frame rates. In our implementation the unbiased solution requires roughly up to 30% more computing power as compared to the biased version.

Illumination in the Presence of Weak Singularities

255

3.3 Interpretation of the Bias Compensation Step The bias compensation step 2.c of the algorithm could be considered as a secondary final gathering [Chr99] as well as Russian roulette for an unbiased path termination. A third interpretation is available in the context of ambient occlusion techniques [IKSZ03, Neu03]: In Fig. 2 the average path length of our eye paths is displayed as a gray image, where pixels are darker as the eye path becomes longer. Because paths are terminated whenever the geometry term is below the bound b, this image in fact looks like an image computed by the ambient occlusion technique. In Sect. 2 we thus provided the missing mathematical facts for why ambient occlusion works so fine and is justified. Moreover, by our technique, we do not only scan the hemisphere around one point, but the whole vicinity that can be reached by short paths. This completely removes the problem of blurry patterns in concave corners as it may occur with final gathering [Chr99].

4 Conclusion We presented a new mathematical framework for robustly computing integrals of integrands containing weak singularities without any numerical exceptions. Based on this concept we derived a robust algorithm for computing local solutions of a Fredholm integral equation of the second kind. In the context of computer graphics our approach is more general than ambient occlusion and secondary final gathering. The implementation exposes the simplicity of a path tracer and the resulting images do not show the artifacts of current stateof-the-art rendering techniques, since the algorithm is unbiased. Compared to other unbiased techniques like classical bidirectional path tracing, our method is more efficient and easily implemented in professional rendering software systems. The method of how the integrands are bounded allows one to efficiently apply hierarchical Monte Carlo methods [Hei00,5]. In future research, we also will investigate how to determine the constant c other than by numerical experiments. Finally the combination of occlusion maps and shadow buffering by our new method can yield more realistic hardware rendering algorithms.

Acknowledgements The first author has been funded by the Stiftung Rheinland- Pfalz f¨ ur Innovation. The paper has been dedicated to Anneliese Oeder. The authors thank Regina Hong for proofreading.

256

T. Kollig and A. Keller

References [BWS03]

[Chr99] [FBG02]

[Hei00]

[IKSZ03]

[Jen01] [Kel97] [Kel98] [Kel01] [Kel02]

[Kel03]

[KH01]

[KK02a]

[KK02b] [Neu03]

[Shr66] [Vea97] [VG95]

C. Benthin, I. Wald, and P. Slusallek, A Scalable Approach to Interactive Global Illumination, Computer Graphics Forum 22 (2003), no. 3, 621– 629. P. Christensen, Faster Photon Map Global Illumination, Journal of Graphics Tools 4 (1999), no. 3, 1–10. S. Fernandez, K. Bala, and D. Greenberg, Local Illumination Environments for Direct Lighting Acceleration, Rendering Techniques 2002 (Proc. 13th Eurographics Workshop on Rendering) (P. Debevec and S. Gibson, eds.), 2002, pp. 7–13. S. Heinrich, Monte Carlo Approximation of Weakly Singular Operators, Talk at the Dagstuhl Seminar 00391 on Algorithms and Complexity for Continuous Problems, Sept. 2000. A. Iones, A. Krupkin, M. Sbert, and S. Zhukov, Fast, Realistic Lighting for Video Games, IEEE Computer Graphics and Applications 23 (2003), no. 3, 54–64. H. Jensen, Realistic Image Synthesis Using Photon Mapping, AK Peters, 2001. A. Keller, Instant Radiosity, SIGGRAPH 97 Conference Proceedings, Annual Conference Series, 1997, pp. 49–56. , Quasi-Monte Carlo Methods for Photorealistic Image Synthesis, Ph.D. thesis, Shaker Verlag Aachen, 1998. , Hierarchical Monte Carlo Image Synthesis, Mathematics and Computers in Simulation 55 (2001), no. 1-3, 79–92. A. Keller, Beyond Monte Carlo – Course Material, Interner Bericht 320/02, University of Kaiserslautern, 2002, Lecture at the Caltech, July 30th – August 3rd, 2001. A. Keller, Strictly Deterministic Sampling Methods in Computer Graphics, SIGGRAPH 2003 Course Notes, Course #44: Monte Carlo Ray Tracing (2003). A. Keller and W. Heidrich, Interleaved Sampling, Rendering Techniques 2001 (Proc. 12th Eurographics Workshop on Rendering) (K. Myszkowski and S. Gortler, eds.), Springer, 2001, pp. 269–276. T. Kollig and A. Keller, Efficient Bidirectional Path Tracing by Randomized Quasi-Monte Carlo Integration, Monte Carlo and Quasi-Monte Carlo Methods 2000 (H. Niederreiter, K. Fang, and F. Hickernell, eds.), Springer, 2002, pp. 290–305. , Efficient Multidimensional Sampling, Computer Graphics Forum 21 (2002), no. 3, 557–563. I. Neulander, Image-Based Diffuse Lighting using Visibility Maps, Proceedings of the SIGGRAPH 2003 Conference on Sketches & Applications, 2003, pp. 1–1. Y. Shreider, The Monte Carlo Method, Pergamon Press, 1966. E. Veach, Robust Monte Carlo Methods for Light Transport Simulation, Ph.D. thesis, Stanford University, 1997. E. Veach and L. Guibas, Optimally Combining Sampling Techniques for Monte Carlo Rendering, SIGGRAPH 95 Conference Proceedings, Annual Conference Series, 1995, pp. 419–428.

Illumination in the Presence of Weak Singularities [War91]

257

G. Ward, Adaptive Shadow Testing for Ray Tracing, 2nd Eurographics Workshop on Rendering (Barcelona, Spain), 1991. [WBS03] I. Wald, C. Benthin, and P. Slusallek, Interactive Global Illumination in Complex and Highly Occluded Environments, Rendering Techniques 2003 (Proc. 14th Eurographics Workshop on Rendering) (P. Christensen and D. Cohen-Or, eds.), 2003, pp. 74–81. [WBWS01] I. Wald, C. Benthin, M. Wagner, and P. Slusallek, Interactive Rendering with Coherent Ray Tracing, Computer Graphics Forum 20 (2001), no. 3, 153–164. [WKB+ 02] I. Wald, T. Kollig, C. Benthin, A. Keller, and P. Slusallek, Interactive Global Illumination using Fast Ray Tracing, Rendering Techniques 2002 (Proc. 13th Eurographics Workshop on Rendering) (P. Debevec and S. Gibson, eds.), 2002, pp. 15–24.

Irradiance Filtering for Monte Carlo Ray Tracing Janne Kontkanen1 , Jussi R¨ as¨anen1,2 , and Alexander Keller3 1

2

3

Helsinki University of Technology/TML, P.O.Box 5400, FIN-02015 HUT, Finland [email protected] Hybrid Graphics, Ltd. [email protected] University of Ulm, Albert-Einstein-Allee 11, D-89069 Ulm, Germany [email protected]

Summary. Stochastic ray tracing algorithms generate photo-realistic images by simulating the global illumination. Typically a rather long computation time is required for decreasing the visible noise to an acceptable level. In this paper we propose a spatially variant low-pass filter for reducing this noise. We analyze the theoretical background of the method and present an efficient implementation that enables the use of a comparatively small number of samples while producing high quality images. Our algorithm can be used to accelerate path tracing and final gathering in photon mapping. We compare the method to irradiance caching and the results show that our algorithm renders images of similar or better quality up to five times faster.

1 Introduction The efficient computation of indirect illumination has long been one of the major challenges of photo-realistic image synthesis. In order to compute the color of a pixel, the reflected radiance  Li (x, ωi )fr (ωr , x, ωi ) cos θdωi (1) Lr (x, ωr ) := Ω

at a surface location x, where ωr is the direction of reflection, has to be determined many times. Here Li is the incident radiance, fr the bidirectional reflection distribution function and θ is the angle between incident direction ωi and the surface normal n at x. The evaluation of integrals of the above form is costly since the incident radiance Li requires casting expensive rays from x in direction ωi . The reflected radiance is a piecewise continuous function. Its discontinuities can be caused by shadow boundaries, surface texture, or edges in the

260

J. Kontkanen et al.

scene geometry. During the process of image generation these discontinuities can be detected and used to identify the domains of continuity. Most models for the bidirectional reflectance distribution function fr = fs + fd are a linear combination of basis functions for specular and glossy surface properties fs and diffuse reflection fd . While specular and glossy surfaces are relatively inexpensive to render by tracing a small number of reflected rays, the diffuse surface interaction requires to scatter rays into the whole hemisphere, which becomes very expensive. The diffuse component fd ≡ fd (x) is characterized by depending only on the surface location x and therefore it can be taken out of the integral (1) yielding the irradiance  E(x) := Li (x, ωi ) cos θdωi . (2) Ω

So far the most widely adopted method that exploits the regions of continuity is irradiance caching [15]. The basic idea of irradiance caching is to compute relatively accurate irradiance estimates at a sparse set of surface locations and to extrapolate the missing values inside the regions of continuity. Compared to computing the irradiance at each location separately a tremendous speedup is obtained. Since extrapolation is used, the cached irradiance values must be precise in order to avoid visible artifacts in the image. This is especially difficult when rendering animations, where imprecisions cause distracting flickering intensity in the movie. In this paper we propose a better method that still efficiently utilizes the piecewise continuity of the irradiance signal, but is faster and produces less artifacts than previous methods. Instead of extrapolation, we show that lowpass filtering is an effective way for reducing noise in the irradiance. In order to account for the regions of continuity, the support of our filter and the sampling rate is spatially variant. Our method can be used as a replacement for irradiance caching [15]. It works with any rendering algorithm that uses stochastic ray tracing to compute indirect illumination, such as path tracing [10], bi-directional path tracing [7], or photon mapping [2]. The method aims at high quality offline rendering and performs best in scenes where indirect illumination is costly to compute. 1.1 Previous Work It was recognized rather early that exploiting the piecewise continuity of the irradiance can help to make rendering algorithms more efficient. We use the term indirect irradiance to denote surface irradiance excluding caustics and direct illumination, both of which are best handled by specialized algorithms [2, 10]. From the vast amount of research to evaluate the indirect irradiance only a few methods are practical, which we briefly discuss here.

Irradiance Filtering

261

Irradiance Caching Ward et al. [15] introduced irradiance caching. Spatial samples of the irradiance (2) are cached in an octree data structure. To determine whether the existing cache entries are suitable for extrapolating the irradiance at another pixel location of an image, Ward et al. derived a continuity metric predicting the change of irradiance with respect to change in surface normal and surface location. Later on Ward and Heckbert [14] improved the extrapolation method by including the first two terms of the Taylor series, i.e. by considering numerical approximations of the directional and spatial derivatives of the integrand in (2) in order to alleviate the extrapolation artifacts. This method then has been summarized in several books (e.g. [2]) and became the state of the art standard. However, there are still severe drawbacks: Since extrapolation is used, all samples must be computed at very high precision, because otherwise noisy samples are extrapolated, which results in distracting flicker in animations. In addition, the continuity metric forces the placement of a high number of expensive samples in regions of concave geometry. Thus corners are overly sampled, although their illumination can be continuous. Therefore Tabellion and Lamorlette [12] introduced several modifications to finally make irradiance caching at least feasible for production use, which could be first seen in the motion picture Shrek 2.The convincing results were obtained by a new metric for continuity, which avoids sampling too densely in concave geometry, and by a new set of extrapolation coefficients. However, the disadvantages of extrapolation remain the same, and thus the irradiance must be computed at very high precision. Filtering Noise Jensen [3] has already tried to reduce the noise in images by applying a median filter to the samples of the indirect irradiance. However, since the method did not utilize continuity information, its performance is limited. McCool [8] applied an anisotropic diffusion filter for reducing noise in images generated by stochastic methods. The filter used information about surface normals, depth, and color in order not to blur over discontinuities caused by shadow boundaries and textures. Since the technique has been applied as a post-process to the final image, the noise removal is not efficient on textured surfaces. In addition, the irradiance estimates must be relatively accurate so that the diffusion process does not detect discontinuities from noise. These weaknesses prohibit the method from practical use. Suykens and Willems [11] proposed adaptive filtering for progressive image synthesis by stochastic ray tracing. Their work is based on kernel density estimation. In the algorithm the contributions of light paths are splatted onto screen with a variable width kernel. The method is theoretically sound, but the performance of the algorithm is much lower than irradiance caching.

262

J. Kontkanen et al.

Merging Irradiance Caching and Filtering Most of the previous methods rely on being able to tell noise from discontinuities, which often fails and then results in visible artifacts in the image. This was accounted for in the discontinuity buffer [4], which uses the method of dependent tests to uniformly sample the integrand of (1) over the whole image plane and then to integrate samples in domains of geometric continuity only. Thus a blurring of discontinuities has been prevented. The discontinuity buffer resembles our new method for filtering, but in addition to detecting geometric discontinuities, our filter is designed to respect the characteristics of the irradiance signal. To summarize the previous work, irradiance caching, even with irradiance gradients and the improvements presented by Tabellion and Lamorlette requires accurate irradiance estimates in the sampling locations and is prone to extrapolation artifacts. In practice this means that it is hard to render artifact-free images with irradiance caching. Existing filtering methods utilize some knowledge of the scene geometry but none of them do it as efficiently as irradiance caching. Also, most of the methods apply filtering to final pixel radiance (1), which exposes smaller regions of continuity as compared to incident indirect illumination (2). Our algorithm uses the efficient discontinuity metric developed for irradiance caching but replaces extrapolation by filtering thus combining the best of both worlds.

2 Irradiance Filtering In this section we develop our filtering algorithm. In order to be able to use a relatively small number of samples for integro-approximation in (2), i.e. the method of dependent tests [5], we have to remove noise by applying a low-pass filter to the irradiance signal. 2.1 Theoretical Model for the Irradiance Estimator As mentioned in previous section, it is well known that indirect irradiance is a piecewise continuous function and that within regions of continuity it is a slowly varying signal [13, 15]. We restrict our considerations to a region of continuity and make the more explicit assumption that the signal is dominated by low frequencies. The central limit theorem [9, p. 219] states that the sample mean of M uncorrelated random samples drawn from any distribution with mean µ and variance σ 2 is approximately normally distributed with equal mean and vari2 ance σM , provided that M is large enough. The theorem holds even if the initial distribution is not normal. This results in

Irradiance Filtering

σ(x) ˆ E(x) = E(x) + (x) √ , M

263

(3)

as a model for the irradiance estimator random function. Here E(x) is the irradiance at location x, σ(x) is the standard deviation of the integrand, M is the number of independently identically distributed random samples, and (x) is a normally distributed random error term with the mean of zero and the standard deviation of one. Because the samples are uncorrelated, the signal (x) represents white noise. How large M should be for (3) to hold depends on the situation, but empirical studies have shown that sample sizes as small as 25 are sufficient for obtaining an approximately normal distribution [9, p. 219]. This is relevant for us since we want to use a small number of samples. The power spectral density of the estimated irradiance signal equals the sum of power spectral densities of the accurate irradiance signal and the error signal: A 2 B σ(x) 2 2 ˆ  = |F(E(x))| + F( (x) √ (4) |F(E(x))| M It turns out that the last term in the equation is constant regardless of frequency. By our assumption, low frequencies dominate the signal E(x). As a consequence, the signal-to-noise ratio is better at low frequencies and becomes worse toward higher frequencies (see Fig. 1 for an example of a typical situation). Thus, if we attenuate the higher frequencies of the signal by low-pass Noise (x) √σ

M

ˆ Sampled irradiance E(x) Filtered irradiance

Frequency domain

Spatial domain

Irradiance E(x)

Fig. 1. A model for noise due to integro-approximation by the method of dependent tests. From left to right: A typical irradiance signal in spatial domain and frequency domain, the noise generated using our error model, the signal with additive noise, and the filtered signal. Note that the expected power of the noise is independent of frequency. Because low frequencies dominate the irradiance signal, the signal-tonoise ratio is greater at low frequencies. With a properly sized low-pass filter it is possible to remove noise without blurring the result too much.

264

J. Kontkanen et al.

filtering, we may improve the perceived image quality. By doing this, we introduce bias into the image in the form of blurring, but in most practical cases this is an acceptable trade off. Note that the lowest frequencies still contain noise after filtering. This is difficult to notice in still images, but shows as flickering in animations. A possible solution to the problem is discussed in Sect. 4.1. 2.2 Filtering the Irradiance Estimator Our low-pass filter should have a frequency spectrum that smoothly decreases towards higher frequencies following approximately the assumed signal-tonoise ratio. A Gaussian filter would fit this specification, but unfortunately its kernel has infinite support. Instead, we chose to use Cook’s filter [1]:  2 2 e−(d/a) − e−r if d/a ≤ r f (d) = , (5) 0 if d/a > r which approximates the Gaussian, but has a compact support and decreases to zero without discontinuity. In the equation, r is the radius of the filter support and a is a user-controlled parameter for adjusting the amount of blur. The constant r should be selected small enough so that spatial support is compact, but on the other hand large enough so that the frequency spectrum of the filter is not too far from the Gaussian. Examining the filter in both spatial and frequency domains, we found that r = 2 is a good choice with respect to both requirements. The filter is parameterized by the distance d, which we derive next. The local frequency spectrum of the irradiance signal varies spatially. In areas where the irradiance signal changes slowly, its spectrum is narrow, and thus a filter with a narrow pass-band can be used. In areas where the irradiance signal changes rapidly, its spectrum is wider, and a filter with wider pass-band must be used in order to preserve detail. A narrow filter in the frequency domain corresponds to a large filter support in the spatial domain, and vice versa. Two important geometric features determining the frequency content of an irradiance signal are contact shadows and geometric discontinuities. The discontinuity metric developed by Ward et al. for irradiance caching [15] adapts to these, stating that the maximum change of indirect irradiance between two surface locations is roughly proportional to d(i, j) =

|xi − xj | : + 1 − ni · nj mi

(6)

where x, n and m are surface location, surface normal in that location, and harmonic mean distance to the surrounding geometry. Subscript i refers to the pivot point and subscript j to another point. Although not designed for

Irradiance Filtering

265

predicting the frequency content of the irradiance signal, we noticed that the metric works well for this purpose. The above metric ranges from zero to infinity, because the harmonic mean distance mi in the denominator can be arbitrarily close to zero in concave corners and infinite in unoccluded surface geometry. For irradiance caching this results in too many samples in concave corners and too little on unoccluded surfaces. This problem was addressed in the improved method by Tabellion and Lamorlette [12], where the authors set upper and lower bounds for the harmonic mean distance to keep the sampling density on the practical range. We adopt this solution in our method. 2.3 Sampling for the Method of Dependent Tests Since our filter is parameterized by distance, uniformly sampling the surface geometry would result in highest filtering quality. However, such a distribution would waste samples on surfaces, which are not even visible in the image. The sample distribution of the irradiance cache is efficient in this respect, but it is highly irregular leading to severe biasing artifacts when used with our filter. Instead, we approximate the desired distribution by the sample distribution created by a classic ray tracer [10] in the image plane. This yields an irradiance sampling location xi whenever a ray hits a surface with a nonzero diffuse material component fd . The algorithm is simple to implement and results in a relatively uniform distribution on the surfaces. The irregularities in the distribution are caused by perspective, occlusion, and geometric discontinuities. However, these seem to be perceptually insignificant. Because the filter is spatially variant, filtering removes noise more effectively in some neighborhoods than in others. To compensate for this, we use a number samples inversely proportional to the sum of filter weights at each location. How this is done in practice is explained in the algorithm Sect. 3. 2.4 Determining the Set of Samples for Filtering To efficiently evaluate the filtered irradiance at xi , we can discard sample locations xj , which are guaranteed to yield zero filter weights f (d(i, j)). By setting the distance d(i, j) equal to the filter cut-off radius r used in (5), we can bound the distance |xi − xj | by the radius rib = a r mi of a conservative bounding sphere around the filter support. Note that the : term 1 − ni · nj from the distance (6) can only reduce rib and thus can be omitted. The indices of the samples xj inside the bounding sphere around the location xi form the index set Wi . The sets Wi can efficiently be identified by range searching a 3d-tree built from the locations xi [2].

266

J. Kontkanen et al.

3 The Rendering Algorithm The samples from the previous section are stored in a data structure with an entry for each pixel of the image. An entry can contain one or more supersamples, while each supersample consists of a tree representing the reflection paths computed by a classic ray tracer [10]. The nodes xi of the tree with a diffuse material component fd = 0 are the locations where irradiance needs to be computed. The algorithm has six phases: 1. By classic ray tracing the irradiance sampling locations xi and the direct illumination are determined. 2. The harmonic mean distance mi to surrounding geometry is computed at each sampling location xi . 3. The filter support at each sampling location is computed. 4. The irradiance is integrated at each sampling location. 5. The noisy irradiance estimate is filtered. 6. The final image is composited. Classic Ray Tracing and Supersampling According to Sect. 2.3, the first stage of the algorithm is classic ray tracing. We perform adaptive supersampling based on the discontinuities in color, normals, and depth. At each ray-surface intersection, we create a new node in the ray tree of the supersample. In case of simultaneous reflection and transmission, the path splits into two. If indirect glossy reflections and caustics are to be included, they can be computed at this stage and the results are stored in the tree nodes. Mean Distance Computation To use Ward’s discontinuity metric for filtering, the harmonic mean distance mi to surrounding geometry is computed at each sampling location by randomly sampling the hemisphere. This is done by casting rays, summing up the inverse distances to the closest hitpoint, and taking its inverse as the harmonic mean. This is a relatively fast operation and the variance of the harmonic mean distance is typically much lower than the variance of incident radiance. In addition, the result can be improved by applying a simple 3x3 median filter to computed mean distances. As a consequence, in most scenes as few as 16 distance samples per supersample are sufficient. Filter Support Analysis Due to a spatially variant filter size, some areas require more accurate sampling than others. For each sampling location xi we calculate the sum

Irradiance Filtering

(

Si :=

f (d(i, j))

267

(7)

j∈Wi

of filter weights and store it with the corresponding sample xi . The fast determination of the index set Wi of candidates has been addressed in Sect. 2.4. In the next stage of the algorithm, Si will be used as a measure of the support for the current sample xi . Irradiance Integration Indirect irradiance is computed by a standard path tracer or by final gathering from a photon map [2, 10]. Direct caustics should be excluded since they contain high frequencies that would be removed along with the noise by the filter. They can be calculated using e.g. photon mapping and added to the result at the compositing stage [2]. For keeping the computation time predictable, we allow the user to specify the average number of samples per supersample Navg . This determines the total sample budget for computing the irradiance. The number of samples Ni =

C . Si

(8)

at each sampling location xi is inversely proportional to the sum of filter weights. In order to solve for C, we require the sum of all Ni to be equal to the total sample budget M −1 ( Ni = Navg M, i=0

where M is the total number of supersamples in the buffer. The previous two equations yield Navg M (9) C = 1M 1 . i=0 Si

Now, the number of samples used for the irradiance integration can be determined by (8). As shown in the left image of Fig. 2, this simple heuristic produces satisfactory results. Filtering 1

The final irradiance Ei

=

j∈Wi

Ej f (d(i, j)) Si

(10)

is computed as a weighted sum of irradiance estimates Ej , which have been computed from (2) using Nj samples. The right image of Fig. 2 illustrates the result of filtering, while Fig. 3 demonstrates how various filter sizes affect the final irradiance.

268

J. Kontkanen et al.

Fig. 2. The left image shows the sampled irradiance estimates Ei , while the right image shows the result after filtering, i.e. Ei
. The brighter spots along the wall are due to color bleeding from the floor texture (see Fig. 4). A filter radius between 10 and 60 pixels was used for the 800 × 600 image.

Fig. 3. Three closeups of the bunny showing indirect irradiance filtered with various filter supports. The leftmost image was filtered with a small kernel, which was not able to remove all the noise. The middle image was filtered with larger kernel which successfully removed the noise and left the detail intact. The rightmost image was filtered with a too large kernel.

Compositing Now the radiance value of each supersample is computed by evaluating the corresponding ray tree. This effectively accumulates the different components of illumination, like e.g. specular paths and caustics, and yields a complete global illumination solution. The final image is computed in a standard manner by tone mapping and applying a reconstruction filter to the supersamples to obtain color values for each pixel. Since the compositing procedure is standard in rendering, we omit more details and instead refer to classic ray tracing books as e.g. [10].

Irradiance Filtering

269

4 Results We tested the algorithm on a PC with a 2.8 GHz Pentium 4 processor and 1GB of memory. For comparison, we implemented two-pass irradiance caching with irradiance gradients, which computes the irradiance sampling locations in the first pass and extrapolates the values in the second. Note that the two-pass method gives better results than the classic single-pass version as described in the original paper by Ward et al. [15], since it has more samples available when reconstructing the irradiance. Although our theoretical results are derived using random sampling, in practice stratified sampling and quasi-Monte Carlo methods also work well. With our algorithm, we used the deterministic sampling framework described by Keller [6]. With both methods, path tracing [10] was used for computing the radiance samples. We rendered two test scenes with both algorithms. The resulting images are compared in Fig. 4. First, we adjusted the number of samples and the parameters of the methods aiming at perceptually similar image quality. As expected, we could not fulfill this goal completely, since it turned out too time consuming to compute images without extrapolation artifacts by irradiance caching. As a result, the images rendered with our method look slightly better than those computed with irradiance caching. In the test scenes our method was five times faster than irradiance caching as can be seen from the timings and statistics in Table 1. This is a direct consequence of being able to use less samples due to the method of dependent tests. The two filtering stages, support analysis and final filtering, took a significant amount of time, but the majority of computation was still spent sampling the indirect irradiance. We also rendered images with irradiance caching using the same amount of time. This resulted in distracting artifacts that can be seen in Fig. 4 (C,G). Table 1. Results for the test scenes. The first row shows the results for our method, the second row for irradiance caching trying to match the quality and the last row for irradiance caching trying to match the time. The corresponding images are shown in Fig. 4. Helicopter Image Method Samples Meandist. Sampling Filtering Total A new Irr. filtering 15 659,950 216s 3 793s 1 454s 5 531s 29 752s 14s 29 830s B Irr. cache(≈qual.) 137 175 000 0s 5 644s 14s 5 723s C Irr. cache(≈time) 25 818,040 0s Bunny Image Method Samples Meandist. Sampling Filtering Total E new Irr. filtering 26 596 061 327s 13 468s 1 428s 15 267s Irr. cache(≈qual.) 162 645 964 0s 79 620s 9s 79 672s F 15 879s 11s 15 934s G Irr. cache(≈time) 31 566 672 0s

270

J. Kontkanen et al.

A. Irradiance Filtering

C. Irr. Caching (≈time)

E. Irradiance Filtering

G. Irr. Caching (≈time)

B. Irr. Caching (≈quality, 5 x time)

D.. Difference of A and C

F. Irr. Caching (≈quality, 5 x time)

H. Difference of E and G

Fig. 4. The test scenes rendered with our new algorithm and with irradiance caching. Images A and E have been rendered with our method aiming at perceptually flawless quality. Images B and F have been computed with irradiance caching trying to match in quality. Images C and G use the same amount of time as the images made with irradiance filtering. The artifacts in C and G pop out in the contrast-enhanced difference images D and H. Darker color indicates more difference.

Irradiance Filtering

271

Compared to irradiance caching, the efficiency of our algorithm depends more directly on the resolution, since filtering takes more time as resolution increases. Thus, at least when rendering simple scenes with high resolution, it might be beneficial to limit the filter radius and to compensate it by more samples. However, in scenes with difficult indirect illumination, sampling almost always dominates the computation time. Our tests with resolutions ranging from 640x480 to 1024x768 indicate that the method scales linearly or slightly better with the number of pixels, as expected. 4.1 Discussion and Future Work The algorithm is straightforward to parallelize as each client can render a tile of an image. Before filter support analysis and filtering, neighboring clients share the part of the sample buffer needed by the other client. The maximum width of the slice shared by neighboring tiles is equal to the maximum filter radius. The discontinuity metric of Ward et al. works well for predicting how fast the indirect irradiance is likely to change, but it might also be possible to derive a metric that is specifically designed for predicting the frequency content. This would enable designing a filter that more accurately attenuates frequencies with low signal-to-noise ratio. Due to the use of a 3D data structure, irradiance caching allows reusing the irradiance samples in animations, provided the scene is static. While our data structure does not directly support this, the reuse is possible by extending the filter to the time domain. Using samples from nearby animation frames allows dynamic scenes and enables filtering with a spatially smaller kernel. This also helps to remove the noise left in the lowest frequencies after spatial filtering. Our preliminary tests look promising.

5 Conclusions We have shown that effective noise-removal is possible for an irradiance signal dominated by low frequencies, and we have designed a filter that works well in practice and has theoretical motivation. For many years, irradiance caching has been a de-facto standard for accelerating the computation of indirect illumination. Our main improvement is that we do not rely on expensive accurate irradiance estimates, but share samples among large neighborhoods thus enabling the use of far less samples. As a result, we are able to render images with better quality up to five times faster.

272

J. Kontkanen et al.

Acknowledgements The first two authors thank Vesa Meskanen for invaluable conversations and for leading them to the topic of irradiance filtering. All authors would also like to thank 3Dr research team, Lauri Savioja, Tuomas Lukka, Ville Miettinen, Brian Budge, and the R&D team of Illuminate Labs for precious comments and discussions. In addition, we are grateful to Samuli Laine for help in programming and ideas, and to Eetu Martola for the helicopter model. This research was funded by National Technology Agency of Finland, Bitboys, Hybrid Graphics, Nokia, and Remedy Entertainment.

References 1. R. Cook. Stochastic Sampling in Computer Graphics. In ACM Transactions on Graphics, volume 5, pages 51–72, January 1986. 2. H. Jensen. Realistic Image Synthesis Using Photon Mapping. AK Peters, 2001. 3. H. Jensen and N. Christensen. Optimizing path tracing using noise reduction filters. Proceedings of WSCG 1995, 1995. 4. A. Keller. Quasi-Monte Carlo Methods for Photorealistic Image Synthesis. PhD thesis, Aachen, 1998. 5. A. Keller. Hierarchical Monte Carlo Image Synthesis. Mathematics and Computers in Simulation, 55(1-3):79–92, 2001. 6. A. Keller. Strictly Deterministic Sampling Methods in Computer Graphics. SIGGRAPH 2003 Course Notes, Course #44: Monte Carlo Ray Tracing, 2003. 7. E. Lafortune and Y. Willems. Bidirectional Path Tracing. In Proc. 3rd International Conference on Computational Graphics and Visualization Techniques (Compugraphics), pages 145–153, 1993. 8. M. McCool. Anisotropic Diffusion for Monte Carlo Noise Reduction. ACM Transactions on Graphics (TOG), 18(2):171–194, 1999. 9. J. Milton and J. Arnold. Introduction to Probability and Statistics. McGraw-Hill, 1990. 10. P. Shirley. Realistic Ray Tracing. AK Peters, Ltd., 2000. 11. F. Suykens and Y. Willems. Adaptive Filtering for Progressive Monte Carlo Image Rendering. In 8th International Conference in Central Europe on Computer Graphics, Visualization and Interactive Digital Media (WSCG 2000), Plzen, Czech Republic, 2000. 12. E. Tabellion and A. Lamorlette. An Approximate Global Illumination System for Computer Generated Films. ACM Trans. Graph., 23(3):469–476, 2004. 13. I. Wald, T. Kollig, C. Benthin, A. Keller, and P. Slusallek. Interactive Global Illumination using Fast Ray Tracing. In P. Debevec and S. Gibson, editors, Rendering Techniques 2002 (Proc. 13th Eurographics Workshop on Rendering), pages 15–24, 2002. 14. G. Ward and P. Heckbert. Irradiance Gradients. In 3rd Eurographics Workshop on Rendering, pages 85–98, 1992. 15. G. Ward, F. Rubinstein, and R. Clear. A Ray Tracing Solution for Diffuse Interreflection. Computer Graphics (Proceedings of ACM SIGGRAPH 88), 22:85–92, 1988.

On the Star Discrepancy of Digital Nets and Sequences in Three Dimensions Peter Kritzer Department of Mathematics, University of Salzburg, Hellbrunnerstr. 34, A-5020 Salzburg, Austria [email protected] Summary. By relating the star discrepancy of a digital (t, m, 3)-net to that of a digital (t, 2)-sequence, we obtain new upper bounds on the star discrepancy of digital (t, m, 3)-nets. From these results, we derive new upper bounds on the star discrepancy of digital (t, 3)-sequences. We also show the existence of (0, m, 4)-nets with particularly low star discrepancy.

1 Introduction ∗ For a given point set x0 , x1 , . . . , xN −1 in [0, 1)s , the star discrepancy DN is defined by ∗ := sup AN (J)N −1 − λ(J) , DN J

where2the supremum is extended over all intervals J ⊆ [0, 1)s of the form s J = j=1 [0, αj ), 0 < αj ≤ 1, AN (J) denotes the number of i with xi ∈ J, and λ is the Lebesgue measure. The concepts of (digital) (t, m, s)-nets and (t, s)-sequences provide a very efficient method to construct point sets with small star discrepancy. An extensive survey on this topic is presented by Niederreiter in [5, 6]. We first give the general definition of a (t, m, s)-net. Definition 1. Let b ≥ 2, s ≥ 1, and 0 ≤ t ≤ m be integers. Then a point set P consisting 2 of bm points in [0, 1)s forms a (t, m, s)-net in base b, if every s subinterval J = j=1 [aj b−dj , (aj + 1)b−dj ) of [0, 1)s , with integers dj ≥ 0 and integers 0 ≤ aj < bdj for 1 ≤ j ≤ s and of volume bt−m , contains exactly bt points of P . Observe that a (t, m, s)-net is extremely well distributed if the quality parameter t is small. The definition of a (t, s)-sequence is based on (t, m, s)-nets and is given in the following.

274

P. Kritzer

Definition 2. Let b ≥ 2, s ≥ 1, and t ≥ 0 be integers. A sequence (xn )n≥0 in [0, 1)s is a (t, s)-sequence in base b if for all l ≥ 0 and m > t the point set consisting of the points xlbm , . . . , x(l+1)bm −1 is a (t, m, s)-net in base b. Again, a (t, s)-sequence is particularly well distributed if the quality parameter t is small. We study digital (t, m, s)-nets and digital (t, s)-sequences for s = 2, 3, and 4 over a finite field F q , where q is a prime power. A digital (t, m, s)-net over a finite field F q , which is a special type of a (t, m, s)-net in base q (cf. [6]), is defined as follows (for a more general definition, see again [6]). Denote the elements of F q , for q a prime power, by 0, 1, . . . , q − 1, where we assume that 0 is the neutral element with respect to addition and 1 is the neutral element with respect to multiplication. Choose a bijection between the elements of F q and the set of integers {0, 1, . . . , q − 1}. In general, this bijection could be chosen arbitrarily. Here, however, we always assume that the bijection is such that k corresponds to k for each k ∈ {0, 1, . . . , q − 1}. For the construction of a digital (t, m, s)-net choose s matrices C1 , . . . , Cs over F q with the following property. For each choice of nonnegative integers d1 , . . . , ds with d1 + · · · + ds = m − t, the system of the first d1 rows of C1 together with the first d2 rows of C2 together with the .. . first ds rows of Cs is linearly independent over F q . For a fixed i ∈ {0, . . . , q m − 1}, let i have base q representation i = i0 + i1 q + · · · + im−1 q m−1 . For j ∈ {1, . . . , s}, multiply the matrix Cj by the vector in F m q that is associated with the vector of digits of i, which gives (j)

(j) (y 1 (i), . . . , y m (i))T := Cj · (ı0 , . . . , ım−1 )T ∈ F m q .

Then we set (j)

xi

:=

m (j) ( y (i) k

k=1 (1)

qk

.

(s)

Finally, let xi := (xi , . . . , xi ). The point set consisting of the points x0 , x1 , . . . , xqm −1 is called a digital (t, m, s)-net over F q with generating matrices C1 , . . . , Cs . The concept of digital nets that are digitally shifted by vectors in F m q offers a generalization of the notion of digital nets. These are constructed by a slight variation in the net generating procedure outlined above. To be more precise, choose s vectors σ 1 , . . . , σ s with (j)

(j) T σ j = (σ 1 , . . . , σ m ) ∈ Fm q

and set, for each i ∈ {0, . . . , q m − 1},

Star Discrepancy of Digital Nets and Sequences in Three Dimensions

275

 (j) (j) (y 1 (i), . . . , y m (i))T := Cj · (ı0 , . . . , ım−1 )T ⊕ σ j ∈ F m q 

for 1 ≤ j ≤ s, where ⊕ denotes coordinatewise addition in F m q . Point sets that are obtained by digitally shifting digital (t, m, s)-nets are still (t, m, s)nets (cf. Lemma 3 in [2]), however, in general they are no digital nets any more since they do not necessarily contain the origin. A digital (t, s)-sequence over F q , which is a special type of a (t, s)-sequence in base q (cf. [6]), is constructed as follows. Choose s ∞ × ∞-matrices C1 , . . . , Cs over F q such that for any m > t the left upper m × m-submatrices of C1 , . . . , Cs generate a digital (t, m, s)-net over F q . For i ≥ 0, let i have base q representation i = i0 + i1 q + · · · . For j ∈ {1, . . . , s}, multiply the matrix Cj by the vector associated with the vector of digits of i, which yields (j)

(j)

(y 1 (i), y 2 (i), . . .)T := Cj · (ı0 , ı1 , . . .)T , and set (j)

xi

:=

∞ (j) ( y (i) k

k=1

qk

. (1)

(s)

Then the sequence consisting of the points x0 , x1 , . . . with xi := (xi , . . . , xi ) is called a digital (t, s)-sequence over F q and C1 , . . . , Cs are its generating matrices. A technical requirement on a digital (t, s)-sequence is that, for each (j) i ≥ 0 and 1 ≤ j ≤ s, we have yk (i) < q − 1 for infinitely many k (cf. [6]). In this paper, it is our aim to study the star discrepancy of digital (t, m, 3)nets over F q . It was shown in [5,6] that for the star discrepancy of an arbitrary (t, m, 3)-net (not necessarily digital) in base b we have C D 2 b−1 9 b−1 m ∗ t 2 (m − t) + b Dbm ≤ b (m − t) + . (1) 2 2 4 Inequality (1) was improved for the special case of digital (0, m, 3)-nets over F 2 in [7], where it was shown that in this case 2m D2∗m ≤

m2 + O(m) . 6

(2)

Here we give new upper bounds on the star discrepancy of digital (t, m, 3)-nets over F q , improving on (1) and (2). The basic tool in our considerations will be to relate the star discrepancy of digital (t, m, 3)-nets to the star discrepancy of digital (t, 2)-sequences. Concerning the star discrepancy of (t, 2)-sequences, upper bounds were recently given in [1] which are as follows. For the star discrepancy of the first N elements of a (not necessarily digital) (t, 2)-sequence ∗ (S), we have S in base b, DN • for b ≥ 2 and bt ≤ N < bt+2 , ∗ (S) ≤ bt c1 (b) , N DN

(3)

276

P. Kritzer

• for even b ≥ 2 and bt+2 ≤ N, ∗ N DN (S) ≤

bt b2 (b − 1) log N + c3 (t, b) , (log N )2 + c2 (t, b) 16(b + 1)(log b)2 log b

(4)

• for odd b ≥ 3 and bt+2 ≤ N, ∗ N DN (S) ≤

bt (b − 1)2 log N + c5 (t, b) , (log N )2 + c4 (t, b) 2 16(log b) log b

(5)

where c1 (b) is a constant depending only on b, and where c2 (t, b), c3 (t, b), c4 (t, b), and c5 (t, b) are constants depending only on t and b. Remark 1. In the case of (0, 2)-sequences in base 2, the leading term in the upper bound given above simplifies to (log N )2 /(12(log 2)2 ). This should be compared to a result by Faure [4], who showed the existence of a (0, 2)-sequence in base 2 such that lim sup N →∞

∗ N DN 1 ≥ . 2 (log N ) 24(log 2)2

Further, we are going to derive new upper bounds on the star discrepancy of the first N elements of digital (t, 3)-sequences. We have the following ex∗ isting result by Niederreiter [5, 6]. The star discrepancy DN (S) of the first N terms of a (t, 3)-sequence S in base b (not necessarily digital) satisfies ∗ (S) ≤ N DN

1 1 (b − 1)3 bt (k − t)3 + (b − 1)2 (b + 5)bt (k − t)2 24 16 1 1 + (b − 1)(b2 + 16b + 61)bt (k − t) + (b2 + 4b + 13)bt (6) 48 8

for N ≥ bt , where k is the largest integer with bk ≤ N . We are going to improve this bound by making use of our results on digital (t, m, 3)-nets. This paper is organized as follows. In the subsequent section, we state our main results. Section 3 is devoted to technical lemmas and auxiliary observations. The proofs of the theorems, finally, are given in Sect. 4.

2 The Main Results Concerning the star discrepancy of digital (t, m, 3)-nets, we have the following theorem, improving on (1) and (2). Theorem 1. (a) Let P be a digital (t, m, 3)-net over F q for an even prime power q. Then we have q m Dq∗m (P ) ≤

q t q 2 (q − 1)m2 + O(m) , 16(q + 1)

where the constant in the O-notation depends only on t and q.

Star Discrepancy of Digital Nets and Sequences in Three Dimensions

277

(b) Let P be a digital (t, m, 3)-net over F q for an odd prime power q. Then we have q t (q − 1)2 m2 q m Dq∗m (P ) ≤ + O(m) , 16 where the constant in the O-notation depends only on t and q. Remark 2. It was shown by Pillichshammer in [7, Theorem 2] that there always exists a digital (0, m, 3)-net over F 2 such that its star discrepancy is bounded above by m2 /12 + O(m). Theorem 1 considerably improves this result, since it implies that this upper bound holds for all digital (0, m, 3)-nets over F 2 . From Theorem 1 we immediately get the following Corollary 1. (a) For any even prime power q and given t ≥ 0, we have lim sup max m→∞

q m Dq∗m q t q 2 (q − 1) , ≤ 2 m 16(q + 1)

where the maximum is extended over all digital (t, m, 3)-nets over F q . (b) For any odd prime power q and given t ≥ 0, we have lim sup max m→∞

q m Dq∗m q t (q − 1)2 , ≤ m2 16

where the maximum is extended over all digital (t, m, 3)-nets over F q . We also have the following proposition, the proof of which will be given in Sect. 4. Proposition 1. The bounds in Theorem 1 are also valid for digital (t, m, 3)nets over F q that are digitally shifted by vectors in F m q . The results stated in Theorem 1 and Proposition 1 can be used for the derivation of new upper bounds on the star discrepancy of the first N terms of digital (t, 3)-sequences. We have the following theorem, improving on (6). Theorem 2. Let S be a digital (t, 3)-sequence over F q . Then for the star discrepancy of its first N elements, N ≥ q t , we have (a) if q is even, ∗ (S) ≤ N DN

  q t q 2 (q − 1)2 (log N )3 + O (log N )2 , 96(q + 1)(log q)3

(b) if q is odd, ∗ (S) ≤ N DN

  q t (q − 1)3 (log N )3 + O (log N )2 , 96(log q)3

where the constants in the O-notation depend only on t and q.

278

P. Kritzer

From Theorem 2 we deduce Corollary 2. (a) For any even prime power q and given t ≥ 0 we have lim sup sup N →∞

∗ N DN q t q 2 (q − 1)2 ≤ , 3 (log N ) 96(q + 1)(log q)3

where the supremum is extended over all digital (t, 3)-sequences over F q . (b) For any odd prime power q and given t ≥ 0 we have lim sup sup N →∞

∗ N DN q t (q − 1)3 ≤ , 3 (log N ) 96(log q)3

where the supremum is extended over all digital (t, 3)-sequences over F q . Finally, we state a further consequence of Theorem 2 with respect to the existence of (0, m, 4)-nets with low star discrepancy. In [5,6], Niederreiter gave a general upper bound on the star discrepancy of arbitrary (t, m, 4)-nets in base b. For the special case t = 0, this upper bound is given by C D 3 b−1 15 3(b − 1)2 2 3(b − 1) m ∗ 3 m + m+ m + b Dbm ≤ . (7) 2 8 8 4 By the help of the results in Theorem 2 we can—for special choices of the base b—construct (0, m, 4)-nets with particularly low star discrepancy compared to the upper bound in (7). We have Theorem 3. For every m ≥ 1 and every prime power q ≥ 3 there exists a digital (0, m, 4)-net over F q such that (a) if q is even, q m Dq∗m ≤

q 2 (q − 1)2 3 m + O(m2 ) , 96(q + 1)

(b) if q is odd, (q − 1)3 3 m + O(m2 ) , 96 where the constants in the O-notation depend only on q. q m Dq∗m ≤

3 Auxiliary Results We introduce some notation. For convenience, we shall in the following frequently omit the bars over the elements of F q . For natural numbers k and l, let 0k×l denote the k × l-matrix over F q consisting only of zeros. Moreover, let E l×l be the l × l-matrix over F q given by

Star Discrepancy of Digital Nets and Sequences in Three Dimensions

E l×l

279

⎛ ⎞ 0 ... 0 1 ⎜0 . . . 1 0⎟ ⎜ ⎟ ⎟ =⎜ ⎜. . . . . . . . .⎟ . ⎝0 1 . . . 0⎠ 1 0 ... 0

We first have Lemma 1. Let P be a digital (t, m, 3)-net over F q with generating matrices m m×m . Then, for every C1 = ((au,v ))m u,v=1 , C2 = ((bu,v ))u,v=1 , and C3 = E k ∈ {t + 1, . . . , m}, the left upper k × k-submatrices of C1 and C2 generate a digital (t, k, 2)-net over F q . Proof. Let k ∈ {t + 1, . . . , m} be given. We denote the left upper k × ksubmatrices of C1 and C2 by C1 (k) and C2 (k) respectively. Let d1 and d2 be nonnegative integers with d1 + d2 = k − t. We consider the (m − t) × m-matrix G consisting of the first d1 rows of C1 , the first d2 rows of C2 , and the first m − k rows of C3 , that is, ⎛ ⎞ a1,1 ... a1,k a1,k+1 ... a1,m ⎜ .. .. .. .. ⎟ ⎜ . . . . ⎟ ⎜ ⎟ ⎜ad1 ,1 ⎟ . . . a a . . . a d ,k d ,k+1 d 1 1 1 ,m ⎟ ⎜ ⎜ b1,1 ⎟ . . . b b . . . b 1,k 1,k+1 1,m ⎜ ⎟ .. .. .. .. ⎟ . G=⎜ ⎜ . . . . ⎟ ⎜ ⎟ ⎜ bd ,1 ... bd2 ,k bd2 ,k+1 ... bd2 ,m ⎟ ⎜ 2 ⎟ ⎜ ⎟ ⎜ ⎟ (m−k)×k (m−k)×(m−k) ⎝ ⎠ 0 E From the (t, m, 3)-net property of P it follows that the rank of G is m − t. Since the right lower (m − k) × (m − k)-submatrix of G is E (m−k)×(m−k) , it follows that the left upper (k − t) × k-submatrix of G must have rank k − t. Therefore, the first d1 rows of C1 (k) together with the first d2 rows of C2 (k) are linearly independent and the result follows.  We also need the following Lemma 2. Let C1 and C2 be the generating matrices of a digital (t, m, 2)-net over F q . Then we can find two (m + 1) × (m + 1)-matrices D1 and D2 such that D1 and D2 generate a digital (t, m + 1, 2)-net over F q , and Cj is the left upper m × m-submatrix of Dj for 1 ≤ j ≤ 2. Proof. We denote the rows of C1 by au and the rows of C2 by bv , 1 ≤ u, v ≤ m. We show how the matrices D1 and D2 can be constructed. Let

280

P. Kritzer

⎛ ⎜ ⎜ ⎜ ⎜ C2 ⎜ ⎜ C1 em+1−t ⎟ ⎜ ⎟ ⎜ , D D1 := ⎝ := 2 ⎜ ⎠ ⎜ ⎜ 0 ... 0 ⎜ ⎝ 0 ... 0 ⎛

⎞

⎞ f1 .. ⎟ . ⎟ ⎟ fm−t ⎟ ⎟ 1 ⎟ ⎟ , 0 ⎟ ⎟ .. ⎟ . ⎠ 0

where em+1−t is the (m+1−t)-th unit vector in F m+1 , and where f1 , . . . , fm−t q are certain elements in F q . We show how to choose f1 , . . . , fm−t such that D1 and D2 generate a digital (t, m + 1, 2)-net. To begin with, it is obvious that the first m + 1 − t rows of D1 are linearly independent. In the next step, we consider the first m − t rows of D1 together with the first row of D2 , that is, we consider the matrix ⎛ ⎞ b1 f1 ⎜ a1 0⎟ ⎜ ⎟ G1 = ⎜ . .. ⎟ . ⎝ .. .⎠ am−t 0 By the linear independence of a1 , . . . , am−t , it follows that G1 is regular if we choose f1 = 0. We now consider the first m − t − 1 rows of D1 together with the first two rows of D2 , that is, we consider the matrix ⎛ ⎞ ⎞ ⎛ g1 b2 f2 ⎜ g2 ⎟ ⎜ b1 f1 ⎟ ⎜ ⎟ ⎟ ⎜ ⎟ ⎟ ⎜ a1 0 ⎟ =: ⎜ G2 = ⎜ ⎜ g3 ⎟ . ⎜ ⎟ ⎟ ⎜ .. .. .. ⎝ ⎠ ⎝ .⎠ . . am−t−1 0

g m+1−t

By the (t, m, 2)-net property of P it is clear that a1 , . . . , am−t−1 , b1 are linearly independent. It follows that G2 has full rank if g 1 is not a linear combination of g 2 , . . . , g m+1−t . Let us in the first place assume that b2 cannot be expressed as a linear combination of a1 , . . . , am−t−1 , b1 . Then it immediately follows that g 1 is not a linear combination of g 2 , . . . , g m+1−t and so G2 must have rank m + 1 − t. Suppose on the other hand that b2 is a linear combination of a1 , . . . , am−t−1 , b1 . This representation is unique due to the linear independence of a1 , . . . , am−t−1 , b1 . If b1 occurs in the representation of b2 , we choose f2 = 0. Otherwise, we choose f2 = 0. In both cases it is guaranteed that g 1 cannot be expressed as a linear combination of g 2 , . . . , g m+1−t . This procedure can be repeated for f3 , . . . , fm−t . In each case, the choice of fi depends on whether bi is a linear combination of a1 , . . . , am+1−t−i , b1 , . . . , bi−1 , and, if this is the case, on the sum of the fj corresponding to the rows bj occurring in this linear combination. In each step, the matrix Gi has rank m + 1 − t and the numbers f1 , . . . , fi−1 are not altered. In this way we  achieve the desired construction of D1 and D2 .

Star Discrepancy of Digital Nets and Sequences in Three Dimensions

281

The following lemma is (in different versions) well known in the theory of uniform distribution. Lemma 3. Let P be a point set consisting of N points x0 , . . . , xN −1 ∈ [0, 1)3 and let Q be a point set consisting of N points y 0 , . . . , y N −1 ∈ [0, 1)3 . Suppose we have, for 0 ≤ i ≤ N − 1,     (1) (2) (3) (1) (2) (3) , y i = y i , yi , yi , xi = xi , xi , xi (j) (j) with xi − yi ≤ ε for 1 ≤ j ≤ 3 and 0 ≤ i ≤ N − 1, where ε > 0 is a constant. Then we have ∗ ∗ |DN (P ) − DN (Q)| ≤ 3ε .

Proof. The assertion follows by proceeding, step by step for each coordinate of the points of P and Q, as in the proof of Lemma 2.5 in [6] (see also [7, Lemma 4]).  We also need the following Lemma 4. Let P be a point set that is obtained by digitally shifting a digital (t, m, 3)-net over F q , generated by matrices C1 , C2 , and C3 = E m×m , by vectors σ 1 , σ 2 , σ 3 ∈ F m q . Then P can also be obtained by shifting the digital (t, m, 3)-net generated by C1 , C2 , C3 by vectors τ 1 = (C1 · σ 0 ) ⊕ σ 1 , τ 2 = (C2 · σ 0 ) ⊕ σ 2 , τ 3 = 0 , where σ 0 ∈ F m q is the unique vector such that (C3 · σ 0 ) ⊕ σ 3 = 0 (note that C3 is regular), and ⊕ denotes coordinatewise addition in F m q . Proof. Let i ∈ F m q be the vector associated with the digit vector of i ∈ {0, . . . , q m − 1} and denote, for short, the vector i ⊕ σ 0 by i0 . Observe that    (C1 ·i) ⊕ σ 1 , (C2 ·i) ⊕ σ 2 , (C3 ·i) ⊕ σ 3 , 0 ≤ i ≤ q m − 1    = (C1 ·i0 ) ⊕ σ 1 , (C2 ·i0 ) ⊕ σ 2 , (C3 ·i0 ) ⊕ σ 3 , 0 ≤ i ≤ q m − 1    = (C1 ·i) ⊕ (C1 ·σ 0 ) ⊕ σ 1 , (C2 ·i) ⊕ (C2 ·σ 0 ) ⊕ σ 2 , C3 ·i , 0 ≤ i ≤ q m − 1 . The result follows.



4 The Proofs The main tool in the proof of Theorem 1 is the following proposition. Proposition 2. Let P be a digital (t, m, 3)-net over F q generated by C1 , C2 , and C3 = E m×m . Then we can find two ∞ × ∞-matrices D1 and D2 such that D1 and D2 generate a digital (t, 2)-sequence over F q , and Cj is the left upper m × m-submatrix of Dj for 1 ≤ j ≤ 2.

282

P. Kritzer

Proof. The construction of D1 and D2 is achieved by repeatedly applying Lemma 2. It is then guaranteed that C1 and C2 are the left upper m × msubmatrices of D1 and D2 . Moreover, Lemma 1 shows that the left upper k×ksubmatrices of D1 and D2 generate a digital (t, k, 2)-net for t + 1 ≤ k ≤ m. By Lemma 2, it is clear that the same holds for all k > m. From this, the result follows.  Remark 3. Observe that D1 and D2 are—due to the construction method outlined in Lemma 2—chosen such that the entries in the matrices D1 and D2 “below” C1 and C2 are 0. It follows that the first q m points of the sequence generated by D1 and D2 are the points of the digital net generated by C1 and C2 . By the help of Proposition 2 and the auxiliary results in Sect. 3, we can now give the proof of Theorem 1. Proof (Theorem 1). We start with showing (a). Let P be a fixed digital (t, m, 3)-net over F q with even q, generated by matrices C1 , C2 , C3 . Multiplying the generating matrices of a digital net by the same regular m × m-matrix from the right does not change the point set except for the order of points. Therefore, by the (t, m, 3)-net property of P , we can assume that ⎞ ⎛ (m−t)×t 0 E (m−t)×(m−t) ⎟ ⎜ ⎟ ⎜ ⎜cm−t+1,1 . . . cm−t+1,m ⎟ C3 = ⎜ ⎟ , ⎟ ⎜ .. .. ⎠ ⎝ . . cm,m cm,1 . . . where the ci,j denote certain elements of F q . We now study the star discrepancy of the point set Q, which we assume to 3 = E m×m . Note that Q is also be the digital net generated by C1 , C2 , and C a digital (t, m, 3)-net with the same quality parameter t as P , since the first m − t rows of C3 and E m×m are the same. By Proposition 2, the projection onto the first two coordinates of the points of Q, Q{1,2} , can be “embedded” in a digital (t, 2)-sequence S such that the first q m points of S form Q{1,2} . The first q m points of S, let us call them x0 , . . . , xqm −1 , are related to the points of Q, let us call them y 0 , . . . , y qm −1 , via   i y i = xi , m q for each i ∈ {0, . . . , q m − 1}. By Lemma 8.9 in [5] it follows that q m Dq∗m (Q) ≤

max

1≤M ≤q m

By making use of (3) and (4), we get

∗ M DM (S) + 1 .

Star Discrepancy of Digital Nets and Sequences in Three Dimensions

q m Dq∗m (Q) ≤

283

q t q 2 (q − 1)m2 + O(m) , 16(q + 1)

where the constant in the O-notation depends only on q and t. Note now that the points of P and Q only differ in the t least significant q-ary digits of the third coordinate. Thus, the difference in the third coordinate of the points of P and Q is bounded by q t−m and the result follows from Lemma 3. The proof of (b) is similar to that of (a), using (3) and (5).  We now use Lemma 4 to show Proposition 1. Proof (Proposition 1). Let P be the point set obtained by digitally shifting a digital (t, m, 3)-net over F q , generated by C1 , C2 , and (without loss of generality) ⎞ ⎛ (m−t)×t 0 E (m−t)×(m−t) ⎟ ⎜ ⎟ ⎜ ⎜cm−t+1,1 . . . cm−t+1,m ⎟ C3 = ⎜ ⎟ , ⎟ ⎜ .. .. ⎠ ⎝ . . cm,m cm,1 . . .  by shift vectors σ 1 , σ 2 , σ 3 ∈ F m q . Moreover, let Q be the digital net generated m×m 3 = E by C1 , C2 , and C and let Q be the point set that is obtained by  by σ 1 , σ 2 , σ 3 . Similar to the proof of Theorem 1 we find digitally shifting Q by Lemma 3 that Dq∗m (P ) and Dq∗m (Q) differ at most by a constant. So it remains to estimate the star discrepancy of Q. By Lemma 4, Q can also be  by τ 1 , τ 2 , and τ 3 as given in Lemma 4. That is, Q is obtained by shifting Q 3 = E m×m , digitally shifted a digital (t, m, 3)-net generated by C1 , C2 , and C only in the first two coordinates.  We obtain a We apply Proposition 2 to the first two coordinates of Q. digital (t, 2)-sequence S that shares its first q m points with the projection  If we digitally shift the m onto the first two coordinates of the points of Q.  most significant q-ary digits of the points of S by τ 1 and τ 2 , we still have a (t, 2)-sequence S, which shares its first q m points with the projection onto the first two coordinates of the points of Q. The result now follows similarly to the results in Theorem 1 by the use of (3)–(5).  We now give the proof of Theorem 2. Proof (Theorem 2). We use a technique introduced by Niederreiter (see, for example, [6, Proof of Lemma 4.11]) and further developed for digital sequences by Pillichshammer (see [7, Proof of Theorem 3]). Let x0 , x1 , x2 , . . . be a digital (t, 3)-sequence over F q generated by the ∞× ∞-matrices C1 , C2 , and C3 . Moreover, let N ≥ q t have base q representation N = b0 + b1 q + · · · + bk q k , with q-ary digits bj (0 ≤ j ≤ k) and bk = 0. Assume that r is maximal such that q r is a divisor of N . We divide the sequence into subsequences ωm,b for b = 0, . . . , bm − 1 and m = 0, . . . , k, where ωm,b contains those xn with

284

P. Kritzer k (

k (

bl q l + bq m ≤ n <

l=m+1

bl q l + (b + 1)q m .

l=m+1

For fixed m ≥ t, divide Cj , 1 ≤ j ≤ 3, into ⎛ ⎜Cj (m) ⎜ Cj = ⎜ ⎜ ⎝

⎞

Dj (m)⎟ ⎟ ⎟ , ⎟ ⎠ Ej (m)

where Cj (m) is the left upper m × m submatrix of Cj . For the xn in ωm,b we have that k m−1 ( ( n= bl q l + bq m + al q l , l=m+1

l=0

with al ∈ {0, . . . , q − 1}. Thus, ⎛

⎛

a0 a1 .. . .. .

⎞⎞

⎛

⎛

b

⎞⎞

⎛

0 0 .. . .. . 0

⎞

⎟ ⎜ ⎜bm+1 ⎟⎟ ⎜ ⎟⎟ ⎜ ⎜ ⎟ ⎜ ⎜ ⎟⎟ ⎜ ⎟⎟ ⎜ ⎜ ⎜ ⎟ ⎜ ⎜ .. ⎟⎟ ⎜ ⎟⎟ ⎜ ⎜ ⎟ ⎜ ⎜ . ⎟⎟ ⎜ ⎟⎟ ⎜ ⎜ ⎟ ⎟⎟ ⎜Dj (m) · ⎜ ⎟⎟ ⎜Cj (m) · ⎜ ⎟ ⎜ ⎜ bk ⎟ ⎟ ⎜ ⎟⎟ ⎜ ⎜ ⎜ ⎟ ⎜ ⎜ ⎟⎟ ⎜ ⎟⎟ ⎜ ⎟ ⎜ ⎜ 0 ⎟⎟ ⎜ ⎟⎟ ⎜ ⎜ ⎜ ⎟ , Cj · n = ⎜ ⎝am−2 ⎠⎟ ⊕ ⎜ ⎝ ⎠⎟ ⊕ ⎜ ⎟ ⎟ ⎜ ⎟ ⎜ .. ⎜ ⎟ ⎜ ⎟ ⎜ ⎜ am−1 ⎟ . ⎟ ⎟ ⎜ ⎟ ⎜ ⎜ ⎟ ⎟ ⎜ ⎟ ⎜ ⎜ 0 0 ⎟ ⎟ ⎜ ⎟ ⎜ ⎜ ⎟ ⎜ ⎟ ⎜Ej (m) · n⎟ ⎜ 0 0 ⎟ ⎠ ⎝ ⎠ ⎝ ⎝ ⎠ .. .. . . where n denotes the vector in F ∞ q associated with the digit vector of n. Hence ωm,b is a shifted digital (t, m, 3)-net generated by C1 (m), C2 (m), and C3 (m) ˜ m,b which is translated by a vector with coordinates less than q −m . Denote by ω the same sequence as ωm,b without the translation. Then, with Lemma 3, we find that 3 ∗ ∗ ˜ m,b Dm,b − D ≤ m , q ∗ ˜ ∗ denote the star discrepancy of ωm,b and ω where Dm,b and D ˜ m,b , respecm,b tively. We then get k b( m −1 ( m=t b=0

q

m

∗ Dm,b

≤

k b( m −1 (

qm

m=t b=0

≤3

k ( m=t

bm +

 3  ∗ ˜ m,b + D qm

k ( m=t

bm ∆q (t, m, 3) ,

Star Discrepancy of Digital Nets and Sequences in Three Dimensions

285

where we used Proposition 1 and where, inspired by the notation used in [6], ∆q (t, m, 3) denotes the upper bound in Theorem 1 (a) or (b), depending on whether q is even or not. We have ∗ N DN ≤3

k (

bm +

m=t

k (

bm ∆q (t, m, 3) +

m=t

t−1 (

bm q m .

(8)

m=0

In the next step, we consider the point set made up of those xn with N ≤ n ≤ q k+1 − 1, consisting of q k+1 − N points. Let k (

q k+1 − N =

cm q m ,

m=0

cm ∈ {0, . . . , q − 1}. We split up the sequence xN , . . . , xqk+1 −1 into subsequences ωm,c for c = 0, . . . , cm − 1 and m = 0, . . . , k, with ωm,c consisting of the xn with q k+1 −

k (

cl q l − cq m − q m ≤ n < q k+1 −

l=m+1

k (

cl q l − cq m .

l=m+1

A similar argument to the one above gives (q k+1 − N )Dq∗k+1 −N ≤ 3

k ( m=t

cm +

k (

cm ∆q (t, m, 3) +

m=t

t−1 (

cm q m .

m=0

Since k ≥ t, the first q k+1 points of our (t, 3)-sequence are a digital (t, k +1, 3)net. The first N points are just the difference between the first q k+1 points and those xn satisfying N ≤ n ≤ q k+1 − 1. Therefore, we get ∗ N DN ≤3

k (

cm +

m=t

k (

cm ∆q (t, m, 3) +

m=t

t−1 (

cm q m + ∆q (t, k + 1, 3) .

Combining (8) and (9) and making use of min {A, B} ≤ ∗ ≤ N DN

(9)

m=0 A+B 2 ,

we finally get

k k 3( 1( (bm + cm ) + (bm + cm )∆q (t, m, 3) 2 m=t 2 m=t

+

t−1 1 ( 1 (bm + cm )q m + ∆q (t, k + 1, 3) . 2 m=0 2

Since q k+1 = q k+1 − N + N =

k (

(bm + cm )q m ,

m=0

(10)

286

P. Kritzer

it follows by the definition of r that bm + cm = 0 if 0 ≤ m ≤ r − 1, br + cr = q, and bm + cm = q − 1 if r < m ≤ k. If r ≤ t − 1, (10) gives ∗ N DN

 1 (q − 1) + (q − 1)∆q (t, m, 3) ≤ 2 2 m=t   t−1 ( 1 1 r+1 m q + (q − 1)q + + ∆q (t, k + 1, 3) 2 2 m=r+1   k ( 3 1 (q − 1) + (q − 1)∆q (t, m, 3) = 2 2 m=t k  ( 3

1 1 + q t + ∆q (t, k + 1, 3) . 2 2

(11)

If r ≥ t, we get ∗ N DN

  k ( 3 1 1 3 (q − 1) + (q − 1)∆q (t, m, 3) ≤ q + q∆q (t, r, 3) + 2 2 2 2 m=r+1 1 + ∆q (t, k + 1, 3) . 2

(12)

Now we distinguish two cases. If q is even, we get, by inserting the bound in Theorem 1 (a) for ∆q (t, m, 3) into (11), and by using the fact that k and r are bounded above by log N/ log q, ∗ N DN ≤

  q t q 2 (q − 1)2 (log N )3 + O (log N )2 , 3 96(q + 1)(log q)

where the constant in the O-notation depends only on t and q. Inserting into (12) yields a similar result with the same leading term. If q is odd, we get, by inserting the bound in Theorem 1 (b) into (11), ∗ N DN ≤

  q t (q − 1)3 (log N )3 + O (log N )2 , 3 96(log q)

where the constant in the O-notation again depends only on t and q. A result with the same leading term is obtained by inserting into (12).  We conclude our observations by giving the proof of Theorem 3. Proof (Theorem 3). Let q be an arbitrary prime power satisfying q ≥ 3. Let x0 , x1 , x2 , . . . be a digital (0, 3)-sequence over F q (the existence of such a sequence is guaranteed by [5, Corollary 6.20]), and choose m ≥ 1. Then the point set consisting of the points y 0 , y 1 , . . . , y qm −1 with   i m y i := , x i , i = 0, 1, . . . , q − 1 , qm

Star Discrepancy of Digital Nets and Sequences in Three Dimensions

287

is a digital (0, m, 4)-net over F q (cf. [5, Lemma 5.15]). By [5, Lemma 8.9] and by Theorem 2 the result follows.  Remark 4. Concrete examples of (0, m, 4)-nets for which the bounds in Theorem 3 hold are given by four-dimensional nets that are sometimes referred to as Faure-Niederreiter nets. These are obtained by taking a Faure-Niederreiter sequence in dimension three and transforming it as outlined in the proof of Theorem 3. Faure-Niederreiter sequences were introduced by Faure for prime bases [3] and further generalized to prime power bases by Niederreiter [5], [6, Remark 4.52]. We thereby have found an upper bound on the star discrepancy of Faure-Niederreiter nets in dimension four which is better than the one in (7).

Acknowledgments This work was supported by the Austrian Science Foundation (FWF), Projects S8311-MAT and P17022-N12. Further, the author is grateful to J. Dick, F. Pillichshammer, and W. Ch. Schmid for valuable suggestions and discussions.

References 1. Dick, J. and Kritzer, P.: A best possible upper bound on the star discrepancy of (t, m, 2)-nets. Submitted (2004) 2. Dick, J. and Pillichshammer, F.: Multivariate integration in weighted Hilbert spaces based on Walsh functions and weighted Sobolev spaces. To appear in J. Complexity (2005) 3. Faure, H.: Discr´epance de suites associ´ees ` a un syst`eme de num´eration (en dimension s). Acta Arith., 41, 337–351 (1982) 4. Faure, H.: Discrepancy lower bound in two dimensions. In: H. Niederreiter et al. (eds.) Monte Carlo and Quasi-Monte Carlo Methods in Scientific Computing, Lecture Notes in Statistics 106, pp. 198–204. Springer, New York (1995) 5. Niederreiter, H.: Point sets and sequences with small discrepancy. Monatsh. Math., 104, 273–337 (1987) 6. Niederreiter, H.: Random Number Generation and Quasi-Monte Carlo Methods, CBMS–NSF Series in Applied Mathematics 63. SIAM, Philadelphia (1992) 7. Pillichshammer, F.: Improved upper bounds for the star discrepancy of digital nets in dimension 3. Acta Arith., 108, 167–189 (2003)

Lattice Rules for Multivariate Approximation in the Worst Case Setting Frances Y. Kuo1 , Ian H. Sloan1 and Henryk Wo´zniakowski2,3 1

2

3

School of Mathematics, University of New South Wales, Sydney NSW 2052, Australia Department of Computer Science, Columbia University, New York, NY 10027, USA Institute of Applied Mathematics and Mechanics, University of Warsaw, ul. Banacha 2, 02-097 Warszawa, Poland [email protected] [email protected] [email protected]

Summary. We develop algorithms for multivariate approximation in weighted Korobov spaces of smooth periodic functions of d variables. Our emphasis is on large d. The smoothness of functions is characterized by the parameter α > 1 that controls the decay of Fourier coefficients in the L2 norm. The weight γj of the Korobov space moderates the behaviour of functions with respect to the jth variable. Small γj means that functions depend weakly on the jth variable. We study algorithms that are linear combinations of trigonometric polynomials of the function values computed at lattice points with a generating vector z. We study two choices of the vector z. The first choice is a vector computed by a known component-by-component construction for multivariate integration, see e.g., [14, 15, 17]. The second choice, proposed in this paper, also selects a vector computed by a component-by-component construction, but one which at each step minimizes an upper bound on the multivariate approximation error. The second choice leads to a better bound on the approximation error. We study tractability and strong tractability of the approximation problem in the worst case setting. That is, we want to guarantee that the minimal number of function values needed to compute an ε approximation depends polynomially on ε−1 and polynomially on the number d of variables (tractability) or is independent of d (strong tractability). Necessary and sufficient conditions on tractability and strong tractability are known, see [12, 25]. These results have been proved by a non-constructive argument, and no constructive algorithms achieving tractability or strong tractability error bounds have been known. We provide such a construction in this paper. We prove that algorithms based on lattice rules with two choices of the generating vectors achieve tractability or strong tractability error bounds. We believe, however, that the exponents of ε−1 corresponding to our error bounds are not sharp.

290

F.Y. Kuo et al.

1 Introduction We are interested in the approximation of functions f : [0, 1]d → R where the number d of variables is large. Our study is for periodic functions belonging to certain weighted Korobov spaces. These spaces are defined in terms of some smoothness parameter α which controls the decay of Fourier coefficients. We will be measuring this decay in the L2 sense. Formally it is enough to assume that α ≥ 0. However, α > 1 is necessary and sufficient to guarantee that function values are well defined, and the corresponding space is then a reproducing kernel Hilbert space. Since we will be studying algorithms that use function values we assume that α > 1. Traditionally, Korobov spaces are unweighted in the sense that all variables are of equal importance. Here, as in [5, 12, 19], we study weighted Korobov spaces in which successive variables may have diminishing importance. This is achieved by introducing weights, that is, a sequence of parameters γ = (γ1 , γ2 , . . .), with each γj moderating the behavior of the jth variable. If all γj = 1 then we have the classical (unweighted) Korobov space. If, however, γj is small then the dependence on the jth variable is weak. In particular, in the limiting case, if γj = 0 then functions do not depend on the jth variable. The main goal of our paper is to develop constructive algorithms for approximation that use lattice rules for which the worst case error satisfies tractability or strong tractability error bounds in weighted Korobov spaces. Tractability means that the minimal number n(ε, d) of function evaluations required to achieve an error ε is bounded polynomially in ε−1 and d; strong tractability means that this bound is independent of d. Tractability and strong tractability depend on the problem setting and on the type of information used by algorithms, see e.g., [12, 24, 25]. In this paper we consider the worst case setting, and focus mainly on standard information, i.e., on function values. Work on the average case setting is left for a future paper. Lattice rules are a family of equal-weight quadrature rules traditionally used for multivariate integration, see [16]. Recently there have been tremendous advances in the construction of lattice rules for integration, see e.g., [14, 15, 17]. The parameters characterizing these lattice rules are constructed component by component: each component is obtained by successive 1-dimensional searches, with all the previous components kept unchanged. The relatively small cost of these algorithms makes them feasible for application in practice. In particular, we can construct n points of lattice rule in the d-dimensional case using O(d n log n) operations as recently shown in [13]. Approximation for unweighted Korobov spaces has been studied in many papers. The optimal rate of convergence has been found if the approximation error is defined in the Lp sense, see papers of Temlyakov, [20–22], as well as papers cited there. The use of lattice rules for approximation has already been suggested in [7] and in papers cited there. It was done for the unweighted Korobov spaces with decaying Fourier coefficients in the L∞ sense. The recent papers [11, 26]

Lattice Rules for Multivariate Approximation in the Worst Case Setting

291

also used lattice rules for approximation in unweighted Korobov spaces. The algorithm in [11] is similar to our algorithm discussed below, but the error criterion is quite different. The paper [26] uses spline interpolations based on lattice points and the error measure is an upper bound to the worst-case error defined in L2 . Both papers study the unweighted case for which tractability does not hold. We will use lattice rules for approximation and for weighted Korobov spaces with decaying Fourier coefficients in the L2 sense. The case of L∞ is left for a future research. Lattice rules of rank-1 are characterized by an integer vector z. We study two choices of the vector z. Since multivariate approximation of a function f can be viewed as approximation of a number of its Fourier coefficients, and since each Fourier coefficient is a multivariate integral of a function related to f , it is clear that multivariate approximation can be solved by known algorithms for multivariate integration. This motivates our first choice z 1 of the generating vector z, as a vector computed by the known component-bycomponent construction for multivariate integration. Not surprisingly, the first choice can be improved by taking into account more properties of the original problem of multivariate approximation. This is the essence of the second choice z 2 of the generating vector z, which we now explain. We first find the exact expression of the worst case error of an algorithm using n function values at lattice points for an arbitrary generating vector z. The essential part of the worst case error is the spectral radius of a non-negative-definite symmetric matrix Tz . The matrix Tz is of a special form, has many zero elements, and its non-zero elements are given as infinite series depending on the vector z and the weights γ. We estimate the spectral radius by the trace of the matrix Tz . This is probably an overestimate, and gives us reason to believe that our results are not best possible. We then compute the average value of the trace of Tz over all possible generating vectors z. It turns out that the component-by-component construction yields a generating vector z 2 with error of the same order as the average of Tz . The cost of computing the vector z 2 is polynomial in d and n under a suitable assumption on the weights γ as explained in Sect. 6.2. Necessary and sufficient conditions on tractability and strong tractability are holds iff 1∞known in the worst case setting, see [12, 25]. Strong tractability std ∗ ∗ γ < ∞. Then there exist positive number C and p ∈ [p , p + 2], j j=1 1∞ with p∗ = 2 max(α−1 , sγ ) and sγ = inf{s > 0 : j=1 γjs < ∞}, such that n(ε, d) ≤ C ε−p

std

∀ ε ∈ (0, 1) ∀ d ≥ 1 .

(1)

The smallest value of pstd is called the exponent of strong tractability, and is not known. The proof of (1) in [12,25] is non-constructive, and no constructive algorithms achieving strong tractability error bounds are presently known. We provide such a construction in this paper. We prove that algorithms based on lattice rules with the above two choices of the generating vector achieve a strong tractability error bound. More precisely, the lattice rule with

292

F.Y. Kuo et al.

the generating vector z j , for j = 1, 2, solves the approximation problem using O(ε−pj ) function values, with the implied factor in the big O notation independent of ε and d, and with the exponents arbitrarily close to p1 ≈ 2p∗ + p2 ≈ 2p∗ ,

p∗ 2 , 2

see Theorem 1 , see Theorem 3 .

√

√ ∗ ∗ Note that √ p1 ≤ p + 2 only if p ≤ 5 − 1, that is when α ≥ ( 5 + 1)/2 and sγ ≤ ( 5−1)/2. As we see, the exponent for the second choice is always better, and always satisfies p2 ≤ p∗ + 2 since p∗ ≤ 2. Hence, we obtain constructive lattice rules which achieve a strong tractability error bound that is better than the non-constructive results known previously. However, we believe that a better exponent than 2p∗ is possible. In fact, we conjecture that p∗ is the exponent of strong tractability. 1d We now discuss tractability, which holds iff a := lim supd→∞ j=1 γj / log(d + 1) < ∞. Then it is proved in [25] that there exists a positive C such that std ∀ ε ∈ (0, 1) ∀ d ≥ 1 , n(ε, d) ≤ Cε−p d q with pstd arbitrarily close to 4 and q arbitrarily close to 2ζ(α) a, where ζ(α) is the Riemann zeta function. Again, the smallest values of pstd and q are not known, and the proof of this result is non-constructive. We prove that algorithms based on lattice rules with two choices of the generating vectors z j achieve a tractability error bound. More precisely, they solve the approximation problem using O(ε−pj dqj ) function values, with the implied factor in the big O notation independent of ε and d, and with the exponents p1 = 6 and p2 = 4, and the exponents qj arbitrarily close to 4ζ(α) a,

see Theorem 1 and Theorem 3 .

As before, we believe that these exponents can be lowered. We finally summarize the content of this paper. In Sect. 2 we precisely define the approximation problem. In Sect. 3 we review known facts needed for our analysis. These relate to weighted Korobov spaces, lattice rules for integration, and a component-by-component algorithm for generating the vector z. Tractability and strong tractability are reviewed in Sect. 3.4. We derive the worst case error of the lattice rule algorithm with an arbitrary generating vector z for the approximation problem in Sect. 4. Section 5 deals with the first choice, whereas Sect. 6 deals with the second choice of the generating vector z. The component-by-component algorithm for the approximation problem is given in Sect. 6.2. Proofs of two lemmas are given in the final Sect. 7.

Lattice Rules for Multivariate Approximation in the Worst Case Setting

293

2 Formulation of the Problem We want to approximate periodic functions f belonging to some weighted Korobov space Hd whose norm depends on a smoothness parameter α, α > 1, and a sequence γ = (γ1 , γ2 , . . .) of positive weights. Details of the function spaces are given in Sect. 3. For now we just need the fact that Hd is embedded in L2 ([0, 1]d ). Multivariate approximation, or simply approximation, is defined in terms of the operator which is the embedding from the weighted Korobov space Hd to the space L2 ([0, 1]d ), i.e., EMBd : Hd → L2 ([0, 1]d ) is given by EMBd f = f . If point evaluation is a bounded linear functional in Hd , then we can approximate EMBd f by linear algorithms4 of the form An,d (f ) =

n (

ak f (tk )

(2)

k=1

for some functions ak ∈ L2 ([0, 1]d ), and deterministically chosen sample points tk ∈ [0, 1]d . The main emphasis in this paper is on algorithms given by lattice rules. Lattice rules are usually used for multivariate integration, see [16] and papers cited there. Their use for approximation was first suggested in [7] and papers cited there. In this paper, we use lattice rules for approximation as a tool to obtain tractability results. This idea has already been used in [12]. We now briefly introduce the idea how lattice rules can be used for approximation. Any function f in Hd can be expressed as the Fourier series ( f (x) = fˆ(h)e2πih·x , h∈Zd

where fˆ(h) is the usual Fourier coefficient  f (x)e−2πih·x dx . fˆ(h) =

(3)

[0,1]d

Here h = (h1 , . . . , hd ) ∈ Zd , x = (x1 , . . . , xd ) ∈ Rd , and h · x = h1 x1 + · · · + hd xd is the usual dot product. Let γ = (γ1 , γ2 , . . .) be a given sequence of real numbers (the weights), and let α > 1 be a fixed number. For h = (h1 , . . . , hd ) ∈ Zd , let

d  1 if hj = 0, (4) rα (γ j , hj ) with rα (γ j , hj ) = |hj |α rα (γ, h) = otherwise, γj j=1 4

It is known that nonlinear algorithms as well as adaptive choice of sample points tk do not help in decreasing the error, see e.g., [23].

294

F.Y. Kuo et al.

and define A(d, M ) := {h ∈ Zd : rα (γ, h) ≤ M } .

(5)

Properties of this set are studied in Sect. 3.3. For M ≥ 1, we approximate f by first dropping terms in the Fourier series with h ∈ / A(d, M ), and then approximating the Fourier coefficients fˆ(h) for h ∈ A(d, M ) by an n-point rank-1 lattice rule with generating vector z. Recall that a rank-1 lattice rule approximates the d-dimensional integral of a function g by  n 1 (  kz  , g(x) dx ≈ g n n [0,1]d k=1

where z is an integer vector having no factor in common with n, and the braces around the vector indicate that we take the fractional part of the components of the vector. A quick review of lattice rules for integration is given in Sect. 3.2. Since fˆ(h) is the integral of g(x) = f (x)e−2πih·x , it seems natural to apply lattice rules for such g. In this way, we approximate f by  n  ( 1 (  kz  −2πih· kz n An,d,M (f )(x) = e f (6) e2πih·x . n n h∈A(d,M )

k=1

That is, f is approximated by a linear algorithm of the form (2), with (   kz 1 ak (x) = for k = 1, 2, . . . , n . e2πih·(x− n ) and tk = kz n n h∈A(d,M )

The error for the approximation is clearly ( (f − An,d,M (f ))(x) = fˆ(h)e2πih·x h∈A(d,M / )

(

+

h∈A(d,M )



1 (  kz  −2πih· kz n e f fˆ(h) − n n n

 e2πih·x .(7)

k=1

In Sects. 5 and 6 we introduce two algorithms for constructing the generating vector z. The first algorithm is actually an existing algorithm for choosing z for integration in the worst case setting. The second algorithm is better tuned for approximation, choosing z to minimize a certain approximation error bound. For both algorithms we study their errors and show that they achieve strong tractability and tractability error bounds.

3 Preliminaries 3.1 Weighted Korobov Spaces The weighted Korobov space Hd = Hd,α,γ is a weighted tensor-product reproducing kernel Hilbert space of 1-periodic real-valued L2 -functions defined on [0, 1]d . Here α > 1 (thus the functions in Hd are continuous) and

Lattice Rules for Multivariate Approximation in the Worst Case Setting

295

γ = (γ1 , γ2 , . . .) is a sequence of weights satisfying 1 ≥ γ1 ≥ γ2 ≥ · · · > 0 . The inner product in Hd is given by ( rα (γ, h)fˆ(h)ˆ g (h) , f, gd = h∈Zd

where rα (γ, h) and the Fourier coefficients fˆ(h) are given by (4) and (3) 1

respectively. The norm in Hd is  · d = ·, ·d2 . Note that since rα (γ, h) ≥ 1, we have ⎞ 12 ⎛ ( |fˆ(h)|2 ⎠ ≤ f d for all f ∈ Hd . f L2 ([0,1]d ) = ⎝ h∈Zd

These spaces have previously been studied in a number of papers, see e.g., [5, 12, 19]. The parameter α measures the rate of decay of the Fourier coefficients, and is intimately related to the smoothness of the functions. For our analysis in this paper we need to assume that point evaluation is a bounded linear functional in Hd , which leads to the condition that α > 1. In general, one may consider weighted Korobov spaces with α ≥ 0. For α = 0, the definition of the inner product is replaced by f, gd = f, gL2 ([0,1]d ) , see [12] for details. The notion of weights was first introduced in [18] to model the situation in which successive variables have diminishing importance. The weight γj moderates the behavior of the functions with respect to the jth variable. If γj = 1 for all j then all variables are of equal importance and we recover the classical unweighted Korobov spaces. Here we have imposed the condition that all weights are at most 1 for simplicity of our subsequent analysis. An alternative would be to assume that the weights are uniformly bounded. One may also wish to consider a more general setting where the weights are allowed to depend on the dimension d, as in [4]. This will be the subject of our future study. For α1 ≤ α2 and fixed d and γ, we have Hd,α2 ,γ ⊆ Hd,α1 ,γ and f d,α1 ,γ ≤ f d,α2 ,γ . Thus the unit ball of Hd,α2 ,γ is contained in the unit ball of Hd,α1 ,γ . On the other hand, for fixed d and α and different sequences of weights γ and η, the spaces Hd,α,γ and Hd,α,η are algebraically the same, with different but equivalent norms. Moreover, if γj ≥ ηj for all j, then f d,α,γ ≤ f d,α,η , and hence the unit ball of Hd,α,η is contained in the unit ball of Hd,α,γ . As in [24], we define the sum-exponent of γ by ∞ (   γjs < ∞ , sγ := inf s > 0 : j=1

with the convention that inf ∅ = ∞. As we shall see, we will need to assume that sγ < ∞ and sometimes that sγ ≤ 1.

296

F.Y. Kuo et al.

The space Hd with α > 1 has a reproducing kernel given by Kd (x, y) =

( e2πih·(x−y) . rα (γ, h) d

(8)

h∈Z

Recall that a reproducing kernel Kd of Hd is a function satisfying Kd (x, y) = Kd (y, x) for all x, y ∈ [0, 1]d , Kd (·, y) ∈ Hd for all y ∈ [0, 1]d , and most importantly, that the reproducing property f (y) = f, Kd (·, y)d

∀f ∈ Hd , ∀ y ∈ [0, 1]d

holds. All these properties are easily verified for the given kernel (8). 3.2 Lattice Rules for Multivariate Integration Lattice rules are a family of equal-weight quadrature rules traditionally used for approximating the d-dimensional integral  Id (f ) = f (x) dx [0,1]d

over the unit cube [0, 1]d , see [16]. A rank-1 lattice rule with n points is of the form n 1 (  kz  Qn,d (f ) = , f n n k=1

where z is an integer vector known as the generating vector, and the braces around the vector indicate that we take the fractional part of each component of the vector. We shall require that every component of z is coprime with n. For simplicity of the analysis, n is often assumed to be prime, and z is then restricted to have components from the set Zn := {1, 2, . . . , n − 1} . It is well-known that the integration error for rank-1 lattice rule satisfies ( Id (f ) − Qn,d (f ) = − fˆ(h) . h∈Zd \{0} h·z≡0 (mod n)

The worst case integration error of Qn,d in the space Hd is defined as wor-int en,d :=

sup f ∈Hd , f d ≤1

|Id (f ) − Qn,d (f )| .

(9)

There is an explicit expression for the squared worst case error in terms of the reproducing kernel of Hd . For our weighted Korobov spaces, the squared

Lattice Rules for Multivariate Approximation in the Worst Case Setting

297

worst case error for a rank-1 lattice rule with generating vector z is given by, see e.g. [19],  wor-int 2 en,d (z) n    1( ,0 = −1 + Kd kz n n k=1   d n−1 d ∞ ( 1 e2πikzj /n 1 ( = −1 + (1 + 2ζ(α)γj ) + 1 + γj , (10) n j=1 n ||α j=1 k=1

=−∞  =0

1∞ where ζ(x) := =1 −x , x > 1, is the Riemann zeta function. It is natural to seek a generating vector z for which the worst case error is minimized. When n is prime there are (n − 1)d possible choices for z, and thus an exhaustive search is practically impossible for large n and d. For this reason, a restricted form of z proposed by Korobov [6] is commonly used, namely 1≤a≤n−1. z = (1, a, a2 , . . . , ad−1 ) mod n, With this restriction, one only needs to search through n − 1 choices of z, instead of (n − 1)d . Recently, a number of component-by-component construction algorithms have been developed. The generating vector z is constructed one component at a time: for s = 2, 3, . . . , d, the sth component of z is obtained by a search through n − 1 possible values of zs to minimize the worst case error, while the previous s−1 components are fixed. The most basic form of such an algorithm is given below. For other variants, see [2, 3, 9, 10, 14, 15, 17]. Algorithm 1 Let n be a prime number. 1. Set z1 = 1. 2. For s = 2, 3, . . . , d, and already known z1 , z2 , . . . , zs−1 , find zs in the set wor-int (z1 , . . . , zs−1 , zs )]2 . {1, 2, . . . , n − 1} which minimizes [en,s

In practice, it is usual to take α to be an even integer, since then the infinite sum in the last term of the squared worst case error (10) can be expressed as ∞ 8 9 ( e2πikzj /n (2π)α kzj , B = α α n ||α (−1) 2 +1 α!

(11)

=−∞  =0

where Bα is the Bernoulli polynomial of degree α. For an even integer α, the cost of the algorithm is O(n2 d2 ) operations, which can be reduced to O(n2 d) operations at the expense of O(n) storage. In the recent paper [13], the cost ˆ is reduced to O(d n log n) operations. of computing the generating vector z

298

F.Y. Kuo et al.

ˆ constructed by this It is proved in [8] and [1] that the generating vector z algorithm satisfies d   1  wor-int 2 1 1 + 2ζ(αλ)γjλ λ z ) ≤ (n − 1)− λ en,d (ˆ

(12)

j=1

for all λ satisfying

1 α

< λ ≤ 1.

3.3 The Set A(d, M ) The set A(d, M ) defined by (5) plays an important role in our subsequent analysis. In the lemma below we present some properties of the set A(d, M ). Lemma 1. For d ≥ 1 and M ≥ 0, let A(d, M ) := {h ∈ Zd : rα (γ, h) ≤ M } and let |A(d, M )| denote the cardinality of A(d, M ). 1

1

(a) If h ∈ A(d, M ), then |hj | ≤ (γj M ) α ≤ M α for all j = 1, 2, . . . , d. ∞8 9  E γd+1 M (h, hd+1 ) : h ∈ A d, |h . (b) A(d+1, M ) = {(h, 0) : h ∈ A(d, M )}∪ α | d+1 hd+1=−∞ hd+1 =0

(c) |A(d + 1, M )| = |A(d, M )| + 2

∞   ( γ M . A d, hd+1 α d+1

hd+1 =1

(d) For arbitrary q >

1 α,

we have |A(d, M )| ≤ M q

d  

 1 + 2ζ(αq)γjq .

j=1

Proof. (a) Recall that rα (γ, h) ≥ 1. We will therefore assume that M ≥ 1, for otherwise the set A(d, M ) is empty and there is nothing to prove. By 2d the definition of A(d, M ), if h ∈ A(d, M ) then j=1 rα (γj , hj ) ≤ M , and in particular, each factor rα (γj , hj ) in the product must not exceed M . This |h |α holds if hj is 0. For hj = 0, we have γjj ≤ M , which leads to the desired result. (b) Again without lost of generality we will assume that M ≥ 1. Let (h, hd+1 ) ∈ A(d + 1, M ) where h is a d-dimensional vector. Depending on the value of hd+1 , we will put a constraint on h. By definition we require that rα (γ, h)rα (γd+1 , hd+1 ) ≤ M . When hd+1 = 0, this simplifies to rα (γ, h) ≤ M γd+1 M which is equivalent to h ∈ A(d, M ). For hd+1 = 0, we have rα (γ, h) ≤ |h α, d+1 |  γd+1 M  which means that h ∈ A d, |hd+1 |α . The result should now be apparent. (c) This follows immediately from (b) since all the sets in the formula (b) are disjoint and the cardinality of the set with hd+1 = 0 is the same as the set with −hd+1 . (d) We prove this result by a simple induction argument. For d = 1, it is clear from our earlier arguments that

Lattice Rules for Multivariate Approximation in the Worst Case Setting

1 1 + 2#(γ1 M ) α $ |A(1, M )| = 0

299

if M ≥ 1, if M < 1

≤ (1 + 2ζ(αq)γ1q ) M q for any q > α1 , where we used ζ(x) > 1 ∀ x > 1. Suppose now that the bound is true for |A(d, M )| and consider the cardinality |A(d + 1, M )|. Using the recurrence (c) and the bound for |A(d, M )| with varying M ’s, we have |A(d + 1, M )| ≤ M q

d  

d ∞  q  (    γd+1 M 1 + 2ζ(αq)γjq +2 1 + 2ζ(αq)γjq hα d+1

j=1

j=1

hd+1 =1

 q  q M = 1 + 2ζ(αq)γd+1

d 



 1 + 2ζ(αq)γjq = M q

j=1

d+1 



 1 + 2ζ(αq)γjq ,

j=1

which is exactly the bound we are looking for. This completes the proof. 1

Later we will assume that n is larger than M α . For example, let n ≥ 1 κM α with a fixed number κ > 1. Then it follows from Lemma 1(a) that the components of h ∈ A(d, M ) will be restricted to |hj | ≤ κ−1 n for all j = 1, 2, . . . , d. 3.4 Approximation in Weighted Korobov Spaces We now take a step back and discuss a general approximation problem for weighted Korobov spaces Hd in the worst case setting, see e.g., [12]. Suppose that we approximate f by a linear algorithm of the form An,d (f ) =

n (

ak Lk (f ) ,

k=1

where the ak ’s are some functions from L2 ([0, 1]d ), and the Lk ’s are some continuous linear functionals defined on Hd . Suppose that Lk ∈ Λ, where Λ is either Λall or Λstd . Here Λall consists of all continuous linear functionals, that is Λall = Hd∗ , whereas Λstd , known as the class of standard information, consists only of function evaluations, i.e., Lk ∈ Λstd iff there exists tk ∈ [0, 1]d such that Lk (f ) = f (tk ) for all f ∈ Hd . In particular, the approximation given by (6) uses only standard information from Λstd . The worst case error of the algorithm An,d is defined as ewor n,d :=

sup f ∈Hd , f d ≤1

f − An,d (f )L2 ([0,1]d ) .

The initial error associated with A0,d = 0 is ewor 0,d :=

sup f ∈Hd , f d ≤1

f L2 ([0,1]d ) .

300

F.Y. Kuo et al.

Since f L2 ([0,1]d ) ≤ f d for all f ∈ Hd , and for f = 1 we have f d = f L2 ([0,1]d ) = 1, we conclude that the initial error ewor 0,d = 1. For ε ∈ (0, 1) and d ≥ 1, define wor nwor (ε, Hd , Λ) := min{ n : ∃ An,d with Lk ∈ Λ such that ewor n,d ≤ εe0,d }

as the smallest number of evaluations from Λ needed to reduce the initial error by a factor ε. As in [12], we say that the approximation problem for weighted Korobov spaces Hd in the worst case setting is tractable in the class Λ iff there are non-negative numbers C, p and q such that nwor (ε, Hd , Λ) ≤ C ε−p d q

∀ ε ∈ (0, 1) and ∀ d ≥ 1 .

(13)

The numbers p and q are called ε- and d-exponents of tractability; we stress that they are not defined uniquely. The approximation problem is strongly tractable iff (13) holds with q = 0. In this case, the infimum of the numbers p is called the exponent of strong tractability, and is denoted by pwor (Λ). In the class Λall , we can compute the Fourier coefficients exactly, and it is well known that the optimal5 algorithm is, see e.g., [23], ( fˆ(h)e2πih·x , An,d (f )(x) = h∈A(d,ε−2 )

with n = |A(d, ε−2 )|, and the worst case error is at most ε. Indeed, we have in this case  2 ( f − An,d (f )2L2 ([0,1]d ) = fˆ(h)e2πih·x dx [0,1]d

=

(

−2 ) h∈A(d,ε /

|fˆ(h)|2

−2 ) h∈A(d,ε /

=

(

−2 ) h∈A(d,ε /

|fˆ(h)|2 rα (γ, h)

1 ≤ f 2d ε2 , rα (γ, h)

with the last step following from the definition (5) for A(d, ε−2 ). Thus we have ewor n,d ≤ ε. It is proved in [24] that strong tractability and tractability in the class Λall are equivalent and they hold6 iff sγ < ∞, in which case the exponent of strong tractability is   pwor (Λall ) = 2 max α1 , sγ . 5

6

Optimality means that the worst case error of An,d is minimal in the class of all algorithms using n evaluations from Λall . For the class Λall we can also consider weighted Korobov spaces with α ≥ 0. Then strong tractability and tractability hold iff α > 0 and sγ < ∞, see [12]. Since the main emphasis in this paper is on lattice rules that use information from the class Λstd we need to assume the stronger condition that α > 1.

Lattice Rules for Multivariate Approximation in the Worst Case Setting

301

For the class Λstd , it is proved in [12] that strong tractability holds iff ∞ (

γj < ∞ .

(14)

j=1

When (14) holds, we have sγ ≤ 1, and it is proved in [25] via a non-constructive argument that the exponent of strong tractability satisfies pwor (Λstd ) ∈ [ pwor (Λall ), pwor (Λall ) + 2 ] . Clearly, in this case pwor (Λall ) ≤ 2 and therefore pwor (Λstd ) ≤ 4. It is also proved in [12] that tractability in the class Λstd holds iff 1d a := lim sup d→∞

j=1

γj

log(d + 1)

<∞.

Moreover, it is proved in [25] using a non-constructive argument that for any δ > 0, nwor (ε, Hd , Λstd ) ≤ Cε−(4+δ) d 2ζ(α)a+δ

∀ ε ∈ (0, 1) and ∀ d ≥ 1 .

A summary of tractability results for approximation in weighted Korobov spaces in the worst case setting can be found in Theorem 1 of [12]. One of our aims is to find a lattice rules algorithm which leads in a constructive way to strong tractability error bounds in the worst case setting with the smallest possible exponent of ε−1 .

4 Worst Case Error For a prime n, let ewor n,d,M (z) denote the worst case error for our proposed approximation An,d,M given by (6). It follows from (7) and the orthonormal property of {e2πih·x }h in L2 that f − An,d,M (f )2L2 ([0,1]d )  = |(f − An,d,M (f ))(x)|2 dx [0,1]d

(

=

|fˆ(h)|2

h∈A(d,M / )

+

( h∈A(d,M )

 2 n 1 (  kz  −2πih· kz −2πih·x n e f (x)e dx − f .(15) n [0,1]d n k=1

The first term of the error in (15) can be bounded by

302

F.Y. Kuo et al.

(

(

|fˆ(h)|2 =

h∈A(d,M / )

|fˆ(h)|2 rα (γ, h)

h∈A(d,M / )

1 1 ≤ f 2d . rα (γ, h) M

We need to find a bound for the second term of the error in (15), which clearly depends on the choice of z. Using the reproducing property of the kernel Kd , we can write the expression between the absolute value signs in the second term of (15) as  n 1 (  kz  −2πih· kz n = f, τ  , e f (x)e−2πih·x dx − f h d n n [0,1]d k=1

where  τh (t) =

   2πih· kz 1( n e Kd t, kz n n n

Kd (t, x)e2πih·x dx − [0,1]d

k=1

e2πih·t − = rα (γ, h)

(

(

e2πiq·t = − rα (γ, q)

q∈Zd (h−q)·z≡0 (mod n)

e2πiq·t . rα (γ, q)

q∈Zd \{h} (h−q)·z≡0 (mod n)

Due to equality in (15) we thus have 

2 β ewor + sup = n,d,M (z) M f ∈Hd , f d ≤1

(

2

|f, τh d |

for some β ∈ [0, 1] .

h∈A(d,M )

Let Tz denote the |A(d, M )| × |A(d, M )| non-negative-definite matrix defined by Tz := (τh , τp d ) . Then sup f ∈Hd , f d ≤1

(

2

|f, τh d | = ρ(Tz ) ,

h∈A(d,M )

where1ρ(Tz ) denotes the spectral radius of Tz . Indeed, it is enough to consider f = h∈A(d,M ) xh τh for complex numbers xh . Then f 2d = (Tz x, x) with x being the |A(d, M )|-vector with components xh , and with the usual dot product (·, ·). On the other hand, ( 2 |f, τh d | = (Tz x, Tz x) . h∈A(d,M )

We need to find supx: (Tz x,x)≤1 (Tz x, Tz x) = supy: (y,y)≤1 (Tz y, y) = ρ(Tz ), as claimed. We now find a more explicit form of the matrix Tz . Since the Fourier coefficients of τh satisfy

0 if q = h or (h − q) · z ≡ 0 (mod n), τˆh (q) = 1 if q = h and (h − q) · z ≡ 0 (mod n), − rα (γ,q)

Lattice Rules for Multivariate Approximation in the Worst Case Setting

303

we obtain τh , τp d =

( q∈Zd

⎧ 0 ⎪ ⎪ ⎨ =

(

rα (γ, q)ˆ τh (q)ˆ τp (q) =

q∈Zd \{h,p} (h−q)·z≡0 (mod n) (p−q)·z≡0 (mod n)

(

⎪ ⎪ ⎩∈Zd \{0,p−h}

1 rα (γ, q)

if (h − p) · z ≡ 0 (mod n), 1 rα (γ, h + )

if (h − p) · z ≡ 0 (mod n) . (16)

·z≡0 (mod n)

Our analysis leads to the following lemma. Lemma 2. The worst case error for the algorithm An,d,M defined by (6) satisfies   12 β wor + ρ(Tz ) for some β ∈ [0, 1] , en,d,M (z) = M where Tz is a non-negative-definite symmetric |A(d, M )| × |A(d, M )| matrix with entries given by τh , τp d in (16) for h, p ∈ A(d, M ).

5 First Approach: Lattice Rules Constructed for Integration In this section, we use lattice rules with the generating vectors constructed for the integration problem by Algorithm 1, and take the approximation defined by (6). We analyze the worst case error of the algorithm An,d,M by estimating the second term of (15) in a different way than in the previous section. From the definition (9) for the worst case integration error of any n-point quadrature rule Qn,d in the space Hd , the integration error for any function f ∈ Hd is bounded by the worst case error of Qn,d times the norm of f . Now the expression between the absolute value signs in the second term of (15) is exactly the integration error, using the n-point rank-1 lattice rule with generating vector z, for the function fh (x) = f (x)e−2πih·x . Thus we have  n 1 (  kz  −2πih· kz −2πih·x n ≤ ewor-int (z)f  , e f (x)e dx − f h d n,d n [0,1]d n k=1 (17) -int (z)]2 is given by (10), and if z = z ˆ is constructed by Algorithm 1 where [ewor n,d wor-int (ˆ z )]2 is estimated by (12). then [en,d

304

F.Y. Kuo et al.

The norm of fh can be estimated as in [12]: ( |fˆh ()|2 rα (γ, ) fh 2d = ∈Zd

=

(

|fˆ(h + )|2 rα (γ, h + )

∈Zd

≤ f 2d rα (γ, h)

d 

rα (γ, ) rα (γ, h + )

max(1, 2α γj ) ,

(18)

j=1

which follows from the estimate proved in [12], d  rα (γ, ) ≤ rα (γ, h) max(1, 2α γj ) . rα (γ, h + ) j=1

ˆ constructed by Algorithm 1 in the lattice rule, then it follows from If we use z (12), (17) and (18) that  2 n (   kˆ z 1 −2πih· n z e f (x)e−2πih·x dx − f kˆ n [0,1]d n k=1 h∈A(d,M ) ⎛ ⎞⎛ ⎞ d d   ( 1   1 ⎝ 1 + 2ζ(αλ)γjλ λ ⎠ ⎝ ≤ max(1, 2α γj )⎠ f 2d rα (γ, h) 1 (n − 1) λ j=1 j=1 h∈A(d,M ) ⎛ ⎞⎛ ⎞ d d  1  M |A(d, M )| ⎝   ≤ 1 + 2ζ(αλ)γjλ λ ⎠ ⎝ max(1, 2α γj )⎠ f 2d 1 (n − 1) λ j=1 j=1 (

< λ ≤ 1. Hence we have ⎛ ⎞ d d  1  wor  2 M |A(d, M )| ⎝   1 λ λ⎠ en,d,M (ˆ + 1 + 2ζ(αλ)γ z) ≤ max(1, 2α γj ) 1 j M (n − 1) λ j=1 j=1

for all λ satisfying

1 α

for all λ satisfying

1 α

d  j=1

< λ ≤ 1. Note that ˆγ ,d) min(k α

ˆγ ,d) α min(k

max(1, 2 γj ) = 2



γj ,

j=1

where the index kˆγ is defined as follows. If there exists an index k such that γk+1 ≤ 21α < γk then kˆγ = k, and if such k does not exist then kˆγ = ∞. We can then estimate the cardinality of the set A(d, M ) by Lemma 1(d) and obtain the following result.

Lattice Rules for Multivariate Approximation in the Worst Case Setting

305

Lemma 3. Let n be prime and suppose that kˆγ is defined as above. Then ˆ ∈ Znd the worst case error for the algorithm An,d,M defined by (6) with z constructed by Algorithm 1 satisfies  wor 2 Cd,q,λ M q+1 1 + en,d,M (ˆ z) ≤ 1 M (n − 1) λ where Cd,q,λ

for all q >

1 α

and

1 α

<λ≤1,

⎞  min(kˆγ ,d) ⎛ d d       1 ˆ 1 + 2ζ(αq)γjq ⎠ 1 + 2ζ(αλ)γjλ λ = 2α min(kγ ,d) γj ⎝ j=1

j=1

j=1

⎞ ⎛  min(kˆγ ,d) d d  ( ( 2ζ(αλ) ˆ ≤ 2α min(kγ ,d) γj exp ⎝2ζ(αq) γjq + γjλ ⎠ . λ j=1 j=1 j=1 In the last estimate of Lemma 3 we used ⎞ ⎛ ⎞ ⎛ d d d  ( ( (1 + xj ) = exp ⎝ log(1 + xj )⎠ ≤ exp ⎝ xj ⎠ , j=1

j=1

j=1

which holds for any non-negative xj . We want to find small M and n for which the upper bound of the error z ) given in Lemma 3 is at most ε. Clearly, M = M (n) should be chosen ewor n,d,M (ˆ with respect to n such that the two terms in the error bound are of the same order, and the choice of n depends on values of q and λ. Furthermore, n must be prime since Algorithm 1 works only for prime numbers. The requirement that n be prime is easy to satisfy at the expense of multiplying n by at most two. Indeed, let us define the function pr(x) as the smallest prime number which is no less than x. Here x is an arbitrary positive number. Clearly, &x' ≤ pr(x) ≤ 2&x' since7 there is a prime number in the interval [k, 2k] for any positive integer k. Hence if n∗ (ε) is needed to guarantee that the error is at most ε, it is enough to take n = pr(n∗ (ε)), and then n is prime and is of the same order as n∗ (ε). We are ready to define appropriate M and n. Let p∗ = pwor (Λall ) = 2 max α1 , sγ . Suppose we have ∞ (

γj < ∞ ,

(19)

j=1

which is the necessary and sufficient condition for strong tractability in the class Λstd . Clearly this condition implies sγ ≤ 1 and thus p∗ ≤ 2. If sγ = 1, then p∗ = 2 and we take q = λ = 1 in Lemma 3 to obtain  wor 2 Cd,1,1 M 2 1 CM 2 1 + ≤ + , en,d,M (ˆ z) ≤ M n−1 M n−1 7

This is known as “Bertrand’s postulate”, proved by Chebyshev in 1850.

306

F.Y. Kuo et al.

where  ˆ

C := C∞,1,1 ≤ 2αkγ

⎛



ˆ

kγ 

exp ⎝4ζ(α)

γj

j=1

∞ (

⎞ γj ⎠ < ∞ .

j=1

Note that kˆγ is finite since γj → 0 as j → ∞. We choose M in relation to n so that 1 CM 2 = . M n−1 This leads to  1 n−1 3 M (n) = (20) C and z) ≤ ewor n,d,M (n) (ˆ

√

2 C 6 (n − 1)− 6 . 1

1

Thus to ensure that ewor z ) ≤ ε we require n − 1 ≥ 8Cε−6 , which can be n,d,M (n) (ˆ achieved by choosing (21) n = pr(8Cε−6 + 1) . For sγ < 1, we have p∗ < 2 and we choose δ ∈ (0, 2 − p∗ ] and take ∗ q = λ = p 2+δ in Lemma 3. Let C(δ) := C∞, p∗ +δ , p∗ +δ < ∞. We have 2

2

p∗ +δ

 wor 2 C(δ)M 2 +1 1 en,d,M (ˆ + z) ≤ , 2 M (n − 1) p∗ +δ and we choose M such that p∗ +δ

C(δ)M 2 +1 1 = . 2 M (n − 1) p∗ +δ This leads to

 M (n) =

and ewor z) ≤ n,d,M (n) (ˆ

√

2

(n − 1) p∗ +δ C(δ)

1  p∗ +δ 2

−

1

2 C(δ) p∗ +δ+4 (n − 1)

+2

(22)

1 (p∗ +δ)2 2(p∗ +δ)+ 2

.

Thus to achieve ewor z ) ≤ ε, we may choose n,d,M (n) (ˆ  p∗ +δ+

n = pr 2

(p∗ +δ)2 4

C(δ)

p∗ +δ 2

ε

  (p∗ +δ)2 − 2(p∗ +δ)+ 2

 +1

.

(23)

Lattice Rules for Multivariate Approximation in the Worst Case Setting

307

This proves, in a constructive way, that the exponent pwor (Λstd ) of strong ∗2 tractability in the class Λstd is at most 2p∗ + p2 . We also know, by the nonconstructive argument, that pwor (Λstd ) ≤ p∗ + 2. Observe that the “construc√ tive” inequality is better one iff p∗ ≤ 5 − 1 = √ √ than the “non-constructive” 1.236 . . . , i.e., for α ≥ ( 5 + 1)/2 and sγ ≤ ( 5 − 1)/2. We now assume that (19) does not hold, but 1d a := lim sup d→∞

j=1

γj

log(d + 1)

<∞.

This is the necessary and sufficient condition for tractability in the class Λstd . Under this assumption, for any δ > 0, there exists dδ ≥ 1 such that 1d j=1

γj

log(d + 1)

≤a+δ

for all d ≥ dδ .

In other words, there exists K > 0 such that 1d dγd j=1 γj ≤ ≤K log(d + 1) log(d + 1)

for all d ≥ 1 .

Hence γj = O(j −1 log(j + 1)), which leads to sγ = 1. We take q = λ = 1 in Lemma 3 to obtain ⎞ ⎛  kˆγ  d  ( ˆ γj exp ⎝4ζ(α) γj ⎠ Cd,1,1 ≤ 2αkγ j=1

 ˆ

= 2αkγ  ˆγ αk

≤2

j=1



ˆ

kγ  j=1

1d j=1

γj / log(d+1)



ˆ

kγ 

(d + 1)4ζ(α)

γj

γj

for all d ≥ dδ .

(d + 1)4ζ(α)(a+δ)

j=1 4ζ(α)(a+δ)   There exists C(δ) > 0 such that Cd,1,1 ≤ C(δ)d for all d ≥ 1. Thus from Lemma 3 we have 4ζ(α)(a+δ)  2  wor C(δ)d M2 1 + . z) ≤ en,d,M (ˆ M n−1

We now choose

 M (n) =

which leads to

n−1

4ζ(α)(a+δ)  C(δ)d

 13 ,

(24)

308

F.Y. Kuo et al.

ewor z) ≤ n,d,M (n) (ˆ and if we choose

√

 6 d 3 ζ(α)(a+δ) (n − 1)− 6 , 2 C(δ) 1

2

1

  −6 4ζ(α)(a+δ)  n = pr 8C(δ)ε d +1 ,

(25)

z ) ≤ ε. then ewor n,d,M (n) (ˆ We summarize the analysis of this section in the following theorem. Theorem 1. Consider the approximation problem for weighted Korobov spaces in the worst case setting.   (a) Let p∗ = 2 max α1 , sγ and suppose that ∞ (

γj < ∞ .

j=1

Given ε > 0, the approximation algorithm An,d,M (n) defined by (6), with ˆ constructed by Algorithm 1, and with n and M (n) generating vector z given by (21) and (20) if sγ = 1, and by (23) and (22) if sγ < 1, achieves z ) ≤ ε using n = O(ε−p ) function values. The the error bound ewor n,d,M (n) (ˆ implied factor in the big O notation is independent of d and the exponent p is arbitrarily close to p∗ 2 . 2p∗ + 2 Hence, we have strong tractability of this approximation problem in the class Λstd with the exponent at most   p∗ 2 ∗ ∗ ,p + 2 . min 2p + 2 (b) Suppose that

1d a := lim sup d→∞

j=1

γj

log(d + 1)

<∞.

Given ε > 0, the approximation algorithm An,d,M (n) defined by (6), with ˆ constructed by Algorithm 1, and with n and M (n) generating vector z z ) ≤ ε using given by (25) and (24), achieves the error bound ewor n,d,M (n) (ˆ n = O(ε−6 d q ) function values. The implied factor in the big O notation is independent of ε and d, and the exponent q can be arbitrarily close to 4ζ(α)a.

6 Second Approach: Lattice Rules Constructed for Approximation In this section we study lattice rules with the generating vectors specially constructed for the approximation problem in the worst case setting. It is perhaps

Lattice Rules for Multivariate Approximation in the Worst Case Setting

309

not surprising that such lattice rules yield smaller error bounds than lattice rules with the generating vectors constructed for the integration problem, as studied in the previous section. Our point of departure is again the second term in (15), which contains the integration error of the function fh (x) = f (x)e−2πih·x when we use an n-point rank-1 lattice rule with the generating vector z. From the theory of lattice rules for integration, we know that this integration error is exactly ( ( fˆh () = fˆ(h + ) . ∈Zd \{0} ·z≡0 (mod n)

∈Zd \{0} ·z≡0 (mod n)

Thus the approximation error of the algorithm An,d,M given by (6) satisfies 2 ( ( ( 2 2 ˆ ˆ |f (h)| + f (h + ) . f − An,d,M (f )L2 ([0,1]d ) = h∈A(d,M / )

As before we have

(

h∈A(d,M )

∈Zd \{0} ·z≡0 (mod n)

1 f 2d . M

|fˆ(h)|2 ≤

h∈A(d,M / )

For the second term, it follows from the Cauchy-Schwarz inequality that 2 ( fˆ(h + ) ∈Zd \{0} ·z≡0 (mod n)



(

≤

 |fˆ(h + )| rα (γ, h + )

∈Z \{0} ·z≡0 (mod n)

∈Zd \{0} ·z≡0 (mod n)

d

(

≤ f 2d

(

2

∈Zd \{0} ·z≡0 (mod n)

1 rα (γ, h + )



1 . rα (γ, h + )

Hence we obtain the following upper bound on the squared worst case approximation error  wor 2 1 en,d,M (z) ≤ + M

(

(

h∈A(d,M )

∈Zd \{0} ·z≡0 (mod n)

1 . rα (γ, h + )

This error bound has an alternative interpretation. Recall from Lemma 2 that an exact expression for the squared worst case error is given by 

2 β ewor + ρ(Tz ) = n,d,M (z) M

for some β ∈ [0, 1] ,

310

F.Y. Kuo et al.

where Tz is a non-negative-definite hermitian matrix with entries τh , τp d given by (16). Clearly, ρ(Tz ) is bounded above by the trace of Tz , and we see from (16) that (

tr(Tz ) =

τh , τh d =

h∈A(d,M )

(

(

h∈A(d,M )

∈Zd \{0} ·z≡0 (mod n)

1 , rα (γ, h + )

which is exactly the second term in our new bound. In other words, we have replaced the largest eigenvalue of Tz in the exact expression by the sum of all eigenvalues. Let us now define ( ( 1 . (26) En,d,M (z) := tr(Tz ) = r (γ, h + ) α d h∈A(d,M )

∈Z \{0} ·z≡0 (mod n)

We wish to find a vector z which gives a small value of En,d,M (z). 6.1 The Existence of a Good Generating Vector In this section we show that there exists a generating vector z which yields improved worst case error bounds. We remind the reader that the vectors z are from the set Znd with Zn := {1, 2, . . . , n − 1}. Lemma 4. Let n be a prime number, and define (

F (h, z) :=

∈Zd \{0} ·z≡0 (mod n)

1 . rα (γ, h + )

(a) For any h ∈ A(d, M ) and z ∈ Znd , we have F (h, z) = −

d n−1 d ∞ 1 e2πikzj /n 1 1 ( ( + . (1 + 2ζ(α)γj ) + rα (γ, h) n j=1 n rα (γj , hj + ) j=1 k=1

=−∞

(b) For any h ∈ A(d, M ), we have ( 1 F (h, z) (n − 1)d d z∈Zn

=− where

d d 1 1 n−1  + (1 + 2ζ(α)γj ) + ∆j (hj ) , rα (γ, h) n j=1 n j=1

Lattice Rules for Multivariate Approximation in the Worst Case Setting

311

⎧ 2ζ(α)γj (1 − n1−α ) ⎪ ⎪ 1 − if h ≡ 0 (mod n), ⎪ ⎪ ⎨  n−1  ∞ ( n γj 1 1 + 2ζ(α)γj ∆j (h) = ⎪ otherwise. − + γj ⎪ α α ⎪ n − 1 |hj | |hj + n| n−1 ⎪ ⎩ =−∞  =0

  1 −α (c) Let κ > 1 be some fixed number and define c := 1 − . For any λ 1 κ  1 1 λ satisfying α < λ ≤ 1, if n ≥ max κM α , 8e(1 + ζ(αλ) + c1 ζ(αλ)) then for any h ∈ A(d, M ) we have d (  1 cd,λ   λ 1 + 2ζ(αλ)γjλ , [F (h, z)] ≤ (n − 1)d n d j=1 z∈Zn

where cd,λ := 4(1 + ζ(αλ) + cλ1 ζ(αλ))   ∗   1 min(kγ ,d) − 1, 2e 1 + × max 2 1 + 2e

∗  1 min(kγ ,d) 2e

with kγ∗ = k if there exists an index k such that γk+1 ≤ kγ∗ = ∞ otherwise.

 − 2e + 1 ,

1 2e

< γk , and

The proof of this lemma is deferred to Sect. 7. We now obtain a bound on min En,d,M (z)

d z∈Zn

1 1 λ by using Jensen’s inequality. This inequality states that ( k ak ) ≤ k aλk for any non-negative ak and any λ ∈ (0, 1]. We have 

λ = min [En,d,M (z)]λ min En,d,M (z)

d z∈Zn

d z∈Zn

⎛

= min ⎝ d z∈Zn

≤ min

d z∈Zn

⎞λ

(

F (h, z)⎠

h∈A(d,M )

(

[F (h, z)]λ

h∈A(d,M )

( ( 1 ≤ [F (h, z)]λ (n − 1)d d z∈Zn h∈A(d,M ) ⎛ ⎞ ( ( 1 ⎝ = [F (h, z)]λ ⎠ . (n − 1)d d h∈A(d,M )

z∈Zn

312

F.Y. Kuo et al.

Using Lemma 1(d) and Lemma 4(c), we obtain, for sufficiently large n, ⎛ ⎞ 1 q d d λ cd,λ Mλ    λ1   1 q ⎝ 1 + 2ζ(αq)γj ⎠ 1 + 2ζ(αλ)γjλ λ , min En,d,M (z) ≤ 1 d z∈Zn nλ j=1 j=1 for all q >

1 α

and

1 α

< λ ≤ 1. This leads to the following lemma.

−α  . Lemma 5. Let κ > 1 be some fixed number and define c1 := 1 − κ1 1 < λ ≤ 1, if n is a prime number satisfying n ≥ For any λ satisfying α   1 max κM α , 8e(1 + ζ(αλ) + cλ1 ζ(αλ)) , then there exists z ∈ Znd such that the worst case error for the algorithm An,d,M defined by (6) satisfies  wor 2 Cd,q,λ M λ 1 en,d,M (z) ≤ + 1 M nλ q

where

⎛ 1 λ

Cd,q,λ = cd,λ ⎝

d  

1 q λ

1 + 2ζ(αq)γj

for all q >

⎞ ⎠

j=1

1 α

d  

,

1 + 2ζ(αλ)γjλ

 λ1

j=1

⎞ d d ( ( 2ζ(αq) 2ζ(αλ) ≤ cd,λ exp ⎝ γq + γjλ ⎠ , λ j=1 j λ j=1 ⎛

1 λ

with cd,λ defined as in Lemma 4(c). 1∞ As in the previous section, we assume first that j=1 γj < ∞. Then sγ ≤ 1. Note that kγ∗ is finite since γj → 0 as j → ∞. If sγ = 1, we take q = λ = 1 in Lemma 5 and choose M (n) =

 n  12

,

C

(27)

where C := C∞,1,1 < ∞. We have ewor n,d,M (n) (z) ≤

√

2 C 4 n− 4 , 1

1

and we require n ≥ 4Cε−4 to achieve ewor n,d,M (n) (z) ≤ ε. Moreover, we require 1

n ≥ κM α = κ

1  n  2α

C

,

which leads to n ≥ κ 2α−1 C − 2α−1 . Combining all the requirements on n, we conclude that n may be chosen as    2α 1 . (28) n = pr max 4Cε−4 , κ 2α−1 C − 2α−1 , 8e(1 + ζ(α) + c1 ζ(α)) 2α

1

Lattice Rules for Multivariate Approximation in the Worst Case Setting

313

∗

If sγ < 1, we take q = λ = p 2+δ with δ ∈ (0, 2 − p∗ ] in Lemma 5, set C(δ) := C∞, p∗ +δ , p∗ +δ < ∞, and choose 2

2

 M (n) = This leads to ewor n,d,M (n) (z) ≤

 12 .

(29)

√ 1 1 2 C(δ) 4 n− 2(p∗ +δ) ,

p∗ +δ

∗

2

n p∗ +δ C(δ)

∗

and we require n ≥ 2p +δ C(δ) 2 ε−2(p +δ) to achieve ewor n,d,M (n) (z) ≤ ε. Taking all requirements on n into account, we may choose   α(p∗ +δ) p∗ +δ p∗ +δ ∗ ∗ n = pr max 2p +δ C(δ) 2 ε−2(p +δ) , κ α(p∗ +δ)−1 C(δ)− 2α(p∗ +δ)−2 ,   ∗    ∗  p∗ +δ 8e 1 + ζ α(p 2+δ) + c1 2 ζ α(p 2+δ) . (30) 1d 1∞ Now we assume j=1 γj < ∞ does not hold, but a := lim supd→∞ j=1 γj / log(d + 1) < ∞. Then sγ = 1 and we take q = λ = 1. Thus for any δ > 0 there exists dδ ≥ 1 such that ⎞ ⎛ d ( Cd,1,1 ≤ cd,1 exp ⎝4ζ(α) γj ⎠ = cd,1 (d + 1)4ζ(α)

j=1 1d j=1

γj / log(d+1)

≤ cd,1 (d + 1)4ζ(α)(a+δ) for all d ≥ dδ ,

4ζ(α)(a+δ)   and there exists C(δ) > 0 such that Cd,1,1 ≤ C(δ)d for all d ≥ 1. We choose  12  n , (31) M (n) = 4ζ(α)(a+δ)  C(δ)d

which gives ewor n,d,M (n) (z) ≤

√

 4 d ζ(α)(a+δ) n− 4 . 2 C(δ) 1

1

−4 4ζ(α)(a+δ)  To achieve again ewor d . Thus we n,d,M (z) ≤ ε, we require n ≥ 4C(δ)ε may choose   4ζ(α)(a+δ) 2α 1 −4 4ζ(α)(a+δ)  − 2α−1  n = pr max 4C(δ)ε d , κ 2α−1 C(δ) d− 2α−1 ,  . (32) 8e(1 + ζ(α) + c1 ζ(α))

We summarize the analysis of this section in the following theorem. Theorem 2. Consider the approximation problem for weighted Korobov spaces in the worst case setting.

314

F.Y. Kuo et al.

(a) Let p∗ = 2 max

1

α , sγ



and suppose that ∞ (

γj < ∞ .

j=1

Given ε > 0, there exists a generating vector z such that the approximation algorithm An,d,M (n) defined by (6), with n and M (n) given by (28) and (27) if sγ = 1, and by (30) and (29) if sγ < 1, achieves the error bound −p ) function values. The implied factor ewor n,d,M (n) (z) ≤ ε using n = O(ε in the big O notation is independent of d and the exponent p is arbitrarily close to 2p∗ . Hence, we have strong tractability of this approximation problem in the class Λstd with the exponent at most 2p∗ . (b) Suppose that 1d j=1 γj <∞. a := lim sup d→∞ log(d + 1) Given ε > 0, there exists a generating vector z such that the approximation algorithm An,d,M (n) defined by (6), with n and M (n) given by (32) −4 q d ) and (31), achieves the error bound ewor n,d,M (n) (z) ≤ ε using n = O(ε function values. The implied factor in the big O notation is independent of ε and d, and the exponent q can be arbitrarily close to 4ζ(α)a. ∗

Note that 2p∗ ≤ 2p∗ + p2 and 2p∗ ≤ p∗ + 2 for p∗ ≤ 2. That is, we have proved the existence of some lattice rule which leads to a better ε-exponent than any result currently known. In the next subsection we will present an algorithm for finding such a lattice rule. 6.2 Component-by-Component Construction We present a new algorithm for constructing the generating vector z for which error bounds similar to those in Theorem 2 hold. Algorithm 2 Let n be a prime number and M a given fixed number, M ≥ 1. 1. Set z1 = 1. 2. For s = 2, 3, . . . , d, find zs in {1, 2, . . . , n − 1} to minimize En,s,M (z1 , . . . , zs−1 , zs ) .

Here En,d,M (z) is defined by (26). We have from Lemma 4(a) the explicit expression

Lattice Rules for Multivariate Approximation in the Worst Case Setting

En,d,M (z) =

(

" −

h∈A(d,M )

315

d 1 1 + (1 + 2ζ(α)γj ) rα (γ, h) n j=1

& n−1 d ∞ e2πikzj /n 1 ( ( . + n rα (γj , hj + ) j=1 k=1

(33)

=−∞

Lemma 6. Let κ > 1 be some fixed number and suppose n is a prime number 1 satisfying n ≥ κM α . Then the worst case error for the algorithm An,d,M defined by (6) with z ∗ ∈ Znd constructed by Algorithm 2 satisfies  wor 2 1 Cd,q,λ,µ M λ en,d,M (z ∗ ) ≤ + for all q > 1 M (n − 1) λ q

1 1 α, α

< λ ≤ 1, and 0 < µ ≤

1 c1

,

−α  where c1 := 1 − κ1 and ⎛ ⎞ d d  1   1 1 ⎝  1 + 2ζ(αq)γjq λ ⎠ 1 + 2(1 + µλ )ζ(αλ)γjλ λ Cd,q,λ,µ = µ j=1 j=1 ⎛ ⎞ d d λ ( ( 2ζ(αq) )ζ(αλ) 1 2(1 + µ ≤ exp ⎝ γq + γjλ ⎠ . µ λ j=1 j λ j=1 The lemma is shown if we can prove that q

∗

En,d,M (z ) ≤

Cd,q,λ,µ M λ 1

(n − 1) λ

for all q >

1 1 α, α

< λ ≤ 1, and 0 < µ ≤

1 c1

.

The proof for this result is long and tedious. For this reason it is deferred to Sect. 7. 1∞ Suppose that j=1 γj < ∞. If sγ = 1, we take q = λ = 1 and µ = c11 in Lemma 6, and choose  1 n−1 2 , (34) M (n) = C where C := C∞,1,1, c1 < ∞. Then 1

∗ ewor n,d,M (n) (z ) ≤

√ 1 1 2 C 4 (n − 1)− 4 ,

and n may be chosen as    2α 1 . n = pr max 4Cε−4 + 1, κ 2α−1 C − 2α−1 ∗

If sγ < 1, we take q = λ = p 2+δ with δ ∈ (0, 2 − p∗ ] and µ = Lemma 6. We set C(δ) := C∞, p∗ +δ , p∗ +δ , 1 < ∞ and then choose 2

2

c1

(35) 1 c1

in

316

F.Y. Kuo et al.

 M (n) = so that

∗ ewor n,d,M (n) (z ) ≤

√

2

(n − 1) p∗ +δ C(δ)

 12 (36)

2 C(δ) 4 (n − 1)− 2(p∗ +δ) . 1

1

Thus we may choose    ∗ +δ α(p∗ +δ) p∗ +δ − 2α(pp∗ +δ)−2 p∗ +δ −2(p∗ +δ) ∗ +δ)−1 α(p 2 n = pr max 2 C(δ) ε + 1, κ C(δ) . (37) This proves, in the constructive way, that the exponent pwor (Λstd ) of strong std is at most 2p∗ . tractability in the class Λ1 ∞ Now we assume that j=1 γj < ∞ does not hold, but a := lim supd→∞ 1d 1 j=1 γj / log(d + 1) < ∞. We take q = λ = 1 and µ = δ with δ ∈ (0, c1 ]. There exists dδ ≥ 1 such that ⎞ ⎛ d ( Cd,1,1,δ ≤ 1 exp ⎝(4 + 2δ)ζ(α) γj ⎠ δ

j=1

= 1δ (d + 1)(4+2δ)ζ(α)

1d j=1

γj / log(d+1)

≤ 1δ (d + 1)(4+2δ)ζ(α)(a+δ)

for all d ≥ dδ ,

(4+2δ)ζ(α)(a+δ)   and there exists C(δ) > 0 such that Cd,1,1,δ ≤ C(δ)d for all d ≥ 1. We choose

 M (n) =

n−1

 12

(4+2δ)ζ(α)(a+δ)  C(δ)d

,

(38)

which gives ∗ ewor n,d,M (n) (z ) ≤

√  14 d (1+ δ2 )ζ(α)(a+δ) (n − 1)− 14 . 2 C(δ)

Thus we may choose   −4 (4+2δ)ζ(α)(a+δ)  n = pr max 4C(δ)ε d + 1, 2α 1  − 2α−1 d− κ 2α−1 C(δ)

(4+2δ)ζ(α)(a+δ) 2α−1

 .

(39)

We summarize the analysis of this section in the following theorem. Theorem 3. Consider the approximation problem for weighted Korobov spaces in the worst case setting.

Lattice Rules for Multivariate Approximation in the Worst Case Setting

(a) Let p∗ = 2 max

1

α , sγ



317

and suppose that ∞ (

γj < ∞ .

j=1

Given ε > 0, the approximation algorithm An,d,M (n) defined by (6), with generating vector z ∗ constructed by Algorithm 2, and with n and M (n) given by (35) and (34) if sγ = 1, and by (37) and (36) if sγ < 1, achieves ∗ −p the error bound ewor ) function values. The n,d,M (n) (z ) ≤ ε using n = O(ε implied factor in the big O notation is independent of d and the exponent p is arbitrarily close to 2p∗ . Hence, we have strong tractability of this approximation problem in the class Λstd with the exponent at most 2p∗ . (b) Suppose that 1d j=1 γj <∞. a := lim sup d→∞ log(d + 1) Given ε > 0, the approximation algorithm An,d,M (n) defined by (6), with generating vector z ∗ constructed by Algorithm 2, and with n and M (n) ∗ given by (39) and (38), achieves the error bound ewor n,d,M (n) (z ) ≤ ε using −4 q n = O(ε d ) function values. The implied factor in the big O notation is independent of ε and d, and the exponent q can be arbitrarily close to 4ζ(α)a. Note that Theorem 3, while constructive, differs only in minor ways from the existence result in Theorem 2. To implement Algorithm 2, we need to rewrite En,d,M (z) into a computable form. The last term in (33) can be rearranged as follows: " & n−1 d n−1 ∞ 1 ( ( e2πik·z/n e2πikzj /n 1 ( ( = n rα (γj , hj + ) n rα (γ, h + ) k=1 j=1 =−∞ k=1 ∈Zd & " n−1 1 ( −2πikh·z/n ( e2πik(h+)·z/n e = n rα (γ, h + ) k=1 ∈Zd "  & n−1 d ∞ ( 1 ( −2πikh·z/n  e2πikzj /n = e 1 + γj . n ||α j=1 k=1

=−∞  =0

When α is an even integer, we can write the sum over  in terms of the Bernoulli polynomial Bα given by (11). For an even integer α, the total cost of Algorithm 2 is O(n2 d2 |A(d, M )|) operations. If we assume that limj→∞ γj = 0, which is a necessary condition for tractability of multivariate approximation, and take M = M (n) as argued above, then M (n) is roughly ∗ ∗ of order n1/p , and |A(d, M (n))| is roughly of order n1/(α p ) . In this case, the total cost of Algorithm 2 is roughly O(n2+1/(2 max(1,α sγ )) d2 ) operations.

318

F.Y. Kuo et al.

It may also be possible to reduce the computation cost using the techniques in [13]. In a practical application of Algorithm 2, one must select a value of ε and possibly a value of δ since the choice of n, and thus M = M (n), depends on ε and may also depend on δ. Determining an appropriate value of δ may be a difficult task.

7 Proofs Proof of Lemma 4 (a) Using the property 1 ( 2πik·z/n e = n n

k=1

1 0

if  · z ≡ 0 (mod n), otherwise,

we can write F (h, z) = −

( 1 + rα (γ, h) d

1 n

∈Z

=− =−

1n

e2πik·z/n rα (γ, h + ) k=1

n−1 1 ( 1 ( ( e2πik·z/n 1 1 + + rα (γ, h) n rα (γ, h + ) n rα (γ, h + ) d d ∈Z

k=1 ∈Z

d 

n−1 d (

1 1 1 + (1 + 2ζ(α)γj ) + rα (γ, h) n j=1 n

∞ (

k=1 j=1 =−∞

e2πikzj /n , rα (γj , hj + )

where the second term in the last equality follows from ( ∈Zd

d d ∞  (  ( 1 1 1 = = = (1 + 2ζ(α)γj ) . rα (γ, h + ) rα (γ, ) j=1 rα (γj , ) j=1 d =−∞

∈Z

(b) We need to prove that for n prime and 1 ≤ k ≤ n − 1, n−1 ∞ 1 ( ( e2πikzj /n = ∆j (hj ) . n − 1 z =1 rα (γj , hj + ) j

=−∞

Since, for 1 ≤ k ≤ n − 1, n−1 1 ( 2πikzj /n e = n − 1 z =1 j

we have

1 1 − n−1

if  ≡ 0 (mod n), otherwise,

Lattice Rules for Multivariate Approximation in the Worst Case Setting

319

∞ n−1 1 ( ( e2πikzj /n n − 1 z =1 rα (γj , hj + ) =−∞

j

∞ (

=

=−∞ ≡0 (mod n)

=

n n−1

1 1 − rα (γj , hj + ) n − 1

∞ ( =−∞ ≡0 (mod n)

∞ ( =−∞  ≡0 (mod n)

1 rα (γj , hj + )

∞ ( 1 1 1 − . rα (γj , hj + ) n − 1 rα (γj , hj + )

(40)

=−∞

For any hj ∈ Z, we have ∞ ( =−∞

∞ ( 1 1 = = 1 + 2ζ(α)γj . rα (γj , hj + ) rα (γj , ) =−∞

If hj is a multiple of n, then ∞ (

1 = rα (γj , hj + )

=−∞ ≡0 (mod n)

∞ (

∞ ( 2ζ(α)γj 1 1 = =1+ . rα (γj , ) rα (γj , n) nα =−∞

=−∞ ≡0 (mod n)

If hj is not a multiple of n, then ∞ (

γj 1 = + γj rα (γj , hj + ) |hj |α

=−∞ ≡0 (mod n)

=

∞ (

1 |hj + |α

=−∞ ≡0 (mod n)  =0 ∞ (

γj + γj |hj |α

=−∞  =0

1 . |hj + n|α

The result follows by substituting these expressions back into (40). λ (c) Since [rα (γ, h + )] = rαλ (γ λ , h + ) with γ λ = {γjλ }, it follows from Jensen’s inequality and the definition of F (h, z) = F (h, z; α, γ) that [F (h, z; α, γ)]λ ≤ F (h, z; αλ, γ λ ) . Thus it follows from (b), with α replaced by αλ and γ replaced by γ λ , that ( 1 [F (h, z)]λ d (n − 1) d z∈Zn

( 1 ≤ F (h, z; αλ, γ λ ) (n − 1)d d z∈Zn

=−

d d  n−1  1  1 λ j (hj ) , ∆ 1 + 2ζ(αλ)γ + + j rαλ (γ λ , h) n j=1 n j=1

(41)

320

F.Y. Kuo et al.

j (h) is ∆j (h) from (b), with α replaced by αλ and γj replaced by γ λ . where ∆ j Clearly the second term in (41) is of order n−1 . We just need to show that the first term and the third term together are also of order n−1 . Let R :=

d d  γjλ n−1   n−1   1 ∆j (hj ) − ∆ = (h ) − , j j n j=1 rαλ (γ λ , h) n j=1 |hj |αλ j∈u

where u = u(h) := { j ∈ [1, d] : hj = 0}. j (hj )| ≤ 1 since n ≥ 1 + ζ(αλ) ≥ 1 + ζ(αλ)γ λ . For each j, if hj = 0 then |∆ j 1 For hj = 0, we have from Lemma 1(a) that |hj | ≤ M α ≤ κ−1 n < n, and ∞ ( =−∞  =0

∞ ( 1 = |hj + n|αλ

1 =−∞ |n|αλ 1 +  =0

≤

≤

∞ ( =−∞  =0 ∞ ( =−∞  =0



1

|n|αλ 1 − 

αλ

hj n

1

|n|αλ 1 −

|hj | n

αλ

 1 αλ

= cλ1

κ

∞ ( =−∞  =0

1 2cλ ζ(αλ) = 1 αλ , αλ |n| n

where c1 := (1 − κ1 )−α . Thus for hj = 0 we have   γjλ 1 + 2ζ(αλ)γjλ 2cλ1 ζ(αλ)γjλ n j (hj )| ≤ |∆ + + αλ αλ n − 1 |hj | n n−1 ≤

γjλ γjλ 2cλ1 ζ(αλ) 1 + 2ζ(αλ) 1 ωλ + + ≤ , + + |hj |αλ n−1 n−1 n−1 |hj |αλ n

where ωλ := 4(1 + ζ(αλ) + cλ1 ζ(αλ)). Hence     γjλ γjλ ωλ + . R≤ − |hj |αλ n |hj |αλ j∈u j∈u It is easy to check that this last expression increases when γjλ /|hj |αλ increases. Thus we have  ωλ   γj + − γj . R≤ n j∈u j∈u Assume first that kγ∗ defined in (c) is at least d. Then γj > β := j = 1, 2, . . . , d. We then bound γj by 1 and obtain R ≤

 1+

ωλ  d −1. n

1 2e

for all

Lattice Rules for Multivariate Approximation in the Worst Case Setting

321

Assume now that kγ∗ < d. Then γkγ∗ +1 ≤ β < γkγ∗ . Bounding γj by 1 for j ≤ kγ∗ and by β for j > kγ∗ , we obtain ∗ ∗  ∗ ωλ |u | ωλ  kγ  β+ R≤ 1+ − β |u | , n n

where u∗ = u∗ (h) := {j ∈ u : j > kγ∗ } and clearly 0 ≤ |u∗ | ≤ d. Hence, in both cases ∗  ∗ ωλ |u | ω λ k  β+ R≤ 1+ − β |u | , n n where k := min(kγ∗ , d). Now consider  ω λ t ω λ k  β+ f (t) := 1 + − βt , n n so that R ≤ f (|u∗ |) ≤ max{f (t) : t = 0, 1, . . . , d}. Since n ≥ 2eωλ , we have ωλ 1 n ≤ 2e = β and      ω λ k  1 1 ω λ t  t β+ f (t) = − 1 + log − β log ωλ n n β+ n β      ω λ t 1 1 ≤− β+ log − β t log . n β + ωnλ β For x ∈ [0, β], define  t

gt (x) := (β + x) log

1 β+x



  1 − β log , β t

1 ) ≥ 1, and so that f  (t) ≤ −gt ( ωnλ ). Clearly gt (0) = 0, log( β+x

  gt (x) = (β + x)t−1 t log

1 β+x



 − 1 ≥ (β + x)t−1 (t − 1) ≥ 0

∀t ≥ 1 ,

implying gt (x) ≥ 0 for t ≥ 1, and in particular gt ( ωnλ ) ≥ 0 for t ≥ 1. This leads to f  (t) ≤ 0 for all t ≥ 1, and hence R ≤ max(f (0), f (1)). We have  ωλ  ω λ k  β+ f (1) = 1 + −β n n k   ( k ω λ j ωλ  ω λ k ωλ [2(1 + β)k − 1] = 1+ , +β ≤ j n n n n j=1 and

 ω λ k ωλ β −1 [(1 + β)k − 1] . f (0) = 1 + −1≤ n n Hence we conclude that cd,λ − 1 , R≤ n

322

F.Y. Kuo et al.

where

  cd,λ := ωλ max 2 1 +

 1 k 2e

 − 1, 2e 1 +

 1 k 2e

 − 2e + 1 .

Thus we have d (  1 1  cd,λ − 1 λ 1 + 2ζ(αλ)γjλ + [F (h, z)] ≤ d (n − 1) n n j=1 d z∈Zn

≤

d  cd,λ   1 + 2ζ(αλ)γjλ . n j=1

This completes the proof. Proof of Lemma 6 We will prove this result by induction on d. It is not hard to check that the result is true for d = 1. Suppose that z ∗ ∈ Znd satisfies ⎛ ⎞ q 1 d d λ   M 1    1 µ q ∗ λ⎠ ⎝ 1 + 2ζ(αq)γj 1 + 2(1 + µλ )ζ(αλ)γjλ λ En,d,M (z ) ≤ 1 (n − 1) λ j=1 j=1 for all q >

1 1 α, α

< λ ≤ 1, and 0 < µ ≤

En,d+1,M (z, zd+1 ) ( =

(

1 c1 .

We have

1 1 . rα (γd+1 , hd+1 + d+1 ) rα (γ, h + )

(h,hd+1 ) (,d+1 )∈Zd+1 \{0} ∈A(d+1,M ) (,d+1 )·(z,zd+1 )≡0 (mod n)

By separating the d+1 = 0 and d+1 = 0 terms, we can write En,d+1,M (z, zd+1 ) = φ(z) + θ(z, zd+1 ) , where φ(z) =

(

(

1 rα (γd+1 , hd+1 ) rα (γ, h + ) 1

(h,hd+1 ) ∈Zd \{0} ∈A(d+1,M ) ·z≡0 (mod n)

and θ(z, zd+1 ) ( =

∞ (

(

1

1 . rα (γd+1 , hd+1 + d+1 ) rα (γ, h + )

d+1 =−∞ (h,hd+1 ) ∈Zd ∈A(d+1,M ) d+1 =0 ·z≡−d+1 zd+1 (mod n)

∗ We wish to obtain an upper bound on En,d+1,M (z ∗ , zd+1 ). This is done in ∗ three steps. We first obtain a bound on φ(z ) and then obtain a bound on ∗ ). Finally we combine these two bounds to get the desired bound θ(z ∗ , zd+1 ∗ ). on En,d+1,M (z ∗ , zd+1

Lattice Rules for Multivariate Approximation in the Worst Case Setting

323

Step 1: Upper Bound for φ(z ∗ ) By separating out the case hd+1 = 0, we can write, using Lemma 1(b), (

(

h∈A(d,M )

∈Zd \{0} ·z≡0 (mod n)

φ(z) =

+2γd+1

∞ (

1

hα hd+1 =1 d+1

1 rα (γ, h + ) (

(

  γd+1 M h∈A d, h α d+1

∈Zd \{0} ·z≡0 (mod n)

1 rα (γ, h + )

≤ (1 + 2ζ(α)γd+1 ) En,d,M (z) . Thus it follows from the given bound on En,d,M (z ∗ ) that ⎛ ⎞ q 1 d λ  1 M   µ ⎝ φ(z ∗ ) ≤ (1 + 2ζ(α)γd+1 ) 1 + 2ζ(αq)γjq λ ⎠ 1 (n − 1) λ j=1 ×

d  

1 + 2(1 + µλ )ζ(αλ)γjλ

 λ1

.

(42)

j=1 ∗ Step 2: Upper Bound for θ(z ∗ , zd+1 ) ∗ Regardless of what z is, we choose zd+1 to minimize En,d+1,M (z, zd+1 ). Since ∗ )≤ the only dependency on zd+1 is in the term θ(z, zd+1 ), we have θ(z, zd+1 θ(z, zd+1 ) for all zd+1 ∈ {1, 2, . . . , n − 1}, which implies that for any λ ≤ 1 we λ  λ ∗ ) ≤ [θ(z, zd+1 )] for all zd+1 ∈ {1, 2, . . . , n − 1}. Thus have θ(z, zd+1

 λ ∗ θ(z, zd+1 ) ≤

( 1 λ [θ(z, zd+1 )] , n − 1 z =1 n−1

d+1

or equivalently, ⎞ λ1 n−1 ( 1 λ ∗ θ(z, zd+1 )≤⎝ [θ(z, zd+1 )] ⎠ . n − 1 z =1 ⎛

(43)

d+1

∗ We will obtain a bound on θ(z, zd+1 ) through this last inequality. λ It follows from Jensen’s inequality and [rα (γ, hj )] = rαλ (γ λ , hj ) that

324

F.Y. Kuo et al. λ

[θ(z, zd+1 )]  ( =

∞ (

(

1 rα (γd+1 , hd+1 + d+1 ) rα (γ, h + ) 1

λ

d+1 =−∞ (h,hd+1 ) ∈Zd ∈A(d+1,M ) d+1 =0 ·z≡−d+1 zd+1 (mod n)

≤

∞ (

(

(

1 λ ,h rαλ (γd+1 d+1

d+1 =−∞ (h,hd+1 ) ∈Zd ∈A(d+1,M ) d+1 =0 ·z≡−d+1 zd+1 (mod n)

1 + d+1 ) rαλ

(γ λ , h

+ )

.

Let Θ(z) denote the average of this last expression over all zd+1 in the set {1, 2, . . . , n − 1}. Then we have ( 1 λ [θ(z, zd+1 )] ≤ Θ(z) , n − 1 z =1 n−1

(44)

d+1

and ∞ (

(

Θ(z) =

(

1 1 λ ,h λ rαλ (γd+1 d+1 + d+1 ) rαλ (γ , h + )

d+1 =−∞ (h,hd+1 ) ∈Zd ∈A(d+1,M ) d+1 ≡0 (mod n) ·z≡0 (mod n) d+1 =0

1 + n−1

∞ (

(

(

1 1 , λ ,h λ , h + ) r (γ rαλ (γd+1 +  ) αλ d+1 d+1

d+1 =−∞ (h,hd+1 ) ∈Zd ∈A(d+1,M ) d+1 ≡0 (mod n) ·z ≡0 (mod n)

where we made used of the fact that when d+1 is not a multiple of n, ( 1 n − 1 z =1

(

n−1

d+1

1 (γ λ , h

rαλ ∈Zd ·z≡−d+1 zd+1 (mod n)

+ )

=

( 1 1 . λ , h + ) n−1 r (γ αλ d ∈Z ·z ≡0 (mod n)

We wish to find a bound on Θ(z) that is independent of z. To simplify the notation, let G(h) :=

( ∈Zd ·z≡0 (mod n)

1 λ rαλ (γ , h + )

and

G :=

d  

 1 + 2ζ(αλ)γjλ .

j=1

Clearly, we have 0 ≤ G(h) ≤ G, and ( d

1 rαλ

(γ λ , h

∈Z ·z ≡0 (mod n)

With the new notation, we rewrite

+ )

= G − G(h) .

Lattice Rules for Multivariate Approximation in the Worst Case Setting ∞ (

(

Θ(z) =

1

d+1 =−∞ (h,hd+1 ) ∈A(d+1,M ) d+1 ≡0 (mod n) d+1 =0

+

=

1 n−1

1 n−1

+

λ ,h rαλ (γd+1 d+1

+ d+1 )

∞ (

(

d+1 =−∞ (h,hd+1 ) ∈A(d+1,M ) d+1 ≡0 (mod n)

(

∞ (

(

 + d+1 ) 

1

λ ,h rαλ (γd+1 d+1 d+1 =−∞ (h,hd+1 )  = 0 ∈A(d+1,M ) d+1

n n−1

G(h)

1 λ ,h rαλ (γd+1 d+1

∞ (

325

+ d+1 )

 G − G(h)

 G − G(h)

1 λ ,h rαλ (γd+1 d+1 + d+1 )

d+1=−∞ (h,hd+1 ) ∈A(d+1,M ) d+1 ≡0 (mod n) d+1 =0

 G(h) −

G n

Following the arguments in the proof of Lemma 4(b), we have ∞ ( d+1 =−∞ d+1 =0

1 1 λ = 1 + 2ζ(αλ)γd+1 , − λ ,h λ ,h rαλ (γd+1 +  ) r (γ d+1 d+1 αλ d+1 d+1 )

for all hd+1 ∈ Z, and ∞ ( d+1 =−∞ d+1 ≡0 (mod n) d+1 =0

1 λ ,h rαλ (γd+1 d+1 + d+1 )

⎧ λ 2ζ(αλ)γd+1 1 ⎪ ⎪ ⎪ 1 + − ⎪ λ ,h ⎪ nαλ rαλ (γd+1 ⎨ d+1 ) ∞ ( = 1 λ ⎪ γd+1 ⎪ ⎪ |hd+1 + d+1 n|αλ ⎪ ⎪ d+1 =−∞ ⎩ d+1 =0

From these we can write

if hd+1 is a multiple of n, otherwise .

 .

326

F.Y. Kuo et al.

1 Θ(z) = n−1



(

1+

(h,hd+1 ) ∈A(d+1,M )



(

n + n−1

(h,hd+1 )∈A(d+1,M ) hd+1 ≡0 (mod n)

(

n + n−1

λ 2ζ(αλ)γd+1

−



1

λ 2ζ(αλ)γd+1 1 − 1+ λ ,h nαλ rαλ (γd+1 d+1 )



∞ (

λ γd+1

d+1 =−∞ d+1 =0

(h,hd+1 )∈A(d+1,M ) hd+1 ≡0 (mod n)



λ ,h rαλ (γd+1 d+1 )

1 |hd+1 + d+1 n|αλ

 G − G(h)





G G(h) − n

G G(h) − n





= W1 + W2 + W3 , where W1 corresponds to the terms with hd+1 = 0, W2 corresponds to those terms with hd+1 = 0 in the first two lines in the above expression, and W3 is exactly the third line in the above expression. Below we will make use of Lemma 1 without further comment. We start by obtaining a bound on W1 . We have 1 n−1

W1 =

+

(

  λ 2ζ(αλ)γd+1 G − G(h)

h∈A(d,M )

n n−1

( h∈A(d,M )

λ 2ζ(αλ)γd+1 = n−1



λ 2ζ(αλ)γd+1 αλ n

1 1 − αλ n

  G G(h) − n

 |A(d, M )|

d  1  1 + 2ζ(αλ)γjλ n j=1

λ  2ζ(αλ)γd+1 n  ( 1 − αλ G(h) n−1 n h∈A(d,M ) ⎞ ⎛ d d q       M ⎝ λ 1 + 2ζ(αq)γjq ⎠ 1 + 2ζ(αλ)γjλ . ≤ 2ζ(αλ)γd+1 n − 1 j=1 j=1

−

Now we obtain a bound on W2 . We have   ∞ λ ( (   γd+1 2 λ W2 = 1 + 2ζ(αλ)γd+1 − αλ G − G(h) n−1 hd+1   h =1 d+1

2n + n−1

∞ (

γd+1 M h∈A d, hα d+1

(

  γd+1 M hd+1 =1 h∈A d, hα hd+1 ≡0 (mod n) d+1



λ λ γd+1 2ζ(αλ)γd+1 − 1+ nαλ hαλ d+1



G G(h) − n

 .

Lattice Rules for Multivariate Approximation in the Worst Case Setting

327

1

Since hd+1 is a multiple of n in the second term and n ≥ κM α , the set A(d, γd+1 M/hα d+1 ) is empty. Therefore the second term above is 0. We have d ∞  λ   (   1 + 2ζ(αλ)γd+1 γ M 1 + 2ζ(αλ)γjλ × 2 A d, hd+1 α d+1 n−1 j=1 hd+1 =1   λ ≤ 1 + 2ζ(αλ)γd+1 ⎛ ⎞ d d q       M q ⎝ 1 + 2ζ(αq)γjq ⎠ 1 + 2ζ(αλ)γjλ . ×2ζ(αq)γd+1 n − 1 j=1 j=1

W2 ≤

Finally we obtain a bound on W3 . When hd+1 is not a multiple of n, we have from the proof of Lemma 4(c) that ∞ ( d+1 =−∞ d+1 =0

1 2cλ ζ(αλ) ≤ 1 αλ . αλ |hd+1 + d+1 n| n

Using this estimate and G(h) ≤ G, we have W3 ≤

d λ    2cλ1 ζ(αλ)γd+1 1 + 2ζ(αλ)γjλ × 2 nαλ j=1

  γ M A d, hd+1 α

∞ (

d+1

hd+1 =1 hd+1 ≡0 (mod n)

λ ≤ 2cλ1 ζ(αλ)γd+1

⎛ ⎞ d d q       M q ⎝ 1 + 2ζ(αq)γjq ⎠ 1 + 2ζ(αλ)γjλ . ×2ζ(αq)γd+1 n − 1 j=1 j=1

Putting the bounds of W1 , W2 and W3 together, we conclude that Θ(z) = W1 + W2 + W3    q λ λ 1 + 2(1 + cλ1 )ζ(αλ)γd+1 ≤ 2ζ(αλ)γd+1 + 2ζ(αq)γd+1 ×

d d    Mq   1 + 2ζ(αq)γjq 1 + 2ζ(αλ)γjλ . n − 1 j=1 j=1

(45)

Hence it follows from (43), (44) and (45) that     λ1 q ∗ λ λ θ(z, zd+1 1 + 2(1 + cλ1 )ζ(αλ)γd+1 ) ≤ 2ζ(αλ)γd+1 + 2ζ(αq)γd+1 d  

q

×

Mλ (n − 1)

1 λ

1+

j=1

 2ζ(αq)γjq λ 1

d  

1 + 2ζ(αλ)γjλ

j=1

Since this bound holds for any z, it must also hold for z ∗ .

 λ1

.(46)

328

F.Y. Kuo et al.

∗ Step 3: Upper Bound for En,d,M (z ∗ , zd+1 )

Now we combine the bounds found in the last two steps. We have from (42) and (46) that ∗ ∗ ) = φ(z ∗ ) + θ(z ∗ , zd+1 ) En,d+1,M (z ∗ , zd+1  ≤ (1 + 2ζ(α)γd+1 ) µ1      λ1 q λ λ λ + 2ζ(αλ)γd+1 + 2ζ(αq)γd+1 1 + 2(1 + c1 )ζ(αλ)γd+1 ⎛ ⎞ q d d   1    1 λ M q λ⎠ ⎝ 1 + 2ζ(αq)γ 1 + 2(1 + µλ )ζ(αλ)γjλ λ × 1 j (n − 1) λ j=1 j=1 ⎛ ⎞ q 1 d+1 d+1 λ   1 1 µM q λ ⎠ ⎝ 1 + 2ζ(αq)γ 1 + 2(1 + µλ )ζ(αλ)γjλ λ . ≤ 1 j (n − 1) λ j=1 j=1

This last inequality is derived by repeated applications of Jensen’s inequality λ together with the properties 2λ ≤ 2, [ζ(α)] ≤ ζ(αλ), and   λ λ 1 + 2(1 + cλ1 )ζ(αλ)γd+1 , ≤ µ1λ 1 + 2(1 + µλ )ζ(αλ)γd+1 which holds since 1 < cλ1 ≤

1 . µλ

This completes the proof.

Acknowledgment The support of the Australian Research Council under its Centres of Excellence program is gratefully acknowledged. The first author was supported by a University of New South Wales Vice-chancellor’s Post-doctoral Research Fellowship. The third author was also partially supported by the National Science Foundation under Grant DMS-0308713.

References 1. J. Dick, On the convergence rate of the component-by-component construction of good lattice rules, J. Complexity, 20, 493–522 (2004). 2. J. Dick and F. Y. Kuo, Reducing the construction cost of the componentby-component construction of good lattice rules, Math. Comp., 73, 1967–1988 (2004). 3. J. Dick and F. Y. Kuo, Constructing good lattice rules with millions of points, Monte Carlo and Quasi-Monte Carlo Methods 2002 (H. Niederreiter, ed.), Springer-Verlag, 181–197 (2004). 4. J. Dick, I. H. Sloan, X. Wang, and H. Wo´zniakowski, Liberating the weights, J. Complexity, 20, 593–623 (2004).

Lattice Rules for Multivariate Approximation in the Worst Case Setting

329

5. F. J. Hickernell and H. Wo´zniakowski, Tractability of multivariate integration for periodic functions, J. Complexity, 17, 660–682 (2001). 6. N. M. Korobov, Properties and calculation of optimal coefficients, Doklady Akademii Nauk SSSR, 132, 1009–1012 (1960); English translation in Soviet Math. Dokl., 1, 696–700 (1960). 7. N. M. Korobov, Number-theoretic Methods in Approximate Analysis, Fizmatgiz, Moscow, 1963. 8. F. Y. Kuo, Component-by-component constructions achieve the optimal rate of convergence for multivariate integration in weighted Korobov and Sobolev spaces, J. Complexity, 19, 301–320 (2003). 9. F. Y. Kuo and S. Joe, Component-by-component construction of good lattice rules with a composite number of points, J. Complexity, 18, 943–976 (2002). 10. F. Y. Kuo and S. Joe, Component-by-component construction of good intermediate-rank lattice rules, SIAM J. Numer. Anal., 41, 1465–1486 (2003). 11. D. Li and F. J. Hickernell, Trigonometric spectral collocation methods on lattices, Recent Advances in Scientific Computing and Partial Differential Equations (S. Y. Cheng, C.-W. Shu, and T. Tang, eds.), AMS Series in Contemporary Mathematics, vol. 330, American Mathematical Society, Providence, Rhode Island, 121–132 (2003). 12. E. Novak, I. H. Sloan, and H. Wo´zniakowski, Tractability of approximation for weighted Korobov spaces on classical and quantum computers, Found. Comput. Math., 4, 121–156 (2004). 13. D. Nuyens and R. Cools, Fast algorithms for component-by-component construction of rank-1 lattice rules in shift-invariant reproducing kernel Hilbert spaces, Math. Comp., to appear. 14. I. H. Sloan, F. Y. Kuo, and S. Joe, On the step-by-step construction of quasiMonte Carlo integration rules that achieve strong tractability error bounds in weighted Sobolev spaces, Math. Comp., 71, 1609–1640 (2002). 15. I. H. Sloan, F. Y. Kuo, and S. Joe, Constructing randomly shifted lattice rules in weighted Sobolev spaces, SIAM J. Numer. Anal., 40, 1650–1665 (2002). 16. I. H. Sloan and S. Joe, Lattice Methods for Multiple Integration, Oxford University Press, Oxford, 1994. 17. I. H. Sloan and A. V. Reztsov, Component-by-component construction of good lattice rules, Math. Comp., 71, 263–273 (2002). 18. I. H. Sloan and H. Wo´zniakowski, When are quasi-Monte Carlo algorithms efficient for high dimensional integrals?, J. Complexity, 14, 1–33 (1998). 19. I. H. Sloan and H. Wo´zniakowski, Tractability of multivariate integration for weighted Korobov classes, J. Complexity, 17, 697–721 (2001). 20. V. N. Temlyakov, Approximate recovery of periodic functions of several variables, Mat. Sbornik., 128, 256–268 (1985); English translation in Math. USSR Sbornik., 56, 249–261 (1987). 21. V. N. Temlyakov, Reconstruction of multivariate periodic functions based on their values at the knots of number-theoretical nets, Anal. Math., 12, 287–305 (1986). 22. V. N. Temlyakov, On approximate recovery of functions with bounded mixed derivative, J. Complexity, 9, 41–59 (1993). 23. J. F. Traub, G. W. Wasilkowski, and H. Wo´zniakowski, Information-Based Complexity, Academic Press, New York, 1988. 24. G. W. Wasilkowski and H. Wo´zniakowski, Weighted tensor product algorithms for linear multivariate problems, J. Complexity, 15, 402–447 (1999).

330

F.Y. Kuo et al.

25. G. W. Wasilkowski and H. Wo´zniakowski, On the power of standard information for weighted approximation, Found. Comput. Math., 1, 417–434 (2001). 26. X. Y. Zeng, K. T. Leung, and F. J. Hickernell, Error analysis of splines for periodic problems using lattice designs, Monte Carlo and Quasi-Monte Carlo Methods 2004 (H. Niederreiter and D. Talay, eds.), to appear.

Randomized Quasi-Monte Carlo Simulation of Markov Chains with an Ordered State Space Pierre L’Ecuyer1 , Christian L´ecot2 , and Bruno Tuffin3 1

2

3

D´epartement d’informatique et de recherche op´erationnelle, Universit´e de Montr´eal, C.P. 6128, Succ. Centre-Ville, Montr´eal (Qu´ebec), Canada, H3C 3J7 [email protected] Laboratoire de Math´ematiques, Universit´e de Savoie, 73376 Le Bourget-du-Lac Cedex, France [email protected] IRISA-INRIA, Campus Universitaire de Beaulieu, 35042 Rennes Cedex, France [email protected]

Summary. We study a randomized quasi-Monte Carlo method for estimating the state distribution at each step of a Markov chain with totally ordered (discrete or continuous) state space. The number of steps in the chain can be random and unbounded. The method simulates n copies of the chain in parallel, using a (d + 1)dimensional low-discrepancy point set of cardinality n, randomized independently at each step, where d is the number of uniform random numbers required at each transition of the Markov chain. The method can be used in particular to get a lowvariance unbiased estimator of the expected total cost up to some random stopping time, when state-dependent costs are paid at each step. We provide numerical illustrations where the variance reduction with respect to standard Monte Carlo is substantial.

1 Introduction A deterministic quasi-Monte Carlo (QMC) method for estimating transient measures over a fixed number of steps, for discrete-time and discrete-state Markov chains with a totally ordered state space, was proposed and studied in [3], based on ideas of [2]. The method simulates n = 2k copies of the chain in parallel (for the same number of steps) using a (0, 2)-sequence in base 2. At step j of the chain, it reorders the n copies according to their states and simulates the transitions (next states) for the n copies by employing the elements nj to nj + n − 1 of the (0, 2)-sequence in place of uniform random numbers to drive the simulation. It assumes that simulating each transition of the chain requires a single uniform random variate. Convergence to the correct value was proved in [3] under a condition on the structure of the transition probability matrix of the Markov chain.

332

P. L’Ecuyer et al.

In this paper, we generalize this method to Markov chains with continuous state space, with a random and unbounded number τ of steps (this permits one to cover regenerative simulation, in particular), and for which the number d of uniform random variates that are required to generate the next state in one step of the Markov chain can be larger than 1. The method uses randomized versions of a single highly-uniform (or low-discrepancy) point set of cardinality n in the d-dimensional unit cube. It provides unbiased mean and variance estimators. We have theoretical results on the rate of convergence of the variance of the mean estimator (as n → ∞) only for narrow special cases, but our empirical results with a variety of examples indicate that this variance goes down much faster with the proposed method than for standard Monte Carlo (MC) simulation or for randomized QMC (RQMC) methods that use a single τ d-dimensional point to simulate each sample path of the chain (as in [2, 3, 7], for example). In the next section, we first define our Markov chain model, then we motivate and state our RQMC sampling algorithm. Section 3 contains convergence results for special settings. Section 4 illustrates the method via numerical examples where it improves the simulation efficiency (by reducing the variance) by large factors compared with standard MC.

2 The Array-RQMC Algorithm 2.1 Markov Chain Model We consider a Markov chain that evolves over a state space X , according to the stochastic recurrence: X0 = x0 ,

Xj = ϕj (Xj−1 , Uj ), j ≥ 1 ,

(1)

where the Uj are i.i.d. random vectors uniformly distributed over the ddimensional unit hypercube [0, 1)d . We want to estimate µ = E[Y ]

where

Y =

τ (

cj (Xj ) ,

(2)

j=1

each cj : X → R is a cost function, τ is a stopping time with respect to the filtration generated by {(j, Xj ), j ≥ 0}, and we assume implicitly that X , the ϕj ’s, and the cj ’s satisfy appropriate measure-theoretic requirements so that all objects of interest in this paper are well-defined (so we hide the uninteresting technical details). We also assume that the functions ϕj and cj are easy to evaluate at any given point, for each j. The random variable Y is easy to generate by standard MC: For j = 1, . . . , τ , generate Uj ∼ U [0, 1)d , compute Xj = ϕj (Xj−1 , Uj ), and add the value of cj (Xj ) to an accumulator, which at the end will contain the value of

RQMC Simulation of Markov Chains

333

Y . This can be replicated n times independently, and the sample mean and variance of the n values of Y are unbiased estimators of the exact mean and variance of Y . From this, one can compute a confidence interval on µ. Let s = supω dτ , where the supremum is taken over all possible sample paths ω, and s = ∞ if τ is unbounded. In this setting, the random variable Y can be written as a function of a sequence of s i.i.d. U (0, 1) random variables, say Y = f (U1 , . . . , Us ), for some complicated function f . If τ is unbounded, we assume that it at least has finite expectation. One way of estimating µ by RQMC is to select an s-dimensional RQMC point set of cardinality n, say Vi = (Ui,1 , . . . , Ui,s ) for i = 1, . . . , n, compute the average value of f over these n points, say Y¯n , and take it as an estimator of µ. To estimate the variance and compute a confidence interval on µ this procedure can be replicated m times, with independent randomizations of the same QMC point set. Under simple conditions on the randomization (e.g., one must have E[f (Vi )] = µ), the sample mean and sample variance of these m averages are unbiased estimators of the exact mean and variance of Y¯n . Further details on this classical RQMC approach can be found in [2, 3, 8] and other references given there. Owen [8] discusses several ways of handling the case where s is large, perhaps infinite (he gives examples of situations when this happens). He proposes an RQMC variant called Latin Supercube Sampling (LSS), where the s coordinates are partitioned into finite subsets of sizes (say) s1 , s2 , . . . , and an sj -dimensional QMC point set Pn,j is used for each subset j, but with the n points randomly permuted, independently across the subsets. Thus, each chain uses a random point from Pn,j for each subset j of coordinates. If all sj ’s are equal, all Pn,j ’s can be the same point set. Our method bears ressemblance with LSS with sj = d for all j, but the points are assigned to the chains in a systematic manner (by sorting the chains according to their states, as described in a moment) rather than by a random permutation. 2.2 Array-RQMC for Comparable Markov Chains We now assume (for the remainder of the paper) that X ⊆ R ∪ {∞}, and that a total order has been defined on X , for which ∞ is the largest state. The state ∞ is an absorbing state used to indicate that we have reached the stopping time τ . That is, Xj = ∞ for j > τ , and cj (∞) = 0. The basic idea of the proposed method is to seek a good estimate of the distribution function Fj of the random variable Xj for each j. For that, we simulate n copies of the chain in parallel and estimate Fj by the empirical distribution of n values of Xj thus obtained. In contrast with classical integration or approximation methods, the states at which Fj is to be “evaluated” need not be selected in advance but are generated automatically by the RQMC algorithm according to a distribution that approximates Fj . The array-RQMC algorithm works as follows. At step 1, we take an RQMC point set Pn,1 = {u0,1 , . . . , un−1,1 } in [0, 1)d , define

334

P. L’Ecuyer et al.

Xi,1 = ϕ1 (x0 , ui,1 )

for i = 0, . . . , n − 1 ,

and estimate the distribution F1 of X1 by the empirical distribution Fˆ1 of X0,1 , . . . , Xn−1,1 . This gives the following approximation, where I denotes the indicator function: F1 (x) = P [X1 ≤ x]  = I(ϕ1 (x0 , u) ≤ x) du

(3)

[0,1)d

≈

n−1 1( I(ϕ1 (x0 , ui,1 ) ≤ x) n i=0

=

n−1 1( def I(Xi,1 ≤ x) = Fˆ1 (x) , n i=0

(4)

which amounts to estimating the integral (3) by RQMC in (4). At step j, we use the empirical distribution Fˆj−1 of X0,j−1 , . . . , Xn−1,j−1 as an approximation of the distribution Fj−1 of Xj−1 . Let Pn,j = {u0,j , . . . , un−1,j } be an RQMC point set in [0, 1)d such that the (d + 1)-dimensional point set  = {ui,j = ((i + 0.5)/n, ui,j ), 0 ≤ i < n} is “highly uniform” (or has “low Pn,j discrepancy”) in [0, 1)d+1 , in a sense that we leave open for the moment (a precise definition of “low discrepancy” in the asymptotic sense, as n → ∞, will be adopted in the propositions of Sect. 3). We estimate Fj by the empirical distribution Fˆj of the values Xi,j = ϕj (X(i),j−1 , ui,j ), i = 0, . . . , n − 1. This −1 is well defined): can be interpreted as follows (if we assume that Fj−1 Fj (x) = P [Xj ≤ x] = E[I(ϕj (Xj−1 , Uj ) ≤ x)]   = I(ϕj (y, u) ≤ x) du dFj−1 (y) X [0,1)d   ≈ I(ϕj (y, u) ≤ x) du dFˆj−1 (y) X [0,1)d  −1 = I(ϕj (Fˆj−1 (v), u) ≤ x) du dv

(5) (6) (7)

[0,1)d+1

n−1 1( −1 I(ϕj (Fˆj−1 ((i + 0.5)/n), ui,j ) ≤ x) ≈ n i=0

=

n−1 1( I(ϕj (X(i),j−1 , ui,j ) ≤ x) n i=0

=

n−1 1( def I(Xi,j ≤ x) = Fˆj (x) . n i=0

(8)

In (6), we have replaced Fj−1 in (5), by its approximation Fˆj−1 . In (8), we approximate the integral in (7) by RQMC over [0, 1)d+1 with the point set

RQMC Simulation of Markov Chains

335

 Pn,j . Observe that this point set gives a perfect stratification of the distribution Fˆj−1 , with exactly one observation per stratum (the strata are the jumps of Fˆj−1 ). Putting these pieces together, we get the following algorithm (the “for” loops are written using the C/C++/Java syntax and indentation alone indicates the scope of the loops):

Array-RQMC algorithm: 1 (Initialization). Select a d-dimensional QMC point set P˜n = ˜ n−1 ) and a randomization of P˜n such that (a) each ran(˜ u0 , . . . , u domized point is uniform over [0, 1)d and (b) if Pn = (u0 , . . . , un−1 ) denotes the randomized version, then Pn = {((i + 0.5)/n, ui ), 0 ≤ i < n} has “low discrepancy”. 2 (Simulate chains). Simulate in parallel n copies of the chain, numbered 0, . . . , n − 1, as follows: For (j = 1; X0,j−1 < ∞; j++) Randomize P˜n afresh into Pn,j = {u0,j , . . . , un−1,j }; For (i = 0; i < n and Xi,j−1 < ∞; i++) Xi,j = ϕj (Xi,j−1 , ui,j ); Sort (and renumber) the chains for which Xi,j < ∞ by increasing order of their states; (The sorted states X0,j , . . . , Xn−1,j provide Fˆj ). 3 (Output). Return the average Y¯n of the n values of Y as an estimator of µ. This entire procedure is replicated m times to estimate the variance and compute a confidence interval on µ.

3 Unbiasedness and Convergence Proposition 1. (a) The average Y¯n is an unbiased estimator of µ and (b) the empirical variance of its m copies is an unbiased estimator of var[Y¯n ]. Proof. The successive steps of the chain use independent randomizations. Therefore, for each chain, the vectors that take place of the Uj ’s for the successive steps j of the chain in the recurrence (1) are independent random variables uniformly distributed over [0, 1)d . Thus, any given copy of the chain obeys the correct probabilistic model defined by (1) and (2), so the value of Y is an unbiased estimator of µ for each chain and also for the average, which proves (a). For (b), it suffices to observe that the m copies of Y¯n are i.i.d. unbiased estimators of µ.  Of course, this proposition implies that the variance of the overall average converges as O(1/m) when m → ∞. A more interesting question is: What about the convergence when n → ∞?

336

P. L’Ecuyer et al.

−1 The integrand I(ϕj (Fˆj−1 (v), u) ≤ x) in (7) is 1 in part of the unit cube, and 0 elsewhere. The shape and complexity of the boundary between these two regions depends on ϕ1 , . . . , ϕj . We assume that these regions are at least measurable sets. For continuous state spaces X , the Hardy-Krause total variation of this indicator function is likely to be infinite, in which case the classical Koksma-Hlawka inequality will not be helpful to bound the integration error in (8). On the other hand, we have proved bounds on the convergence rate for the two following (narrow) special cases: (1) when the chain has a finite number of states (Proposition 2), and (2) when  = d = 1 and the ϕj ’s satisfy a number of conditions (Proposition 3). Detailed proofs of these (and other) propositions will be given in the expanded version of the paper.

Proposition 2. Suppose that the state space X is finite, say X = {1, . . . , L}, that the Markov chain is stochastically increasing (i.e., P [Xj ≥ x | Xj−1 = y] is non-decreasing in y for each j), and that at each step j, we use inversion from a single uniform to generate the next state Xj from its condi1L−1 tional distribution given Xj−1 (so d = 1). Let Γj = =1 |cj ( + 1) − cj ()|,  = {((i + 0.5)/n, ui,j ), 0 ≤ i < n}, and suppose that the star discrepancy Pn,j   satisfies Dn∗ (Pn,j ) = O(n−1 log n) w.p.1 (this can easily be achieved by of Pn,j taking a (0, 2)-sequence in some base b). Then, 1 n−1 ( cj (Xi,j ) − E[cj (Xj )] ≤ jΓj KLn−1 log n n i=0 for some constant K. This implies that the variance of the cost estimator for step j converges as O((jLn−1 log n)2 ) = O((jL)2 n−2+ ) when n → ∞. Proposition 3. Let  = d = 1. Define Dn (Fˆj , Fj ; x) :=

1 ( I(Xi,j ≤ x) − Fj (x) n 0≤i
and

Dn∗ (Fˆj , Fj ) := sup |Dn (Fˆj , Fj ; x)| , x∈X

where I(A) denotes the indicator function of A. Suppose that n = bk , that Pn is a (0, k, 2)-net in base b, that ∀j ≥ 1 ∀x ∈ X ∀u ∈ [0, 1)

V (Ix ◦ ϕj (·, u)) ≤ 1 ,

and that there exists a sequence mj of integers such that ∀j ≥ 1 ∀x ∈ X ∀y ∈ X

V (Ix ◦ ϕj (y, ·)) ≤ mj ,

where V (·) means the total variation and Ix denotes the indicator function of Ax := {y ∈ X : y ≤ x}. Then,

RQMC Simulation of Markov Chains

337

Dn∗ (Fˆj , Fj ) ≤ Mj b−k/2 = O(jn−1/2 ) where Mj :=

1j

h=1 (mh

+ 1). Moreover, if cj has bounded variation, 1 n−1 ( cj (Xi,j ) − E[cj (Xj )] ≤ V (cj )Dn∗ (Fˆj , Fj ) . n i=0

This proposition can be proved by generalizing the arguments of [3]. It only gives a bound of O(1/n) on the variance, just as for ordinary MC, but here this bound is also a worst-case deterministic bound on the squared error. Moreover, the variance seems to converge at a faster rate than O(1/n) (empirically) in the examples that we tried, as illustrated in the next section.

4 A Numerical Illustration 4.1 An M/M/1 Queue with d = 1 Consider an M/M/1 queue with arrival rate λ = 1, i.e., a single-server queue with i.i.d. exponential interarrival times with mean 1 and i.i.d. exponential service times with mean 1/µ < 1. The utilization factor for the server is ρ = 1/µ. We want to estimate the expected average waiting time of the first t customers, denoted µt . (This µt could be computed numerically without simulation; we just use this simple academic example to illustrate our method.) Let Wj denote the waiting time of customer number j in this system, where the first customer (who arrives to the empty system) has number 0. These Wj ’s satisfy the well-known Lindley’s recurrence: W0 = 0 and Wj = max(0, Wj−1 + Sj−1 − Aj ) for j ≥ 1. We estimate µt by the sample average Y = (W0 + · · · + Wt−1 )/t. To compute Y , we need to generate the 2(t − 1) random variates S0 , A1 , . . . , St−1 , At . This estimator Y (and the corresponding f in Sect. 2.1) is unbounded, but it has bounded variance. We first consider a Markov chain that moves by one step each time one of these random variates is generated. That is, X0 = W0 , X1 = W0 + S0 , X2 = W1 , X3 = W1 + S1 , and so on. In this case, d = 1 and s = 2(t − 1). Then, we also consider the case where the chain moves by one step every d/2 customers (where d is even), so Xj = Wjd/2 and s = (t − 1)/d. We tried both classical RQMC and array-RQMC for this example, with t = 100. Table 1 gives the estimated variance reduction factors compared with standard MC, i.e., the empirical variance per observation for the MC estimator divided by that of the method considered, for some RQMC methods. For the RQMC methods, the sample variance of the averages Y¯n is multiplied by n to get the variance per observation, i.e., per simulated copy of the Markov chain. All values are rounded to the closest integer. If the required CPU time to simulate the mn copies of the chain with RQMC is approximately α times that required to simulate mn independent

338

P. L’Ecuyer et al.

Table 1. Empirical variance reduction factors of RQMC with respect to MC, for the average waiting time of 100 customers, estimated with m = 100 Korobov, n = Sobol, n =

1021 4093 16381 65521 262139 1024 4096 16384 65536 262144

Classical-Korobov, ρ = 0.2 3 Classical-Sobol, ρ = 0.2 1 Array-Korobov, ρ = 0.2 52 Array-Sobol, ρ = 0.2 53 Classical-Korobov, ρ = 0.5 3 Classical-Sobol, ρ = 0.5 1 Array-Korobov, ρ = 0.5 39 Array-Sobol, ρ = 0.5 72 Classical-Korobov, ρ = 0.8 4 Classical-Sobol, ρ = 0.8 2 Array-Korobov, ρ = 0.8 80 Array-Sobol, ρ = 0.8 208

4 8 10 22 1 2 1 15 125 336 826 2991 303 748 3247 7964 4 7 8 9 1 4 5 7 133 442 810 2464 259 1340 3642 12460 2 8 10 11 2 5 10 10 322 1064 1329 3674 563 2333 11860 61290

copies, and if the variance reduction factor is γ, then we can say that MC requires γ/α times more CPU time to achieve a given precision for the estimator of µ. For this example, we have α ≈ 2 for array-RQMC (due to the overhead of maintaining several copies of the chain in parallel and sorting them) and α ≈ 1 for classical RQMC. For array-RQMC, we tried several possibilities for the d-dimensional RQMC point set Pn . Here we give the results for (a) a (d + 1)-dimensional Korobov lattice rule with its first coordinate skipped, randomized by a random shift modulo 1 followed by a baker’s transformation [1] (denoted ArrayKorobov ) and (b) the first n points of a Sobol sequence randomized by a left (upper triangular) matrix scrambling followed by a random digital shift [3, 9] (denoted Array-Sobol ). For Array-Korobov, the multiplier a of the two-dimensional Korobov lattice rule was selected so that n is prime, a is a primitive element modulo n (this requirement is actually not needed), and a/n is close to the golden ratio. With this choice of a, the rule performs quite well in the two-dimensional spectral test. The points of the Korobov lattice are enumerated by order of the (skipped) first coordinate, so Pn becomes the original Korobov lattice point set. For instance, if d = 1, the points of Pn before the shift are 0, a, 2a, . . . , (n−1)a in that order. The points of the Sobol net are enumerated by order of their Gray code, which makes their enumeration faster. This is effectively equivalent to applying a permutation to their second coordinate. We repeated the same experiment, but enumerating the points in their natural order (without using the Gray code) and for d = 1 the resulting variance reduction was generally smaller! Thus, using the Gray code implements some form of scrambling that seems beneficial. Further investigation would be needed to understand exactly why.

RQMC Simulation of Markov Chains

339

For the classical RQMC, we used a randomly shifted 2(t − 1)-dimensional Korobov lattice rules using parameters taken from Table 1 of [2] (where n is prime and close to a power of 2). This is denoted by Classical-Korobov in the table. We also tried 2(t − 1)-dimensional Sobol nets with a left matrix scrambling and a random digital shift (denoted Classical-Sobol). All the variance reduction factors were estimated by making 100 × 218 independent simulation runs to estimate the MC variance and m = 100 independents replicates for each n, for each RQMC method. The simulations were performed on a standard laptop computer with 1.7 GHz processor, in Java, using the SSJ simulation library [4]. The array-RQMC methods clearly outperform both MC and classical RQMC in this example, even though classical RQMC is already significantly more efficient than MC (about 10 times more efficient for the Korobov rules with n ≥ 216 ). The improvement factor is larger when the queue has more traffic (i.e., for larger ρ, which is also when the variance is larger) and larger for the Sobol nets than for the Korobov rules. For classical RQMC, it is slightly better for the Korobov rules than for the Sobol nets. 4.2 Increasing d Table 2 gives estimated variance reduction factors of two RQMC methods compared with standard MC, when d increases, with n ≈ 217 . Here, at each step of the Markov chain, we generate d random variates to compute the waiting times of d/2 customers. Note that for “Classical-Korobov,” the exact variance reduction factor does not depend on d; the variation observed in the table is only statistical noise. It gives an idea of the precision of our varianceimprovement estimators. For Array-Sobol, the variance reduction factors decrease with d, but not so fast. The gains are still substantial even for d = 8, where the RQMC method approximates 9-dimensional integrals at each step of the Markov chain. Table 2. Estimated variance reduction factors of d-dimensional classical RQMC and array-RQMC with respect to MC, for selected values of d and n ≈ 217 d=1 d=2 d=4 d=8 Classical-Korobov, ρ = 0.2 34 30 25 24 Array-Sobol, ρ = 0.2 6300 3300 1700 1300 Classical-Korobov, ρ = 0.5 11 12 21 18 Array-Sobol, ρ = 0.5 7000 7000 6600 2900 Classical-Korobov, ρ = 0.8 11 10 12 15 Array-Sobol, ρ = 0.8 24000 8200 13000 10000

340

P. L’Ecuyer et al.

4.3 Random Dimension: a Regenerative System So far in this example, s was fixed to 2(t − 1). We now modify the example so that s = ∞. Recall that the M/M/1 queue is a regenerative system that regenerates whenever a customer arrives to an empty system. Each regenerative cycle contains a random and unbounded number of customers. Suppose we want to estimate µ = E[Y ], where we take the following two possibilities for Y : (i) the total waiting time of all customers in a regenerative cycle and (ii) the number of customers in a cycle whose waiting time exceeds c, for some constant c > 0. Note that changing the uniforms slightly may split or merge regenerative cycles, making Y highly discontinuous in both cases. Moreover, in the second case, Y is integer-valued, so it is not as smooth as in the first case. For our numerical illustration of case (ii), we take c = 1. The exact value of µ for case (i) is 1 for ρ = 0.5 and 16 for ρ = 0.8. For case (ii), it is approximately 0.368 for ρ = 0.5 and 3.116 for ρ = 0.8. Tables 3 and 4 give the estimated variance reduction factors of arrayRQMC compared with standard MC, again with m = 100. The improvement factors are not as large as in the two previous tables, but they are still significant and also increase with n. Table 3. Estimated variance reduction factors of array-RQMC with respect to MC, for the regenerative example, case (i) Korobov, n = Sobol, n = Array-Korobov, ρ = 0.5 Array-Sobol, ρ = 0.5 Array-Korobov, ρ = 0.8 Array-Sobol, ρ = 0.8

1021 4093 16381 65521 262139 1024 4096 16384 65536 262144 4 8 7 5

17 13 9 5

47 30 25 16

80 70 36 34

174 174 115 87

Table 4. Estimated variance reduction factors of array-RQMC with respect to MC, for the regenerative example, case (ii) Korobov, n = Sobol, n = Array-Korobov, ρ = 0.5 Array-Sobol, ρ = 0.5 Array-Korobov, ρ = 0.8 Array-Sobol, ρ = 0.8

1021 4093 16381 65521 262139 1024 4096 16384 65536 262144 26 14 12 9

62 23 45 32

134 77 109 74

281 172 86 177

627 659 415 546

RQMC Simulation of Markov Chains

341

4.4 Summary of Other Numerical Experiments We have performed numerical experiments with various other examples. They will be reported in the detailed version of the paper. In particular, we tried examples with multidimensional state spaces and others with integrands of high variability. Generally speaking, as expected, we observed empirically that the performance of the array-RQMC method tends to degrade when the integrand has higher variability, or when the dimension of the state space becomes larger than 1 and there is no obvious “natural order” for the states. But even in these cases, there can still be significant gains in efficiency compared with MC and classical RQMC. For example, the payoff of an Asian option can be simulated by a Markov chain with state Xj = (Sj , S¯j ) where Sj is the underlying asset price at observation time j and S¯j is the average of S1 , . . . , Sj . The final payoff is a function of S¯s only, where s is the number of observation times. One possible way of ordering the states (which is not necessarily the best way) is simply by their values of Sj . With this order, in a numerical example where the asset price evolves as a geometric Brownian motion and the number of observation times varies from 10 to 120, we observed empirical variance reduction factors (roughly) from 1500 to 40000 for array-RQMC compared with MC. With classical RQMC, the factors were (roughly) 5 to 10 times smaller. Our empirical results suggest better convergence rates than those implied by the (worst-case) bounds that we have managed to prove. Getting better convergence bounds for the variance is a topic that certainly deserves further investigation. From the practical viewpoint, an interesting challenge would be to find good ways of ordering the states for specific classes of problems where the Markov chain has a multidimensional state space. In the future, we also intend to study the application of array-RQMC to other settings that fit a general Markov chain framework. For instance, we think of Markov chain Monte Carlo methods and stochastic approximation algorithms.

Acknowledgments The work of the first author has been supported by NSERC-Canada grant No. ODGP0110050, NATEQ-Qu´ebec grant No. 02ER3218, and a Canada Research Chair. The work of the third author has been supported by EuroNGI Network of Excellence and SurePath ACI s´ecurit´e project. The paper benefited from the comments of an anonymous reviewer.

References 1. F. J. Hickernell. Obtaining o(n−2+ ) convergence for lattice quadrature rules. In K.-T. Fang, F. J. Hickernell, and H. Niederreiter, editors, Monte Carlo and Quasi-Monte Carlo Methods 2000, pages 274–289, Berlin, 2002. Springer-Verlag.

342

P. L’Ecuyer et al.

2. C. L´ecot and S. Ogawa. Quasirandom walk methods. In K.-T. Fang, F. J. Hickernell, and H. Niederreiter, editors, Monte Carlo and Quasi-Monte Carlo Methods 2000, pages 63–85, Berlin, 2002. Springer-Verlag. 3. C. L´ecot and B. Tuffin. Quasi-Monte Carlo methods for estimating transient measures of discrete time Markov chains. In H. Niederreiter, editor, Monte Carlo and Quasi-Monte Carlo Methods 2002, pages 329–343, Berlin, 2004. SpringerVerlag. 4. P. L’Ecuyer. SSJ: A Java Library for Stochastic Simulation, 2004. Software user’s guide, Available at http://www.iro.umontreal.ca/∼lecuyer. 5. P. L’Ecuyer and C. Lemieux. Variance reduction via lattice rules. Management Science, 46(9):1214–1235, 2000. 6. P. L’Ecuyer and C. Lemieux. Recent advances in randomized quasi-Monte Carlo methods. In M. Dror, P. L’Ecuyer, and F. Szidarovszky, editors, Modeling Uncertainty: An Examination of Stochastic Theory, Methods, and Applications, pages 419–474. Kluwer Academic Publishers, Boston, 2002. 7. C. Lemieux and P. L’Ecuyer. A comparison of Monte Carlo, lattice rules and other low-discrepancy point sets. In H. Niederreiter and J. Spanier, editors, Monte Carlo and Quasi-Monte Carlo Methods 1998, pages 326–340, Berlin, 2000. Springer-Verlag. 8. A. B. Owen. Latin supercube sampling for very high-dimensional simulations. ACM Transactions on Modeling and Computer Simulation, 8(1):71–102, 1998. 9. A. B. Owen. Variance with alternative scramblings of digital nets. ACM Transactions on Modeling and Computer Simulation, 13(4):363–378, 2003.

Experimental Designs Using Digital Nets with Small Numbers of Points Kwong-Ip Liu1 and Fred J. Hickernell2 1

2

Department of Mathematics, Hong Kong Baptist University, Kowloon Tong, Hong Kong [email protected] Department of Mathematics, Hong Kong Baptist University, Kowloon Tong, Hong Kong. Present address: Department of Applied Mathematics, Illinois Institute of Technology, Room 208, Bldg. E1, 10 W. 32nd St., Chicago, IL 60616 USA [email protected]

Summary. Digital nets can improve upon traditional fractional factorial designs because for a budget of n = bm runs, they can allow b, . . . , bm−1 , or bm levels per factor, while retaining the good balance properties of orthogonal arrays. However, the t-value typically used to characterize the quality of digital nets is not adequate for the purposes of experimental design. Rather, concepts from the experimental design literature, such as strength, resolution and aberration should be used. Moreover, the known number-theoretic constructions of digital nets are optimized for large m, whereas for laboratory experiments one typically has n = bm less than 100. This article describes some recent work on constructing digital nets with small numbers of points that are suitable for experimental designs. Coding theory provides some bounds on the quality of designs that may be expected. The generating matrices for the designs are found by computational search. The quality of the designs obtained is compared with the coding theory bounds.

1 Introduction An experiment with s factors is considered. It is desired to understand how a response y depends on the controllable factors xj . Although the original experimental domain may be a set of continuous values, in laboratory experiments it is more practical to fix a finite number of discrete levels, i.e., qj levels for factor j. We carry out n experiments at the settings x1 , . . . , xn , where xi = (xi1 , . . . , xis )T . The results of experiments are fit by a regression model, 

This research was supported by a Hong Kong RGC grant HKBU2007/03P.

344

K.-I. Liu and F.J. Hickernell

⎞ ⎛ ⎞ y1 β1 g1 (x1 ) + · · · + βp gp (x1 ) + ε1 ⎟ ⎜ ⎟ ⎜ .. y = ⎝ ... ⎠ = ⎝ ⎠ = Gβ + ε , . yn β1 g1 (xn ) + · · · + βp gp (xn ) + εn ⎛

ˆ = (GT G)−1 GT y = β + (GT G)−1 GT ε . β The model together with the design determine G. To estimate β, G must be of full rank. Furthermore, the estimation of β is less affected by noise if the condition number of G is small. If the regression model, i.e., the gk (x), is known in advance, the design can be chosen optimally [10]. However, in practice the model is determined only after the experiment is performed. In this case, one should choose the design points independently of the model by spreading them uniformly in the design space. A good digital net is such a design.

2 Orthogonal Arrays and Digital Net Designs If there are qj levels per factor, the design with all possible combinations of levels would require q1 × · · · × qs experiments. This number of experiments is usually too large even for moderately large qj and s. An alternative is to choose the design points that are full combinations when considering only r out of s factors. Such a design is an orthogonal array of strength r denoted OA(n, q1 × · · · × qs , r) [1, 2]. If the generating matrices are chosen carefully, a digital net [6,8,10] can be considered as a generalized orthogonal array OA(bm , bm × · · · × bm , 1), where b is the base. It has good strengths for different number of factors and levels. The level of a factor is said to be collapsed if the original labels are mapped to fewer numbers of labels by the most significant digits. For example, the original label is a number l(m) ≡ l1 · · · lm where the li are digits of l(m) in base b. It can be collapsed to be the new label l(m−p) ≡ l1 · · · lm−p between 0 and bm−p − 1, for any 0 < p ≤ m. Thus a digital net design can be considered as an orthogonal array with a flexible number of levels. A digital net also has a flexible number of factors. The strengths of orthogonal arrays formed by a smaller number of factors tend to be larger. In the following sections, there are tables of the strengths of orthogonal arrays formed from digital nets by collapsing to bm1 × · · · × bms levels. For the sake of simplicity, it is assumed that m1 ≥ · · · ≥ ms .

3 Quality Measures and Bounds of Digital Net Designs Let ψ(i) and φ(i) be vectors of digits of an integer i ≡ i1 . . . im in base b defined as,

Experimental Designs Using Digital Nets with Small Numbers of Points

345

ψ(i) = (im , . . . , i1 )T , φ(i) = (i1 , . . . , im )T . The j-th coordinate of the i-th point in the digital net pij can be found by, φ(pij ) = Cj ψ(i) mod b,

i = 0, . . . , bm − 1,

(1)

where Cj is the generating matrix of the j-th coordinate. If the coordinates of the digital net are scaled within [0, 1), pij would be divided by bm . The quality of a digital net for QMC methods is measured by the t-value, which can be determined from the generating matrices [10]. Proposition 1. For a (t, m, s)-net and any mi ≥ 0 with m1 +. . .+ms = m−t, the row vectors of the generator matrices of [C1 ]m1 ∪· · ·∪[Cs ]ms are all linearly independent. Here and below [C]j represents the first j rows of the matrix C. In general, the strengths of a collapsed digital net can be determined from the generating matrices. Proposition 2. A digital net design has strength r when the levels are collapsed to bm1 × · · · × bms if for any 1 ≤ g1 ≤ · · · ≤ gr ≤ s the row vectors [Cg1 ]mg1 ∪ · · · ∪ [Cgr ]mgr are linearly independent. Proof. For any given fixed gj , the above assumption about the linear independence of the row vectors implies that the (mg1 + · · · + mgr ) × m matrix C = ([Cg1 ]Tmg1 , . . . , [Cgr ]Tmgr )T has full rank, since mg1 +· · ·+mgr ≤ m. Thus, m

+···+m

g1 gr exactly bm−mg1 −···−mgr times. Cx, x ∈ Fm b takes on each value in Fb From the defining relationship (1) for a digital net this in turn implies that the collapsed points (φ(mg1 ) (pi,g1 ), . . . , φ(mgr ) (pi,gr )), i = 0, . . . bm − 1, take on each possible value in {0, . . . , bmg1 − 1} × · · · × {0, . . . , bmgr − 1} exactly bm−mg1 −···−mgr times. Since this is true regardless of the choice of gj , the collapsed points form an orthogonal array of strength r. 

A good digital net design can satisfy Proposition 2 with several different choices of strengths, numbers of factors and collapsed levels. The largest values of s for which Proposition 2 holds as a function of r and q1 × · · · × qs can be summarized in a strength table, an example of which is given in the upper part of Table 1. The base of the digital net design in this example is b = 2. The numbers in the upper table are the maximum numbers of factors, s, of the collapsed design with specific strength. The entry ‘×’ means the strength is impossible to achieve for that choice of collapsed levels. The entry ‘–’ means the number of factors are equal to the entry on the right. The entries attaining the Rao’s bound are bold (see (3) and (4) below). The lower part of Table 1 is an example of a t-table. The numbers in the t-table are the maximum numbers of factors, s, of the design with a specific t-value [6]. Table 2 shows a digital net design with this strength table. The corresponding generating matrices are given in Figure 1. The digital net design in Table 2 with strength table

346

K.-I. Liu and F.J. Hickernell

Table 1. The strength table and t-table for the digital net design in Table 2 with 16 levels and 16 runs Largest s Levels, q1 × · · · × qs s

2 4 × 2s−1 42 × 2s−2 43 × 2s−3 44 × 2s−4 4s s−1 8×2

Largest s

2

Strength, r 3

4

15 13 11 9 7 5 6

– 3 × × × × ×

5 × × × × × ×

2

Quality, t 1

0

15

5

1

Table 2. An example of digital net design with 16 levels and 16 runs Factor 1 2 3 4 5 6 7 8 8 10 11 12 13 14 15 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15

0 8 4 12 1 9 5 13 2 10 6 14 3 11 7 15

0 8 12 4 5 13 9 1 10 2 6 14 15 7 3 11

0 8 4 12 9 1 13 5 6 14 2 10 15 7 11 3

0 8 12 4 13 5 1 9 6 14 10 2 11 3 7 15

0 8 1 9 2 10 3 11 12 4 13 5 14 6 15 7

0 1 8 9 10 11 2 3 12 13 4 5 6 7 14 15

0 1 8 9 2 3 10 11 12 13 4 5 14 15 6 7

0 8 9 1 10 2 3 11 12 4 5 13 6 14 15 7

0 1 8 9 10 11 2 3 4 5 12 13 14 15 6 7

0 8 1 9 10 2 11 3 12 4 13 5 6 14 7 15

0 1 8 9 2 3 10 11 4 5 12 13 6 7 14 15

0 8 9 1 2 10 11 3 4 12 13 5 6 14 15 7

0 1 2 3 8 9 10 11 4 5 6 7 12 13 14 15

0 1 2 3 8 9 10 11 12 13 14 15 4 5 6 7

in Table 1 is a (0, 4, 1)-net, a (1, 4, s)-net for 2 ≤ s ≤ 5, and a (2, 4, s)-net for 6 ≤ s ≤ 15. A (t, m, s)-net in base b may be collapsed to an orthogonal array OA(bm , bm1 × · · · × bms , r) with m1 ≥ · · · ≥ ms provided that m − m1 − · · · − mr ≥ t. Thus, the t-value of a net is related to the strength table. However, the t-value is too crude as a measure of net quality for experimental design purposes. Based on the t-value, the net design with 2s levels and m = 4 cannot be

Experimental Designs Using Digital Nets with Small Numbers of Points

0 0 0 1

C1 00 01 10 00

1 0 0 0

1 0 0 0

C2 00 10 00 01

0 0 1 0

1 0 0 0

C3 10 11 00 01

1 0 1 0

1 0 0 0

C4 01 10 00 01

0 1 1 0

1 0 0 0

C5 11 11 00 01

1 0 0 0

C6 00 00 01 10

1 1 0 0

0 0 0 1

C7 11 00 01 00

1 1 0 0

0 0 0 1

C8 10 00 01 00

1 1 0 0

1 0 0 0

C9 11 00 01 10

1 1 0 0

0 0 0 1

C10 110 001 010 000

1 0 0 0

C11 011 001 010 100

0 0 0 1

C12 100 001 010 000

1 0 0 0

C13 100 001 010 100

0 0 0 1

C14 010 001 100 000

0 0 0 1

C15 011 001 100 000

347

0 1 1 0

Fig. 1. The generating matrices of the digital net design example with 16-level and 16-run.

guaranteed to have strength 4 since m − m1 − m2 − m3 − m4 = 4 − 1 − 1 − 1 − 1 = 0 < 1 = t However, in fact one does have a strength 4 design with 5 factors. The coding theoretical orthogonal array literature supplies a number of bounds on the values on the strength tables. Two prominent bounds on strength, r, are given in [2]: Table 3. The upper bounds for a 16-run digital net design Largest s Levels, q1 × · · · × qs s

2 4 × 2s−1 42 × 2s−2 43 × 2s−3 44 × 2s−4 4s 8 × 2s−1

Largest s

2

Strength, r 3

4

15 13 11 9 7 5 9

8 4 × × × × ×

5 × × × × × ×

2

Quality, t 1

0

15

7

3

348

K.-I. Liu and F.J. Hickernell

• Singleton bound, m≥

r (

mi .

(2)

i=1

• Rao’s bound, bm ≥

u (

(

(bmj1 − 1) · · · (bmji − 1),

(3)

i=0 1≤j1 ≤···≤ji ≤s

bm ≥

if r = 2u, u ( (

(bmj1 − 1) · · · (bmji − 1)

i=0 1≤j1 ≤···≤ji ≤s

(

+(bm1 − 1)

(bmj1 − 1) · · · (bmju − 1),

(4)

2≤j1 ≤···≤ju ≤s

if r = 2u + 1. Another bound on the t-value is the generalized Rao’s bound for nets in [5]: b

m

≥ 1+

 u ( i   ( i−1 s i=1 j=1

bm

j−1

j

(b − 1)j bi−j

(5)

if m − t = 2u,  u ( i   ( i−1 s ≥ 1+ (b − 1)j bi−j j j−1 i=1 j=1

+

u+1 ( j=1

s−1 j−1



u j−1

 (b − 1)j bu−j+1

(6)

if m − t = 2u + 1. Table 3 shows these bounds for the case of 16-run digital net designs. In the upper table, the entries with ‘×’ represent the Singleton bounds and the numbers represent Rao’s bounds. Since the values in Table 3 are obtained by considering only a single choice of numbers of levels, tighter bounds should exist when considering all choices of numbers of levels together. In the lower table, the numbers represent the generalized Rao’s bound for nets. Three entries in Table 1 do not attain the Rao’s bounds on strength: 2s levels with strength 3, 4 × 2s−1 levels with strength 3 and 8 × 2s−1 levels with strength 2. Other digital net designs are found to attain the bounds in these entries and their strength tables are shown in Table 4–7. The values in parentheses are the differences to the corresponding entries in Table 1. In all cases, it is impossible to attain the Rao’s bound on strength for all entries. The details can be found in www.uniformdesign.org.

Experimental Designs Using Digital Nets with Small Numbers of Points

349

Table 4. The strength table and t-table of a design attains the bound in 8 × 2s−1 levels with strength 2 Largest s Levels, q1 × · · · × qs 2s 4 × 2s−1 42 × 2s−2 43 × 2s−3 4s s−1 8×2

Largest s

Strength, r 3

4

– 3 × × × ×

5 × × × × ×

2

Quality, t 1

0

15

5

1

2 15 13 8 (–3) 7 (–2) 5 9 (+3)

Table 5. The strength table and t-table of a design attains the bound in 2s levels with strength 3 and 4 × 2s−1 levels with strength 3 Largest s Levels, q1 × · · · × qs 2s 4 × 2s−1 42 × 2s−2 43 × 2s−3 4s s−1 8×2

Largest s

2

Strength, r 3

15 7 6 5 4 3

8 (+3) 4 (+1) × × × ×

(–6) (–5) (–4) (–1) (–3)

4 4 (–1) × × × × ×

2

Quality, t 1

0

15

4

1

4 Search Algorithm The search for a good digital net is an optimization problem on the domain of generating matrices. The genetic algorithm has been successfully applied to find (t, m, s)-nets [4]. A different approach is taken to search the generating matrices of a digital net design that satisfies a given strength table. It has been found that the efficiency of the search process improves if the target strength table is achievable. Otherwise, the search time would be similar to an exhaustive search. The following are the definitions of the variables and the search algorithm. m – 2m runs. s – The number of factors.

350

K.-I. Liu and F.J. Hickernell

Table 6. The strength table and t-table of a design attains the bound in 2s levels with strength 4 and 4 × 2s−1 levels with strength 3 Largest s Levels, q1 × · · · × qs 2s 4 × 2s−1 4s 8 × 2s−1

Largest s

2 15 – (–9) 4 (–1) 3 (–3)

Strength, r 3

4

– 4 (+1) × ×

5 × × ×

2

Quality, t 1

0

15

4

1

Table 7. The strength table and t-table of a design attains the bound in 4 × 2s−1 levels with strength 3 and 4s levels with strength 2 Largest s Levels, q1 × · · · × qs s

2 4 × 2s−1 42 × 2s−2 43 × 2s−3 44 × 2s−4 4s s−1 8×2

Largest s

2 15 13 11 9 7 5 3 (–3)

Strength, r 3 – (–1) 4 (+1) × × × × ×

4 4 (–1) × × × × × ×

2

Quality, t 1

0

15

5

1

Cj – The jth coordinate generator matrix. [C]k – The first k row vectors of the matrix C. (r, (l1 , l2 , · · · , lj )) – The specification that requests level 2l1 × 2l2 × · · · × 2lj to be strength r. Span(v) – The set of vectors that is spanned by v. RelSpec(i,j) – The set of specification such that lj ≥ i. FindGenerator(l,f ) if (l == m) l=0 f =f +1 if (f == s) return DONE

Experimental Designs Using Digital Nets with Small Numbers of Points

351

CV = CandidateVector(l,f ) for (V in CV ) Vl,f = V FindGenerator(l + 1,f ) CandidateVector(i,j) ret = AllVectors() for (r, (l1 , l2 , · · · , lj )) in RelSpec(i,j) for all 1 ≤ j1 < · · · < jr−1 ≤ j − 1, ret = ret - Span([Cj1 ]l1 ∪ · · · ∪ [Cjr−1 ]lr−1 ∪ [Cjr ]i−1 ) return ret The basic procedure of the search algorithm is to find the set of possible candidate vectors for each row in the generating matrices according to Proposition 2 for all collapsed levels in the target strength table. A test vector is selected from each candidate vector set. If the resulting digital net design has a strength table that matches the target strength table, the algorithm stops. The procedure is started from the generating matrix of factor 1. For each generating matrix, the candidate vector set is searched starting from the first row. The candidate vectors of latter rows depend on the test vectors assigned in the former rows. The size of the candidate vector set for the latter rows is reduced as the number of test vectors increases. Many different assignments of the test vectors are equivalent since the dependences between the test vectors are the same. Therefore, it is not necessary to test all combinations of test vectors. For example, the four row vectors in C1 (called a, b, c and d) in Fig. 1 can be considered as the basis to produce the row vectors in other generating matrices. Figure 2 shows the row vectors of C1 –C5 in this way. If these four vectors are replaced by other four linearly independent vectors and the other row vectors are recalculated according to Fig. 2, the resulting digital net design would have the same strength table. The new design is a scrambling of the original design [9]. Consider the search of the generating matrices according to the strength table in Table 1. The specifications are defined in Fig. 3. Some of RelSpec(i,j) and CandidateVector(i,j) are shown in Table 8 and Table 9 respectively. Decimal numbers are used to represent the binary vectors in Table 9. Consider C1 a b c d

C2 d c a b

C3 a+c+d b+c a b

C4 b+d a+c a b

C5 b+c+d a+b+c a b

Fig. 2. The relationship of the row vectors of C1 – C5 in Table 1.

352

K.-I. Liu and F.J. Hickernell s1: s2: s3: s4: s5: s6: s7: s8: s9: s10: s11:

(2,(1,1,1,1,1,1,1,1,1,1,1,1,1,1,1)) (3,(1,1,1,1,1)) (4,(1,1,1,1,1)) (2,(2,1,1,1,1,1,1,1,1,1,1,1,1)) (3,(2,1,1)) (2,(2,2,1,1,1,1,1,1,1,1,1)) (2,(2,2,2,1,1,1,1,1,1)) (2,(2,2,2,2,1,1,1)) (2,(2,2,2,2,2)) (2,(3,1,1,1,1,1)) (1,(4,4,4,4,4,4,4,4,4,4,4,4,4,4,4))

Fig. 3. The specifications of the example digital net design. Table 8. An example of RelSpec(i,j) j=1 i= 1

2 3 4

2

3

4

5

6

s1 to s11 s1 to s11 s1 to s11 s1 to s4, s1 to s4, s1, s4, s6 to s11 s6 to s11 s6 to s8, s10, s11 s4 to s11 s6 to s9, s7 to s9, s8, s9, s9, s11 s11 s11 s11 s11 s10, s11 s11 ··· ··· ··· ··· s11 ··· ··· ··· ··· ···

7

···

s1, s4, · · · s6 to s8, s11 ··· ··· ··· ···

··· ···

6

7

···

9

7

···

Table 9. An example of CandidateVector(i,j) j=1

2

i= 1

1 to 15

8 to 15

2

2 to 15

3

4 to 15

4

8 to 15

3

4

5

13, 14, 10, 14, 14 15 15 4 to 7, 6, 7, 10, 5, 15 7, 9 12 to 15 11 1 to 3, 5 · · · ··· ··· to 7, 9 to 11, 13 to 15 ··· ··· ··· ···

1 to 8, · · · 10 to 15 ··· ···

···

···

···

···

i = 2, j = 4 as an example. The results of RelSpec(2,4) are the specifications which has a number larger than or equal to 2 in the factor 4. The test vectors in the former entries are the first vector in candidate vectors in this case. By RelSpec(2,4) and the test vectors, the CandidateVectors(2,4) is found. For s8 and s9, they request that the first two row vectors of any one of the first three generating matrices are independent to the row vectors of the fourth generating matrix. Factor 1 and 4 (1, 2, 10) spans the vectors 1, 2, 3, 8, 9, 10, 11. Factor 2 and 4 (4, 8, 10) spans the vectors 2, 4, 6, 8, 10, 12, 14. Factor

Experimental Designs Using Digital Nets with Small Numbers of Points

353

Table 10. Execution time for s = 15 m 4 5 6

Execution time (s) < 0.1 < 10.0 < 60.0

3 and 4 (6, 10, 13) spans the vectors 1, 6, 7, 10, 11, 12, 13. So the candidate vectors are 5, 15. The execution times of programs to find digital net design with different m and s = 15 are shown in Table 10. The program runs on a commodity computer. The designs and strength tables can be found in www.uniformdesign.org. Even with the same values of s and m, the execution time depends on the target strength table. In general, the execution time decreases if there are more specifications needed to be satisfied.

5 Conclusion and Open Questions Using digital nets in experimental designs gives the advantage of flexibility in the number of levels over orthogonal arrays. It provides good balance for both coarse grain for a large number of factors and fine grain for a few number of factors. Coarse grain and fine grain are useful for factor screening and accurate robust model estimation respectively [3, 12]. The requirements for a good net for experimental design are somewhat different from the quality criteria of a good net for numerical integration and the quality criteria for orthogonal arrays. The Sobol’, Niederreiter and Niederreiter-Xing nets [6,8,11] are constructed with a large number of points, n = bm , in mind. Orthogonal arrays are optimized for one specific level combinations of factors. The number of levels in orthogonal arrays are usually small and the order of factors is unimportant. For the digital net designs constructed here, the number of runs is small, the number of levels per factor is large and flexible, and the order of the factors is significant. There are several open questions for digital net designs: 1. Are there any tighter bounds on the strength table, especially considering several entries simultaneously? 2. Are there better numerical or analytic way to construct digital net design? 3. What are the combinatorial relationships among the generating matrices?

References 1. Dey, A. & Mukerjee, R. (1999), Fractional Factorial Plans, John Wiley & Sons.

354

K.-I. Liu and F.J. Hickernell

2. Hedayat, A. S., Sloane, N. J. A. & Stufken, J. (1999), Orthogonal Arrays: Theory and Applications, Springer Series in Statistics, Springer–Verlag. 3. Hickernell, F. J. & Liu, M. Q. (2002), Uniform designs limit aliasing, Biometrika 89, 893–904. 4. Hong, H. S. (2002), Digital nets and sequences for Quasi-Monte Carlo methods, PhD thesis, Hong Kong Baptist University. 5. Martin, W. J. & Stinson, D. R. (1999), A generalized Rao bound for ordered orthogonal arrays and (t, m, s)-nets, Canadian Mathematical Bulletin 42(3), 359– 370. 6. Niederreiter, H. (1988), Low-discrepancy and low-dispersion sequences, J. Numb. Theor. 30, 51–70. 7. Niederreiter, H. (1992), Random Number Generation and Quasi-Monte Carlo Methods, Society for Industrial and Applied Mathematics. 8. Niederreiter, H. & Xing, C. (1998) , Nets, (t, s)-sequences and algebraic geometry, in Random and Quasi-Random Point Sets, Vol. 138 of Lecture Notes in Statistics, Springer-Verlag, pp. 267–302. 9. Owen, A. B. (1995), Randomly permuted (t, m, s)-nets and (t, s)-sequences, in Monte Carlo and Quasi-Monte Carlo Methods in Scientific Computing, Vol. 106 of Lecture Notes in Statistics, Springer-Verlag, pp. 299–317. 10. Pukelsheim, F. (1993), Optimal Design of Experiments, Wiley. 11. Sobol’, I. M. (1967), The distribution of points in a cube and the approximate evaluation of integrals, U.S.S.R. Comput. Math. Math. Phys. 7, 86–112. 12. Yue, R. X. & Hickernell, F. J. (1999), Robust designs for fitting linear models with misspecification, Statist. Sinica 9, 1053–1069.

Concentration Inequalities for Euler Schemes Florent Malrieu1 and Denis Talay2 1

2

IRMAR, UMR 6625, Universit´e Rennes 1, France [email protected] INRIA Sophia-Antipolis, 2004 route des Lucioles, B.P. 93, 06902 Sophia-Antipolis Cedex, France [email protected]

Summary. We establish a Poincar´e inequality for the law at time t of the explicit Euler scheme for a stochastic differential equation. When the diffusion coefficient is constant, we also establish a Logarithmic Sobolev inequality for both the explicit and implicit Euler scheme, with a constant related to the convexity of the drift coefficient. Then we provide exact confidence intervals for the convergence of Monte Carlo methods.

1 Poincar´ e and Logarithmic Sobolev Inequalities To describe and control the statistical errors of probabilistic numerical methods, one can use better results than limit theorems such as Central Limit Theorems. Indeed, it is worthy having non asymptotic error estimates in order to choose numerical parameters (number of Monte Carlo simulations, or number of particles, or time length of an ergodic simulation) in terms of the desired accuracy and confidence interval. To this end, concentration inequalities are extremely useful and accurate. As reminded in the Sect. 6 below, sufficient conditions for concentration inequalities are Poincar´e (or spectral gap) and Logarithmic Sobolev inequalities. Such inequalities consist in bounding from above a variance or an entropy by an energy quantity. We start by defining Poincar´e and Logarithmic Sobolev inequalities for measures on Rd . Remark 1. In what follows, we call “smooth” function a C ∞ function with polynomial growth. Definition 1 (Poincar´ e inequality). A probability measure µ on Rd satisfies a Poincar´e (or spectral gap) inequality with constant C if   2 (1) Varµ (f ) := Eµ (f 2 ) − (Eµ f )2 ≤ C Eµ |∇f | for all smooth functions f with bounded derivatives.

356

F. Malrieu and D. Talay

Definition 2 (Logarithmic Sobolev inequality). The probability measure µ on Rd satisfies a Logarithmic Sobolev inequality with constant C if      2 (2) Entµ f 2 := f 2 log f 2 dµ − f 2 dµ log f 2 dµ ≤ C Eµ |∇f | for all smooth functions f bounded derivatives. The Logarithmic Sobolev inequality implies the Poincar´e inequality and a better concentration inequality (see (18) and (19)) below. One can easily check that the Gaussian measure N (m, S) on Rd satisfies a Poincar´e (respectively Logarithmic Sobolev) inequality with constant ρ (respectively 2ρ), where ρ is the largest eigenvalue of the covariance matrix S. We now consider a much less elementary example and we follow [3]. Let (Xt ) be a time continuous Markov process with infinitesimal generator L. Set   2 α(s) := Ps (Pt−s f ) . As the time derivative of Pt f is Pt Lf , one has α (s) = 2Ps Γ Pt−s f ,

(3)

where Γ (f, g) :=

1 [L(f g) − f Lg − gLf ] 2

and Γ f := Γ (f, f ) .

Suppose that the semigroup Pt satisfies the commutation relation ∃ρ ∈ R, Γ Pt f ≤ e−2ρt Pt Γ f .

(4)

Then it also satisfies the Poincar´e inequality since  t   1 − e−2ρt 2 Pt f 2 − (Pt f ) = α(t) − α(0) = Pt Γ f . α (s) ds ≤ ρ 0 It now remains to get sufficient conditions for (4). Set Γ 2 f :=

1 [L(Γ f ) − 2Γ (f, Lf )] , 2

´ and notice that α (s) = 4Ps Γ 2 Pt−s f . Suppose that the Bakry–Emery criterion with curvature ρ holds, that is, Γ 2f ≥ ρ Γ f . Then α (s) ≥ 2ρα (s), from which one can deduce (4).

(5)

Concentration Inequalities for Euler Schemes

357

We end this section by considering the special case of diffusion processes. Let (Xt )t≥0 be the Rd valued diffusion process solution of the stochastic differential equation  t  t√ b(Xs ) ds + 2σ(Xs ) dBs , (6) Xt = X0 + 0

0

where (Bt )t≥0 is a Brownian motion on Rd , σ(x) is a d × d matrix valued function, and b(x) is a Rd valued function. A straightforward computation provides 2 Γ (f ) = |σ∇f | . Using the fact that L is the generator of a diffusion process, one can prove that the logarithmic Sobolev inequality EntPt (f 2 ) ≤

 2 1 − e−2ρt Pt (Γ f ) ρ

is implied by the reinforced commutation relation :  : Γ Pt f ≤ e−ρt Pt Γf , that is,

|σ∇Pt f | ≤ e−ρt Pt (|σ∇f |) .

(7)

In addition, one can show that this reinforced commutation relation is equiv´ alent to the Bakry–Emery curvature criterion. In the case of one–dimensional diffusions, this criterion is equivalent to the condition   σ  (x) b(x) − b (x) ≥ ρ . ∃ρ ∈ R, inf σ(x)σ  (x) + (8) x∈R σ(x) Observe that this condition obviously holds true when σ, σ  , σ  , b and b are bounded functions, and σ is bounded from below by a positive constant. We now aim to get Poincar´e and Logarithmic Sobolev inequalities for approximation schemes of diffusion processes and particle systems for McKean– Vlasov partial differential equations. Complete proofs will appear in [10].

2 Poincar´ e Inequalities for Multidimensional Euler Schemes Consider the Euler scheme (Xnγ )n∈N on Rd with discretization step γ: √ γ Xn+1 := Xnγ + b(Xnγ )γ + 2σ(Xnγ ) (Bn+1 − Bn ) .

(9)

This scheme discretizes (6) and defines a Markov chain on Rd with transition kernel

358

F. Malrieu and D. Talay

   : K(f )(x) := E f x + b(x)γ + 2γσ(x)Y , where Y is Gaussian N (0, Id ). We conjecture that, under appropriate hypotheses on the functions b and σ, the law of Xnγ satisfies a Poincar´e inequality with a constant uniform in γ < 1 and 1 ≤ n ≤ γ1 . However, at the time being, we have succeeded to only get a partial version of this result. The extension is in progress. Proposition 1. If d = 1, suppose that the functions σ and b have continuous and bounded derivatives and σ is bounded. If d > 1, suppose in addition that σ is constant. Then, for all n ∈ N and all smooth functions f ,     2 2 K n f 2 (x) − (K n f (x)) ≤ Cγ,n K n |∇f | (x) . The constant Cγ,n can be chosen as Cγ,n = γc

(Cγ )n − 1 , Cγ − 1

(10)

where Cγ satisfies ∃C > 0, ∀0 < γ < 1, Cγ1/γ ≤ C. Proof. We mimic the continuous time semigroup argument. Observe that 2

K n (f 2 ) − (K n f ) =

n 8  (   2 9 K i (K n−i f )2 − K i−1 K n−i+1 f i=1

=

n (

8     2 9 . K i−1 K (K n−i f )2 − K K n−i f

i=1

Therefore, Var

Kn

(f ) =

n (

K i−1 VarK (K n−i f ) .

(11)

i=1

Notice that the operator VarK () is the discrete time version of the operator Γ . The kernel K is the Gaussian law with mean x + b(x)γ and covariance matrix 2γσ(x)σ ∗ (x). Thus, since σ is bounded, it satisfies the Poincar´e inequality     2 2 VarK (f )(x) ≤ 2γ K |σ(x)∇f | (x) ≤ cγ K |∇f | (x) . (12) In addition,  √ ∇Kf (x) = E Id + γJac b(x) + γJac (σ(x)Y )  √ ×∇f (x + γb(x) + γσ(x)Y ) .

Concentration Inequalities for Euler Schemes

359

Therefore, the Cauchy–Schwarz inequality leads to 2

|∇Kf (x)| ≤

( i

⎞2   ( ∂σij √ 2 E ⎝1 + bi (x)γ + γ (x)Yj ⎠ K |∇f | (x) , ∂xk ⎛

j,k

from which the desired result easily follows. In the next sections we prove that, under the above restrictive hypotheses, we even can get Logarithmic Sobolev inequalities.

3 Logarithmic Sobolev Inequalities for One-Dimensional Euler and Milstein Schemes The aim of this section is to establish Logarithmic Sobolev inequalities for numerical schemes in dimension one and to make the constants explicit in the inequalities in terms of the curvature of the solution of (6). The Commutation Relation for the Bernoulli Scheme Consider the approximation scheme with transition kernel   : Jf (x) := E f (x + γb(x) + 2γσ(x)Z) , where the law of Z is the probability measure 12 δ−1 + 12 δ1 . Then (σ(Jf ) )(x) is equal to     : : E σ(x)(1 + γb (x) + 2γσ  (x)Z) f  x + γb(x) + 2γσ(x)Z . Thus

   : , σ(x)(Jf ) (x) = E (1 − αx (γ))(σf  ) x + γb(x) + 2γσ(x)Z

where αx (γ) :=

σ(x + γb(x) +

√ √ 2γσ(x)Z) − σ(x)(1 + γb (x) + 2γσ  (x)Z) √ . σ(x + γb(x) + 2γσ(x)Z)

In view of the Taylor formula,     : : σ x + γb(x) + 2γσ(x)Z = σ(x) + σ  (x) b(x)γ + 2γσ(x)Z + σ  (x)σ(x)2 Z 2 γ + O(γ 3/2 ) . Therefore

360

F. Malrieu and D. Talay

  σ  (x)b(x) − b (x) γ + O(γ 3/2 ) , αx (γ) = σ(x)σ  (x) + σ(x) since Z 2 = 1 almost surely. The curvature criterion (8) leads to αx (γ) ≥ ργ + O(γ 3/2 ). Consequently, for all γ small enough it holds that   |σ(x)(Jf ) (x)| ≤ 1 − ργ + O(γ 3/2 ) J (|σf  |) (x) . Now, the Bernoulli law satisfies a Logarithmic Sobolev inequality with constant 2 (see [1]). We thus deduce that the iterated kernel J n of the Bernoulli scheme satisfies a Logarithmic Sobolev inequality with constant   2 3/2 2n 1 − (1 − ργ + O(γ . )) ρ + O(γ 1/2 ) The Milstein Scheme The previous result seems surprising since we have used that Bernoulli r.v. satisfy Z 2 = 1 a.s. Consider the new Markov chain with kernel    : Jf (x) := E f x + γb(x) + 2γZ + σ  (x)σ(x)(Z 2 − 1)γ , where the law of Z is a probability measure with compact support, mean 0 and variance 1. This chain is the one-dimensional Milstein scheme for (6). For a comparison with the Euler scheme, see, e.g. [15]. Similar arguments as above lead to the following result. Proposition 2. Let Z have a law with compact support, mean 0 and variance 1 which satisfies a Logarithmic Sobolev inequality with constant c. Then the iterated kernel J n of the Milstein scheme satisfies a Logarithmic Sobolev inequality with constant   c 3/2 2n )) 1 − (1 − ργ + O(γ . ρ + O(γ 1/2 )

4 Logarithmic Sobolev Inequalities for Multidimensional Euler Schemes with Constant Diffusion Coefficient and Potential Drift Coefficient In this section, we are given a smooth function U and we consider the equation √ dXt = 2dBt − ∇U (Xt ) dt .

Concentration Inequalities for Euler Schemes

361

4.1 The Explicit Euler Scheme Assume in this subsection that ∇U is a uniformly Lipschitz function on Rd . 2 For U (x) = |x| /2 one gets the Ornstein–Uhlenbeck process. The transition kernel of the explicit Euler scheme is    : Kf (x) = E f x − ∇U (x)γ + 2γY , where Y is a d dimensional Gaussian vector N (0, Id ). Let λ ∈ R be the largest real number such that 2

Hess U (x)v, v ≥ λ |v|

(13)

for all x and v in Rd . We now assume that λγ < 1. This technical assumption is not restrictive since the discretization step γ is small. Theorem 1. For all n ∈ N, x ∈ R and smooth functions f from Rd to R,   2 EntK n (f 2 ) ≤ Dγ,n K n |∇f | , where Dγ,n :=

  4 1 − (1 − λγ)2n . λ(2 − λγ)

(14)

Remark 2. If λ is equal to 0, Dγ,n needs to be understood as 4nγ. Proof. The kernel K satisfies a Logarithmic Sobolev inequality with constant 4γ. Moreover, ∇Kf (x) = (Id − γHess U (x)) K (∇f ) (x) . Therefore |∇Kf (x)| ≤ (1 − γλ)K (|∇f |) (x) .

(15)

Observe that EntK n (f 2 ) := K n (f 2 log f 2 ) − K n (f 2 ) log K n (f 2 ) is equal to n ( 

    K i K n−i (f 2 ) log K n−i (f 2 ) − K i−1 K n−i+1 (f 2 ) log K n−i+1 (f 2 ) .

i=1

In the sequel, gn−i will stand for EntK n (f 2 ) =

n ( i=1

:

K n−i (f 2 ). We have

n   (   2 2 K i−1 EntK (gn−i ) ≤ 4γ K i |∇gn−i | , i=1

362

F. Malrieu and D. Talay

since K satisfies a Logarithmic Sobolev inequality with constant 4γ. Now, in view of the commutation relation (15), we get 2

|∇gn−i | =

∇K n−i (f 2 ) 2 4K n−i (f 2 )

2 K ∇K n−i−1 (f 2 ) ≤ (1 − λγ) 4KK n−i−1 (f 2 ) 

2

for all 1 ≤ i ≤ n. Therefore, using Cauchy–Schwarz inequality,  2 f (Kf )2 ≤K , K(g) g from which " 2 &    ∇K n−i−1 (f 2 ) 2 K ∇K n−i−1 (f 2 ) 2 . ≤ K = K |∇g | n−i−1 4KK n−i−1 (f 2 ) 4K n−i−1 (f 2 ) A straightforward induction shows that   2 2 |∇gn−i | ≤ (1 − λγ)2(n−i) K n−i |∇f | . Consequently, EntK n (f ) ≤ 4γ 2

"n−1 (

&

i=0

=

   1 − (1 − λγ)2n n  2 2 K n |∇f | = 4γ K |∇f | 2 1 − (1 − λγ)     2 1 − (1 − λγ)2n K n |∇f | ,

(1 − λγ)

2i

4 λ(2 − λγ)

which ends the proof. 4.2 The Implicit Euler Scheme In this subsection we assume that U is a uniformly convex function, that is, there exists λ > 0 such that 2

Hess U (x)v, v ≥ λ |v| for all x, v ∈ Rd . Since the drift coefficient −∇U is not necessarily globally Lipschitz, we consider the implicit Euler scheme :  γ  γ Xn+1 γ + 2γY , = Xnγ − ∇U Xn+1 where Y is a standard Gaussian variable on Rd . Setting ϕ(x) := (I + ∇U (x)γ) ¯ of the implicit Euler scheme is the kernel K

−1

(x) ,

Concentration Inequalities for Euler Schemes





¯ (x) = E f ◦ ϕ x + Kf

:

363

 2γY

.

Let N (x, 2γI) be the Gaussian distribution with mean x and covariance matrix 2γId . We have   2 EntK¯ (f 2 ) = EntN (x,2γI) ((f ◦ ϕ)2 ) ≤ 4γEN (x,2γI) |∇(f ◦ ϕ)| . In view of the definition of ϕ we get −1

Jac ϕ(x) = [Id + γHess U (x)] and thus Jac ϕ(x)v, v ≤ (1 + γλ)

−1

,

2

|v|

for all v in Rd , from which |∇(f ◦ ϕ)| = |(Jac ϕ) (∇f (ϕ))| ≤

1 |(∇f ) ◦ ϕ| . 1 + λγ

¯ Consequently, the kernels (K(·)(x)) x satisfy a Logarithmic Sobolev inequality 4γ with constant 1+λγ . On the other hand, ¯ (x) = EN (x,2γI) [(Jac ϕ)(∇f ) ◦ ϕ] . ∇Kf ¯ and ∇ satisfy the commutation relation Then K ¯ )(x) ≤ (1 + γλ)−1 K ¯ (|∇f |) (x) . ∇K(f Obvious adaptations of the proof of Theorem 1 lead to Theorem 2. For all n ∈ N, x ∈ R and smooth functions f from Rd to R one has   ¯ n |∇f |2 , EntK¯ n (f 2 ) ≤ Dγ,n K where Dγ,n

4(1 + λγ) = λ(2 + λγ)

 1−

1 (1 + λγ)2n

 .

(16)

5 Uniform Logarithmic Sobolev Inequalities for One–Dimensional Euler Schemes with Constant Diffusion Coefficient and Convex Potential Drift Coefficient Let V be a smooth functions from R to R. Let (Xt )t≥0 be the solution of √ dXt = 2 dBt − V  (Xt ) dt .

364

F. Malrieu and D. Talay

Notice that

   ∇Pt f (x) = E ∇f (Xt ) exp −

t

x





V (Xs ) ds

.

(17)

0

When V  ≥ λ > 0 we easily get the commutation relation |∇Pt f | ≤ e−λt Pt (|∇f |) . We now consider the less obvious case where V  is supposed nonnegative only. 5.1 Poincar´ e Inequality for the Diffusion Process Lemma 1. Let

    t V  (Xs ) ds . D(t, x) := Ex exp −2 0

Then it exist t0 > 0 such that D(t0 ) := sup D(t0 , x) < 1. x∈R

Proof. One has D(t + s) ≤ D(t)D(s) for all t ≥ 0 and s ≥ 0. Indeed, for all t ≥ 0, s ≥ 0 and x ∈ R, the Markov property ensures that  '    t    s x  Xt  V (Xu ) du E V (Xu ) du D(t + s, x) = E exp −2 exp −2 0 0   t   = Ex D(s, Xt ) exp −2 V  (Xu ) du 0     t ≤ D(s)Ex exp −2 V  (Xu ) du = D(s)D(t, x) ≤ D(s)D(t). 0

For x ≥ a, set τa := inf {t ≥ 0, Xtx = a}. Then,   t   D(t, x) = Ex 1 {τa
The second term on the r.h.s. is bounded from above by exp(−λt). The first one can be bounded from above as follows:   t   x  V (Xs ) ds E 1 {τa
  a = E 1 {τa
≤ D(t/2, a) + e−λt/2 .

Concentration Inequalities for Euler Schemes

365

One can easily show that sup D(t, x) ≤ D(t/2, a) + e−λt/2 + e−λt . x≥a

The right hand side is bounded from above by 1 for all t large enough. By symmetry, one also has sup D(t, x) < 1 . |x|≥a

Finally, the continuity of x → D(t, x) ensures that sup {D(t, x), x ∈ [−a, a]} < 1 for all t > 0, which ends the proof. Proposition 3. Assume that V is convex and it exists a ≥ 0 and λ > 0 such that sup {V  (x), |x| ≥ a}. It then exists t0 ≥ 0 such that D(t0 ) < 1 and     1 − D(t0 )t/t0 −1 2 VarPt (·)(x) (f ) ≤ 2t0 1 + Pt |∇f | 1 − D(t0 ) for all t > t0 . Moreover, the invariant measure µ of (Xt ) satisfies a Poincar´e inequality with constant 2t0 (1 + 1/(1 − D(t0 ))). Observe that the proposition implies that, for all t0 > 0 and n ∈ N,     2 2 2 |∇Pnt0 f (x)| ≤ D(nt0 )Pnt0 |∇f | (x) ≤ D(t0 )n Pnt0 |∇f | (x) . Proof. Let t0 > 0 be as in Lemma 1 and set K(f )(x) := Pt0 f (x). Arbitrarily choose t > 0. Let n be the integer part of t/t0 . We have VarPt (·)(x) (f ) = Pt (f 2 )(x) − (Pt (f )(x))2   = Pnt0 VarPt−nt0 (f ) + VarPnt0 (Pt−nt0 f ) . Since t − nt0 < t0 , Pt−nt0 satisfies a Poincar´e inequality with a constant bounded by 2t0 . Therefore,   2 VarPt (·)(x) (f ) ≤ 2t0 Pt |∇f | + VarK n (Pt−nt0 (f )) . Moreover, K n satisfies the Poincar´e inequality VarK n (f ) =≤

2t0 (1 − D(t0 )n ) n 2 K (|∇f | ). 1 − D(t0 ) 2

2

In view of the commutation relation |∇Pt−nt0 (f )| ≤ Pt−nt0 (|∇f | ), we finally get   2t (1 − D(t )n ) 0 0 2 2 K n Pt−nt0 (|∇f | ) . VarPt (·)(x) (f ) ≤ 2t0 Pt |∇f | + 1 − D(t0 )

366

F. Malrieu and D. Talay

5.2 Uniform Poincar´ e Inequality for the Euler Scheme We now get a uniform Poincar´e inequality for the Euler scheme with kernel K(f )(x) = E {f (x − V  (x)α + Y )} , where Y is Gaussian N (0, 2α). Consider the commutation relation ∇K 2 f (x) = (1 − αV  (x))Ex [(1 − αV  (X1 ))∇f (X2 )] . As 1 − αV  (x) ≤ 1, we have   ∇K 2 f (x) 2 ≤ Ex (1 − αV  (X1 ))2 K 2 (|∇f |2 )(x) . Since  the support of  the law of X1 is the entire real line, for all x ∈ R, Ex (1 − αV  (X1 ))2 < 1. Moreover   sup Ex (1 − αV  (X1 ))2 < 1 x∈R

since V  (y) ≥ λ > 0 if |y| ≥ a. This observation leads to Poincar´e inequalities for both K n and the invariant measure of the Euler scheme; in these inequalities the constants are uniform w.r.t. time. 5.3 Uniform Logarithmic Sobolev Inequality In view of (17) we have   t   x   |∇Pt (f )(x)| ≤ E |f (Xt )| exp − V (Xs ) ds 0     t  ' = E |f  (Xt )| E exp − V  (Xs ) ds X0 = x, Xt . 0

To get the commutation relation |∇Pt (f )| ≤ ρPt (|f  |), one needs to suppose that the following property holds true: Property 1. There exists t0 > 0 such that D(t0 ) := supx,y∈R D(t0 , x, y) < 1, where    t0   V  (Xs ) ds X0 = x, Xt0 = y D(t0 , x, y) := E exp − 0

for all t > 0 and x, y in R. This property holds true when V  is nonnegative. We are now trying to relax the convexity condition on V , assuming only that V is strictly convex out of a compact set.

Concentration Inequalities for Euler Schemes

367

6 Applications 6.1 Monte Carlo Simulations Poincar´e and Logarithmic Sobolev inequalities are important for applications because they provide concentration inequalities for empirical means. The proof of this claim uses a tensorization argument and the Herbst’s argument that we now remind. Theorem 3. Let µ be a probability measure on Rd . If µ satisfies a spectral gap (respectively Logarithmic Sobolev) inequality with constant C, then the measure µ⊗N on RdN satisfies a spectral gap (respectively Logarithmic Sobolev) inequality with constant C. Theorem 4. If µ satisfies a Logarithmic Sobolev inequality with constant c, then for all Lipschitz functions f with Lipschitz constant and all λ > 0,   2 2 K eλf ≤ ecλ  /4 eλKf . The Herbst’s argument ensures that a measure which satisfies a Logarithmic Sobolev inequality has Gaussian tails (see [9]). One then deduces Theorem 5. Let the measure µ on Rd satisfy the Logarithmic Sobolev inequality (2) with constant C. Let X1 , . . . , XN be i.i.d. random variables with law µ. Then, for all bounded Lipschitz functions on Rd , it holds   N 1 ( 2 f (Xi ) − E(f (X1 )) ≥ r ≤ 2e−N r /C . (18) P N i=1 One can also show Theorem 6. Assume that the measure µ on Rd satisfies the Poincar´e inequality (1) with constant c. Let X1 , . . . , XN be i.i.d. random variables with law µ. Then, for all bounded Lipschitz functions on Rd with Lipschitz constant α, it holds      N 1 ( r r2 N , f (Xi ) − E(f (X1 )) ≥ r ≤ 2 exp − min . (19) P N K α α2 i=1 6.2 Ergodic Simulations Let (Yn )n be a Markov chain on Rd with transition kernel K such that, for all smooth functions f , |∇Kf | (x) ≤ αK(|∇f |)(x) ,

(20)

for some α < 1. For example, fix t0 > 0 and set K = Pt0 , where (Pt ) is the semi-group of the diffusion

368

F. Malrieu and D. Talay

dXt = dBt − ∇U (Xt ) dt with Hess U (x) ≥ ρI and ρ > 0. One can then choose α = e−ρt0 . Alternatively, K can be chosen as the transition kernel of the implicit Euler scheme which discretizes (Xt ). Using Herbst’s argument one can show Proposition 4. For all 1-Lipschitz functions f on Rd ,      N 1 ( dx N (1 − α)2 2 Px r ≤ 2 exp − f (Yi ) − f dµ ≥ r + , N N c i=1 where dx =

(21)

α Ex (|x − X1 |). 1−α

6.3 Stochastic Particle Methods for McKean–Vlasov Equations Consider the McKean–Vlasov equation d d ( ∂2 ∂ 1 ( ∂ Pt = (aij [x, Pt ] Pt ) − (bi [x, Pt ] Pt ) , ∂t 2 i,j=1 ∂xi ∂xj ∂x i i=1

(22)

where Pt is a probability measure on Rd and, for some functions b and σ,  b [x, p] = b(x, y) p(dy), Rd  σ(x, y) p(dy), σ [x, p] = Rd

∗

a [x, p] = σ [x, p] σ [x, p]

for all x in Rd and all probability measures p. The functions b and σ are the interaction kernels. This equation has been introduced by [13] and then widely studied from both probabilistic and analytic points of view (see, e.g., [14] for a review). Under appropriate conditions one can show that Pt is the marginal law at time t of the law of the solution of the nonlinear stochastic differential equation

 

 ¯0 + tσ X ¯ s , Qs dBs + t b X ¯ s , Qs ds, ¯t = X X 0 0 ¯ t ) = Qt , L(X ¯ t : one thus has Pt = Qt . This probabilistic ¯ t ) stands for the law of X where L(X interpretation suggests to consider the stochastic particle system in mean field interaction

1N 1N dXti,N = N1 j=1 σ(Xti,N , Xtj,N )dBti + N1 j=1 b(Xti,N , Xtj,N ) dt, X0i,N = X0i , i = 1, . . . , N ,

Concentration Inequalities for Euler Schemes

369

where (B.i )i are independent Brownian motions on Rd . One aims to approximate Pt by the empirical measure µN t of the particle system: µN t =

N 1 ( δ i,N . N i=1 Xt

The convergence of the particle system to the nonlinear process has been deeply studied (see [14]). Suppose now that σ is constant (for the same reason as in Sect. 2). It can also be shown that the law of the particle system at time t satisfies a Logarithmic Sobolev inequality with a constant which does not depend on the number of particles. However the corresponding confidence intervals are not fully satisfying for numerical purposes since the particle system needs to be discretized in time to be simulated. The convergence rate of the Euler scheme in terms of N and the discretization step are studied in [2, 5–7]. Refining the proof of Theorem 1 by precisely expliciting the diffusion matrix of the particle system, one can also show that the Euler scheme satisfies a spectral gap inequality with a constant independent of N : Proposition 5. Suppose that the coefficient b is a bounded Lipschitz function, and σ is constant. Then the Euler scheme for the above particle system satisfies     N 1 (     N γ,i,N γ,i,N f Xt (x) − Ef Xt (x) ≥ r ≤ 2 exp − γ r2 P N Ct i=1

for all Lipschitz functions f with Lipschitz constant equal to 1 and all r ≥ 0, We again conjecture that, when the diffusion kernel is not constant, under appropriate conditions the particle system and the corresponding discretizated system satisfy a Poincar´e inequality. Then the above inequality would still hold true with min(r, r2 ) instead of r2 . We now consider the granular media equation: ∂u = div [∇u + u(∇V + ∇W ∗ u)] , ∂t where ∗ stands for the convolution and V and W are convex potentials on 2 3 Rd . This equation in R with V = |x| /2 and W = |x| has been introduced by [4] to describe the evolution of media composed of many particles colliding inelastically in a thermal bath. One can show that the solution ut of the nonlinear partial differential equation converges to an equilibrium distribution u∞ . Indeed, define the generalized relative entropy as    1 η(u) = u log u + uV + W (x − y)u(x)u(y) . 2 One has

370

F. Malrieu and D. Talay

Theorem 7. [8] If V is uniformly convex, i.e. Hess V ≥ λI and W is even and convex then η(ut ) − η(u∞ ) ≤ Ke−2λt where u∞ is the unique minimizer of η or equivalently the unique solution of u∞ =

1 exp (−V (x) − W ∗ u∞ (x)) , Z



with Z=

exp (−V (x) − W ∗ u∞ (x)) dx .

The granular media equations can be viewed as McKean–Vlasov equations. The particle system well defined and the propagation of chaos result holds uniformly in time (see [11]):   ¯ ti ≤ √c , E Xti,N − X N ¯ i ’s are independent copies of the solution of the nonlinear equawhere the X tion. As the interaction kernels are not globally Lipschitz, one needs to use the implicit Euler scheme to discretize the particle system. Let (YnN,γ )n∈N be this implicit Euler scheme with discretization step γ. We have (see [12]): Theorem 8. There exists c > 0 such that    N 1 ( 2 c √ i,N,γ −λt ≤ 2e−N λr /2 f (Yt ) − f du∞ ≥ r + c γ + √ + ce P N N i=1 for all Lipschitz functions f with Lipschitz constant 1.

References 1. C. An´e, S. Blach`ere, D. Chafa¨ı, P. Foug`eres, I. Gentil, F. Malrieu, C. Roberto, and G. Scheffer. Sur les in´egalit´es de Sobolev logarithmiques, volume 10 of Panoramas et Synth` eses. Soci´et´e Math´ematique de France, Paris, 2000. 2. F. Antonelli and A. Kohatsu-Higa. Rate of convergence of a particle method to the solution of the McKean-Vlasov equation. Ann. Appl. Probab., 12:423–476, 2002. 3. D. Bakry. On Sobolev and logarithmic Sobolev inequalities for Markov semigroups. In New trends in stochastic analysis (Charingworth, 1994), pp. 43–75, River Edge, NJ, 1997. Taniguchi symposium, World Sci. Publishing. 4. D. Benedetto, E. Caglioti, and M. Pulvirenti. A kinetic equation for granular media. RAIRO Mod´el. Math. Anal. Num´ er., 31(5):615–641, 1997. 5. M. Bossy. Optimal rate of convergence of a stochastic particle method to solutions of 1D viscous scalar conservation laws. Math. Comp., 73(246):777–812, 2004.

Concentration Inequalities for Euler Schemes

371

6. M. Bossy and D. Talay. Convergence rate for the approximation of the limit law of weakly interacting particles: application to the Burgers equation. Ann. Appl. Probab., 6(3):818–861, 1996. 7. M. Bossy and D. Talay. A stochastic particle method for the McKean-Vlasov and the Burgers equation. Math. Comp., 66(217):157–192, 1997. 8. J. Carrillo, R. McCann, and C. Villani. Kinetic equilibration rates for granular media. Rev. Matematica Iberoamericana, 19:1–48, 2003. 9. M. Ledoux. Concentration of measure and logarithmic Sobolev inequalities. In S´eminaire de Probabilit´ es XXXIII. Lectures Notes in Math., vol 1709, pp. 120– 216. Springer, Berlin, 1999. 10. F. Malrieu and D. Talay. Poincar´e inequalities for Euler schemes. In preparation. 11. F. Malrieu. Logarithmic Sobolev inequalities for nonlinear PDE’s. Stochastic Process. Appl., 95(1):109–132, 2001. 12. F. Malrieu, Convergence to equilibrium for granular media equations and their Euler schemes. Ann. Appl. Probab., 13(2): 540–560, 2003. 13. H. P. McKean, Jr. Propagation of chaos for a class of non-linear parabolic equations. In Stochastic Differential Equations (Lecture Series in Differential Equations, Session 7, Catholic Univ., 1967), pp. 41–57. Air Force Office Sci. Res., Arlington, Va., 1967. 14. S. M´el´eard. Asymptotic behaviour of some interacting particle systems; McKean-Vlasov and Boltzmann models. In Probabilistic models for nonlinear partial differential equations (Montecatini Terme, 1995), D. Talay and L. Tubaro (Eds.), pp. 42–95. Springer, Berlin, 1996. 15. D. Talay. Probabilistic numerical methods for partial differential equations: elements of analysis. In Probabilistic models for nonlinear partial differential equations (Montecatini Terme, 1995), D. Talay and L. Tubaro (Eds.), pp. 148– 196. Springer, Berlin, 1996.

Fast Component-by-Component Construction, a Reprise for Different Kernels Dirk Nuyens and Ronald Cools Dept. of Computer Science, K.U.Leuven, B-3001 Heverlee, Belgium [email protected], [email protected]

Summary. In [16] (Nuyens and Cools) it was shown that it is possible to generate rank-1 lattice rules with n points, n being prime, in a fast way. The construction cost in shift-invariant tensor-product reproducing kernel Hilbert spaces was reduced from O(sn2 ) to O(sn log(n)), with s the number of dimensions. This reduction in construction cost was made possible by exploiting the algebraic structure of multiplication modulo a prime. Here we look at the applications of the fast algorithm from a practical point of view. Although the choices for the function space are arbitrary, in practice only few kernels are used for the construction of rank-1 lattices. We will discuss componentby-component construction for the worst-case Korobov space, the average-case Sobolev space, the weighted lattice criterion Rn,γ and polynomial lattice rules based on the digital Walsh kernel, of which the last two were presented at MC2 QMC 2004 by Joe [11] and Dick, Leobacher and Pillichshammer, see e.g. [7]. We also give an example implementation of the algorithm in Matlab.

Key words: Numerical integration, Quasi-Monte Carlo, Rank-1 lattice rules, Component-by-component construction, Fast algorithms, Digital nets

1 Introduction We want to approximate an s-dimensional integral over the unit cube by an equal weight cubature rule,  ( 1 f (xk ) , I(f ) = f (x) dx ≈ Q(f ) = (1) n [0,1)s xk ∈Pn

where the n evaluation points are a rank-1 lattice '  k·z :0≤k
(2)

374

D. Nuyens and R. Cools

and by k · z we mean (componentwise) multiplication modulo n. The integer vector z is called the generating vector of the lattice and its components are taken from the set Zn = {0 < z < n : gcd(z, n) = 1}, i.e. z ∈ Zns . In this paper we assume that n is prime and thus this set simplifies to Zn = {1, . . . , n − 1}. The generating vector is chosen such as to minimize a certain error measure for the cubature rule given in (1). Here we minimize the worst-case error of functions in the unit ball of a shift-invariant tensor-product reproducing kernel Hilbert space H(K) with a given reproducing kernel K: e(z) = sup {|I(f ) − Q(f )| : f ∈ H(K), f  ≤ 1} .

(3)

A particularly nice algorithm to construct a good generating vector z is the component-by-component algorithm, see e.g. [10, 13, 19, 20]. In this algorithm the components of the generating vector z are found component by component. This was first deemed not likely to be a successful strategy by Niederreiter [14, page 987] but later shown possible by Sloan and Reztsov [20]. This construction method makes the generated lattice rule naturally extensible in the number of dimensions. The typical construction cost of this algorithm is O(sn2 ), but can be significantly reduced to O(sn log(n)), as described in previous work for n prime [16]. This fast algorithm makes it possible to construct lattice rules with a large number of points (say n = 108 ), which was practically intractable with the original algorithm. As an added advantage of the fast algorithm, we note that, due to the reduced number of arithmetic operations, its numerical qualities are also better. In Sect. 2 we will present the component-by-component algorithm and rewrite it as a repeated matrix-vector multiplication with a fixed matrix Ωn . Then in Sect. 3 we will show that this matrix-vector multiplication can be done in time O(n log(n)) instead of the classical O(n2 ). Since the complete algorithm consists of s times this matrix-vector multiplication (when constructing a generating vector for s dimensions), the total time is O(sn log(n)) as promised. In Sect. 4 we will show some of the applications in which this algorithm can be used. We will describe the worst-case Korobov space setting, the average-case Sobolev space setting, the weighted lattice criterion Rn,γ and even the construction of polynomial lattice rules. At MC2 QMC2004 the component-by-component algorithm was introduced for these last two, and in particular for a combination of those in the talk by Leobacher. This paper is written from a more practical point of view and therefore we present a possible implementation of the fast algorithm in Matlab in Sect. 5 and finally make some general remarks in Sect. 6.

2 The Component-by-Component Algorithm The component-by-component algorithm will construct the generating vector z for increasing dimension s up to smax . Hereby in each iteration s, the s-th

Fast Component-by-Component Construction

375

component zs is chosen and all previously chosen components remain fixed. We choose zs ∈ Zn which minimizes the worst-case error (3) in our shift-invariant tensor-product reproducing kernel Hilbert space H over all functions in the unit ball. The advantage of using a shift-invariant tensor-product reproducing kernel Hilbert space is that the expression for this worst-case error can be written explicitly as ⎛ ⎞   1/2 s n−1 s   ( k · z 1 j ⎠ βj + . (4) es,γ,β (z1 , . . . , zs ) = ⎝− βj + γj ω n n j=1 j=1 k=0

See e.g. [9] for the derivation of such formulae in different settings and specifically in the case of a shift-invariant kernel. Some words about notation are in order here. The kernel of an s-dimensional tensor-product space is the product of 1-dimensional kernels Ks,γ,β (x) =

s 

(βj + γj ω(xj )) ,

(5)

j=1

where ω is the variable kernel part and the γj are positive (product-type) weights which denote the importance of the different dimensions. Since we consider only shift-invariant kernels the variable kernel function has only one argument. The γj are taken as a decaying sequence of positive weights, γ1 ≥ γ2 ≥ · · · ≥ γs ≥ 0 , which means that the successive coordinates are less and less important. The function ω was dubbed the variable part of the kernel in the previous work [16], since it can be seen to correspond to the non-constant frequency part of the Fourier series of the 1-dimensional kernel (up to scaling): ( ( K1,γ,β (x) = kˆh exp(2πi hx) = kˆ0 + kˆh exp(2πi hx) . h∈Z

h∈Z\{0}

Sometimes the constant part of the kernel is weighted by weights βj . However a simple calculation shows that the worst-case error (4) for a function space with weights γj and βj can be written in terms of the worst-case error in a function space with weights γ 3j = γj /βj and β3j = 1. We then obtain the worstcase error as a scaled version of the worst-case error in this simpler function space as ⎛ ⎞   s s n−1 s    (  k · zj 1 γj ⎠. βj e2s,3γ ,β3 = βj ⎝−1 + ω 1+ e2s,γ,β = n β n j j=1 j=1 j=1 k=0

From this we see that (up to scaling) only the ratios γj /βj matter. We want to emphasize however, that the function ω can be any function in our derivations, and so the choice of the reproducing kernel is completely free.

376

D. Nuyens and R. Cools

Given the expression for the worst-case error (4) and the description of the component-by-component construction, we can present the original algorithm as Algorithm 1. In the algorithm we consider the worst-case error in dimension s as a function of all the possible choices of zs ∈ Zn . We then pick the minimum and keep this choice fixed for the following iterations. Under certain conditions on the weights γj , it can be shown that the constructed lattice rule achieves the optimal rate of convergence, see [6, 10, 13], and thus this construction method is justified. Algorithm 1 (Original) CBC for shift-invariant tensor-product RKHS for s = 1 to smax do for all zs ∈ Zn do   n−1 s  s  1 ( k · zj 2 es (zs ) = − βj + β j + γj ω n n j=1 j=1 k=0

end for zs = argmin e2s (z) z∈Zn

end for

A quick inspection of Algorithm 1 shows that a direct implementation would have a construction cost of O(s2 n2 ). But since this is an iterative process in s, the product over s terms can be split in two parts: a product over the first s − 1 terms and just the last term:     s−1    s    k · zj k · zj k · zs βj + γj ω βj + γj ω = βs + γs ω . n n n j=1 j=1 (6) In iteration s the product up to dimension s − 1 is fixed, since we do not change the zj , j < s, anymore. Consequently we can store the product in an n-vector denoted ps−1 . Equation (6) then becomes      s   k · zj k · zs βj + γj ω = ps−1 (k) βs + γs ω , n n j=1 which needs to be calculated for each k and each possible choice for zs . This immediately allows us to rewrite the expression for the worst-case error as    s n−1  k·z 1( 2 es (z) = − βj + ps−1 (k) βs + γs ω , ∀z ∈ Zn . (7) n n j=1 k=0

It is not hard to see that (7) is a matrix-vector multiplication. Define the matrix   k·z [Ωn ]z,k = ω , n

Fast Component-by-Component Construction

377

which has at most n different elements (depending on the function ω), and define the worst-case error vector n−1 1 βs ( γs Ωn ps−1 , ps−1 (k) + e2s = −β¯s + (βs + γs Ωn ) ps−1 = −β¯s + n n n k=0

were a scalar plus2a matrix means adding this scalar to all elements of the s matrix and β¯s = j=1 βj , the cumulative product up to dimension s. Then we can restate the component-by-component algorithm in its matrix-vector form as Algorithm 2. The only extra ingredient is the updating of the product vector after we have fixed a zs . A simple calculation shows that this is done by multiplying each element of ps−1 by the corresponding elements of the row zs of Ωn , denoted in a Matlab-like notation by Ωn (zs , :), after multiplication by the weight γs and addition of the constant βs , which minimized e2s (z). Algorithm 2 CBC in matrix-vector form p0 = 1 for s = 1 to smax do e2s = −β¯s + n1 (βs + γs Ωn ) ps−1 zs = argmin e2s (z) z∈Zn

ps = diag(βs + γs Ωn (zs , :)) ps−1 end for

The matrix-vector form in Algorithm 2, shows the real beauty of the component-by-component algorithm. A short description of the algorithm could now be given in terms of the variable kernel matrix Ωn as: 1. Initialize an n-vector p by all ones. Create a representation of the matrix Ωn , i.e. evaluate the function ω in the points {0, 1, . . . , n − 1}/n. 2. For each dimension s do the following: a) Calculate the matrix-vector product of Ωn and p and assign this to a ˜2 . (Note that it is not necessary to multiply with γs , add the vector e constant βs , divide by n and subtract β¯s . We only need to find the index z associated with the minimal value.) ˜2 has its minimum. b) Pick as choice for zs the index z where the vector e c) Update the vector p as the elementwise product of row zs of (βs + γs Ωn ) with the current elements of p. (Note that only here the γs , as well as the constant kernel part βs , is of importance.) 3. Return the generating vector of the lattice rule. The cost of step 1 is O(n cω ), where cω is the cost of evaluating the function ω. We will consider cω constant. The cost of step 2a is the cost of a matrixvector product, this is O(n2 ). Finding the minimum in step 2b is O(n) and updating p in step 2c is also O(n). The total is thus O(sn2 ). From a computational point of view it is now very clear that we could reduce the total construction cost of O(sn2 ) if we could find a matrix-vector

378

D. Nuyens and R. Cools

multiplication algorithm for our given matrix Ωn with cost less than O(n2 ). Such a technique was introduced in [16] and is shortly restated in the next section.

3 Fast Matrix-Vector Multiplication The technique introduced in [16] is reformulated in this section in a way that allows a generalization of the fast algorithm to the case that n is not a prime, see [17]. An exception where the fast algorithm is also directly applicable for n = bm , with b prime, is presented in Sect. 4.4 on polynomial lattice rules. Recall that the set Zn = {1, . . . , n − 1} here. This set together with multiplication modulo n forms the cyclic group Z n . Since the matrix Ωn is defined as the function ω elementwise over elements of the form (k · z)/n, we can identify part of this matrix with the Cayley table of this cyclic group. Before we explain the fast matrix-vector multiplication with the matrix Ωn we introduce the following notation: a prime on a symbol will denote that the 0-index is left out. Thus with p we mean the vector p without its 0-th element, i.e. assuming that p is defined as p(k) with k ∈ Zn , this means that p is defined as p(k) with k ∈ Zn \ {0} = Z n . The same holds for the matrix Ωn , where Ωn is defined as    k·z  . (8) Ωn = ω k∈Z n n  z∈Zn

Thus Ωn has size (n − 1) × (n − 1), where the original matrix Ωn has size (n − 1) × n. In other words we dropped the first column of Ωn . Using a trick originally introduced by Rader for deriving a fast Fourier transform when the number of points is prime [18], we can now permute the rows of Ωn by the positive powers of a generator g for Z n and the columns by the negative powers of this same generator. This will give us constant powers of g along the diagonals and bring Ωn in circulant form: · 1

1 g −1 g −2 · · · 1 g −1 g −2 · · ·

g g

g

g

1 g −1 · · ·

g2

g2 .. .

g2 .. .

g .. .

1 .. .

· · · g3 . .. . ..

g −1 g −1 g −2 g −3 · · ·

1

It is well known that a circulant matrix has a similarity transform which has the Fourier matrix as its eigenvectors, see e.g. [3].

Fast Component-by-Component Construction

379

Theorem 1 (Diagonalization of a circulant matrix). A circulant matrix −1 DFm , where Fm is the Cm of order m has a similarity transform Cm = Fm Fourier matrix of order m and the eigenvalues are defined by the discrete Fourier transform of the first column c of Cm as D = diag(Fm c). Corollary 1 (Fast matrix-vector multiplication with a circulant matrix). A matrix-vector multiplication of a vector x with a circulant matrix Cm can be done in O(m log(m)) by using the similarity transform of Cm : −1 Cm x = Fm diag(Fm c)Fm x = IFFT(diag(FFT(c))FFT(x)) ,

where each discrete Fourier transform (as well as its inverse) is done using an FFT (or IFFT). Note that for each m an m-point FFT (as well as an IFFT) is possible in time O(m log(m)). A possible implementation is [8], which is also used by Matlab. Plugging in this knowledge into Algorithm 2 results in a fast algorithm for component-by-component construction with cost O(sn log(n)). We do not need to consider a matrix-vector multiplication with Ωn , since the first column of Ωn has constant value ω(0) and thus will not influence the minimum. The circulant matrix after the Rader permutation on Ωn will be denoted by Ψn . It is completely defined by its first column ψ  which, given a generator g for Z n , can be calculated as   ψ  = ω(g −1 /n) . (9) 1≤≤n−1

The permutation defined by the positive powers of g is also needed to map the index ws of the minimum in the row permuted error vector, call this Es2 , back to the unpermuted space as zs = g ws −1 . We can be oblivious of the permutation on the columns of Ωn , since we are not really interested in the value of the p vector, and since on initialization this vector only contains the constant value 1, so the permutation would have no effect. For clarity we will however call the product vector in the permuted space q. This brings us to the following practical formulation of Algorithm 2: 1. Initialize an n-vector q by all ones. Find a generator g for Z n and initialize an (n − 1)-vector ψ  as given in (9). Preferably: precalculate the FFT of ψ . 2. For each dimension s do the following: a) Calculate the matrix-vector product of Ψn and q  using FFT’s and ˜ 2. assign this to a vector E

380

D. Nuyens and R. Cools

˜ 2 where the vector has its minimum. Pick b) Pick ws as the index in E zs as g ws −1 . c) Update the vector q  as the elementwise product of row ws of (βs + γs Ψn ) with the current elements of q  . The row corresponding to ws can be formed as a circular up shift over ws of ψ  , and this update rule is thus easily implementable (e.g. as a for-loop from ws down to 1 and a second for-loop from n − 1 down to ws + 1). 3. Return the generating vector of the lattice rule. It can be seen that this algorithm has construction cost O(sn log(n)). The only changes of cost are in step 1 and step 2a. The cost of step 1 is O(n log(n)) if precalculation of the FFT is done. The cost of step 2a is now O(n log(n)). An example implementation of this algorithm is given in Sect. 5.

4 Applications In this section we will use weights βj for the constant part of the kernel to easily accommodate other kernels, but we still only use γj weights to define the actual function space setting. 4.1 Worst-Case Korobov Space Setting The Korobov function space is the space of [0, 1)s -periodic functions for which a converging Fourier series exists with Fourier coefficients decaying with a certain rate α > 1. The 1-dimensional kernel is given by (

K1,γ (x) = 1 + γ

h∈Z\{0}

exp(2πi hx) . |h|α

The ω function in this case is ω(x) =

( h∈Z\{0}

exp(2πi hx) . |h|α

The infinite sum can be written in terms of a Bernoulli polynomial if α is even [1, page 805], ω(x) =

(2π)α Bα (x) . (−1)α/2−1 α!

In practice α is taken to be 2, which is the smallest possible value. Larger values of α correspond to function spaces with smoother functions. So the typical choice is ω(x) = 2π 2 (x2 − x + 1/6) ,

with α = 2 .

Fast Component-by-Component Construction

381

4.2 Average-Case Sobolev Space Setting We consider the Sobolev space in s dimensions which is a tensor-product of 1-dimensional Sobolev spaces of absolutely continuous functions over [0, 1] whose first derivatives are in L2 ([0, 1]). The 1-dimensional reproducing kernel is given by K1,γ (x, y) = 1 + γ σa (x, y) , where

min(|x − a|, |y − a|) , σa (x, y) = 0,

if (x − a)(y − a) > 0 , if (x − a)(y − a) ≤ 0 .

The parameter a is the anchor of this kernel, and is typically chosen as 0, 1 or 1/2. Note that this kernel is not shift-invariant and so is a function in two variables x and y. The study of this function space in the worst-case setting involves so-called shifted lattice rules. For a shifted lattice rule with shift ∆ ∈ [0, 1)s , the point set takes the form  '  k·z + ∆ mod 1 : 0 ≤ k < n . Pn = n Since this kernel defines a function ω in two variables, it is not immediately applicable for the fast algorithm. Following [9, 19], we can define a shift-invariant kernel for this function space

K1,γ (x) = 1 + γ (x2 − x + a2 − a + 1/2)

= 1 + γ (B2 (x) + a2 − a + 1/3) . When using this shift-invariant kernel K , we are actually calculating the worst-case error in a Korobov space with α = 2 and adjusted weights β3j = 1 + γj (a2j − aj + 1/3) ,

γ 3j = γj /(2π 2 ) .

The error now corresponds to the average-case error in the original Sobolev space by using random shifts ∆ for the point set. Indeed, the shift-invariant kernel is obtained by taking the mean over all possible shifts, see [19] for details. Taking a typical choice for the anchor of the space and adjusting the weights we can use the fast algorithm to construct randomly shifted lattice rules in a weighted Sobolev space by taking ω(x) = x2 − x + 1/6 ,

with a = 1 and βj = 1 + γj /3 .

382

D. Nuyens and R. Cools

4.3 The Lattice Rule Criterion Rn and the Connection with the Weighted Star Discrepancy Joe considered a component-by-component construction method for rank-1 lattice rules which achieve the optimal rate of convergence O(n−1+δ ), under certain conditions on the weights, for the weighted star discrepancy at MC2 QMC2004, see [11]. Also see [10] for the unweighted version. There is a link with the classical Koksma-Hlawka inequality by a bound on the star discrepancy Dn involving the quantity Rn (see [15]) Dn (z) ≤

Rn (z) s + . n 2

So it suffices to only consider Rn to obtain a bound on Dn . Here, as in [11], we will consider the weighted star discrepancy, and thus the weighted quantity Rn,γ . This quantity Rn,γ in s dimensions can be associated with the worstcase error of a product of the 1-dimensional kernels (consult [11] for details) K1,γ (x) = (1 + γ) + γ

( −n/2
exp(2πi hx) . |h|

This kernel transforms easily into the needed form, and so we have that ω(x) =

( −n/2
exp(2πi hx) , |h|

with βj = 1 + γj .

To reduce the evaluation cost of the ω function, the asymptotic techniques in [12] should be used. 4.4 Polynomial Lattice Rules and the Digital Walsh Kernel Another component-by-component construction method presented also at MC2 QMC2004 by Dick, Leobacher and Pillichshammer, is that of polynomial lattice rules based on the digital Walsh kernel, see [6, 7]. These polynomial lattice rules are in fact (t, m, s)-nets and were introduced by Niederreiter, see [15, Sect. 4.4] for a more thorough description. Here the number of points is n = bm which is not prime, but we will show that the algorithm works due to the algebraic structure of polynomial multiplication over a field modulo an irreducible polynomial. For polynomial lattice rules in base b the generating vector z consists of polynomials over a finite field, zj ∈ (Fb [x]/p) , with p an irreducible polynomial over Fb with deg(p) = m. This means that there are bm − 1 possible choices for the components of the generating vector of the lattice rule, which look like a0 + a1 x + · · · + am−1 xm−1 , with a ∈ Fb and not all a zero.

Fast Component-by-Component Construction

383

Generalizing the construction of the point set (2) we consider the b-adic expansion k = k0 + k1 b + · · · + km−1 bm−1 and associate a polynomial k(x) = k0 +k1 x+· · ·+km−1 xm−1 with each such k, 0 ≤ k < bm . The multiplication is now defined modulo p and thus q(x) = k(x)·zj (x) ∈ Fb [x]/p. To map q(x)/p(x) into [0, 1), we consider the formal Laurent series expansion of q(x)/p(x) up to x−m and then evaluate at x = b. This is denoted by the function vm . We thus have that component j of point k is generated as ∞    m ( ( k(x) · zj (x) − u x u b− , u ∈ Fb . = xk,j = vm = vm p(x) =1

=1

See [6, 15] for further details. The theory for component-by-component construction of these rules is similar to that of the classical lattice rules. Here the components zj of the generating vector are chosen from the set (Fb [x]/p) , where the multiplier k is from the set Fb [x]/p. As in the scalar case we can define a matrix Ωn as     k(x) · z(x)  , Ω n = ω vm k(x)∈(Fb [x]/p(x)) p(x)  z(x)∈(Fb [x]/p(x))

compare with (8). The product of k(x) and z(x) is done in the field Fb [x]/p and is thus modulo the irreducible polynomial p(x) – this is the equivalence of the product modulo the prime n in the scalar case. Since the multiplicative group of every finite field is cyclic, we can find a primitive polynomial g which generates all of Fb [x]/p and thus we can transform the matrix Ωn into circulant form. This is all we need for the fast algorithm. In [6] a digital Walsh kernel is used to define the function space. The 1-dimensional kernel defined for a given base b and smoothness α is given as K1,γ (x) = 1 + γ φwal,b,α (x) . Here φwal,b,α is defined as

µb (α) , φwal,b,α = µb (α) − blogb (x)(α−1) (µb (α) − 1) ,

if x = 0 , otherwise ,

and µb (α) = bα (b − 1)/(bα − b), see [6] for more details. For polynomial lattice rules the logical choice is to take base b = 2 and then define a suitable power m to have a practical number of points n = 2m . Since b−∞ = 0 and logb (0) = −∞, we thus have for a typical choice that ω(x) = φwal,b,α is given as ω(x) = 2 − 2log2 (x) ,

with b = 2, α = 2 .

384

D. Nuyens and R. Cools

5 Example Implementation in Matlab In this section we give a possible implementation in Matlab of the gorithm for the scalar lattice rules. In this example implementation assume that the variable kernel function ω is symmetric around 1/2, ω(x) = ω(1 − x). Using this assumption we only need half of the q  vectors, and so the needed FFT’s are only half the size. See [16] for thorough explanation. function [z, e2] = fastrank1(n, s_max, omega, gamma, beta) if ~isprime(n), error(’n must be prime’); end; z = zeros(s_max, 1); e2 = zeros(s_max, 1); m = (n-1)/2; % assume the ω function symmetric around 1/2 q = ones(m, 1); % permuted product vector q (without zero index) q0 = 1; % zero index of permuted product vector: q(0) ˜ 2 in the text E2 = zeros(m, 1); % the vector E cumbeta = cumprod(beta); g = generator(n); % generator g for {1, 2, . . . , n − 1} perm = zeros(m, 1); % permutation formed by positive powers of g perm(1) = 1; for j=1:m-1, perm(j+1) = mod(perm(j)*g, n); end; perm = min(n - perm, perm); % map everything back to [1, n/2) psi = feval(omega, perm/n); % the vector ψ psi0 = feval(omega, 0); % zero index: ψ(0) fft_psi = fft(psi); for s = 1:s_max % step 2a: circulant matrix−vector multiplication E2 = ifft(fft_psi .* fft(q)); E2 = real(E2); % Matlab uses complex fft’s: remove this noise % step 2b: choose ws and zs which give minimal value [min_E2, w] = min(E2); % pick index of minimal value if s == 1, w = 1; noise = abs(E2(1) - min_E2), end; z(s) = perm(w); % extra: we want to know the exact value of the worst−case error e2(s) = -cumbeta(s) + ( beta(s) * (q0 + 2*sum(q)) + ... gamma(s) * (psi0*q0 + 2*min_E2) )/n; % step 2c : update q q = (beta(s) + gamma(s) * psi([w:-1:1 m:-1:w+1])) .* q; q0 = (beta(s) + gamma(s) * psi0) * q0; fprintf(’s=%4d, z=%4d, w=%4d, e2=%.5e, e=%.5e\n’, ... s, z(s), w, e2(s), sqrt(e2(s))); end;

fast alwe will so that and ψ  a more

Fast Component-by-Component Construction

385

The implementation we give is not the best possible. The FFT’s could in fact be taken as real to complex transforms, instead of the complex to complex transforms provided by Matlab. Likewise the IFFT could be taken as a complex to real transform. This functionality is not provided in Matlab, but could be exploited in a C implementation resulting in less memory usage and a faster construction. ˜ 2 should be constant in the first Also note that in theory the vector E iteration. This means that we can always pick z1 = w1 = 1. In practice there will be numerical noise and so we force this decision. This immediately gives us some information about the noise level for the FFT, but nothing is done with this information in the example implementation. We did not list any code for the function generator which is used for finding a generator for Z n , but such an algorithm can be found in [2, page 25] and is relatively easy to implement. We give an example usage of the routine for constructing a lattice rule for the average-case Sobolev space setting: omega = inline(’x.^2 - x + 1/6’); n = 4001; s = 100; gamma = 0.9.^[1:s]; [z, e2] = fastrank1(n, s, omega, gamma, 1+gamma/3);

For polynomial lattice rules the initialization part of the algorithm becomes only a little bit harder. We have to find a generator for Fb [x]/p and calculate all its positive powers, then find the Laurent series expansion for g k /p and apply the functions vm and ω. However, the core of the algorithm, which is the matrix-vector multiplication, remains the same.

6 Critical Remarks We have shown that one simple implementation framework works for any shift-invariant kernel and that the algorithm is practically implementable by giving an example implementation in Matlab. This algorithm has construction cost O(sn log(n)). Although it is possible to do fast construction for a composite number of points, the results in [4,5,16] show that rules constructed with a prime number of points have better worst-case errors (i.e. the constant is better, while the rate is the same). We also would like to note that the prime number algorithm is a lot easier to implement and to understand. The reader is referred to [17] for details about the general algorithm. In the case of the polynomial lattice rules we arrive at an algorithm for a number of points of the form n = bm , with b prime. Because of the parallels between prime numbers and irreducible polynomials, the presented algorithm is immediately applicable, although bm is not prime. In [16] it was shown that with a relatively standard PC configuration, lattice rules with a number of points almost up to 108 can be calculated by

386

D. Nuyens and R. Cools

this algorithm. The question however is: how many points do we need? Or is it enough that we can generate the lattice rule on the fly for a moderate amount of points? Depending on the number of points n, but also depending on the function space, the algorithm can exhibit numerical problems. It must be noted however that these numerical problems will manifest themselves a lot sooner when using the classical O(sn2 ) algorithm due to the significantly larger number of operations. An open question is therefore: can we improve the numerical qualities of the component-by-component algorithm?

Acknowledgements This research is part of a project financially supported by the Onderzoeksfonds K.U. Leuven/Research Fund K.U. Leuven.

References 1. M. Abramowitz and I. A. Stegun, editors. Handbook of Mathematical Functions with Formulas, Graphs and Mathematical Tables, volume 55 of National Bureau of Standards Applied Mathematics Series. U.S. Government Printing Office, Washington, D.C., 1964. 2. H. Cohen. A Course in Computational Algebraic Number Theory. Graduate Texts in Mathematics. Springer-Verlag, 3rd edition, 1996. 3. P. J. Davis. Circulant Matrices. Wiley, 1979. 4. J. Dick and F. Kuo. Constructing good lattice rules with millions of points. In H. Niederreiter, editor, Monte-Carlo and quasi-Monte Carlo Methods – 2002, pp. 181–197. Springer-Verlag, Jan. 2004. 5. J. Dick and F. Kuo. Reducing the construction cost of the component-bycomponent construction of good lattice rules. Math. Comp., 73:1967–1988, 2004. 6. J. Dick, F. Kuo, F. Pillichshammer, and I. H. Sloan. Construction algorithms for polynomial lattice rules for multivariate integration. Math. Comp., 2005. To appear. 7. J. Dick, G. Leobacher, and F. Pillichshammer. Construction algorithms for digital nets with small weighted star discrepancy. Report AMR04/13, School of Mathematics, University of New South Wales, Sydney, Apr. 2004. 8. M. Frigo and S. G. Johnson. FFTW: An adaptive software architecture for the FFT. In Proc. 1998 IEEE Intl. Conf. Acoustics Speech and Signal Processing, Vol. 3, pp. 1381–1384. IEEE, 1998. 9. F. J. Hickernell. Lattice rules: How well do they measure up? In P. Hellekalek and G. Larcher, editors, Random and Quasi-Random Point Sets, pp. 109–166. Springer-Verlag, 1998. 10. S. Joe. Component by component construction of rank-1 lattice rules having O(n−1 (ln(n))d ) star discrepancy. In H. Niederreiter, editor, Monte-Carlo and quasi-Monte Carlo Methods – 2002, pp. 293–298. Springer-Verlag, Jan. 2004.

Fast Component-by-Component Construction

387

11. S. Joe. Construction of good rank-1 lattice rules based on the weighted star discrepancy. In H. Niederreiter and D. Talay, editors, Monte Carlo and QuasiMonte Carlo Methods 2004. Springer-Verlag. This volume. 12. S. Joe and I. H. Sloan. On computing the lattice rule criterion R. Math. Comp., 59(200):557–568, Oct. 1992. 13. F. Kuo. Component-by-component constructions achieve the optimal rate of convergence for multivariate integration in weighted Korobov and Sobolev spaces. J. Complexity, 19:301–320, June 2003. 14. H. Niederreiter. Quasi-Monte Carlo methods and pseudo-random numbers. Bull. Amer. Math. Soc., 84(6):957–1041, Nov. 1978. 15. H. Niederreiter. Random Number Generation and Quasi-Monte Carlo Methods. Number 63 in Regional Conference Series in Applied Mathematics. SIAM, 1992. 16. D. Nuyens and R. Cools. Fast algorithms for component-by-component construction of rank-1 lattice rules in shift-invariant reproducing kernel Hilbert spaces. Math. Comp., 2005. Accepted. 17. D. Nuyens and R. Cools. Fast component-by-component construction for (non-)primes. Submitted. Presented at Dagstuhl Seminar 04401: ‘Algorithms and Complexity for Continuous Problems’, 2004. 18. C. M. Rader. Discrete Fourier transforms when the number of data samples is prime. Proc. IEEE, 5:1107–1108, 1968. 19. I. H. Sloan, F. Kuo, and S. Joe. Constructing randomly shifted lattice rules in weighted Sobolev spaces. SIAM J. Numer. Anal., 40(5):1650–1665, 2002. 20. I. H. Sloan and A. V. Reztsov. Component-by-component construction of good lattice rules. Math. Comp., 71(237):263–273, 2002.

A Reversible Jump MCMC Sampler for Object Detection in Image Processing Mathias Ortner, Xavier Descombes, and Josiane Zerubia Ariana Research Group (INRIA/I3S), INRIA, 2004 route des Lucioles BP 93, 06902 Sophia Antipolis Cedex, France [email protected] Summary. To detect an unknown number of objects from high resolution images, we use spatial point processes models. The method is adapted to our image processing applications since it describes images as realizations of a point process whose points represent geometrical objects. We consider models made of two parts: a data term which quantifies the relevance of a set of objects with respect to the image and a prior term, containing strong geometrical interactions between objects. We use the Maximum A Posteriori estimator, which is obtained by combining a reversible Markov chain monte carlo (RJMCMC) point process sampler with a simulated annealing procedure. The quality of the results and the speed of the algorithm strongly depend on the used sampler. We present here an adaptation of Geyer-Møller sampler for point processes and show that the resulting Markov Chain keeps the required convergence properties. In particular, we design an updating scheme which allows the generation of points in the neighborhood of some others, and check the relevance of such moves on a toy example. We present experimental results on the difficult problem of the detection of buildings in a Digital Elevation Model of a dense urban area. Key words: Spatial point process, RJMCMC, non homogeneous Poisson point process, image processing, building detection.

1 Introduction A natural way to model images in a probabilistic framework is to consider Markov Random Fields [2]. Such models allow the use of smoothing priors like the Ising model, but require to consider images as a collection of pixels. In the early nineties, A. Baddeley and M.N.M. Van Lieshout [1] proposed to model images as realizations of spatial point processes. Since a spatial point process realization can be a random configuration of disks, rectangles, lines or any other kind of geometric shapes, spatial point process models appear to be especially amenable to the introduction of a geometric knowledge in the prior term. Spatial point process models in image processing have been used

390

M. Ortner et al.

by Green [11] and Rue [12] for object detection and have been applied to real applications in remote sensing [7, 9, 13] and medical imaging [3]. Point process models are especially adapted to object detection as they allow the introduction of a probability distribution on configurations of geometrical shapes. Such distributions are usually sub-divided into two parts: a prior term acting on the configuration of objects to favor or penalize specific geometrical patterns, and a likelihood comparing configurations of objects to the data. The estimator commonly used is the Maximum a Posteriori (MAP) estimator. Simulated annealing algorithms provide a practical way to compute the MAP despite numerous local maxima. However, the algorithm performance strongly depends on the mixing ability of the sampler. In this paper we present a new updating scheme that improves the quality of the Markov Chain.

2 Set Up Images and Point Processes Let (K, B, λ) be a measure space where K is a compact subset of R2 , B the Borel algebra and λ the Lebesgue measure. Let ν(.) be a non-atomic measure on K with 0 < ν(K) < ∞. In our application, K ⊂ R2 represents the support of an image. Let Cconf be the space of unordered configurations of points in K. Then x in Cconf can be written x = {x1 , . . . , xn } where n = n(x) describes the number of points in x. We can write Cconf as the union n ∪∞ n=0 Cn with Cn = {{x1 , . . . , xn } x1 , . . . , xn ∈ K } describing the set of unordered configurations of n points in K. Let C be the space of configurations x = {x1 , . . . , xn(x) } such that x is locally finite (every bounded set of K contains a finite number of points) and x is simple : xi = xj for i = j. C = {x ∈ Cconf s.t. x simple and locally finite}

These restrictions allow to build a σ-algebra N lf and a measure µ on C [14]. For F ∈ N lf , we consider: ⎤ ( 1  ⎣1 (∅ ∈ F ) + 1 ({x1 , . . . , xn } ∈ F ) dν(x1 ) . . . dν(xn )⎦ n! K n ⎡

−ν(K)

µ(F ) = e

n≥1

Object Processes A point process X on K is a measurable mapping from a probability space (Ω, A, P) to C, an example of which is the Poisson point process with intensity ν. Its distribution is given by the measure µ(.) previously defined, and satisfies ν(F ) = Eµ [NF (X)] where NF (X) corresponds to the number of points of X falling in F . We now consider objects ui = (xi , mi ) ∈ K × M , composed of a point xi and a mark mi (one or several real or integer parameters). M is

An RJMCMC Sampler for Object Detection in Image Processing

391

called the mark space. A marked point process is a point process on S such that the process of locations in K is a well defined point process. From now on, C describes configurations of points in S. Density, MAP and MCMC The notion of density of a point process w.r.t. a reference Poisson process allows to specify point process
distributions. A density is a measurable function h : C → [0, ∞( such that C h(x)dµ(x) = 1. In [9] we use this notion to design probability distributions over C including interactions between points. We actually define the energy U (x) associated with a configuration x resulting in the Gibbs distribution:  1 P(X ∈ F ) = h(x)dµ(x) h(x) = exp(−U (x)) Z F ˆ = Argmax h(x). We The goal is then to find the maximizing configuration: x use the framework presented in [14] to perform a simulated annealing scheme on MCMC simulations.

3 An RJMCMC Algorithm for Point Processes Let π(.) be the distribution of a point process on S defined by a density h(.) and the Poisson distribution µ(.) with intensity ν(.). We aim at sampling π(.). Several general methods could be used, including the birth and death algorithm, RJMCMC or perfect simulations. For the purpose of this paper, we designed an RJMCMC algorithm that is specifically adapted to our applications. 3.1 General Scheme We refer here to the work of Geyer in [4]. In his paper, he presents an extension of the algorithm he previously developed jointly with Møller [5] to sample point processes. His extension is partly based on the work of Green [6] on Metropolis Hastings samplers for general state spaces. We use his framework to adapt the Geyer-Møller sampler to our models. An RJMCMC sampler constructs a Markov Chain (Xt ) from a mixture of proposition kernels. For a given state Xt = x, a proposition kernel Qm (., .) is chosen with probability pm . A new state y is then proposed from this kernel: y ∼ Qm (x, .), and finally the proposition is accepted (Xt+1 = y) with probability αm (x, y) and rejected (Xt+1 = x) with a probability 1 − αm (x, y). The acceptance ratio is defined in order to insure the detailed balance of the Markov Chain w.r.t. the desired equilibrium distribution π(.). The usual way to obtain detailed balance is to take

392

M. Ortner et al.

αm (x, y) = min(1, Rm (x, y)) where Rm (x, y) is the Green ratio between two Radon Nykodym derivatives: Rm (x, y) =

π(dy) Qm (y, dx) π(dx) Qm (x, dy)

This ratio can be obtained by choosing a proposal density Qm such that the product density π(dy)Qm (y, dx) has a finite density w.r.t. a symmetric measure. It follows that π(dy)Qm (y, dx) and π(dx)Qm (x, dy) should be of the same dimensionality a property that Green refers to as the dimension matching condition. We denote by P the transition kernel of the obtained Markov chain. 3.2 Proposition Kernels We present different kinds of perturbation kernels Qm (., .) and derive the corresponding Green ratios Rm . Metropolis-Hastings updates are known to be useful when the normalizing constant of density h(.) is intractable. Birth or Death of a Point Let x = {u1 , . . . , un(x) } be the current configuration where n(x) is the number of points ui in the configuration x. The birth or death kernel proposes either to add a new point to x with probability pb (x) > 0 or to remove one of the existing points with probability pd (x) = 1 − pb (x). In the case of a birth, the new object is generated according to ν(.)/ν(S). When death has been selected, an object is removed using a uniform discrete law. Consider the following symmetric measure ξ (for A and B in N lf ):  

ξ(A × B) =

C



1A (x)1B (x ∪ u)ν(du)dµ(x) + u∈S

C

1A (x)

(

1B (x \ u)dµ(x) (1)

u∈x

the corresponding Green’s ratio in case of birth is then: RBD (x, x ∪ u) =

pd (x ∪ u) h(x ∪ u) ν(S) pb (x) h(x) n(x) + 1

In the case of a death the expression is inverted. Non Jumping Perturbations Non jumping perturbations randomly select one point in the current configuration and proposes a new version of it. In the case of configurations of rectangles, a typical example of such a move is the rotation of a randomly selected rectangle. Since such kernels do not involve any jump between spaces of different dimensions, the acceptance ratio is given by the usual Metropolis Hasting scheme. Let u ∈ x be the original object to perturb and v the new

An RJMCMC Sampler for Object Detection in Image Processing

393

version obtained by generating an auxiliary random variable Z according to a density q(x, u, z). The ratio is then given by y =x\u∪v

RN J (x, y) =

h(y) j y (v) q(y, v, z(u)) h(x) j x (u) q(x, u, z(v))

where j x (u) stands for the probability of choosing u in the configuration x. More examples of such transformations can be found in [10]. For process of rectangles, we use translations, rotations, dilation and some other moves. Birth or Death in a Neighborhood The main contribution of this paper is the proof of convergence of the algorithm using the Birth and Death in a Neighborhood (BDN) update move described in this section. For our image processing applications we define point process densities through interactions between points. Let ∼ be a symmetric, reflexive relation on S. It appeared to be useful to design a perturbation kernel which proposes to create or remove an interacting pair of points. Let ∂(u) ⊆ S be the neighborhood of a point: ∂(u) = {v ∈ S s.t. v ∼ u} and R(x) the set of interacting pairs: R(x) = {{u, v} ⊆ x s.t. u ∼ v and u = v}. The birth update first selects uniformly a point u in x, generates an auxiliary random variable z ∈ Σ that has the same dimension as S (dimension matching), applies a diffeomorphism ηu (z) = v in order to obtain a new point v in ∂(u), and finally proposes y = x ∪ v. On the other hand, the removal update first chooses a pair of interacting points in R(x) and then proposes to remove one of its two points with probability 0.5. The two selection schemes yield the following discrete distributions: jbx (u) =

1 n(x)

jdx (u) =

card {c ∈ R(x) s.t. u ∈ c} 2 card R(x)

(2)

The derivation of Green’s ratio requires a change of variable and Jηu , the Jacobian of ηu , is involved in the final expression. Let Λu be the following map on S :  Λu (v) =

0 if v ∈ ηu (Σ) = ∂(u) q(ηu−1 (v))|Jηu−1 (v)| otherwise

where q(z) corresponds to the density of the auxiliary random variable used to generate the new point w.r.t. Lebesgue measure on Σ. The final Green’s ratio for the birth update then follows: RBDN (x, x ∪ v) =

h(x ∪ v) pd jdx∪v (v) fν (v) 1 h(x) pb u∈x jbx (u) Λu (v)

where fν (.) is the Radon-Nykodym derivative of ν(.) w.r.t. the canonical measure on S (Lebesgue measure in the case of real vectors). For instance, in our applications we take ν = (λK × λM )/λM (M ) leading to fν (.) = 1/λM (M ).

394

M. Ortner et al.

3.3 Convergence of the Markov Chain π-Stationarity π reversibility is directly obtained since the Green ratios were defined to satisfy it. Since π-reversibility implies π-stationarity, we only need to check the stability properties of the Markov Chain. Ergodic Convergence In this section we use the framework proposed by Geyer [4] and adapt it to our setup in order to check the convergence properties of the Markov Chain when the birth and death in a neighborhood update is used. We assume the following conditions: [C1] [C2] [C3a] [C3b] [C3c] [C4]

Ruelle’s stability: ∃Rh h(x ∪ u)/h(x) ≤ Rh ∀(x, u) ∈ C × S Aperiodicity P(Xt+1 = Xt ) = pI > 0 , ∃Rν ∈ R such that fν (.) ≤ Rν , ∃(rq , Rq ) ∈ R2 such that 0 < rq ≤ q(.) ≤ Rq , (.)| < RJ , ∃(rJ , RJ ) ∈ R2 such that 0 < rJ < |Jη−1 u ∃ (S1 , . . . , Sk ) a partition of S such that ∀i, (u, v) ∈ Si2 ⇒ u ∼ v.

Conditions C3 are technical conditions and are easy to insure in practice. We show that under these conditions the Markov Chain is aperiodic, irreducible, Harris recurrent and geometrically ergodic. If S is bounded, the dimension matching condition and the reflexivity of the relation ∼ imply condition C4 since we then have ∃rs s.t λ(∂(u)) ≥ rs > 0. By successively adding points in S it is therefore possible to build a partition of S which verifies condition 4. ϕ-Irreducibility and Small Sets ϕ-Irreducibility: A Markov chain (Xn )n≥0 on C is sad to be ϕ-irreducible if ϕ is a non-zero measure on C, and for all x ∈ C and B ⊂ C such that ϕ(B) > 0, there exists an integer n such that the probability Pn (x, B) of hitting B at time n while starting in x is strictly positive Pn (x, B) > 0. Small sets: A set C is small if there exists a non zero measure ϕ and an integer n such that Pn (x, B) ≥ ϕ (B) for all x ∈ C and B ∈ B. First result: If the density satisfies condition C1, and if the algorithm verifies condition C2 then it simulates a ϕ-irreducible Markov chain and every bounded set is small. We follow [4] and consider ϕ0 (.) defined on C as the measure that assign a mass equal to 1 to the empty set, and 0 to others: ϕ0 (x) = 1 if x = ∅, and ϕ0 (x) = 0 otherwise. It is indeed possible to choose Rh to satisfy condition C1 and large enough so that:

An RJMCMC Sampler for Object Detection in Image Processing RBD (x, x \ v) ≥

pb 1 Rh ν(S) pd

and 1 ≥

395

pb 1 Rh ν(S) pd

For a fixed configuration x and a integer m ≥ n(x) one obtains:  m m Pm (x, {∅}) ≥ pm BD pb pI

1 Rh ν(S)

m

which shows that the Markov chain is ϕ0 -irreducible since Pm (x, {∅}) > 0 if m ≥ n(x). Similar calculation shows that every bounded set is small. For a given m, let C = {x ∈ C : n(x) ≤ m} and introduce  c=

pm BD

pm b

pm I

1 Rh ν(S)

m

then by taking ϕ = cϕ0 , we conclude that C is small. Harris Recurrence and Geometric Ergodicity Harris Recurrence: A Markov chain with a stationary distribution π(.) is called Harris recurrent, if for all x ∈ C, and all π-positive set A, Px (τA < ∞) = 1 where τA is the stopping time τA = inf{n s.t. Xn ∈ A}. Geometric Ergodicity: A Markov Chain is geometrically ergodic if there exists a constant r > 1 such that: ∞ (

rn Pn (x, .) − π(.)T V < ∞

∀x ∈ C

n=1

Second result: The Markov chain is Harris recurrent and geometrically ergodic. A convenient way of showing Harris Recurrence is to use a drift condition for recurrence. Geometric ergodicity can be obtained by showing that the Markov Chain satisfies the geometric drift condition. In the aperiodic case, the latter implies the former, and we therefore focus on the geometric drift condition. We need to show (see [8]) that there exists a function W : C → [1, ∞), constants b < ∞ and " < 1 , and a small set C such that: PW (x) = E[W (Xt+1 )|Xt = x] ≤ W (x) + b1C (x)

∀x ∈ C

(3)

To derive the geometric drift condition we use the same function as Geyer: W (x) = An(x) , with an arbitrary A > 1. There are two transformations that can add a point to the current configuration x, “birth or death” and “birth or death in a neighborhood” (BDN). The latter is the main difference with + Geyer and Møller result. Denote by αBD (x, y) the probability of accepting a birth using the usual birth and death. Using condition C1 and the expression for Green’s ratio we obtain: + αBD (x, y) ≤

pd Rh ν(S) pb (n(x) + 1)

396

M. Ortner et al.

∈ (0, 1), there exists KBD depending on Rh , pb ,pd

Therefore, for a given and ν(S) such that

+ αBD (x, y) <  when n(x) ≥ KBD

The expression of the ratio for a death also allows to conclude that the probability of accepting a death is equal to one for n(x) large enough: − αBD (x, y) = 1 when n(x) ≥ KBD

+ We now denote by αBDN (x, y) the probability of accepting a birth using birth or death in a neighborhood. Using conditions C1 and C3: + αBDN (x, x ∪ v) ≤

jdx∪v (v) pd Rh Rν 1 x pb rZ rJ u∈x jb (u)1(u ∈ ∂(v))

The term 1(u ∈ V1(v)) comes from the expression of Λu (v). We focus on the ratio jdx∪v (v)/ u∈x jbx (u)1(u ∈ V (v)) and show that it tends to zero as n(x), the number of points in configuration x, tends to infinity. We first use the expression of jd and jb given by (2). Denoting by s(x) the number of interacting pairs of points in x (i.e. s(x) = card{R(x)}), we obtain after calculations: x∪v 1

jd (v) 1 n(x) = x j (u)1(u ∈ V (v)) 2 (s(x) + 1) u∈x b

(4)

We then use condition C4 and partition S into k subsets Si such that two points belonging to the same subset are related (in ∼ sense). We denote by ni the number of points of x falling in Si , and therefore: n(x) =

k ( i=1

ni

s(x) ≥

 k  ( ni i=1

2

= s(n1 , . . . , nk )

Considering the k-tuples (n1 , . . . ,1 nk ) in Rk , it is well known that the minimum of s(n1 , . . . , nk ) constrained to ni = n is achieved if ni = n/k for all i. We thus obtain: s(n1 , . . . , nk ) ≥ k(n/k)(n/k − 1)/2 which leads to s(x) ≥ + n(x)(n(x) − k)/2k if n(x) ≥ k. Hence for αBDN we may conclude that for a BDN given ∈ (0, 1), there exists K depending on Rh , k, Rν , rZ , rJ , pb , pd and ν(S) such that: + αBDN (x, x ∪ v) <  when n(x) ≥ KBDN

And by symmetry of the ratio, if n(x) is large enough, the probability of accepting a death in a neighborhood is equal to 1: − αBDN = 1 when n(x) ≥ KBDN

Let pst denote the probability of selecting a move that leaves the number of points unchanged. This probability pst depends on the probability of proposing a birth (as usual or in a neighborhood) and rejecting the proposition, the probability of proposing a death (simple or in a neighborhood) and rejecting the proposition, and the probability of proposing a non-jumping transformation. If we define K = max(KBD , KBDN ) then for n(x) ≥ K we have:

An RJMCMC Sampler for Object Detection in Image Processing

397

PW (x) = E[W (Xt+1 )|Xt = x] + + + pBDN pb αBDN ) + ... ≤ An(x)+1 (pBD pb αBD

 ≤

− − + pBDN pd αBDN ) . . . An(x) pst + An(x)−1 (pBD pd αBD  1 A(pBD pb + pBDN pb ) + pst + (pBDN pd + pBD pd ) W (x) A

Choose small enough to insure the existence of " < 1 such that PW (x) ≤ "W (x) for n(x) ≥ K (by taking " = A (pBD pb + pBDN pb ) + pst + 1 A (pBDN pd + pBD pd )). Let C = {x ∈ C : n(x) < K } which is small. Then PW (x) ≤ AK +1 K +1

and thus, taking b = A

for

x∈C

the geometric drift condition (3) is established.

4 Results 4.1 Test of the Sampler In this section, we present tests on an inhomogeneous Poisson process and compare the empirical distribution obtained with the theoretical one. Such tests allow to detect coding errors. We also check that the BDN kernel is useful to improve the exploration capability of the Markov Chain. We consider configurations of points on K = [0, 1] × [0, 1]. In the following sections, we consider a Poisson point process X with density h(.) against the 1 reference Poisson point process of intensity ν(.) = λ(.) where h(x) ∝ exp( u∈x β(u)) with:  β(u) =

ρe if u ∈ Ssub = [0, 12 ] × [0, 12 ] e else

For ρ = 1, the Poisson point process is homogeneous, with intensity e. For ρ > 1, the area Ssub is favored. We choose e = 20, and consider several cases (ρ = 1, ρ = 3, ρ = 6). For each case, we did two experiments, each of them consisting of 10000 trajectories, the first one using only usual birth or death proposition kernel, the second one using also birth or death in a close neighborhood (pBDN = 0.5). Distributions of Interest For a given A ⊆ S we compute a distance between the empirical law of NA (Xt ) at time t and the target distribution. Let (ˆ pt(n,A) )n≥0 be the empirical discrete distribution of the number of points of (Xt ) falling in A and (pn ) the theoretical target distribution pˆtn =

N 1 ( 1(NA (Xti ) = n) N i=1

max dm mmin (t, A) =

m( max

|pn − pˆtn |

n=mmin

Here, mmin and mmax are two relevant parameters defining a truncated support on which both distributions are compared. In practice, we use the two

398

M. Ortner et al.

following distances dS = d(t, Ssub ) and dS c = d(t, S \ Ssub ) to verify that the corresponding random variables are Poisson distributed with mean ρe and e respectively. Birth or Death in a Neighborhood We use the neighborhood u ∼ v ⇐⇒ max(|xu − xv |, |yu − yv |) ≤ dmax and a parameter dmax = 0.1. Results Figures 2, 3 and 4 show results with respectively ρ = 1, ρ = 3 and ρ = 6. The starting point is always the empty configuration. These results show that the Markov Chain ergodically converges faster when BDN (Birth or Death in a Neighborhood) is used, if inhomogeneity is important enough. In Fig. 4, it may seen that BDN makes the convergence fast on Ssub which turn to be useful in our applications. A complete study should take into account the mixing probabilities pBD , pBDN , pb and the parameter dmax .

(a) Laser DEM

(b) Estimated land register

c Fig. 1. Extraction of rectangles on a Digital Elevation Model ( IGN).

4.2 Building Detection We focus on configurations of rectangles: M = (− π2 , π2 ] × [Lmin , Lmax ] × [lmin , lmax ]. We then define a density as the product of a prior and a data term: U (x) = Uint (x) + ρUext (x)

The prior term Uint favors alignments and of rectangles. The data term uses a discontinuity detector and quantifies the relevance of a rectangle hypothesis w.r.t the data. The parameter ρ tunes the relative importance of both terms.

An RJMCMC Sampler for Object Detection in Image Processing E(N(S))=5

and

399

E(N(Sc))=15

2 c

d(S ) with BD only d(Sc) with BD+BDN d(S) with BD only d(S) with BD+BDN

1.8 1.6 1.4 1.2 1 0.8 0.6 0.4 0.2 0

0

200

400

600

800

1000

Fig. 2. Distances dS and dS c function of time with ρ = 1 using N = 10000 runs for two experiments: the first one using only Birth or Death kernel (BD), the second one using also Birth or Death in a Neighborhood (BDN). E(N(S))=15

and

c

E(N(S ))=15

2 d(Sc) with BD only d(Sc) with BD+BDN d(S) with BD only d(S) with BD+BDN

1.8 1.6 1.4 1.2 1 0.8 0.6 0.4 0.2 0

0

200

400

600

800

1000

Fig. 3. Distances dS and dS c function of time with ρ = 3 using N = 10000 runs for two experiments: the first one using only Birth or Death kernel, the second one using also Birth or Death in a Neighborhood.

400

M. Ortner et al. E(N(S))=30

and

E(N(Sc))=15

2 d(Sc) with BD only c d(S ) with BD+BDN d(S) with BD only d(S) with BD+BDN

1.8 1.6 1.4 1.2 1 0.8 0.6 0.4 0.2 0

0

200

400

600

800

1000

Fig. 4. Distances dS and dS c function of time with ρ = 6 using N = 10000 runs for two experiments: the first one using only Birth or Death kernel, the second one using also Birth or Death in a Neighborhood.

The configuration of rectangles maximizing the density is obtained by a simulated annealing applied on an RJMCMC sampler that uses birth or death in neighborhood of different types (of which alignment is an example) and different pre-explorative moves. The data are Digital Elevation Models i.e. images of an urban scene whose pixels values are proportional to the height of corresponding points. Figure 1 shows an example of result on a DEM describing the French town of Amiens. Details on the model and moves employed can be found in [9] where we also prove that the model verifies conditions C1 to C4. Thanks to the geometric prior we were able to obtain results on different kind of data with different levels of noise.

Conclusion We have adapted the Geyer-Møller sampler of point processes to our image processing problems. In particular we have proposed a Birth and Death in a neighborhood update, derived its acceptance ratio using Green’s framework, checked the stability conditions of the Markov Chain and verified its usefulness on a toy example. In [9] we extensively employ the BDN update proposed in this paper. As a result we are able to automatically perform high level analysis on a difficult image processing application, and obtain results for different kinds of data.

An RJMCMC Sampler for Object Detection in Image Processing

401

Acknowledgment The authors would like to thank the French National Geographic Institute (IGN) for providing the data and M. N. M. Van Lieshout from CWI, Amsterdam for several interesting discussions. The work of the first author has been partially supported by the French Defense Agency (DGA) and CNRS.

References 1. A. Baddeley and M.N.M. Van Lieshout. Stochastic geometry models in highlevel vision. Statistics and Images, 1:233–258, 1993. 2. J. Besag. On the statistical analysis of dirty pictures. Journal of Royal Statistic Society, B(68):259–302, 1986. 3. X. Descombes, F. Kruggel, G. Wollny, and H.J. Gertz. An object based approach for detecting small brain lesions: application to virchow-robin spaces. IEEE Transactions on Medical Imaging 23(2):246–255, feb 2004. 4. C.J. Geyer. Likehood inference for spatial point processes. In O.E. BanorffNielsen, W.S Kendall, and M.N.M. Van Lieshout, editors, Stochastic Geometry Likehood and computation. Chapman and Hall, 1999. 5. C.J. Geyer and J. Møller. Simulation and likehood inference for spatial point processes. Scandinavian Journal of Statistics, Series B, 21:359–373, 1994. 6. P.J. Green. Reversible jump Markov chain Monte-Carlo computation and Bayesian model determination. Biometrika, 57:97–109, 1995. 7. C. Lacoste, X. Descombes, and J. Zerubia. A comparative study of point processes for line network extraction in remote sensing. INRIA Research Report 4516, 2002. 8. S. P. Meyn and R.L. Tweedie. Markov Chains and Stochastic Stability. SpringerVerlag, London, 1993. 9. M. Ortner, X. Descombes, and J. Zerubia. Automatic 3D land register extraction from altimetric data in dense urban areas. INRIA Research Report 4919, August 2003. 10. M. Ortner, X. Descombes, and J. Zerubia. Improved RJMCMC point process sampler for object detection by simulated annealing. INRIA Research Report 4900, August 2003. 11. A. Pievatolo and P.J. Green. Boundary detection through dynamic polygons. Journal of the Royal Statistical Society, B(60):609–626, 1998. 12. H. Rue and M. Hurn. Bayesian object identification. Biometrika, 3:649–660, 1999. 13. R. Stoica, X. Descombes, and J. Zerubia. A gibbs point process for road extraction from remotely sensed images. Int. Journal on Computer Vision, 37(2):121– 136, 2004. 14. M.N.M. Van Lieshout. Markov Point Processes and their Applications. Imperial College Press, London, 2000.

Quasi-Monte Carlo for Integrands with Point Singularities at Unknown Locations Art B. Owen Stanford University, Stanford CA 94305, USA [email protected]

Summary. This article considers quasi-Monte Carlo sampling for integrands having isolated point singularities. It is usual for such singular functions to be approached via importance sampling. Indeed one might expect that very uniform sampling, such as QMC uses, should be unhelpful in such problems, and the Koksma-Hlawka inequality seems to indicate as much. Perhaps surprisingly, we find that the expected errors in randomized QMC converge to zero at a faster rate than holds for Monte Carlo sampling, under growth conditions for which 2 +  moments of the integrand are finite. The growth conditions do place constraints on certain partial derivatives of the integrand, but unlike importance sampling, they do not require knowledge of the locations of the singularities.

1 Introduction The core problem in quasi-Monte Carlo (QMC) integration is to compute an estimate of  f (x)dx . (1) I= [0,1]d

The estimates typically take the form 1( Iˆ = f (xi ) , n i=1 n

(2)

for carefully chosen xi . In applications of QMC, some problem specific manipulations are usually made to render the problem and solution into the forms given by (1) and (2) respectively. The classical Monte Carlo method (MC) takes the
xi to be independent random vectors with the U (0, 1)d distribution. When f (x)2 dx < ∞, MC achieves a root mean square error (RMSE) that is O(n−1/2 ). Hlawaka [7] proved the Koksma-Hlawka inequality: for Riemann integrable f , and x1 , . . . , xn ∈ [0, 1]d ,

404

A.B. Owen

|Iˆ − I| ≤ VHK (f )Dn∗ (x1 , . . . , xn ) ,

(3)

where Dn∗ is the star discrepancy of x1 , . . . , xn , and VHK denotes total variation in the sense of Hardy and Krause. Many constructions are known to satisfy Dn∗ (x1 , . . . , xn ) = O(n−1+ ) for all > 0. If VHK (f ) < ∞ then Hlawka’s theorem assures the asymptotic superiority of QMC over MC. Now suppose that VHK (f ) = ∞. Then (3) reduces to |Iˆ − I| ≤ ∞, and so is completely uninformative. We cannot tell whether QMC is asymptotically better than MC, nor even whether the QMC error will converge to zero. Infinite variation is not an obscure issue. It arises commonly in applications. The indicator function for a subset of [0, 1]d for d ≥ 2, has infinite variation, unless that subset has boundaries aligned to the axes of the unit cube. Similarly, functions formed piecewise such as max(g1 (x), g2 (x)) commonly contain a cusp along the set where g1 (x) = g2 (x) that gives them infinite variation, when d ≥ 3. A detailed investigation of the variation of such functions is given in [12]. The integrands we consider here are unbounded. An unbounded integrand necessarily has infinite variation in the sense of Hardy and Krause. Unbounded integrands arise in some problems with Feynmann diagrams from physics, as in [1]. They can also arise from variable transformations such as importance sampling and mappings used to convert problems on infinite domains such as [0, ∞)d or Rd to the unit cube. For some unbounded functions, QMC still attains the O(n−1+ ) rate of convergence. For others, the QMC error |Iˆ − I| diverges to infinity as n → ∞, even as the star discrepancy satisfies Dn∗ = O(n−1+ ). Reference [13] includes examples of both cases. Section 2 reviews the literature on QMC for unbounded functions. Most of that work considers singularities located on the boundary faces or corners of the unit cube, and in the exceptions we must know something about where the internal singularities are. The emphasis here is on functions that diverge to infinity at a finite number of interior points, whose locations are not necessarily known. Section 3 outlines some error bounds obtained from approximating f by another function f that has finite Hardy-Krause variation while remaining close to f . Section 4 presents the notation and some background material. Section 5 develops the error rates that are the main result of this paper. Section 6 ends with some more general comments on randomized QMC versus Monte Carlo.

2 Literature Lehman [10] may have been the first to investigate integration of general unbounded functions by equidistributed sequences. He considered periodic functions on R with f (x + 1) = f (x) and sampling at points xi = iξ for irrational ξ. His functions can be singular either as x ↑ 1 or as x ↓ 0. His

QMC for Point Singularities

405

Theorem 4 also allows finitely many internal singularities, though one has to check that the points iξ for 1 ≤ i ≤ n avoid the singularities suitably. Sobol’ [14] made another large step in the development of quasi-Monte Carlo methods for singular functions. His paper has many results for the case d = 1, but we focus here on the multidimensional results. Sobol’ [14] considers functions f (x) that are singular as x approaches the origin of [0, 1]d , such as products of negative powers of the components of x. These functions are in fact singular as x approaches the entire ‘lower’ boundary of [0, 1]d . He shows that quasi-Monte Carlo points which avoid a hyperbolic region near that boundary lead to consistent integral estimates, that is |Iˆ − I| → 0 as n → ∞. Sobol’s work has been extended recently. Like [14], de Doncker and Guan [3] consider power law singularities around the origin, and they find some benefit from extrapolation schemes. Klinger [9] shows that Halton points as well as certain digital nets avoid cubical regions around the origin, making them suited to integrands with point singularities at (0, . . . , 0). In his dissertation, [8, Theorem 3.17], he gives conditions under which the points of a Kronecker sequence, a d dimensional version of the points studied in [10], avoid a hyperbolic region around the origin. Hartinger, Kainhofer & Tichy [6] show how to replace the uniform density implicit in (1) with a more general bounded probability density function, and they show how to generate low discrepancy points with respect to that density. In that paper the sample points have every component bounded away from zero, effectively avoiding an Lshaped region around the origin. Reference [13] extends Sobol’s work to get rates of convergence, and to consider functions singular around any or all of the 2d corners of [0, 1]d . The rate can be as good as O(n−1+ ) if the singular function obeys a strict enough growth rate. Both hyperbolic and L-shaped regions are considered. That paper also shows how the Halton sequences avoid the boundaries of the unit cube as do randomized QMC points. Hartinger & Kainhofer [5] consider non-uniform densities, with singularities arising at any corner and with hyperbolic or L-shaped avoidance regions, and obtain the same rates as in [13]. As motivation, Sobol’ [14] observed back in 1973 makes the observation that his quadrature methods were being used in practice with success on singular integrands, despite the fact that the supporting theory is limited to bounded integrands. Much of the recent work is motivated in part by the observation that good results are obtained on certain Asian option valuation problems, despite the unboundedness of the corresponding integrands.

3 Use of Low Variation Approximations Suppose that f has infinite variation, but is close to another function f that  Then a three epsilon argument gives has finite variation and integral I.

406

A.B. Owen

( ˆ ≤ |I − I|  + Dn∗ (x1 , . . . , xn )VHK (f) + 1 |I − I| |f (xi ) − f(xi )| . n i=1 n

(4)

We want the sum on the right side of (4) to converge to zero as n increases. To achieve this we will let f change as n increases, and bound all three terms by a common rate in n. Because Dn∗ → 0 we can let VHK (f) increase with n in order to obtain a reduction in the first and third terms. In principle, the infimum of (4) taken over functions f is also an upper bound on the error. In practice, we need strategies for constructing specific f that we can study. One strategy is that of “avoiding the singularity”. We take a set K on which f is bounded, and arrange to sample only within K. The ideal set K might be a level set {x ∈ [0, 1]d | |f (x)| ≤ η}. When such a set is hard to work with, K might be geometrically chosen to avoid the origin or some other place where the singularity arises. Suppose that xi are constructed to belong to a subset K of [0, 1]d , and that f is not singular in K. The function f is an extension of f from K ⊆ [0, 1]d to [0, 1]d if f(x) = f (x) whenever x ∈ K. Then, for deterministic points xi ∈ K we obtain ˆ ≤ |I − I|  + Dn∗ (x1 , . . . , xn )VHK (f) . |I − I|

(5)

In a closely related strategy, suppose that xi are randomized QMC points, where each xi individually has the U [0, 1]d distribution and for any > 0 there is D < ∞ such that Dn∗ (x1 , . . . , xn ) ≤ D n−1+ always holds. Then taking expectations in (4) we get  ˆ ≤2 |f(x) − f (x)|dx + D n−1+ VHK (f) . (6) E(|I − I|) [0,1]d

Random points generally have some positive probability of landing outside K, and as a result, the last term in (4) contributes to the first term in (6). Both strategies require a function f that is close to f and has a small variation. Ilya M. Sobol’ invented such a low variation extension. It was used in [14] but Sobol’ did not publish it. An account of it appears in [12] and also in the next section. Our present purpose is to study integration of functions with point singularities at possibly unknown locations. It is then hard to construct QMC points that avoid the singularity. Accordingly, the randomized QMC strategy is the one that we choose to employ here.

4 Notation A point x ∈ Rd is written as x = (x1 , . . . , xd ). For a, b ∈ Rd with aj ≤ bj , the 2d hyperrectangle [a, b] is the Cartesian product j=1 [aj , bj ]. Most of our work

QMC for Point Singularities

407

will be with the unit hyperrectangle defined via aj = 0 and bj = 1. The unit hyperrectangle is denoted [0, 1]d . We need more general hyperrectangles [a, b], particularly for subsets of [0, 1]d . For any points a, b ∈ Rd the rectangular hull of a and b is rect[a, b] =

d 

[min(aj , bj ), max(aj , bj )] ,

j=1

a kind of d dimensional bounding box determined by points a and b. For u ⊆ 1 : d ≡ {1, . . . , d} we write |u| for the cardinality of u, and −u for the complement {1 ≤ j ≤ d | j ∈ u} of u. The set difference is v − u = {j ∈ v | j ∈ u}. For disjoint sets u and v their union may be written u + v. The symbol xu denotes the |u|-tuple consisting of all xj for j ∈ u. The domain of xu is written [0u , 1u ] or [0, 1]u . For x, y ∈ [0, 1]d the expression xu : y −u denotes the point z ∈ [0, 1]d with j z = xj for j ∈ u and z j = y j for j ∈ u. Similar more general expressions are used. For example, with disjoint sets u, v, w ⊆ 1 : d, the expression xu : y v : z w designates a point in [0, 1]u+v+w . When x−u is held fixed then f can be used to define a function of xu over [0, 1]u . This function is denoted by f (xu ; x−u ), with the argument xu appearing before the semi-colon, and the parameter x−u after it. The value of f (xu ; x−u ) is f (xu : x−u ). We let ∂ u f denote the mixed partial derivative of f taken once with respect to each xj for j ∈ u. By convention ∂ ∅ f = f . The variation of f over the hyperrectangle [a, b] in the sense of Vitali, denoted by V[a,b] (f ) is defined in [11]. We will use some properties of it, as surveyed in [12]. Suppose that ∂ 1:d f exists. Then Fr´echet [4] showed in 1910 that  V[a,b] (f ) ≤ |∂ 1:d f (x)|dx , (7) [a,b]

with equality when ∂ 1:d f is continuous. The variation of f in the sense of Hardy and Krause is ( V[0v ,1v ] f (xv ; 1−v ) . (8) VHK (f ) = v =∅

Sobol’s extension begins with an extendable region. Such a region K, must be accompanied by an anchor point c ∈ K such that z ∈ K implies rect[c, z] ⊆ K. Figure 1 illustrates some extendable regions. The function f is extendable if ∂ 1:d f exists at every point of K. A more general definition of extendable function is available if x ∈ K implies xj = cj for some 1 ≤ j ≤ d (see [12]), but we do not need it here. Sobol’s extension is defined as follows. First, the function f can be written as

408

A.B. Owen

Fig. 1. This figure shows some extendable regions. In the left panel, the region K is above a hyperbolic branch, and the anchor point is at (1, 1). In the right panel, the region K excludes four small corner squares, and the anchor point is at (1/2, 1/2). For each panel one point is selected interior to K and one is selected on the boundary of K. The dashed lines indicate the rectangular hulls joining each selected point to its anchor.

f (x) = f (c) +

( u =∅

∂ u f (y u : c−u )dy u .

(9)

[cu ,xu ]

In integrals like those in (9), the region
of integration is not
a proper hyperrectangle if some cj > xj . We take [cu ,xu ] g(y u )dy u = ± rect[cu ,xu ] g(y u )dy u where the sign is negative if and only if cj > xj holds for an odd number of j ∈ u. Sobol’s extension is then ( ∂ u f (y u : c−u )1yu :c−u ∈K dy u . (10) f(x) = f (c) + u =∅

[cu ,xu ]

For x ∈ K the factor 1yu :c−u ∈K in (10) always equals 1, so that f(x) = f (x) for x ∈ K, and then the term “extension” is appropriate. Reference [12]

shows that V[a,b] (f) ≤ K |∂ 1:d f (x)|dx = K |∂ 1:d f(x)|dx. In view of Fr´echet’s result (7), Sobol’s extension has as low a variation as one could expect. To compute variation, we will need derivatives of f(x). First notice that the f (c) term in (10) corresponds to u = ∅ in a natural convention. Also f(x) depends on x only through the role x plays in limits of integration. Accordingly, for w ⊆ 1 : d, ( ∂ u f (y u−w : xw : c−u )1yu−w :xw :c−u ∈K dy u−w . (11) ∂ w f(x) = u⊇w

In particular

[cu−w ,xu−w ]

QMC for Point Singularities

∂ 1:d f (x), 1:d  ∂ f (x) = 0,

409

x∈K else.

5 Integrable Point Singularities Suppose that the function f has a finite number L of integrable singularities at distinct points z1 , . . . , zL ∈ (0, 1)d . Without loss of generality, we may suppose that L = 1 because integration errors sum. Let the function f have a point singularity at z ∈ (0, 1)d . We do not assume that the position of z is known. where 0 < A < d, We will suppose that f is such that |f (x)| ≤ x − z−A p and 1 ≤ p < ∞. The lower limit on A provides the singularity, while the upper limit ensures that the singularity is integrable:  x − [0,1]d

z−A p dx

 ≤ Sd,p

(1,...,1)p

y −A+d−1 dy

0

 S d(d−A)/p Sd,p  (d−A)/p d,p d , ≤ − 0d−A = d−A d−A

where Sd,p is the d − 1 dimensional volume of the set {x ∈ Rd | xp = 1}. If A < d/2, then f 2 is integrable. In this case Monte Carlo sampling has root mean squared error O(n−1/2 ). 5.1 Growth Conditions We need a notion of a singularity “no worse than” x − z−A p , and to obtain it we impose growth conditions on the partial derivatives of f . If f = x − z−A p for 1 < p < ∞, then for x = z, ∂f = −Ax − z−A−p |xj − z j |p−1 sign(xj − z j ) p ∂xj and generally for u ⊆ {1, . . . , d}, ∂uf =

 |u|−1  k=0

  (−A − kp) x − z−A−|u|p |xj − z j |p−1 sign(xj − z j ) . p j∈u

(12) An easy Lagrange multiplier argument gives, for non-empty u,  −|u|/p |xj − z j | ≤ |u|−|u|/p xu − z u |u| x − z|u| p ≤ |u| p .

(13)

j∈u

Now applying (13) to (12) and absorbing the constants, we obtain the growth condition

410

A.B. Owen

|∂ u f (x)| ≤ Bx − z−A−|u| p

(14)

required to hold for all u ⊆ {1, . . . , d}, all x = z, some 0 < A < d, some for f is B < ∞, and some 1 ≤ p < ∞. For p = 1 the upper bound x − z−A 1 not differentiable, and hence does not itself satisfy the growth conditions. Yet, growth condition (14) for p = 1 may still be of use, when a specific integrand of interest can be shown to satisfy it. 5.2 Extension from x − zp ≥ η To avoid the singularity in x − z−A p , x must satisfy x − zp ≥ η for some η > 0. But if z ∈ (0, 1)d , then the level set K = {x | x − zp ≥ η}, for small η, is not an extendable region: there is no place to put the anchor c so that rect[c, z + η(1, 0, . . . , 0)] and rect[c, z − η(1, 0, . . . , 0)] are both subsets of K. We will make 2d extensions, one from every orthant of [0, 1]d , as defined with respect to an origin at z. These orthants are, for u ⊆ {1, . . . , d}, Cartesian products of the form   C(u) = C(z, u) ≡ [0, z j ] [z j , 1] . (15) j∈u

j ∈u

The set C(u) may also be written as [0u : z −u , z u : 1−u ] or [0, z]u × [z, 1]−u . For small positive η and z ∈ (0, 1)d , the region Ku = Ku (η) ≡ C(u)∩{x− zp ≥ η} is extendable to C(u) with respect to the anchor at cu = 0u : 1−u . Figure 2 illustrates how [0, 1]d can be split into orthants, for d = 2. We let fu (x) be the Sobol’ extension of f from Ku to C(u) with anchor cu . The orthants in (15) overlap, and so we cannot simply take f to be fu (x) for x ∈ C(u). The sets   C(u) = C(z, u) ≡ [0, z j ) [z j , 1] , (16) j∈u

j ∈u

form a partition of [0, 1]d . We may now define f(x) = fu (x),

for x ∈ C(z, u) ,

(17)

and then f(x) is well defined on [0, 1]d . Lemma 1. If f satisfies (14) with z ∈ (0, 1)d , then for fu as defined above, fu (x) − f (x)p ≤ Bu x − z−A p

(18)

holds for all x ∈ C(u) and some Bu < ∞, and  − z−A f(x) − f (x)p ≤ Bx p  < ∞. for all x ∈ [0, 1]d and some B

(19)

QMC for Point Singularities

411

Fig. 2. This figure illustrates the orthants used to extend a function towards a point singularity. The solid point at z = (0.6, 0.4), represents the site of the singularity. It is surrounded by a curve x − zp = η, where p = 3/2 and η = 1/10. The orthants C(u) are rectangles with one corner at z and another at a corner cu of [0, 1]2 . They are labeled by the set u, taking values {}, {1}, {2}, and {1, 2}. The part Ku of C(u) outside of the curve around z is a region extendable to C(u) with respect to an anchor at cu .

 = maxu Bu , so we only Proof: Equation (19) follows from (18) by taking B need to show (18). For x ∈ C(u), subtracting representation (10) from (9) term by term gives ( −A−|v|  |f (x) − f (x)| ≤ B 1yv : c−v y v : c−v dy v u − zp u ∈Kv ≤B ≤B

v =∅

v [cv u ,x ]

v =∅

v [cv u ,x ]

( (

v =∅

≤

−A−|v| y v : c−v dy v u − zp

cu −zp

xv :c−v u −zp

α−A−|v| Sp,|v| α|v|−1 dα

B( −A Sp,|v| xv : c−v u − zp . A v =∅

For v = {1, . . . , d}, we have −A xv : c−v ≤ min |z j − cju |−A ≤ min |z j − cju |−A . u − zp j ∈v

1≤j≤d

  Thus |f (x) − f(x)| ≤ BSp,d x − z−A + B  /A where p

412

A.B. Owen

( d Sp,|v| 1≤j≤d |v| 0<|v|
B  = B min |z j − cj |−A ≤

inf y∈C(u) y − z−A p

x − z−A p ,

so that a finite Bu exists with |f (x) − f(x)| ≤ Bu x − z−A for x ∈ C(u). p



Lemma 1 will allow us to find an upper bound to the first term in the expected absolute quadrature error (6). To handle the second term, we need to consider the variation of f, in the sense of Hardy and Krause: ( VHK (f) = V[0v ,1v ] (f(xv ; 1−v )) v =∅

=

((

V[0,z]u ×[z,1]v−u (f(xv ; 1−v )).

(20)

v =∅ u⊆v

The second equality in (20) comes from an additive property of Vitali variation stated in [15] and repeated in [12]. Apart from boundary points, all of [0, z]u × [z, 1]v−u is inside C(u). But we cannot simply replace f by fu in (20). The reason is that f is ordinarily discontinuous along the boundaries between orthants and such a discontinuity contributes to the Vitali variation of f. For d = 2 and x ∈ C(∅), we find that ⎧ f (x), x ∈ [0, z 1 ) × [0, z 2 ) ⎪ ⎪ ⎪ ∅ ⎨ x ∈ {z 1 } × [0, z 2 ) f{1} (x), f(x) =  (21) ⎪ x ∈ [0, z 1 ) × {z 2 } f{2} (x), ⎪ ⎪ ⎩ f{1,2} (x), x ∈ {z 1 } × {z 2 }. For general d and general u, the function f(x) on C(u) takes on values from each of the 2d−|u| functions fv with v ⊇ u. The small example in (21) can be written as a sum of inclusion-exclusions     f(x) = f∅ (x) + f{1} (x) − f∅ (x) 1x1 =z1 + f{2} (x) − f∅ (x) 1x2 =z2   + f{1,2} (x) − f{1} (x) − f{2} (x) + f∅ (x) 1x{1,2} =z{1,2} , x ∈ C(∅), and more generally, for x ∈ C(u), ( f(x) = gu,w (x), w⊆−u

gu,w (x) = 1xw =zw

( t⊆w

where,

(22)

(−1)|w−t| fu∪t (x) ,

(23)

QMC for Point Singularities

where by convention, 1x∅ =z∅ = 1. Therefore (( ( V[0,z]u ×[z,1]v−u ( gu,w (xv ; 1−v )) VHK (f) ≤

413

(24)

v =∅ u⊆v w⊆−u

because the Vitali variation of a sum of functions is no larger than the sum of the variations of the functions. Because z is an interior point, no z j = 1. If w intersects −v, then gu,w (xv ; 1−v ) = 0 for all xv ∈ [0, z]u × [z, 1]v−u . The sum over w in (24) can therefore be reduced to one over v − u. Many of the functions gu,w vanish except on one “face” of the hyperrectangle on which they are defined. Variation of such functions is covered by the next lemma. Lemma 2. Let a, b ∈ Rd with a ≤ b componentwise. Let u, v, w ⊆ {1, . . . , d} be disjoint with u ∪ v ∪ w = {1, . . . , d}. Suppose that f (x) is defined on [a, b] with f (x) = 0 unless xu = au and xv = bv . Then ⎧ ⎪ Vol([a, b]) = 0 ⎨0, u v V[a,b] (f ) = |f (a : b )|, (25) Vol([a, b]) > 0, w = ∅ ⎪ ⎩ w u v V[aw ,bw ] (f (x ; a : b )), Vol([a, b]) > 0, w = ∅. Proof: This follows from Propositions 18 and 19 of [12].



Lemma 3. Let f satisfy the growth conditions (14) for some z ∈ (0, 1)d . Let f(x) be the extension of f (x) from K = {x ∈ [0, 1]d | x − zp ≥ η} given by (17). Then as η → 0, VHK (f) = O(η −A ) .

(26)

Proof: In view of Lemma 2, we may write   (( ( ( VHK (f) ≤ V[0,z]u ×[z,1]v−u fu+t (xv ; 1−v )1xt =zt v =∅ u⊆v w⊆v−u t⊆w

=

(( ( (

  V[0,z]u ×[z,1](v−t)−u fu+t (xv−t ; z t : 1−v )

v =∅ u⊆v w⊆v−u t⊆w

≤

(( ( (

v =∅ u⊆v w⊆v−u t⊆w

Applying (11) and (14)

|∂ v−t fu+t (xv−t ; z t : 1−v )| dxv−t .

[0,z]u ×[z,1](v−t)−u

414

A.B. Owen

|∂ v−t fu+t (xv−t ; z t : 1−v )| (  ≤ |∂ s f (y s−(v−t) : xv−t : c−s u+t )|1y s−(v−t) :xv−t :c−s s−(v−t)

s⊇v−t

≤B

[cu+t

( 

≤B

dy s−(v−t)

−A−|s| y s−(v−t) : xv−t : c−s 1ys−(v−t) :xv−t :c−s u+t − zp

u+t ∈K

s−(v−t) s−(v−t) [cu+t ,x ]

s⊇v−t

(

u+t ∈K

,xs−(v−t) ]

 Sp,|s|

s⊇v−t

−(v−t)

xv−t :cu+t

xs :c−s u+t −zp

−zp

dy s−(v−t)

α−A−|s|+|s|−1 1α≥η dα

−A  B ( Sp,|s| max η, xs : c−s ≤ u+t − zp A s⊇v−t

= O(η −A ). The estimate O(η −A ) propagates through the integral and four summations in VHK (f), to give VHK (f) = O(η −A ). . Theorem 1. Let f be a Lebesgue measurable function with integral I, and a singularity at z ∈ (0, 1)d for which the growth conditions (14) hold. Let x1 , . . . , xn be randomized quasi-Monte Carlo points: each xi ∼ U [0, 1]d , and ∗ −1+ )=1 for any > 0 there exists D1  < ∞ with Pr(Dn (x1 , . . . , xn ) ≤ D n n for all n ≥ 1. Let Iˆ = (1/n) i=1 f (xi ). Then E(|Iˆ − I|) = O(n(−1+)(d−A)/d ) . Proof: For small η > 0, define f by extending f from {x ∈ [0, 1]d | x−zp ≥ η} as described above. Then  ˆ E(|I − I|) ≤ 2 |f(x) − f (x)|dx + E(Dn∗ (x1 , . . . , xn ))fHK . Combining the hypothesis of the Theorem with Lemma 3 shows that the second term is O(n−1+ η −A ). From Lemma 1,    |f(x) − f (x)|dx ≤ B x − z−A p 1x−zp ≤η dx [0,1]d

[0,1]d



η

≤ BSp,d = O(η

α−A+d−1 dα

0 d−A

).

Taking η proportional to n(−1+)/d gives E(|Iˆ − I|) = O(n(−1+)(d−A)/d ).  Suppose that the integrand f has L singularities at points z ∈ (0, 1)d , for  = 1, . . . , L. Under the following mild conditions on f , the worst of L growth

QMC for Point Singularities

415

conditions will determine the error rate. First, there are L hyperrectangles  Suppose [a , b ] ⊂ (0, 1)d with z ∈ (a , b ) and [a , b ] ∩ [a, b] = ∅ for  = .  holds for x ∈ [a , b ] − {z } and B that |∂ u f (x)| ≤ B x − z −A     < ∞, and p 1:d 0 < A < d, and 1 ≤ p < ∞. Suppose also that |∂ f (x)| is bounded on  [0, 1]d − ∪L =1 (a , b ). We may take f to equal a low variation extension within each of the [a , b ] and to equal f outside of them. Then for the randomized QMC points in Theorem 1 we get E(|Iˆ − I|) = O(n(−1+)(d−max1≤≤L A )/d ) . The next result (whose proof is immediate) shows that randomized QMC is asymptotically superior to independent Monte Carlo sampling in this setting,
when |f (x)|2+ dx < ∞ for some > 0. Corollary 1. Suppose that the conditions of Theorem 1 hold with A < d/2. Then E(|Iˆ − I|) = o(n−1/2 ).

6 Discussion It is surprising to find that even for functions with unknown point singularities, that randomized QMC has superior asymptotics to ordinary Monte Carlo sampling, which after all, includes importance sampling. One might have thought that spreading points uniformly through space would be wasteful for such functions. The key measure of strength for our growth conditions is the scalar A. When A < d/2, then Monte Carlo has the O(n−1/2 ) rate and randomized QMC has expected absolute error O(n(d−A)/d ) = o(n−1/2 ). In fairness, we should note that the QMC approach does make some stronger assumptions than Monte Carlo. The randomized QMC approach requires conditions on mixed partial derivatives of f that are unnecessary in MC. Perhaps a bigger loophole is that the Koksma-Hlawka bound can be extremely conservative. For large d it is certainly plausible that the superior rate for randomized QMC will be slow to set in, and that independent Monte Carlo with importance sampling will beat randomized QMC in practical samples sizes. Perhaps the best approach is to employ an importance sampling strategy in conjunction with randomized QMC points. The importance sampling strategy may well leave an integrand with singularities. Then, the results here show that singular integrands need not be a serious drawback for randomized QMC. Randomized QMC has one important practical advantage compared to importance sampling. We do not need to know the locations of the singularities. It would be interesting to know how generally randomized QMC is asymptotically superior to Monte Carlo for singular functions. From the BorelCantelli theorem [2] we can find that randomized QMC points will avoid high

416

A.B. Owen

level sets, in that
max1≤i≤n |f (xi )| > n will only happen finitely often with probability one, if |f (x)|dx < ∞. It is not however clear that one can always find a good extension f for f , from {x ∈ [0, 1]d | |f (x)| ≤ n} to [0, 1]d .

Acknowledgments I thank Harald Niederreiter and Denis Talay for organizing MC2QMC2004 and for editing this proceedings volume. I also thank Harald Niederreiter and one anonymous reviewer for their careful scrutiny of this paper. Further thanks are due to Reinhold Kainhofer for discussions on QMC for unbounded integrands, Bernhard Klinger for providing a copy of his dissertation, and again to Harald Niederreiter for telling me about Lehman’s results. This work was supported by grant DMS-0306612 from the U.S. National Science Foundation.

References 1. Aldins, J., Brodsky, S. J., Dufner, A. J. & Kinoshita, T. (1970), ‘Moments of the muon and electron’, Physical Review D 1(8), 2378–2395. 2. Chung, K.-L. (1974), A Course in Probability Theory, 2nd edn, Academic Press, New York. 3. de Doncker, E. & Guan, Y. (2003), ‘Error bounds for the integration of singular functions using equidistributed sequences’, Journal of Complexity 19(3), 259– 271. 4. Fr´echet, M. (1910), ‘Extension au cas des int´egrales multiples d’une d´efinition de l’int´egrale due ` a Stieltjes’, Nouvelles Annales de Math´ematiques 10, 241–256. 5. Hartinger, J. & Kainhofer, R. F. (2005), Non-uniform low-discrepancy sequence generation and integration of singular integrands, in H. Niederreiter & D. Talay, eds, ‘Proceedings of MC2QMC2004, Juan-Les-Pins France, June 2004’, Springer-Verlag, Berlin. 6. Hartinger, J., Kainhofer, R. F. & Tichy, R. F. (2004), ‘Quasi-Monte Carlo algorithms for unbounded, weighted integration problems’, Journal of Complexity 20(5), 654–668. 7. Hlawka, E. (1961), ‘Funktionen von beschr¨ ankter Variation in der Theorie der Gleichverteilung’, Annali di Matematica Pura ed Applicata 54, 325–333. 8. Klinger, B. (1997a), Discrepancy of Point Sequences and Numerical Integration, PhD thesis, Technische Universit¨ at Graz. 9. Klinger, B. (1997b), ‘Numerical integration of singular integrands using lowdiscrepancy sequences’, Computing 59, 223–236. 10. Lehman, R. S. (1955), ‘Approximation of improper integrals by sums over multiples of irrational numbers’, Pacific Journal of Mathematics 5, 93–102. 11. Niederreiter, H. (1992), Random Number Generation and Quasi-Monte Carlo Methods, S.I.A.M., Philadelphia, PA. 12. Owen, A. B. (2005), Multidimensional variation for quasi-Monte Carlo, in J. Fan & G. Li, eds, ‘International Conference on Statistics in honour of Professor KaiTai Fang’s 65th birthday’.

QMC for Point Singularities

417

13. Owen, A. B. (2006), ‘Halton sequences avoid the origin’, SIAM Review 48. To appear. 14. Sobol’, I. M. (1973), ‘Calculation of improper integrals using uniformly distributed sequences’, Soviet Math Dokl 14(3), 734–738. 15. Young, W. H. (1913), Proceedings of the London Mathematical Society (2) 11, 142.

Infinite-Dimensional Highly-Uniform Point Sets Defined via Linear Recurrences in F2w Fran¸cois Panneton and Pierre L’Ecuyer D´epartement d’informatique et de recherche op´erationnelle, Universit´e de Montr´eal, C.P. 6128, Succ. Centre-Ville, Montr´eal (Qu´ebec), H3C 3J7, Canada [email protected] and [email protected] Summary. We construct infinite-dimensional highly-uniform point sets for quasiMonte Carlo integration. The successive coordinates of each point are determined by a linear recurrence in F2w , the finite field with 2w elements where w is an integer, and a mapping from this field to the interval [0, 1). One interesting property of these point sets is that almost all of their two-dimensional projections are perfectly equidistributed. We performed searches for specific parameters in terms of different measures of uniformity and different numbers of points. We give a numerical illustration showing that using randomized versions of these point sets in place of independent random points can reduce the variance drastically for certain functions.

1 Introduction Quasi-Monte Carlo (QMC) methods estimate an integral of the form  f (u)du , µ=

(1)

[0,1)t

for a given function f , by the average Qn =

n−1 1( f (ui ) , n i=0

(2)

for a highly-uniform (or low-discrepancy) point set Pn = {u0 , . . . , un−1 } ⊂ [0, 1)t . Randomized QMC (RQMC) randomizes the point set Pn before computing Qn , in a way that each individual point is uniformly distributed over [0, 1)t even though the point set as a whole keeps its high uniformity [3,6,8,12]. In many practical simulation settings, f depends on a random and unbounded number of uniforms [8]. This can be covered by taking t = ∞, with the understanding that f would typically depend only on a finite number of coordinates of u with probability 1, if we interpret u as an infinite sequence of independent uniform random variables over [0, 1). On the other hand, most

420

F. Panneton and P. L’Ecuyer

popular point set constructions (e.g., digital nets and lattice rules) usually assume a fixed (finite) value of t. There are exceptions, e.g., Korobov lattice rules and Korobov polynomial lattice rules [2, 4], where the dimension can be infinite. In this paper, we introduce a method for constructing infinite-dimensional point sets Pn via a linear recurrence in the finite field F2w and a mapping from F2w to the interval [0, 1). The construction is similar to the one used in [11] for random number generation. These point sets are dimension-stationary, i.e., their projections over a subset of coordinates depend only on the spacings between these coordinates. Moreover, most of their two-dimensional projections have maximal equidistribution. We provide a formula that gives the precise number in terms of the parameters of the recurrence. We define several measures of uniformity for Pn in terms of its equidistribution properties, its q-value, and the distance between the closest points, in several dimensions. We report partial results of a search for good point sets in terms of these criteria. Then we try randomized versions of these point sets on a few test problems and compare them, in terms of variance reduction with respect to standard Monte Carlo (MC) simulation, with Sobol’ nets randomized in the same way. In certain settings, the new point sets perform much better than the Sobol’ nets.

2 Definition of the Point Sets Our point sets are constructed as follows. The successive coordinates of each point are defined in essentially the same way as the successive random numbers in [11]. Let q = 2w for some integer w ≥ 1 and Fq the finite field with q elements. We consider a linear recurrence of order r in Fq , mn =

r (

bi mn−i ,

(3)

i=1

br = 0, and where r is a positive integer, b1 , . . . , br and m0 , m1 , . . . are in Fq , 1 r all arithmetic is performed in Fq . The polynomial P (z) = z r − i=1 bi z r−i is a characteristic polynomial of this recurrence. It is well-known that (3) has period length q r − 1 = 2rw − 1 (full period) for any nonzero initial state (m−r+1 , . . . , m0 ) ∈ Frq if and only if P (z) is primitive over Fq . Regardless of the primitivity of P (z), the recurrence (3) is purely periodic, in the sense that it has no transient state. See, e.g., [5, 6] for an account of linear recurrences in finite fields. To construct a point set from such a recurrence, we must define a mapping from the state space Frq to the real interval [0, 1). This requires an explicit representation of the elements of Fq . As in [11], we represent these elements in terms of an ordered polynomial basis, defined as follows. Let M (z) = z w +

Highly-Uniform Point Sets Defined via Linear Recurrences in F2w

421

1w

ai z w−i ∈ F2 [z] be an irreducible polynomial over F2 . Then there exists an algebraic element ζ of Fq whose minimal polynomial over F2 is M (z) and the ordered set (1, ζ, . . . , ζ w−1 ) is an ordered polynomial basis of Fq over F2 (see [5], Chap. 1.4). This means that any element v ∈ Fq can be written uniquely as a linear combination v = v1 +v2 ζ +· · ·+vw ζ w−1 where v = (v1 , . . . , vw )T ∈ Fw 2. Here, we identify F2 with the set {0, 1} in which addition and multiplication are performed modulo 2. Thus, after M (z) has been chosen, each element v of Fq can be represented by its corresponding binary column vector v, called its vector representation. Then, as explained in [11], the recurrence (3) can be implemented by r ( Abi mn−i (4) mn = i=1

i=1

where mn is the vector representation of mn and Abi performs the multiplication by bi in the vector representation, for 1 ≤ i ≤ r. Under this representation, the state at step n can be written as the rw-bit column vector T T sn = (mT n−r+1 , . . . , mn ) .

From recurrence (4), we define an output sequence u0 , u1 , . . . in [0, 1) as follows: T T y i = (mT = (yi,0 , yi,1 , . . .)T , iν , miν+1 , . . .) ∞ ( ui = yi,j−1 2−j

(5)

j=1

for i ≥ 0, where ν is a fixed positive integer and yi,0 , yi,1 , . . . are the successive bits of y i . In practice, y i and the expansion in (5) are necessarily truncated to a finite number of bits, but here we neglect the impact of this truncation. Let (6) Pn = {(u0 , u1 , u2 , . . .) : s0 ∈ F2rw } be the set of all sequences of successive output values ui , from all possible initial states s0 = (m−r+1 , . . . , m0 ) in F2rw . Since the number of states is 2rw and the recurrence (3) is purely periodic, the cardinality of Pn is n = 2rw . This Pn is our infinite-dimensional point set. Each point u ∈ Pn is in fact a periodic infinite sequence, whose period length is that of the cycle of the recurrence that corresponds to the initial state s0 . In the case where P (z) is primitive, for example, there are two cycles: one contains the single state s0 = 0 and has period 1 (it gives the point u = 0) while the other contains all nonzero states and has period length 2rw − 1. In this case, all nonzero points can be enumerated as follows: to get the next point, discard the first ν coordinates of the current point and shift all other coordinates by ν position to the left. If P (z) is not primitive, there will be more cycles. It is easily seen that this Pn is a digital net in base 2. Indeed, because of (4), each bit vector y i is a linear function of the bit vector s0 . That is, we can

422

F. Panneton and P. L’Ecuyer

write y i = C (i) s0 for some ∞ × rw binary matrix C (i) , for i = 0, 1, 2, . . . . A quick examination of the definition of Pn immediately tells us that it satisfies the definition of a digital net (see [6, 7]) with generating matrices C (0) , C (1) , . . . . This net is infinite-dimensional. The sequence of generating matrices is periodic and the successive rows of any C (i) also form a periodic sequence. If we replace s0 by the jth canonical vector ej , the corresponding y i gives us the jth column of C (i) . Since the recurrence is purely periodic, there must be a one-to-one correspondence between s0 and the first rw bits of y i for each i. This implies that the first rw rows of C (i) must be linearly independent over F2 . Thus, the first rw bits of any given coordinate uj of the points of Pn take all possible 2rw values exactly once. That is, if the binary expansion in (5) is truncated to its first rw bits, then each one-dimensional projection of Pn is the set {0, 1/n, . . . , (n − 1)/n}. Some may argue that this type of infinite-dimensional point set is not very interesting because of the periodicity of the point coordinates. However, in practice, Pn would typically be randomized to get an unbiased estimator of µ, and the randomization would normally destroy the periodicity. For example, one simple randomization is a random binary digital shift: generate a single random point U uniformly distributed in [0, 1)∞ and add it to each point of Pn by a bitwise exclusive-or of each coordinate [3]. After this randomization, every individual point of Pn is a random point uniformly distributed over [0, 1)∞ , whereas Pn preserves all its p-equidistribution properties, as defined in the next section. The successive coordinates of the randomized points are no longer periodic.

3 Measures of Uniformity To measure the uniformity of Pn , we will examine its projections over finite subsets of the coordinates. For each such projection, we obtain a finitedimensional point set, say a point set Qn over the t-dimensional hypercube [0, 1)t . Several figures of merit can be adopted to measure the uniformity of such a point set Qn [3,6]. The measures considered in this paper are based on p-equidissections of the unit hypercube [0, 1)t and on the minimal distance between the points of Qn . We recall definitions that can be found, e.g., in [3] and at other places. Let p = (p1 , . . . , pt ) be a vector of positive integers such that p = p1 +. . .+ pt ≤ k. A p-equidissection is a partition of the unit hypercube in rectangular cells aligned with the axes, of equal volume 2−p , defined by dividing the interval [0, 1) along the i-th coordinate into 2pi equal parts, for each i. A p-equidissection such that p1 = . . . = pt =  is called an -equidissection. A set Qn with n = 2k is said to be p-equidistributed if every cell defined by the p-equidissection contains exactly 2k−p points from Qn . It is -equidistributed if it is p-equidistributed for p1 = . . . = pt = .

Highly-Uniform Point Sets Defined via Linear Recurrences in F2w

423

A point set Qn ⊂ [0, 1)t with n = 2k points is a (q, k, t)-net (in base 2) if it is p-equidistributed for every p-equidissection such that p1 + . . . + pt ≤ k − q. The smallest q such that Qn forms a (q, k, t)-net is called the q-value of Qn . We denote it by qt . Generally speaking, a smaller q-value means a more uniform point set. The largest  such that Qn is -equidistributed is called its resolution and def is denoted t (in t dimensions). We have the upper bound t ≤ ∗t = #k/t$. We define the resolution gap in t dimensions as Λt = #k/t$ − t . A smaller resolution gap means a more uniform point set. Equidistribution in p-equidissections has its limitations in measuring the uniformity of a point set. For example, if a point u is a common corner for 2t cells in t dimensions, then up to 2t distinct points of Qn can be arbitrarily close to u, one in each cell. Thus, despite good equidistribution properties, one may have a cluster of several points that are almost identical to each other. To prevent this, one may consider the minimal distance of Qn under the Lp norm, defined as d∗p (Qn ) = min{dp (x, y) : x, y ∈ Qn , x = y} , where dp (x, y) is the Lp -distance between x and y. A large value of d∗p (Qn ) means that all points are far away from each other, and are thus more evenly spread over the hypercube. Here, instead of d∗p (Qn ), we use a related figure of merit defined as follows. Two cells defined by a p-equidissection are adjacent if they have at least one corner in common. A point set Qn ⊂ [0, 1)t is said to be neighbor-free in resolution  if in the -equidissection, no cell contains more that one point from Qn and every cell that contains one point is adjacent to no other such cell. The smallest value of  such that Qn is neighbor-free is called the neighbor-free resolution and is denoted by vt . A lower bound on vt is &k/t' + 1. We define the neighbor-free gap as Γt = vt − &k/t' − 1 . The neighbor-free resolution is linked to the minimal distance by the inequalities √ 2−vt < d∗2 (Qn ) < 2−vt +2 t, 2−vt < d∗∞ (Qn ) < 2−vt +2 , proved in [10]. We want vt (or equivalently, Γt ) to be as small as possible. We now return to our infinite-dimensional point set Pn . For any subset of coordinates J = {j1 , j2 , . . . , ji }, where 0 ≤ j1 < j2 < · · · < ji < t, we define Pn (J) as the i-dimensional projection of Pn over these coordinates. Figures of merit that take into account the uniformity of projections are discussed

424

F. Panneton and P. L’Ecuyer

in [2, 3], for example. Giving special attention to the most important projections often has a significant impact on the performance of RQMC. The most important projections depend on the problem in general, but they are often of small dimension, and associated with coordinate numbers that are close to each other. For any given family J of projections, we define ∆(Pn , J , C) = max C(Pn (J)) J∈J

and Θ(Pn , J , C) =

(

C(Pn (J)) ,

J∈J

where C(Pn (J)) can be either qi , Λi , or Γi , for i = |J|. The criterion ∆(Pn , J , C) looks at the worst-case projection in J , whereas Θ(Pn , J , C) considers the average instead.

4 Guaranteed Uniformity of Certain Projections For the point sets defined in (6), each one-dimensional projection contains exactly one point in each of the intervals [0, 1/n), [1/n, 2/n), . . . , [(n−1)/n, 1). Moreover, because of the way Pn is defined via a recurrence, for any given set of non-negative integers J = {j1 , j2 , . . . , ji }, the projections Pn ({j1 +j, . . . , ji + j}) are identical for all j ≥ 0. That is, the point set is dimension-stationary [2]. The following proposition, on the equidistribution of two-dimensional projections, is proved in [10]. Proposition 1. Suppose that the minimal polynomial P (z) of the recurrence (3) over F2w is a primitive polynomial. Let h = lcm((2k − 1)/(2w − 1), ν)/ν, where lcm means the least common multiple. Then, the two-dimensional projection Pn ({j1 , j1 + j}) is w-equidistributed if and only if j is not a multiple of h. As an illustration, consider a point set Pn of cardinality n = 216 , obtained by taking r = 2, w = 8, and ν = 13. In that case, h = lcm((216 − 1)/(28 − 1), 13)/13 = lcm(257, 13)/13 = 257. This means that among all twodimensional projections of the form Pn ({0, j}), exactly 65280 out of 65535 (i.e., all but 1 out of every 257) are 8-equidistributed (which is the best possible two-dimensional equidistribution for 216 points).

5 A Search for Good Point Sets We made extensive computer searches for good point sets in terms of the general figures of merit defined in Sect. 3, for various values of n. A small

Highly-Uniform Point Sets Defined via Linear Recurrences in F2w

425

Table 1. Point sets with cardinality 214 and 216 Number 1 2 3 4 5 6 7 8 9 10 11 12

r 2 4 7 4 7 4 7 4 7 4 2 2

w 7 4 2 4 2 4 2 4 2 4 7 8

k M (z) 14 77 16 9 14 3 16 c 14 3 16 9 14 3 16 9 14 3 16 c 14 5f 16 d8

ν 152 842 548 286 468 883 236 816 199 675 101 702

b1 73 3 2 4 2 0 3 0 1 b 30 88

b2 52 e 0 9 0 4 2 3 0 f 1f da

b3 – 0 0 e 1 e 0 d 3 0 – –

b4 – e 2 4 1 b 0 3 0 9 – –

b5 – – 1 – 0 – 0 – 1 – – –

b6 – – 0 – 1 – 3 – 1 – – –

b7 ∆(S, C) Θ(S, C) – 1 12 – 1 32 1 12 – 31 3 7 934 – 9 989 1 889 – 959 1 4 303 – 4 295 – 302 – 294

S J1 J1 J1 J1 J1 J1 J1 J1 J2 J2 J2 J2

C Λt Λt Λt Λt qt qt qt qt Γt Γt Γt Γt

subset of the results, for n = 214 and 216 , is given in Table 1. The elements of the finite field F2w are represented using the hexadecimal notation and the polynomial basis (as in [11]). The sets of projections considered in the figures of merit were of the form  s  E {{j1 , . . . , ji }, 0 = j1 ≤ · · · ≤ ji < ti } J = J (s, t1 , . . . , ts ) = i=1

E

{{0, . . . , j}, 0 ≤ j < t1 } .

They are the projections defined by j successive coordinates for j up to t1 , the two-dimensional projections with coordinates less that t2 , the threedimensional projections with coordinates less that t3 , and so on. This type of J was also considered in [2]. Let us denote J (5, k, 24, 16, 8, 8) by J1 (k) and J (3, 3, 24, 16) by J2 . The parameters reported in Table 1 are for the criteria ∆(Pn , J1 (k), Λt ), Θ(Pn , J1 (k), Λt ), ∆(Pn , J1 (k), qt ), Θ(Pn , J1 (k), qt ), ∆(Pn , J2 , Γt ), and Θ(Pn , J2 , Γt ). More extensive tables of parameters are given in [10]. The effectiveness of these point sets will be assessed empirically for simple examples in the next section.

6 Examples We report the results of simple numerical experiments where the point sets of Table 1 perform quite well for integrating certain multivariate functions in a RQMC scheme. We compare their performance with that of Sobol’ nets when both are randomized by a random binary digital shift only (see, e.g., [3] and [9] for a definition and discussions of other randomization methods). In

426

F. Panneton and P. L’Ecuyer

both cases, we estimate the variance per run, i.e., n times the variance of the average over the n points, and compare it with the empirical variance of standard MC. The variance reduction factor reported is the ratio of the MC variance over the RQMC variance per run. 6.1 A Markov Chain We consider a Markov chain with state (i, c, U) where i ∈ {0, 1, 2}, c is an integer, and U = (u1 , u2 , . . .) is an infinite sequence with elements in [0, 1). The chain starts in state i = 1, c = 0 and U = (1, 1, . . .). To determine the next state, we generate U ∼ U (0, 1), a uniformly distributed random variable. If U < pi,i+1 then i ← (i + 1) mod 3, otherwise i ← (i − 1) mod 3. At each step, we increase c by one and update U as U = (U, u1 , u2 , . . .). When c ≥ 300, i = 2, and 1 − p3 < U ≤ 1, the chain terminates. In our numerical experiments, we also terminate the chain whenever i = 360, in order to be able to compare with the Sobol’ nets, for which we have an implementation only for up to 360 dimensions. At each step, there is a cost fi (U), for some functions fi that depend on only two coordinates of U . The goal is to estimate the expected total cost, µ = E[C]. Figure 1 illustrates the behavior of the chain. We can view this Markov chain as a way of randomly sampling twodimensional projections of the point set Pn , and summing up the values of ui uj observed on these projections.

Fig. 1. Evolution of i for our Markov chain.

We consider two cases for the choice of the fj ’s in our experiments. In both cases, pi,j = 1/2 for 0 ≤ i, j ≤ 2 and p3 = 1/2. In the first case, we take f0 (U) = u1 u9 , f1 (U) = u2 u8 , and f2 (U) = u3 u7 . In the second case, we take f0 (U) = u1 u2 , f1 (U) = u2 u3 , and f2 (U) = u1 u3 .

Highly-Uniform Point Sets Defined via Linear Recurrences in F2w

427

We also give the results when we do not stop the chain when i = 360 (“Case 1(b)” and “Case 2(b)”). In these cases, the dimension is not bounded and our implementation of the Sobol’ nets cannot be used. The empirical variance reductions of RQMC compared with MC are given in Table 2. These improvement factors are quite large, and much larger for our new point sets than for the Sobol’ nets. For most point sets, the variance reduction factors is slightly lower in the “(b)” cases but, for some, the trend is reversed (like point set number four). Table 2. Variance reduction factors of RQMC compared with MC for the Markov chain Number Case 1 Case 2 Case 1(b) Case 2(b) Sobol, n = 214 5 28 X X 39 37 X X Sobol, n = 216 1 1000 1400 1200 1300 2 4900 2500 4600 2100 3 1500 1200 1400 910 4 1300 1400 1800 2100 5 1300 730 1100 910 6 1900 160 1800 180 7 550 1200 470 1000 8 1400 1200 1200 900 9 10 680 8 880 10 4200 1500 3900 1400 11 22 870 20 900 12 470 270 430 250

6.2 Some Multivariate Functions Here, we consider the following two functions f , defined over the unit hypercube [0, 1)t : ) j−1 t−1 ( ( 2 g(ui )g(uj ) f (u) = f1 (u) = t(t − 1) j=0 i=0 where g(x) = 27.20917094x3 − 36.19250850x2 + 8.983337562x + 0.7702079855, and ⎛ ⎞ n−1 m−1 (  ⎝1 − f (u) = f2 (u) = 2uim+j ⎠ i=0

j=0

for m = 5, n = 20, and t = 100. Function f1 , which is from [1], is a sum of functions defined on two-dimensional projections and f2 , taken from [10], is a

428

F. Panneton and P. L’Ecuyer Table 3. Variance reduction factors for functions f1 and f2 Number Sobol, n = 214 Sobol, n = 216 1 2 3 4 5 6 7 8 9 10 11 12

f1 1.7 0.9 5 × 104 24 370 9500 19 80 10 1 × 105 630 7700 580 4 × 105

f2 820 220 2 × 104 2 × 105 4 × 107 800 2 × 108 1 × 104 1 × 109 1 × 109 1 × 109 8 × 105 2 × 105 5 × 108

sum of functions that depend on projections in five dimensions. Table 3 reports the empirical variance reduction factors observed for these two functions. For certain point sets, the reduction factors are enormous and much better than for Sobol’ nets.

7 Conclusion In this paper, we have introduced new point sets for quasi-Monte Carlo integration that are very flexible because of their infinite dimensionality. We have provided parameters for point sets that are uniform for many preselected projections and tested them with simple functions to integrate. Tables 2 and 3 show that the point sets selected are efficient in integrating the selected functions. A nice surprise revealed by these tables is the relatively good performance of the point sets (numbers 8 to 12) selected via the minimal distance criteria. It indicates that this uniformity criterion is worth considering for quasi-Monte Carlo applications.

Acknowledgments This work has been supported by NSERC-Canada and FQRNT-Qu´ebec scholarships to the first author and by NSERC-Canada grant No. ODGP0110050, FQRNT-Qu´ebec grant No. 02ER3218, and a Canada Research Chair to the second author. We thank Alexander Keller, who suggested considering the minimal distance to measure uniformity, and the Editor Harald Niederreiter.

Highly-Uniform Point Sets Defined via Linear Recurrences in F2w

429

References 1. L. Kocis and W. J. Whiten. Computational investigations of low-discrepancy sequences. ACM Transactions on Mathematical Software, 23(2):266–294, 1997. 2. P. L’Ecuyer and C. Lemieux. Variance reduction via lattice rules. Management Science, 46(9):1214–1235, 2000. 3. P. L’Ecuyer and C. Lemieux. Recent advances in randomized quasi-Monte Carlo methods. In M. Dror, P. L’Ecuyer, and F. Szidarovszky, editors, Modeling Uncertainty: An Examination of Stochastic Theory, Methods, and Applications, pages 419–474. Kluwer Academic Publishers, Boston, 2002. 4. C. Lemieux and P. L’Ecuyer. Randomized polynomial lattice rules for multivariate integration and simulation. SIAM Journal on Scientific Computing, 24(5):1768–1789, 2003. 5. R. Lidl and H. Niederreiter. Introduction to Finite Fields and Their Applications. Cambridge University Press, Cambridge, revised edition, 1994. 6. H. Niederreiter. Random Number Generation and Quasi-Monte Carlo Methods, volume 63 of SIAM CBMS-NSF Regional Conference Series in Applied Mathematics. SIAM, Philadelphia, 1992. 7. H. Niederreiter. Constructions of (t, m, s)-nets and (t, s)-sequences. Finite Fields and Their Applications, 2005. To appear. 8. A. B. Owen. Latin supercube sampling for very high-dimensional simulations. ACM Transactions on Modeling and Computer Simulation, 8(1):71–102, 1998. 9. A. B. Owen. Variance with alternative scramblings of digital nets. ACM Transactions on Modeling and Computer Simulation, 13(4):363–378, 2003. 10. F. Panneton. Construction d’ensembles de points bas´ ee sur des r´ecurrences lin´eaires dans un corps fini de caract´ eristique 2 pour la simulation Monte Carlo et l’int´egration quasi-Monte Carlo. PhD thesis, D´epartement d’informatique et de recherche op´erationnelle, Universit´e de Montr´eal, Canada, 2004. 11. F. Panneton and P. L’Ecuyer. Random number generators based on linear recurrences in F2w . In H. Niederreiter, editor, Monte Carlo and Quasi-Monte Carlo Methods 2002, pages 367–378, Berlin, 2004. Springer-Verlag. 12. I. H. Sloan and S. Joe. Lattice Methods for Multiple Integration. Clarendon Press, Oxford, 1994.

Monte Carlo Studies of Effective Diffusivities for Inertial Particles G.A. Pavliotis1 , A.M. Stuart2 , and L. Band3 1

2

3

Department of Mathematics, Imperial College London, London SW7 2AZ UK [email protected] Mathematics Institute, Warwick University, Coventry CV4 7AL UK [email protected] School of Mathematical Sciences, University of Nottingham, University Park, Nottingham NG7 2RD UK [email protected]

Summary. The transport of inertial particles in incompressible flows and subject to molecular diffusion is studied through direct numerical simulations. It was shown in recent work [9, 15] that the long time behavior of inertial particles, with motion governed by Stokes’ law in a periodic velocity field and in the presence of molecular diffusion, is diffusive. The effective diffusivity is defined through the solution of a degenerate elliptic partial differential equation. In this paper we study the dependence of the effective diffusivity on the non–dimensional parameters of the problem, as well as on the streamline topology, for a class of two dimensional periodic incompressible flows.

1 Introduction Inertial particles play an important role in various applications such as atmosphere science [6, 16] and engineering [5, 8]. The presence of inertia leads to many exciting phenomena and in particular to the fact that the distribution of particles in turbulent flows is homogeneous, i.e. particles tend to cluster [1, 2, 17, 18]. Whilst the problem of passive tracers, where inertia is neglected, has attracted the attention of many scientists over the last decades, cf. [7,11], the number of theoretical investigations concerning inertial particles is still rather small. The purpose of this paper is to study the long time behavior of particles which move in steady, periodic two–dimensional incompressible velocity fields and are subject to molecular diffusion, using Monte Carlo simulations. The particle motion is governed by Stokes’s law ˙ τx ¨ = v(x) − x˙ + σ β.

(1)

432

G.A. Pavliotis et al.

Here τ > 0 is the Stokes’ number, which can be thought of as the non– dimensional particle relaxation time. The field v(x) is the (given) fluid velocity, σ > 0 is the molecular diffusivity and β˙ stands for white noise, i.e. a mean zero Gaussian process with covariance β˙ i (t)β˙ j (s) = δij δ(t − s), where · stands for ensemble average. It was shown in recent work [9,15] that, for periodic velocity fields v(x), the long time behavior of inertial particles which move according to (1) is governed by an effective Brownian motion. To be more precise, let   t  x (t) := x 2 . (2) The process x (t) satisfies the rescaled equation ¨= τ 2x

1 x ˙ v − x˙ + σ β.

(3)

The results of [9, 15], see Theorem 1 in Sect. 2, imply that x (t) converges, as tends to 0, to a Brownian motion with covariance K, the effective diffusivity. Now, the effective diffusivity is defined in terms of the solution of a degenerate Poisson equation, see equations (8) and (9) below. It is expected that K depends on the parameters of the problem τ and σ in a complicated, highly non linear way. Moreover, the diffusivity is also expected to depend non–trivially on the topology of the streamlines of v(x), as happens for passive tracers [11]. Our goal is to gain some insight into such dependencies by means of direct numerical simulations for a class of two dimensional flows. Similar problems have been investigated within the context of massless tracer particles which move according to equation (1) with τ = 0: ˙ x˙ = v(x) + σ β.

(4)

It has been known for a long time [3] that the long time behavior of passive tracers moving in periodic flows is diffusive, with an effective diffusivity K0 which can be computed in terms of a Poisson equation with periodic boundary conditions, the cell problem. It is a well documented fact that the functional dependence of the effective diffusivity on the molecular diffusivity depends crucially on the streamline topology. For example, in the case of cellular flows–i.e. flows with closed streamlines–both diagonal components of K0 scale linearly with σ: 1. K0 ∼ σ, σ On the contrary, for shear flows the component of K0 along the direction of the shear is inversely proportional to the square of σ [19]: K0 ∼

1 , σ2

σ

1.

Monte Carlo Studies of Effective Diffusivities

433

In [12] lower and upper bounds on the dependence of K0 for σ 1 were derived and the concepts of maximally and minimally enhanced diffusion were introduced. Recall that for pure molecular diffusion (i.e. Brownian motion) the 2 diffusion coefficient is K0 = σ2 . The problem of studying the properties of the effective diffusivity becomes even more involved in the case of the inertial particles for two reasons. First, there are two non–dimensional parameters to consider, the Stokes number τ , together with the molecular diffusivity σ. Second, the Poisson equation that we need to solve in order to compute K, equation (9) below, is degenerate and is posed on 2πTd × Rd , where Td denotes the d–dimensional unit torus; this renders analytical investigations on the dependence of K on τ and σ very difficult. Furthermore, the direct numerical solution of the cell problems is a non–trivial issue and hence Monte Carlo methods are natural. In order to gain some insight into this difficult and interesting problem we resort to direct numerical simulations of (1) for a two–dimensional one– parameter velocity field, the Childress–Soward flow [4] vCS (x) = ∇⊥ ψCS (x),

ψCS (x) = sin(x1 ) sin(x2 ) + δ cos(x1 ) cos(x2 ).

(5)

Here δ ∈ [0, 1]. This family of flows is useful for numerical experiments because as, δ ranges from 0 to 1, the flow ranges from pure cellular to pure shear. In Figs. 1a, 1b and 1c we present the contour plots of the Childress–Soward stream function ψCS (x) for δ = 0.0, δ = 0.5 and δ = 1.0, respectively. 1

1

1

0.8

0.8

0.8

0.6

0.6

0.6

0.4

0.4

0.4

0.2

0.2

0.2

0

0

0

−0.2

−0.2

−0.2

−0.4

−0.4

−0.4

−0.6

−0.6

−0.6

−0.8

−0.8

−0.8

−1 −1 −0.8 −0.6 −0.4 −0.2

a.

0

0.2

δ = 0.0

0.4

0.6

0.8

1

−1 −1 −0.8 −0.6 −0.4 −0.2

b..

0

0.2

δ = 0.5

0.4

0.6

0.8

1

−1 −1 −0.8 −0.6 −0.4 −0.2

c..

0

0.2

0.4

0.6

0.8

1

δ = 1.0

Fig. 1. Contour plot of ψCS (x) for δ = 0.0, δ = 0.5 and δ = 1.0.

The numerical results reported in this paper indicate that the presence of inertia can lead to a tremendous enhancement in the effective diffusivity, beyond the enhancement in the absence of inertia, for certain values of the parameters of the problem. However, it is shown that the effective diffusivity depends very sensitively on the streamline topology. In particular, for shear flows the presence of inertia seems to have a negligible effect on the effective diffusivity. The rest of the paper is organized as follows: in Sect. 2 we review the results on periodic homogenization for inertial particles that were obtained in [9, 15].

434

G.A. Pavliotis et al.

In Sect. 3 we present the results obtained through Monte Carlo simulations concerning the dependence of K on τ, σ and δ for the Childress–Soward flow. Section 4 is reserved for conclusions.

2 Periodic Homogenization for Inertial Particles We consider the motion of inertial particles moving in incompressible velocity fields v(x) subject to molecular diffusion. Under the assumption that the density of the particles ρp is much greater than the density of the surroundρ 1, this gives rise to equation (1) [14]. Generalizations of ing fluid ρf , ρfd ρ equation (1) which are valid for the case ρfd ∼ 1 can also be treated by augmenting v(x) to include added mass effects. We refer to [14] for the model and to [15, sec. 4.7] for details of the homogenization result in this case. √ ˙ as well as the auxiliary Upon introducing the particle velocity y = τ x, variable z = x/ , we can rewrite the rescaled equation (3) as a first order system: √1 y , x˙ = τ y˙ = √τ12 v(z) − τ12 y + √στ  β˙ , (6) √ 1 2 y, z˙ = τ with the understanding that z ∈ 2πTd and x, y ∈ Rd . The ”fast” process {z, y} ∈ 2πTd × Rd is Markovian with generator   1 1 σ2 ∆y . L = √ (y · ∇z + v(z) · ∇y ) + −y · ∇y + τ 2 τ It is proved in [15], using the results of [13], that the process {z, y} is ergodic and that the unique invariant measure possesses a smooth density ρ(y, z) with respect to Lebesgue measure. This density satisfies the stationary Fokker– Planck equation L∗ ρ = 0 , where L∗ is the adjoint of the generator of the process:   1 1 σ2 ∆y . L∗ = √ (−y · ∇z − v(z) · ∇y ) + ∇y (y ) + τ 2 τ The main result of [9,15] is that, provided that the fluid velocity is smooth and centered with respect to the invariant measure ρ(y, z), the long time behavior of the inertial particles is diffusive, with an effective diffusivity which can be computed through the solution of a degenerate Poissson equation. More precisely we have the following theorem. Theorem 1. Let x (t), defined in (2), be the solution of the rescaled equation (3), in which the velocity field v(z) ∈ C ∞ (2πTd ) satisfies

Monte Carlo Studies of Effective Diffusivities



v(z) ρ(y, z) dzdy = 0. 2πTd

435

 (7)

Rd

Then the process x (t) converges weakly, as → 0, to a Brownian motion on Rd with covariance 12 K. Here   1 √ K= y ⊗ Φ(y, z)ρ(y, x) dydz (8) τ 2πTd Rd and the function Φ(y, z) is the solution of the Poisson equation 1 −LΦ(y, z) = √ y. τ

(9)

The notation ⊗ denotes the tensor product between two vectors in Rd . We will sometimes refer to K as the inertial effective diffusivity. The proof of this theorem, which is based on the martingale central limit theorem, can be found in [9], together with bounds on the rate of convergence. It is straightforward to show that the effective diffusivity K is a nonnegative matrix. We also remark that the centering condition (7) ensures the absence of a large scale mean drift. Sufficient conditions for (7) to hold are derived in [15]. In the case where (7) does not hold, a Galilean transformation with respect to the mean drift brings us back to the situation described in Theorem 1. The asymptotic behavior of K as τ tends to 0 was also investigated in [15]. It was shown that, as τ tends to 0, K converges to K0 , the effective diffusivity for the passive tracers case: K = K0 + O(τ ).

(10)

The effective diffusivity K0 is also computed through the solution of a Poisson equation: σ2 ∆, (11a) −L0 χ = v(x), L0 = v(x) · ∇ + 2  σ2 I+ K0 = v(x) ⊗ χ(x) dx. (11b) 2 2πTd Here I stands for the identity matrix on Rd . We will refer to K0 as the tracer effective diffusivity. It still an open question whether the higher order corrections in (10) are of definite sign. Notice that for K0 the cell problem (11a) is a uniformly elliptic PDE with periodic boundary conditions and is amenable to direct numerical simulation, e.g. by means of a spectral method. This is no longer true for the cell problem (9), which is a degenerate elliptic equation posed on Rd × (2πTd ), and we use Monte Carlo methods in this case.

3 Numerical Results In this section we study numerically the effective diffusivity K for equation (1) for the Childress–Soward velocity field (5). We are particularly interested

436

G.A. Pavliotis et al.

in analyzing the dependence of K on the non–dimensional parameters of the problem τ, σ and δ. The results of [15] enable us to check that the Childress–Soward flow satisfies condition (7) and hence the absence of ballistic motion at long scales is ensured. Moreover, the symmetry properties of (5) imply that the two diagonal components of the effective diffusivity are equal, whereas the off– diagonal components vanish. In the figures presented below we use the notation K := K11 = K22 . We compute the effective diffusivity using Monte Carlo simulations: we solve the equations of motion (1) numerically for different realizations of the noise and we compute the effective diffusivity through the formula K = lim

t→∞

1 (x(t) − x(t)) ⊗ (x(t) − x(t)), 2t

where · denotes ensemble average. We solve the stochastic equations of motion using Milstein’s method, appropriately modified for the second order SDE (1) [10, p. 386]: xn+2 = (2 − r)xn+1 − (1 − r)xn + r∆tv(xn+1 ) + σr∆tN (0, 1) . where r = ∆t τ . This method has strong order of convergence 1.0. We use N = 1024 uniformly distributed particles in 2πT2 with zero initial velocities and we integrate over a very long time interval (which is chosen to depend upon the parameters of the problem) with ∆t = 5.10−4 min{1, τ }. We are interested in comparing the effective diffusivities for inertial particles with those for passive tracers. The latter are computed by solving the cell problem (11a) by means of a spectral method similar to that described in [12]. We perform two sets of experiments: first, we fix σ = 0.1 and compute the effective diffusivity for τ taking values in [0.1, 10]. Then, we fix τ = 1.0 and compute K for σ taking values in [0.1, 10]. We perform these two experiments for various values of the Childress–Soward parameter δ ∈ [0, 1]. The choice δ = 0.0 corresponds to closed streamlines, whereas the choice δ = 1.0 leads to a flow with completely open streamlines, i.e. a shear flow. The results of our numerical simulations for δ = 0.0, 0.25, 0.5, 0.75, 1.0 are reported in Figs. 2 to 6. Several interesting results can be drawn from our numerical simulations. First, a resonance occurs when the Stokes’ number τ = O(1), which leads to a tremendous enhancement in the effective diffusivity. In particular, for δ = 0.25 and δ = 0.5, Figs. 3 and 4, the effective diffusivity for τ = O(1) is several orders of magnitude greater than the one for τ = 0.0. On the other hand, the effect of inertia on K becomes negligible as the streamlines become completely open, Fig. 6. In this case, δ = 1.0 the effective diffusivity for inertial particles behaves very similarly to the effective diffusivity for passive tracers.

Monte Carlo Studies of Effective Diffusivities 2

10

101

τ = 1.0 τ = 0.0 σ2/2

10

3

10

2

437

τ > 0.0 τ = 0.0 σ2/2

101 100

K

K

100 10−1 10−1 10−2

10−3 10−1

10−2

100

10−3

100

10−1

σ a. K vs σ

100

100

τ b. K vs τ

Fig. 2. Effective diffusivity versus σ (with τ = 1.0) and τ (with σ = 0.1) for δ = 0.0.

103

104

τ = 1.0 τ = 0.0 σ2/2

2

10

τ > 0.0 τ = 0.0 σ2/2

103 102

K

K

101

101

100 100 10−1

10−1

10−2

10−3 10−1

10−2

100

σ a. K vs σ

100

10−3 10−1

100

100

τ b. K vs τ

Fig. 3. Effective diffusivity versus σ (with τ = 1.0) and τ (with σ = 0.1) for δ = 0.25.

The effective diffusivity as a function of δ for inertial particles as well as passive tracers is plotted in Fig. 7, for τ = 1.0 and σ = 0.1. It becomes clear from this figure that, for τ and σ fixed, the effective diffusivity for inertial particles reaches its maximum for δ ≈ 0.30: in contrast to the passive tracers case, the dependence of the inertial effective diffusivity on δ is not monotonic.

438

G.A. Pavliotis et al. 104

103

τ = 1.0 τ = 0.0 σ2/2

102

τ > 0.0 τ = 0.0 σ2/2

103 102

K

K

101 100

101 100

10−1

10−1

10−2

10−2

10−3 −1 10

100

10−3 10−1

100

σ a. K vs σ

100

100

τ b. K vs τ

Fig. 4. Effective diffusivity versus σ (with τ = 1.0) and τ (with σ = 0.1) for δ = 0.5. 103

τ > 0.0 τ = 0.0

τ = 1.0 τ = 0.0 σ2/2

2

10

102

K

K

101 100

10−1 10−2

10−3 −1 10

100

σ a. K vs σ

100

10−1 10−1

100

100

τ b. K vs τ

Fig. 5. Effective diffusivity versus σ (with τ = 1.0) and τ (with σ = 0.1) for δ = 0.75.

4 Conclusions The dependence of the effective diffusivity for inertial particles on the particle relaxation time τ and the molecular diffusivity σ was investigated in this paper by means of Monte Carlo simulations, for a one parameter family of steady two dimensional flows. We illstrated several phenomena of interest: • the inertial effective diffusivity can be much greater than the tracer effective diffusivity, for certain values of the parameters of the problem;

Monte Carlo Studies of Effective Diffusivities 103

102

τ = 1.0 τ = 0.0 σ2/2

102

439

τ > 0.0 τ = 0.0

K

K

101 100

10−1 10−2

10−3 10−1

100

100

10−1 −1 10

100

100

τ b. K vs τ

σ a. K vs σ

Fig. 6. Effective diffusivity versus σ (with τ = 1.0) and τ (with σ = 0.1) for δ = 1.0. 103 τ = 1.0 τ = 0.0 102

K

101

100

10−1

10−2 0

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

1

δ Fig. 7. Effective diffusivity versus δ for σ = 0.1, τ = 1.0.

440

G.A. Pavliotis et al.

• inertia creates interesting effects of enhanced diffusivity, especially for small molecular diffusion, and these effects depend non-trivially on the topology of the streamlines of the velocity field; • for velocity fields with most, or all, streamlines open the effect of inertia is negligible; • the effective diffusivity is not montonic in the Stokes number, which measures the strength of the inertial contribution – maxima are observed for Stokes numbers of order 1. Many questions are open for further study, both analytical and numerical: • it would be of interest to develop asymptotic studies of the effective diffusivity, in particular to understand the effects of small molecular diffusion and small Stokes number; • it would be of interest to develop variational characterization of effective diffusivities, as has been very effective for passive tracers [11]; • it is of interest to investigate effective diffusivities for time dependent velocity fields v(x, t), with randomness introduced in space and/or time.

References 1. J. Bec. Multifractal concentrations of inertial particles in smooth random flows. Submitted to J. Fluid Mech., 2004. 2. J. Bec, A. Celani, M. Cencini, and S. Musacchio. Clustering and collisions of heavy particles in random smooth flows. Submitted to Phys. fluids, 2004. 3. A. Bensoussan, J.L. Lions, and G. Papanicolaou. Asymptotic analysis of periodic structures. North-Holland, Amsterdam, 1978. 4. S. Childress and A.M. Soward. Scalar transport and alpha-effect for a family of cat’s-eye flows. J. Fluid Mech., 205:99–133, 1989. 5. C.T. Crowe, M. Sommerfeld, and Y. Tsuji. Multiphase flows with particles and droplets. CRC Press, New York, 1998. 6. G. Falkovich, A. Fouxon, and M.G. Stepanov. Acceleration of rain initiation by cloud turbulence. Nature, 419:151–154, 2002. 7. G. Falkovich, K. Gaw¸edzki, and M. Vergassola. Particles and fields in fluid turbulence. Rev. Modern Phys., 73(4):913–975, 2001. 8. J.C.H Fung and J.C. Vassilicos. Inertial particle segregation by turbulence. Phys. Rev. E, 68:046309, 2003. 9. M. Hairer and G. A. Pavliotis. Periodic homogenization for hypoelliptic diffusions. J. Stat. Phys., 117(1/2):261–279, 2004. 10. P.E. Kloeden and E. Platen. Numerical solution of stochastic differential equations, volume 23 of Applications of Mathematics (New York). Springer-Verlag, Berlin, 1992. 11. A.J. Majda and P.R. Kramer. Simplified models for turbulent diffusion: Theory, numerical modelling and physical phenomena. Physics Reports, 314:237–574, 1999. 12. A.J. Majda and R.M. McLaughlin. The effect of mean flows on enhanced diffusivity in transport by incompressible periodic velocity fields. Studies in Applied Mathematics, 89:245–279, 1993.

Monte Carlo Studies of Effective Diffusivities

441

13. J.C. Mattingly and A. M. Stuart. Geometric ergodicity of some hypo-elliptic diffusions for particle motions. Markov Processes and Related Fields, 8(2):199– 214, 2002. 14. M.R. Maxey and J.J. Riley. Equation of motion for a small rigid sphere in a nonuniform flow. Phys. Fluids, 26:883–889, 1983. 15. G.A. Pavliotis and A.M. Stuart. Periodic homogenization for inertial particles. Physics D, 204(3–4): 161–187, 2005. 16. R. A. Shaw. Particle-turbulence interactions in atmosphere clouds. Annu. Rev. Fluid Mech., 35:183–227, 2003. 17. H. Sigurgeirsson and A. M. Stuart. Inertial particles in a random field. Stoch. Dyn., 2(2):295–310, 2002. 18. H. Sigurgeirsson and A. M. Stuart. A model for preferential concentration. Phys. Fluids, 14(12):4352–4361, 2002. 19. G. I. Taylor. Dispersion of soluble matter in solvent flowing slowly through a tube. Proc. R. Soc. Lond. A, 219:186–203, 1953.

An Adaptive Importance Sampling Technique Teemu Pennanen and Matti Koivu Department of Management Science, Helsinki School of Economics, PL 1210, 00101 Helsinki, Finland [pennanen,koivu]@hkkk.fi

Summary. This paper proposes a new adaptive importance sampling (AIS) technique for approximate evaluation of multidimensional integrals. Whereas known AIS algorithms try to find a sampling density that is approximately proportional to the integrand, our algorithm aims directly at the minimization of the variance of the sample average estimate. Our algorithm uses piecewise constant sampling densities, which makes it also reminiscent of stratified sampling. The algorithm was implemented in C-programming language and compared with VEGAS and MISER.

1 Introduction This paper presents an adaptive importance sampling algorithm for numerical approximation of integrals (expectations wrt the uniform distribution U on (0, 1)d ) of the form  EU ϕ =

ϕ(ω)dω .

(1)

(0,1)d

Restricting attention to the uniform distribution on the unit cube is not as restrictive as one might think. Indeed, integrals of the more general form  ϕ(ξ)π(ξ)dξ , (a,b)

where π is a positive density function and (a, b) = (a1 , b1 ) × · · · × (ad , bd ), with possibly ai = −∞ or bi = +∞, can be written in the form (1) with a change of variables; see Hlawka and M¨ uck [6]. Instead of direct sampling from the uniform distribution U , we will follow the importance sampling strategy. If p is a density function on (0, 1)d , then EU ϕ = EU 

ϕ(ω) ϕ(ω) p(ω) = E P , p(ω) p(ω)

The work of this author was supported by Finnish Academy under contract no. 3385

444

T. Pennanen and M. Koivu

where P (A) := E U p(ω)χA (ω) for every measurable set A. If {ωi }N i=1 is a random sample from P , the expectation and variance of the sample average AN (ϕ, p) = are E U ϕ and

1 NV

N 1 ( ϕ(ωi ) N i=1 p(ωi )

(ϕ, p), respectively, where 

V (ϕ, p) = E

P

ϕ(ω) p(ω)

2



ϕ(ω) − E p(ω)

2

P

= EU

ϕ(ω)2 − (E U ϕ)2 . p(ω)

If p ≡ 1, V (ϕ, p) is just the variance V (ϕ) of ϕ wrt the uniform distribution. The general idea in importance sampling is to choose p so that 1. V (ϕ, p) < V (ϕ), 2. P is easy to sample from, 3. p(ω) is easy to evaluate. The extreme cases are p ≡ 1, which satisfies 2 and 3 but not 1, and p = |ϕ|/E U |ϕ|, which satisfies 1 but not 2 and 3. Coming up with a good importance sampling technique is to find a good balance between the two. Adaptive importance sampling (AIS) algorithms update the sampling density as more information about the integrand is accumulated. In many AIS algorithms proposed in the literature, the sampling density is a “mixture density”, i.e. K ( αk q k , (2) p= k=1 k

where q are density functions and αk are positive constants that add up to one. For appropriate choices of q k , such a p is easy to sample from and easy to evaluate so that criteria 2 and 3 are met. Indeed, to sample from (2), one chooses an index k with probability αk , and then samples from q k . In the VEGAS algorithm of Lepage [7] (see also [9]), the component densities q k are uniform densities over rectangles. In VEGAS, p is also required to be of the product form p(ω) = p1 (ω1 ) · · · pd (ωd ), which facilitates sampling, but restricts the ability to adapt to integrands that are not of the product form. A more flexible approach is to use so called kernel densities, where q k (the kernels) are identical up to a shift; see Zhang [12] and the references therein. In [8], Owen and Zhou proposed an AIS strategy where q k are products of univariate β-distributions. In the AIS algorithm presented in this paper, the sampling densities are piecewise constant functions of the form p=

K ( k=1

p k χΩ k ,

(3)

An Adaptive Importance Sampling Technique

445

where the sets Ω k form a partitioning of (0, 1)d into rectangular regions. If ϕ is Riemann integrable, V (ϕ, p) can be made arbitrarily close to zero by choosing the partitioning and the weights pk appropriately; see Sect. 2. The piecewise constant density obviously meets criterion 3, and since it can be written in the form (2), it satisfies criterion 2 as well. A crucial question in an AIS algorithm is how to update p. In many existing algorithms, V (ϕ, p) is reduced indirectly by trying to update p so that it follows ϕ as closely as possible according to a given criterion. The algorithm of [8] minimizes a sample approximation of the mean squared deviation of p from ϕ. Kernel density based methods rely on asymptotic properties of kernel density approximations. The algorithm proposed in this paper, aims directly at minimizing V (ϕ, p). Our algorithm was implemented in C-programming language and compared with VEGAS and MISER implementations from [9]. Comparisons were made on five different integrands over the unit cube with dimensions ranging from 5 to 9. In the tests, our algorithm outperformed MISER in accuracy on every test problem. Compared to VEGAS our algorithm was more accurate on only two problems, but exhibited more robust overall performance. The rest of this paper is organized as follows. The general idea of the algorithm is presented in Sect. 2. Section 3 outlines the implementation of the algorithm and Sect. 5 presents numerical experiments.

2 Adapting p to ϕ When using piecewise constant sampling densities of the form (3), the variance of the sample average can be written as ϕ(ω)2 − (E U ϕ)2 p(ω) K
K ( ( ϕ(ω)2 dω U (Ω k )mk2 U 2 Ωk = − (E ϕ) = − (E U ϕ)2 . k p pk

V (ϕ, p) = E U

k=1

k=1

Here, and in what follows, mkj

1 := U (Ω k )

 ϕ(ω)j dω

j = 1, 2, . . .

Ωk

denotes the jth moment of ϕ over Ω k . For a fixed number K of elements in the partition, the optimal, in the sense of criterion 1, piecewise constant sampling density is the one whose weights pk and parts Ω k solve the following minimization problem

446

T. Pennanen and M. Koivu

minimize Ω k ,pk

subject to

K ( U (Ω k )mk 2

pk

k=1 {Ω k }K k=1 K (

k

− (E U ϕ)2

is a partition of (0, 1)d ,

(4)

k

p U (Ω ) = 1 ,

k=1 k

p ≥ 0,

k = 1, . . . , K.

This is a hard problem in general since optimizing over partitions (even just over rectangular ones) leads to nonconvexities and multiple local minima. The AIS procedure developed in this paper is based on the observation that problem (4) has the same characteristics as the problem of adaptive finite element (FEM) approximation of partial differential equations; see e.g. [1, Section 6]. There also, the aim is to find a finite partition (usually into tetrahedrons) of the domain along with optimal weights for associated kernel functions. In adaptive FEM and in our importance sampling procedure, the optimization is done by iterating three steps: 1. adjust the partition based on the information collected over the previous iterations; 2. compute (numerically) certain integrals over the elements of the new partition; 3. optimize over the weights. In our integration procedure, step 2 consists of numerically computing the first two moments of ϕ over each Ω k . In both adaptive FEM and our integration procedure, step 3 is a simple convex optimization problem. In FEM, it is a quadratic minimization problem which can be solved by standard routines of numerical linear algebra. In our procedure, the optimal weights can be solved analytically; see Subsect. 2.1. In both methods, step 1 is the most delicate one. Our heuristic for updating the partitions is described in Subsect. 2.2. 2.1 Optimizing the Weights If the partition P = {Ω k }K k=1 is fixed, problem (4) becomes a convex optimization problem over the weights, and it can be solved analytically by the technique of Lagrange multipliers. A necessary and sufficient condition for optimality is ∂p L(p, λ) = 0 , ∂λ L(p, λ) = 0 , where the Lagrangian L is given by

An Adaptive Importance Sampling Technique

L(p, λ) =

K ( U (Ω k )mk 2

pk

k=1

=

K (

 U (Ω k )

k=1

 − (E U ϕ)2 + λ

mk2 + λpk pk

K (

447

 pk U (Ω k ) − 1

k=1



− (E U ϕ)2 − λ;

see Rockafellar [11]. The optimality conditions become − K (

mk2 + λ = 0 k = 1, . . . , K , (pk )2

pk U (Ω k ) − 1 = 0 ,

k=1

which can be solved for the optimal weights 1

(mk2 ) 2

pko = 1K

1

k=1

U (Ω k )(mk2 ) 2

.

The optimal sampling density is thus po =

K (

pko χΩ k

(5)

k=1

and the corresponding variance (the minimum over p in (4)) is "

K (

&2 U (Ω

k

1 )(mk2 ) 2

− (E U ϕ)2 .

(6)

k=1

2.2 Optimizing the Partition Our algorithm works recursively, much like MISER, by maintaining a partitioning of (0, 1)d into rectangular subregions [ak , bk ] and splitting its elements in half along one of the co-ordinate axes. However, since the computing time of our sampling technique increases with the number of parts in the partition, we will make splits more sparingly. 1 Each region Ω k contributes to the variance through the term U (Ω k )(mk2 ) 2 in the brackets in (6). Partitioning Ω k into {Ω l }, this term gets replaced by ( 1 U (Ω l )(ml2 ) 2 . l

Making {Ω l } fine enough, we can reduce this term arbitrarily close to
the Riemann-integral Ω k ϕ(ω)dω. Thus, the optimal reduction of the term 1 U (Ω k )(mk2 ) 2 by partitioning the region Ω k is

448

T. Pennanen and M. Koivu 1

U (Ω k )[(mk2 ) 2 − mk1 ] .

(7)

Our recursive partitioning strategy proceeds by splitting in half regions Ω k for which (an approximation of) (7) is greatest. After we have chosen a region for splitting, we need to choose along which of the d axis to split it. The simplest idea would be to choose the axis j for which the number 1 1 (8) U (Ω l )(ml2 ) 2 + U (Ω r )(mr2 ) 2 , where Ω l and Ω r are the regions obtained by splitting Ω k in half along the jth axis, is smallest. The motivation would be that (8) is the number by which 1 the contribution U (Ω k )(mk2 ) 2 of Ω k would be replaced in (6). However, if ϕ 1 is symmetric about the dividing line along axis j, (8) is equal to U (Ω k )(mk2 ) 2 and there is no reduction of variance, even though further splitting along the jth axis might result in a significant reduction of variance. We can effectively avoid this kind of short-sightedness by choosing instead the axis for which the number L ( 1 U (Ω l )(ml2 ) 2 , (9) l=1 l

where Ω are the regions obtained by partitioning Ω k along axis j into L > 2 equal-size subregions, is smallest. In the test problems of Sect. 5, our algorithm was rather insensitive to the value of L as long as it was greater than 2.

3 The AIS Algorithm The above sketch of AIS requires knowledge of mk1 and mk2 in each region Ω k generated in the course of the algorithm. These will be approximated by sample averages. The algorithm works iteratively, much like VEGAS, by sampling N points from the current sampling density pit and using the new sample in updating the partitioning and the density. The idea is that, as the algorithm proceeds, the density should concentrate in the important areas of the domain, so that when sampling from the latest density, a greater fraction of sample points will be available for approximating the moments in these areas. In early iterations, when the moment approximations are not very accurate, the algorithm refines the partitioning less eagerly, but as the sampling density is believed to converge toward the optimal one, more parts are generated. In each iteration, the algorithm uses the latest sample to compute the integral estimate AN (ϕ, pit ) and an estimate of its variance " & N 1 ( ϕ(ωi )2 1 it it 2 VN (ϕ, p ) = − AN (ϕ, p ) . N − 1 N i=1 pit (ωi )2 It also computes a weighted average of the estimates obtained over the previous iterations using the weighting strategy of Owen and Zhou [8]. The latest

An Adaptive Importance Sampling Technique

449

sample is then used to update the moment approximations in each region. The algorithm splits regions for which an estimate of the expression (7) is greater than certain constant εit . A new sampling density based on the new partitioning is formed and εit is updated. We set it+1 equal to a certain percentage of the quantity 1 ˜ k2 ) 2 − m ˜ k1 ] , min U (Ω k )[(m k=1,...,K

m ˜ k1

m ˜ k2

where and are sample average approximations of mk1 and mk2 , respectively. The sampling density for iteration it will be pit pit = (1 − α)˜ o +α ,

(10)

where p˜it o is an approximation of p0 for the current partition and α ∈ (0, 1). k The approximation p˜it o will be obtained by approximating each po in (5) by 1

1K k=1

(m ˜ k2 ) 2 1

U (Ω k )(m ˜ k2 ) 2

.

The reason for using α > 0 is that, due to approximation errors, the sampling density p˜it o might actually increase the variance of the integral estimate. Bounding the sampling density away from zero, protects against such effects; see [5]. Our AIS-algorithm proceeds by iterating the following steps. 1. sample N points from the current density pit ; 2. compute AN (ϕ, pit ) and VN (ϕ, pit ); ˜ k2 of mk1 and mk2 and refine the parti3. compute approximations m ˜ k1 and m tioning; 4. set K ( p˜ko χΩ k + α , pit+1 = (1 − α) k=1

where

1

p˜ko = 1K k=1

(m ˜ k2 ) 2 1

U (Ω k )(m ˜ k2 ) 2

.

The algorithm maintains a binary tree whose nodes correspond to parts in a partitioning that has been generated in the course of the algorithm by splitting larger parts in half. In the first iteration, the tree consists only of the original domain (0, 1)d . In each iteration, “leaf nodes” of the tree correspond to the regions in the current partitioning. When a new sample is generated, the sample points are propagated from the “root” of the tree toward the leafs where the quotients ϕ(ωi )/p(ωi ) are computed. The sum of the quotients in each region is then returned back toward the root. In computing estimates of mk1 and mk2 in each iteration, our algorithm uses all the sample points generated so far. The old sample points are stored in

450

T. Pennanen and M. Koivu

the leaf nodes of the tree. When a new sample is generated and the quotients ϕ(ωi )/p(ωi ) have been computed in a region, the new sample points are stored in the corresponding node. New estimates of mk1 and mk2 are then computed and the partitioning is refined by splitting regions where an estimate of the expression (7) is greater than εit . When a region is split, the old sample points are propagated into the new regions.

4 Relations to Existing Algorithms A piecewise constant sampling density makes importance sampling reminiscent of the stratified sampling technique. Let I k be the set of indices of the points falling in Ω k . When sampling N points from p, the cardinality of I k has expectation E|I k | = U (Ω k )pk N and the sample average can be written as AN (ϕ, p) =

K K 1 ( ( ϕ(ωi ) ( 1 ( = U (Ω k ) ϕ(ωi ) . k N p E|I k | k k k=1 i∈I

k=1

i∈I

If, instead of being random, |I k | was equal to E|I k |, this would be the familiar stratified sampling estimate of the integral. See Press and Farrar [10], Fink [3], Dahl [2] for adaptive integration algorithms based on stratified sampling. In many importance sampling approaches proposed in the literature, the sampling density is constructed so that it follows the integrand as closely as possible, in one sense or the other. For example, minimizing the mean squared deviation (as in [8] in case of beta-densities and in [3] in case of piecewise constant densities) would lead to mk1

p k = 1K k=1

U (Ω k )mk1

instead of pko above. Writing the second moment as mk2 = (σ k )2 + (mk1 )2 , where σ k is the standard deviation, shows that, as compared to this sampling density, the variance-optimal density po puts more weight in regions where the variance is large. This is close to the idea of stratified sampling, where the optimal allocation of sampling points puts U (Ω k )σ k N 1K k k k=1 U (Ω )σ points in Ω k . On the other hand, sampling from po puts, on the average, 1

U (Ω

k

)pko N

U (Ω k )(mk2 ) 2

= 1K

k=1

1

U (Ω k )(mk2 ) 2

N.

An Adaptive Importance Sampling Technique

451

points in Ω k . Looking again at the expression mk2 = (σ k )2 +(mk1 )2 , shows that, as compared to stratified sampling, our sampling density puts more weight in regions where the integrand has large values. Our sampling density can thus be viewed as a natural combination of traditional importance sampling and stratified sampling strategies.

5 Numerical Experiments In the numerical test, we compare AIS with two widely used adaptive integration algorithms VEGAS and MISER; see e.g. [9]. We consider five different test integrands. The first three functions are from the test function library proposed by [4], the double Gaussian function is taken from [7], and as the last test integrand we use an indicator function of the unit simplex; see Table 1. In the first three functions, there is a parameter a that determines the variability of the integrand. For test functions 1–4, we chose d = 9 and we estimate the integrals using 2 million sample points. For test function 5, we used 1 million points with d = 5. Table 1. Test functions Integrand ϕ(x) √ cos(ax), a = 110/ d3 2d 1 a = 600/d2 i=1 a−2 +(xi −0.5)2 , ,  a = 600/d2 (1 + ax)−(d+1)  d    −d  2  d  1 1 xi −2/3 2 xi −1/3 1 10 √ exp − + exp − 2 0.1 0.1 π i=1 i=1

1d u ≤ 1, d! i=1 i 0 otherwise

Attribute name Oscillatory Product Peak Corner Peak Double Gaussian Indicator

In all test cases, AIS used 50 iterations with 40000 and 20000 points per iteration for test functions 1–4 and 5, respectively. The value of the parameter α in (10) was set to 0.01. For all the algorithms and test functions we estimated the integrals 20 times. The results in Table 2 give the mean absolute error (MAD) and the average of the error estimates provided by the algorithms (STD) over the 20 trials. AIS produces the smallest mean absolute errors with Double Gaussian and Product Peak test functions and never loses to MISER in accuracy. VEGAS is the most accurate algorithm in Oscillatory, Corner Peak and Indicator functions, but fails in the Double Gaussian test. This is a difficult function for VEGAS, because the density it produces is a product of 9 univariate bimodal densities having 29 = 512 modes; see Fig. 1 for a two dimensional illustration

452

T. Pennanen and M. Koivu Table 2. Results of the numerical tests

Function Oscillatory Product Peak Corner Peak Double Gaussian Indicator

d 9 9 9 9 5

AIS MAD STD 0.009171 0.012717 0.006411 0.016944 0.000735 0.000981 0.011022 0.019228 0.005601 0.003391

VEGAS MAD STD 0.002244 0.002877 0.014699 0.002791 0.000115 0.000161 0.500516 0.006204 0.001066 0.001039

MISER MAD STD 0.014355 0.020256 0.022058 0.031235 0.001786 0.002209 0.021274 0.018452 0.017896 0.021690

MC MAD STD 0.021432 0.023249 0.017091 0.018515 0.006692 0.009311 0.219927 0.220510 0.011208 0.010907

Fig. 1. Sample produced by VEGAS for double Gaussian.

with 22 = 4 modes. The error estimates (STD) given by AIS and MISER are consistent with the actual accuracy obtained but VEGAS underestimates the error in the case of Product Peak and Double Gaussian. The partition rule used by MISER does not work well with the Indicator function. In choosing the dimension to bisect, MISER approximates the variances with the square of the difference of maximum and minimum sampled values in each subregion. This causes MISER to create unnecessary partitions, which do not reduce the variance of the integral estimate. Figure 2 displays the partition and sample of a 2 dimensional Indicator function created by MISER. The sample generated by MISER resembles a plain MC sample. For comparison and illustration of the behavior of our algorithm, the partitions and the corresponding samples generated after 20 integrations of AIS are displayed in Figs. 3 and 4. Average computation times used by the algorithms are given in Table 3 for all test functions. Maintenance of the data structures used by AIS causes the algorithm to be somewhat slower compared to VEGAS and MISER but the difference reduces when the integrand evaluations are more time consuming. In summary, our algorithm provides an accurate and robust alternative to VEGAS and MISER with somewhat increased execution time. For integrands that are slower to evaluate the execution times become comparable.

An Adaptive Importance Sampling Technique 1

1

0.8

0.8

0.6

0.6

0.4

0.4

0.2

0.2

0

0

0.2

0.4

0.6

0.8

1

0

0

0.2

(a) Partition

0.4

0.6

0.8

453

1

(b) Sample

Fig. 2. Partition and sample produced by MISER for the 2 dimensional Indicator function. 1

1

1

0.8

0.8

0.8

0.6

0.6

0.6

0.4

0.4

0.4

0.2

0.2

0.2

0 0

0.2 0.4 0.6 0.8

1

(a) Corner peak

0 0

0.2 0.4 0.6 0.8

(b) Double Gaussian

1

1

0.8

0.8

0.6

0.6

0.4

0.4

0.2

0.2

0 0

0.2 0.4 0.6 0.8

(d) Oscillatory

1

0 0

1

0 0

0.2 0.4 0.6 0.8

1

(c) Indicator

0.2 0.4 0.6 0.8

1

(e) Product Peak

Fig. 3. Two dimensional partitions produced by AIS.

Acknowledgments We would like to thank Vesa Poikonen for his help in implementing the algorithm. We would also like to thank an anonymous referee for several suggestions that helped to clarify the paper.

454

T. Pennanen and M. Koivu 1

1

1

0.8

0.8

0.8

0.6

0.6

0.6

0.4

0.4

0.4

0.2

0.2

0.2

0

0

0.2 0.4 0.6 0.8

(a) Corner peak

1

0 0

0.2 0.4 0.6 0.8

0 0

0.2 0.4 0.6 0.8

(b) Double Gaussian

1

1

0.8

0.8

0.6

0.6

0.4

0.4

0.2

0.2

0 0

1

0.2 0.4 0.6 0.8

(d) Oscillatory

1

0 0

1

(c) Indicator

0.2 0.4 0.6 0.8

1

(e) Product Peak

Fig. 4. Two dimensional samples produced by AIS. Table 3. Computation times in seconds Function Oscillatory Product Peak Corner Peak Double Gaussian Indicator

AIS

VEGAS

MISER

MC

24.30 56.34 38.21 30.13 6.06

4.19 9.57 3.78 11.29 0.97

2.45 8.54 2.07 10.81 0.45

1.78 7.81 1.35 10.46 0.30

References 1. I. Babuˇska and W. C. Rheinboldt. Error estimates for adaptive finite element computations. SIAM J. Numer. Anal., 15(4):736–754, 1978. 2. Lars O. Dahl. An adaptive method for evaluating multidimensional contingent claims: Part I. International Journal of Theoretical and Applied Finance, 6:301– 316, 2003. 3. Daniel Fink. Automatic importance sampling for low dimensional integration. Working paper, Cornell University, 2001. 4. A. C. Genz. Testing multidimensional integration routines. Tools, Methods and Languages for Scientific and Engineering Computation, pp. 81–94, 1984.

An Adaptive Importance Sampling Technique

455

5. Tim C. Hesterberg. Weighted average importance sampling and defensive mixture distributions. Technometrics, 37(2):185–194, 1995. ¨ 6. E. Hlawka and R. M¨ uck. Uber eine Transformation von gleichverteilten Folgen. II. Computing (Arch. Elektron. Rechnen), 9:127–138, 1972. 7. G. P. Lepage. A new algorithm for adaptive multidimensional integration. Journal of Computational Physics, 27:192–203, 1978. 8. A. B. Owen and Y. Zhou. Adaptive importance sampling by mixtures of products of beta distributions, 1999. URL http://wwwstat.stanford.edu/∼owen/reports/. 9. W. H. Press, S. A. Teukolsky, W. T. Vetterling, and B. P. Flannery. Numerical recipes in C, The art of scientific computing. Cambridge University Press, Cambridge, 2nd edition, 1992. 10. W. H. Press and G. R. Farrar. Recursive stratifield sampling for multidimensional monte carlo integration. Computers in Physics, 27:190–195, 1990. 11. R. T. Rockafellar. Convex analysis. Princeton Mathematical Series, No. 28. Princeton University Press, Princeton, N.J., 1970. 12. Ping Zhang. Nonparametric importance sampling. J. Amer. Statist. Assoc., 91(435):1245–1253, 1996.

MinT: A Database for Optimal Net Parameters Rudolf Sch¨ urer1 and Wolfgang Ch. Schmid2 Department of Mathematics, University of Salzburg, Hellbrunnerstr. 34, A-5020 Salzburg, Austria 1 [email protected] 2 [email protected] Summary. An overwhelming variety of different constructions for (t, m, s)-nets and (t, s)-sequences are known today. Propagation rules as well as connections to other mathematical objects make it a difficult task to determine the best net available in a given setting. We present the web-based database system MinT for querying best known (t, m, s)-net and (t, s)-sequence parameters. This new system provides a number of hitherto unavailable services to the research community.

1 Introduction (t, m, s)-nets and (t, s)-sequences [8,9] are among the best known methods for the construction of low-discrepancy point sets in the s-dimensional unit cube. A problem for the practitioner is that an overwhelming variety of different methods exist. A recent survey of important approaches can be found in [10]. Choosing an optimal net is further complicated by the fact that the existence of nets and sequences is often linked to other mathematical objects, e.g. algebraic function fields (see [12]), linear codes (see [1, 5]), or even other nets. Connections of the last type are usually referred to as “propagation rules”, and a large number of such rules are available. Hence it has become an almost impossible task to determine the best net available in a given setting. This problem led to the publication of tables of net parameters, with [3] and its predecessor [6] being the best-known examples. However, parts of these tables had been outdated before the articles appeared in print. As a more flexible solution the web-based database system MinT for querying best known (t, m, s)-net and (t, s)-sequence parameters has been developed at the Department of Mathematics at the University of Salzburg. It is available on the Internet at the address http://mint.sbg.ac.at/

458

R. Sch¨ urer and W.Ch. Schmid

MinT is an acronym for “Minimal t”, referring to the common task of finding a (t, m, s)-net with minimal t for given m and s. This new system provides a number of hitherto unavailable services to the scientific community. The following list gives a short overview of its advantages compared with any printed version of such tables: • MinT allows distinction between digital and general constructions • MinT allows distinction between constructive and non-constructive methods • MinT gives bounds on existence • MinT allows different views on the data by appropriately choosing dependent and independent parameters • MinT shows complete construction trees • MinT allows comparing different types • MinT has a flexible viewport • MinT gives extensive literature references • MinT allows fast and dynamic updates • MinT is available to everybody In this article we discuss the unique features and design issues of MinT.

2 Basic Definitions and Results Throughout this article the following notation is used: the integer s ≥ 1 denotes the dimension of the s-dimensional half-open unit cube [0, 1)s , the prime power b is the base of a construction, the integer m ≥ 0 parameterizes the size bm of an object, and the integer t with 0 ≤ t ≤ m is called quality parameter. Finally, the integer k is defined as k := m−t and is called strength. 2.1 Nets and Sequences Following [8] and [9], we have the following definitions: Definition 1. A multi-set of bm points x0 , . . . , xbm −1 ∈ [0, 1)s is a (t, m, s)net in base b if any elementary interval "  s  ai ai + 1 , bdi bdi i=1 with ai , di ∈ Z, di ≥ 0, 0 ≤ ai < bdi for i = 1, . . . , s, and volume 1/bk ) contains exactly bt points of the multi-set.

1s

i=1

di = k (i.e.,

The primary goal of MinT is to determine for which quadruple (b, t, m, s) a (t, m, s)-net in base b can exist and for which it cannot. Closely related to nets are digital nets1 : 1

This definition is different but equivalent to the one in [8] and [9].

MinT: A Database for Optimal Net Parameters

459

Definition 2. A set of s m × m matrices C (1), . . . , C (s) over F b with ⎛ (i) ⎞ c1 ⎜ (i) ⎟ ⎜ c2 ⎟ ⎟ for i = 1, . . . , s C (i) = ⎜ ⎜ .. ⎟ ⎝ . ⎠ (i)

cm

generates a digital (t, m, s)-net over F b if for all 0 ≤ di ≤ m, i = 1, . . . , s and 1s (1) (1) (s) (s) m i=1 di = m − t the m − t vectors c1 , . . . , cd1 , . . . , c1 , . . . , cds ∈ F b are linearly independent. The matrices C (1), . . . , C (s) are called generator matrices of the digital net. In [8, Theorem 6.10] it is shown that every digital (t, m, s)-net over F b is a (t, m, s)-net in base b. Therefore digital nets are a special subclass of nets. An important tool for the construction of (digital) nets are (digital) sequences, which can be thought of as an infinite nesting of (digital) nets with increasing size m. The definitions given here are equivalent to the one in [15] for sequences and to the one in [11] for digital sequences, which are more general than the definitions in [8] and [9]. Definition 3. A sequence of points x0 , x1 , . . . ∈ [0, 1]s with a fixed b-adic expansion is a (t, s)-sequence in base b if, for all integers j ≥ 0 and m ≥ t, the point set {[xn ]b,m : jbm ≤ n < (j + 1)bm } is a (t, m, s)-net in base b, where [x]b,m denotes the coordinate-wise m-digit truncation of x in base b. Definition 4. A set of s ∞ × ∞ matrices C (1) , . . . , C (s) over F b generates a digital (t, s)-sequence over F b if their s upper left m × m submatrices generate a digital (t, m, s)-net over F b for all m ≥ t. In [8, Theorem 5.15] it is shown that every (t, s)-sequence in base b yields (t, m, s + 1)-nets in base b for all m ≥ t. From [11, Lemma 1 and 2] it follows that every digital (t, s)-sequence over F b yields digital (t, m, s + 1)-nets over F b for all m ≥ t, and that every digital (t, s)-sequence is also a (t, s)-sequence. 2.2 Linear Codes and Orthogonal Arrays Nets and digital nets are closely related to orthogonal arrays and linear codes. These connections are summarized by the following definitions and results. m Definition 5. A set S ⊆ F m b of cardinality s is an (s, k)-set in F b if any selection of k vectors from S is linearly independent.

Definition 6. A subspace C ⊆ F sb with dimension s − m is a linear [s, s − m, k + 1]-code if the Hamming distance between any two different vectors in C is at least k + 1.

460

R. Sch¨ urer and W.Ch. Schmid

An (s, k)-set in F m b exists if and only if a linear [s, s − m, k + 1]-code over F b exists, because the (s, k)-set forms the parity check matrix of the code [2]. For k ≤ s and by choosing di ≤ 1 it follows directly from the definition of (1) (s) digital (m − k, m, s)-nets over F b that the first row vectors c1 , . . . , c1 of the s generator matrices form an (s, k)-set in F m b . Therefore, the generator matrices of every digital (m − k, m, s)-net over F b (with s ≥ m) yield a linear [s, s − m, k + 1]-code over F b . Linear codes are again a special case of an even more general structure, namely orthogonal arrays: Definition 7. An array with s columns and elements from a set with cardinality b is an orthogonal array OAλ (k, s, b) if in the projection onto any set of k columns each k-tuple of entries occurs exactly λ times. Let H be the parity check matrix of a linear [s, s − m, k + 1]-code over F b , i.e., an s × m matrix with its rows forming an (s, k)-set in F m b . Then it is easy to see that the bm × s matrix produced by listing all vectors of the subspace spanned by the columns of H is an OAbt (k, s, b). Therefore, (s, k)sets and linear codes can be identified with a special class of (namely linear) orthogonal arrays. In addition to that, for every (t, m, s)-net in base b with points xn = (1) (s) (xn , . . . , xn ) for n = 0, . . . , bm − 1, the bm × s matrix A = (ani ) defined by @ ? for n = 0, . . . , bm − 1 and i = 1, . . . , s ani = bxn(i) is an orthogonal array OAbt (k, s, b), i.e., the OA is constructed using the leading digit of the b-adic expansions of the coordinates of the points [13]. 2.3 Ordered Orthogonal Arrays The connection between nets, digital nets, linear codes, and orthogonal arrays is even more explicit in the terminology of ordered orthogonal arrays (OOA), introduced in [4] and [7]. In this setting, nets and digital nets are OOAs with depth T = k whereas linear codes and OAs have depth T = 1. Digital nets and linear codes are linear OOAs whereas nets and OAs do not necessarily have vector space structure. Figure 1 sums up the dependencies between these four classes of objects. The existence of a digital net implies the existence of a net as well as the existence of a linear code. Each of them implies the existence of an orthogonal array. To establish bounds on the existence of these objects, implications run in the opposite direction using proofs by contradiction.

MinT: A Database for Optimal Net Parameters

461

Digital (t, m, s)-Net Linear OOA with T = m – t

(t, m, s)-Net OOA with T = m – t

Linear [s, s – m, d]-Code Linear Orthogonal Array Linear OOA with T = 1

Orthogonal Array OOA with T = 1 Fig. 1. The four classes of objects tracked by MinT.

3 Constructions, Existence, and Bounds For all four classes of mathematical objects introduced in the previous section MinT tracks three different sets of parameters, leading to a total of 12 sets of parameters: • Constructions are explicit and effective methods for creating the object, i.e., the generator matrices for digital nets and linear codes, the point sets for nets, and the runs of an orthogonal array. A method is considered constructive if a computer implementation or at least an algorithm that allows such an implementation is available. If generator matrices are available explicitly (tabulated in print or electronically), the method is also considered constructive. • Existence Results provide proof of the existence of an object with certain parameters without giving an explicit method for its construction. A brute force search through the finite space of possible matrices for given parameters is not considered an effective method. Of course, a construction implies an existence result. • Bounds are proofs ruling out the existence of objects with certain parameters.

4 Sets of Possible Parameters and Their Defining Functions ∗ We are interested in Htype for type ∈ {net, dignet, code, OA}, the set of all (b, t, m, s) such that a type-object with these parameters exists. Even though ∗ is mathematically well defined, there is no efficient routine known for Htype ∗ ∗ . Therefore, Htype determining whether a given quadruple (b, t, m, s) is in Htype has to be bounded by more concrete sets.

462

R. Sch¨ urer and W.Ch. Schmid

mode Let Htype for mode ∈ {constructive, existent, potential} and type as above be the set of all quadruples (b, t, m, s) ∈ N × N 0 × (N 0 ∪ {seq}) × N with b a prime power and t ≤ m, such that an object of type type can “exist”, where the mode of existence is further specified by mode in the following way: potential is defined as the set of all (b, t, m, s) such that no bound rules Htype existent out the existence of a type-object with these parameters; Htype is the set of constructive all parameters for which a type-object is known to exist; finally, Htype is the set of all (b, t, m, s) such that a type-object with these parameters can be constructed explicitly. (Digital) (t, s)-sequences in base b can be represented in this framework by the tuple (b, t, seq, s). This notation allows a consistent handling of (digital) sequences as special cases of (digital) nets without introducing additional types. Obviously, we have certain relations between these 16 sets. To be specific, mode mode mode mode mode mode ⊆ Hnet ⊆ HOA and Hdignet ⊆ Hcode ⊆ HOA for all we have Hdignet potential constructive existent ∗ ⊆ Htype ⊆ Htype ⊆ Htype for all types. modes, and Htype mode is the main goal of MinT. Providing information about the shape of Htype

4.1 Projections of Hmode type mode Each H = Htype has a very regular structure. For instance, if (b, t, m, s) is in  H, so is (b, t , m, s) for t ≤ t ≤ m. For all four types this result follows directly from the definition of the particular object. Therefore, the exact shape of H is determined completely by a function yielding the minimum t for given b, m, and s. Maps of this type are usually referred to as t-tables. Furthermore, if (b, t, m, s) is in H, so is (b, t, m, s ) for 1 ≤ s ≤ s. This result follows from reducing the dimension s for nets and digital nets [8, Lemma 2.7], from shortening a linear code, and from dropping arbitrary columns from an OA. Therefore, the shape of H is also determined by a function yielding the maximum s for given b, m, and t, which is usually referred to as an s-table. Note that s may be unbounded for certain b, m, and t. In this case an s-table returns ∞. Some other tables are possible: for instance, a minimum-m-table yielding the smallest possible m for given b, k = m − t, and s is well defined because (b, t, m, s) ∈ H implies (b, t+u, m+u, s) ∈ H for all u ≥ 0, i.e., (b, m−k, m, s) ∈ H implies (b, m − k, m , s) ∈ H for all m ≥ m. This follows from replicating a net bu times, from adding u arbitrary rows and columns to a digital net, from taking a subcode of a linear code, and from duplicating each row of an orthogonal array bu times. Tables for arbitrary projections are not possible in general. For instance, a maximum-m-table yielding the largest possible m for given b, t, and s makes perfect sense for nets and digital nets because (b, t, m, s) ∈ H implies (b, t, m , s) ∈ H for all t ≤ m ≤ m due to the propagation rule for m-reduction for nets [8, Lemma 2.8] and digital nets [14, Lemma 3]. On the other hand, for

MinT: A Database for Optimal Net Parameters

463

Table 1. MinT creates tables for the following projections Display . . . Depending on . . . minimal t maximal k maximal s maximal s maximal s maximal m maximal k minimal m minimal t

s, m s, m m, t m, k k, t t, s t, s k, s k, s

linear codes and orthogonal arrays no such propagation rule exists. It should also be noted that a maximum-m-table may return two values in addition to integers: ∞ if m is unbounded and “seq” if a (t, s)-sequence also exists. 4.2 An Efficient Representation of Hmode type mode As outlined in the previous section the shape of Htype is completely determined by certain functions, for instance a t-table, an s-table, or a minimumm-table. These functions, however, have quite different properties and are not equally well suited for tabulation or for a computer implementation. A t-table is a comparatively smooth function, because its entries start with t = 0 for small s and m and are bounded by

t ≤ min{m, tb (s − 1)} , where tb (s − 1) is a function growing linearly in s (tb (s − 1) is the t-parameter of an optimal (t, s − 1)-sequence, see [11]). This is contrary to an s-table, which approaches t−1 b (t) + 1 for m → ∞, but is unbounded for m = t. It is easy to see that the information contained in a finite number of t-table entries can always be stored in a finite number of s-table entries, while the opposite may not be possible. For this reason, MinT uses only s-tables internally and generates all other requested tables based thereupon. 4.3 Parameter Range Considered by MinT mode Htype is an infinite set, and also its s-table representation has an unbounded domain and image. Thus truncation must occur when the table is represented in the finite memory of a computer system. As far as the s-values are concerned, finite numbers exceeding about 224 are truncated and stored as a special entry signalizing overflow. This bound seems reasonable because a successful evaluation of integrals with larger dimensions is very unlikely.

464

R. Sch¨ urer and W.Ch. Schmid

The domain of the s-tables suffers more severe restrictions, because the number of tuples (b, t, m) taken into account directly affects the required amount of memory. At the moment, MinT considers all prime powers b = pr with p ≤ 7 and b ≤ 32, i.e., b = 2, 3, 4, 5, 7, 8, 9, 16, 25, 27, and 32. The size m is restricted to 0 ≤ m ≤ 80, which (even for base b = 2) exceeds the size of nets that can be enumerated by any computer imaginable today. Finally, t is between 0 and m for nets, so the full range is taken into account. (t, s)-sequences are only considered for t ≤ 150. 4.4 Tables Generated by MinT Even though only an s-table (storing the largest possible s for given b, t, and m) is used internally, the MinT front-end can produce tables for all nine reasonable projections based on the variables t, k, m, and s (see Table 1). Providing different projections is an important feature because it allows the user to see the data in the way that is most natural to her or him. While earlier tables of net parameters appeared as t-tables, [3, 6] already contain stables in addition to t-tables. Depending on the methods used for the creation of new nets, researchers may prefer different formats (for instance, [1] contains tables listing s depending on m and k). Tables are produced for a fixed base b. They can contain data about any mode with type ∈ {net, dignet} and mode ∈ {constructive, existent, potential}. Htype It is also possible to produce a table containing data from two sets H and H , making it easy to investigate the differences between both sets. Thus, e.g., potential entries that can possibly be improved can be found by comparing Htype existent with Htype . existent Figure 2 contains a screen shot of an s-table for Hnet and b = 2. Note that there are no entries for t > m. For t = m and t = m − 1, s is unbounded (entry “∞”), while values exceeding 224 are denoted by “##”. Other large values are written in exponential notation. Entries for sequences as well as the limits for m → ∞ can be found at the right side of the table. Using the arrow buttons or the input fields in the first row below the table, the viewport can be moved to any desired position or its size can be changed. Additional controls allow transposing the table and changing the base b as mode that is currently displayed. well as the set Htype potential Figure 3 contains a screen shot of a t-table for Hdignet in base b = 2. In this case entries for sequences are displayed at the bottom of the table. By default each cell has a colored background depending on its value, making it easy to identify parameters yielding identical t-values. Unfortunately, coloring cannot be reproduced here.

MinT: A Database for Optimal Net Parameters

465

MinT

Maximal s-table for base 2 — Arbitrary 1 5 1 6 1 7 1 8 1 9 2 0 2 1 2 2 2 3 2 4 2 5 2 6 2 7 2 8 2 9 3 0 3 1
Seq 16 14 13 12 11 11 10 10 10 10 10 10 10 10 10 10 10 10 9

m = t = 7 t = 8

23

19

17

15

14

13

12

11

11

11

11

11

11

11

11

11 11 11

10

t = 9

36

28

22

20

17

16

15

12

12

12

12

12

12

12

12

12 12 12

11

t = 10

77

42

34

26

23

20

18

15

13

13

13

13

13

13

13

13 13 13

12

t = 11

151 129

48

41

30

26

23

18

15

14

14

14

14

14

14

14 14 14

13

t = 12

1e4 257 140

64

47

t = 13

3e 4 3 e4 259 257

35

29

23

14

14

14 14 14

13

24

16

16

15

15

15

15 15 15

14

29

26

26

26

18

16

16

16 15 15

14

t = 15





1e 5 1 e5 513 513 127

64

39

29

27

27

27

19

17

17 17 17

16

2e 5 2 e5 1e3 513 127 127

39

34

28

28

28

20

18 18 17

16

5e 5 5 e5 1e 3 1 e3 127 127

44

37

31

29

29

22 20 17

16

1e 6 1 e6 2e 3 1 e3 130 127

49

40

33

30

30 23 18

17

2e 6 2 e6 2e 3 2 e3 255 133

54

46

35

33 30 20

19

4e 6 4 e6 4e 3 2 e3 255 137

64

60

40 36 20

19

4e 3 4 e3 255 195





















t = 20





t = 21

##





t = 22

, m = 15

Arbitrary

, with width

17

Optional second type colorize

## ##





t = 23

.

14

24

39

t = 19

Base b = 2

14

29

64

t = 18

t= 7

14

39

79

t = 17

Type

15

58


6e4 6e4 510 257

t = 16

Move table to

23

72

t = 14

, height

69

68 43 21

20

##

8e 3 4 e3 511 511

78 71 21

20

##

##

8e 3 8 e3 511 511 89 22

21

17

.

—

Submit

The entry “
” marks unbounded s, while “##” stands for a bounded, but very large s (exceeding 8388602) where MinT does not know the exact value. Values above 1000 are given in exponential notation: an entry of the form “xey” denotes a value of s with x 10y s < (x + 1) 10y. Created by MinT, Dept. of Mathematics, University of Salzburg Please send comments to mint|at|sbg|dot|ac|dot|at

Last change 24.01.05

Fig. 2. Screen shot of MinT: an s-table showing largest possible s for given m and existent . t in base 2 for Hnet

5 Construction Trees MinT provides detailed information about each table entry by simply clicking on it. Figure 4 gives an example showing the result of a query for optimal (t, 52, 120)-nets in base b = 3. The upper part of the screen lists a chain of explicit constructions leading to a digital (37, 52, 120)-net over F 3 : starting with a digital net over F 27 derived from a Niederreiter/Xing sequence based on a certain global function field, the final digital net over F 3 is obtained using a basis-reduction method and two trivial propagation rules. The next part of the result shows that a digital (35, 52, 120)-net over F 3 does also exist. However, this net is not constructive and its existence is only guaranteed by the Gilbert–Varshamov bound for ordered orthogonal arrays. As far as optimality is concerned MinT shows that even a (21, 52, 120)net cannot be ruled out by any bounds2 . A (20, 52, 120)-net, however, cannot exist, because s-reduction would yield a (20, 52, 107)-net, which cannot exist due to the generalized Rao bound for nets. Figure 5 contains another example, demonstrating the close relationship between linear codes and digital nets, especially if s is large compared to m. 2

For bases b > 2 differences between bounds and existence results are in general much larger than for b = 2.

466

R. Sch¨ urer and W.Ch. Schmid

MinT

Minimal t-table for base 2 — Lower bound on t (digital)

s= m = 9 m = 10

4

4

4

5

5

5

5

5

5

5

5

5

5

6

6

6

6

6

6

6

6

6

6

9

4

4

4

5

5

5

5

5

5

5

6

6

6

6

6

6

6

6

6

6

6

6

6

6 10

m = 12

4

4

5

5

5

5

5

5

6

6

6

6

6

6

6

7

7

7

7

7

7

7

7

7 11

m = 13

1 2 3 3 4

4

4

5

5

5

6

6

6

6

6

6

6

6

6

7

7

7

7

8

8

8

8

8

8 12

m = 14

1 2 3 3 4

4

5

5

5

5

6

6

6

7

7

7

7

7

7

7

7

7

7

8

8

8

8

8

8 13

m = 15

1 2 3 3 4

4

5

5

6

6

6

6

6

7

7

7

8

8

8

8

8

8

8

8

8

8

8

8

8 14

m = 16

1 2 3 3 4

5

5

5

6

6

6

6

7

7

7

7

8

8

8

9

9

9

9

9

9

9

9

9

9 15

m = 17

1 2 3 3 4

5

5

6

6

6

6

7

7

7

8

8

8

8

8

9

9

9

9 10 10 10 10 10 10 16

5

6

6

6

7

7

7

7

8

8

8

8

9

9

9

9

9 10 10 10 10 10 10 17

1 2 3 3 4

5

m = 19

1 2 3 3 4

5

5

6

6

7

7

7

7

8

8

8

8

9

9

9

9

9 10 10 10 10 10 10 10 18

m = 20

1 2 3 3 4

5

5

6

6

7

7

7

8

8

8

8

9

9

9

9

9

9 10 10 10 10 10 11 11 19

9 10 10 10 10 10 10 10 10 11 11 20

9

9

m = 22

1 2 3 3 4

5

5

6

7

7

8

8

8

8

9

9

9

9 10 10 10 10 10 11 11 11 11 11 11 21

m = 23

1 2 3 3 4

1 2 3 3 4

5

6

6

7

7

8

8

8

9

9

9

9 10 10 10 10 10 10 11 11 12 12 12 12 22

5

8

9 10 10 10 10 11 11 11 11 11 12 12 12 13 23

6

7

7

7

7

8

8

8

9

m = 24

1 2 3 3 4

6

6

7

7

8

8

9

9

1 2 3 3 4

5

6

6

7

7

8

8

8

9

9 10 10 10 10 11 11 11 11 11 11 12 12 12 13 24

m =


1 2 3 3 4

5

6

6

7

7

8

8

9

9 10 11 11 12 13 13 14 14 14 14 14 14 15 16 16

Seq

2 3 3 4 5

6

6

7

7

8

8

9

9 10 11 11 12 13 13 14 14 14 14 14 14 15 16 16 17

m = 9

, s= 5

. Show

quality t t quality

5

5

m = 25

Lower bound Lower boundon ont (digital t (digital) )

Base b = 2

4

1 2 3 3 4 1 2 3 3 4

m = 21

Type

1 2 3 3 3

m = 11

m = 18

Move table to

5 6 7 8 9 1 0 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33
1 2 3 3 3 3 4 4 4 4 4 4 4 5 5 5 5 5 5 6 6 6 6 6 6 6 6 6 6 8

29

, height

Optional second type

, with width

Digita l Digital

.

Created by MinT, Dept. of Mathematics, University of Salzburg Please send comments to mint|at|sbg|dot|ac|dot|at

colorize

17

.

Submit

Last change 24.01.05

Fig. 3. Screen shot of MinT: a t-table showing smallest possible t for given s and potential . m in base 2 for Hdignet

The constructive as well as the non-constructive existence result for (12, 12 + 7, s)-nets in base b = 3 is based on a linear code: in the constructive case on a [20, 15, 4]-code, which is obtained from the [10, 6, 4]-ovoid code; in the nonconstructive case on a [90, 71, 8]-code, which is an extended or lengthened BCH code. Figure 5 serves also as an example for a construction method with two parents. The (12, 19, 54)-net is constructed using the (u, u + v)-construction for nets, which produces a new net based on two other nets. In these examples, all existence results yield digital nets, and the bounds are applicable to general nets. If better constructions for non-digital nets or sharper bounds for digital nets were available, these results would also be displayed on this page. While printed versions of net parameters can usually only show a single key referring to the construction method, MinT lists the full construction chain – or, in the case of propagation rules depending on several nets, the complete construction tree. For each node in the tree the parameters of the object as well as the construction method are given. Each node is a link to a page providing detailed information about the particular construction, including formulas for the resulting parameters and literature references. A form at the bottom of the page facilitates a quick change of the parameters.

MinT: A Database for Optimal Net Parameters

467

MinT

Best known ( t , 52, 120)-nets in base 3 ( 37 , 52, 120)-net over F 3 — Constructive and digital Digital (37, 52, 120)-net over F3 , using s-reduction based on digital (37, 52, 144)-net over F3 , using net duplication based on digital (36, 51, 144)-net over F3 , using base reduction of the second kind based on digital (2, 17, 48)-net over F27 , using net from sequence based on digital (2, 47)-sequence over F27 , using Niederreiter/Xing sequence construction II/III based on function field F/F 27 with g( F ) = 2 and N( F )  48, using fields by Garica/Quoos

( 35 , 52, 120)-net over F 3 — Digital Digital (35, 52, 120)-net over F3 , using s-reduction based on digital (35, 52, 123)-net over F3 , using Gilbert–Varshamov bound for OOAs

( 21 , 52, 120)-net in base 3 — Lower bound on t There is no (20, 52, 120)-net in base 3, because s-reduction would yield (20, 52, 107)-net in base 3, but generalized Rao bound for nets Go to b = 3

, m = 52

, s = 120

. Show

Created by MinT, Dept. of Mathematics, University of Salzburg Please send comments to mint|at|sbg|dot|ac|dot|at

quality t

.

Submit

Last change 24.01.05

Fig. 4. Screen shot of MinT: Details about the existence of (digital) (t, 52, 120)-nets in base b = 3.

6 Outlook MinT has reached a stage of maturity that makes it useful for a broad audience. The user interface is full-featured and easy to use. All important construction methods are included, which guarantees up-to-date data in the tables. However, MinT is still an active project. Our main goal is to fill the remaining gaps in the data, even though most of these gaps are for large values of s or for uncommon bases b. In this quest we hope for help from the scientific community: information about missing constructions, bounds, or other results is highly appreciated and can be sent to the e-mail address [email protected]. In addition to getting and keeping the data up-to-date, some improvements to the user interface are planned. In the long term MinT will be able to provide

468

R. Sch¨ urer and W.Ch. Schmid

MinT

Best known (12, 12+7, s)-nets in base 3 (12, 12+7, 54)-net over F 3 — Constructive and digital Digital (12, 19, 54)-net over F3 , using ( u, u+ v)-construction for OOAs based on 1. digital (2, 5, 20)-net over F3 , using net-embeddable linear codes with strength k = 3 based on linear [20, 15, 4]-code over F3 , using ( u, u+ v )-construction with parity-check code based on linear [10, 6, 4]-code over F3 , using ovoid 2. digital (7, 14, 34)-net over F3 , using linear code embeddings found in a computer search

(12, 12+7, 64 )-net over F 3 — Digital Digital (12, 19, 64)-net over F3 , using net from net-embeddable linear code based on linear [90, 71, 8]-code over F3 , using extended or lengthened BCH codes

(12, 12+7, 659)-net in base 3 — Upper bound on s There is no (12, 19, 660)-net in base 3, because m -reduction would yield (12, 18, 660)-net in base 3, but generalized Rao bound for nets Go to b = 3

, k= 7

, t = 12

.

Created by MinT, Dept. of Mathematics , University of Salzburg Please send comments to mint|at|sbg|dot|ac|dot|at

Submit Last change 24.01.05

Fig. 5. Screen shot of MinT: Details about the existence of (digital) (12, 12 + 7, s)nets in base b = 3.

generator matrices for selected classes of digital nets. This feature will be implemented based on HIntLib, the high-dimensional integration library3 .

Acknowledgments The development of MinT and this work were supported by the Austrian Science Foundation (FWF), project no. S 8311-MAT.

References 1. J. Bierbrauer, Y. Edel, and W. Ch. Schmid. Coding-theoretic constructions for (t, m, s)-nets and ordered orthogonal arrays. J. Combin. Designs, 10:403–418, 2002. 3

HIntLib is available at http://www.cosy.sbg.ac.at/∼rschuer/hintlib/

MinT: A Database for Optimal Net Parameters

469

2. R. C. Bose. On some connections between the design of experiments and information theory. Bull. Internat. Statist. Inst., 48:257–271, 1961. 3. A. T. Clayman, K. M. Lawrence, G. L. Mullen, H. Niederreiter, and N. J. A. Sloane. Updated tables of parameters of (t, m, s)-nets. J. Combin. Designs, 7:381–393, 1999. 4. K. M. Lawrence. A combinatorial characterization of (t, m, s)-nets in base b. J. Combin. Designs, 4:275–293, 1996. 5. K. M. Lawrence, A. Mahalanabis, G. L. Mullen, and W. Ch. Schmid. Construction of digital (t, m, s)-nets from linear codes. In S. D. Cohen and H. Niederreiter, editors, Finite Fields and Applications, volume 233 of Lect. Note Series of the London Math. Soc., pages 189–208, Cambridge, UK, 1996. Cambridge University Press. 6. G. L. Mullen, A. Mahalanabis, and H. Niederreiter. Tables of (t, m, s)-net and (t, s)-sequence parameters. In H. Niederreiter and P. J.-S. Shiue, editors, Monte Carlo and Quasi-Monte Carlo Methods in Scientific Computing, volume 106 of Lecture Notes Statistics, pages 58–86. Springer-Verlag, 1995. 7. G. L. Mullen and W. Ch. Schmid. An equivalence between (t, m, s)-nets and strongly orthogonal hypercubes. J. Combin. Theory A, 76:164–174, 1996. 8. H. Niederreiter. Point sets and sequences with small discrepancy. Monatsh. Math., 104:273–337, 1987. 9. H. Niederreiter. Random Number Generation and Quasi-Monte Carlo Methods, volume 63 of CBMS-NSF Regional Conference Series in Applied Mathematics. SIAM Society for Industrial and Applied Mathematics, Philadelphia, PA, USA, 1992. 10. H. Niederreiter. Constructions of (t, m, s)-nets. In H. Niederreiter and J. Spanier, editors, Monte Carlo and Quasi-Monte Carlo Methods 1998, pages 70–85. Springer-Verlag, 2000. 11. H. Niederreiter and C. P. Xing. Low-discrepancy sequences and global function fields with many rational places. Finite Fields Appl., 2:241–273, 1996. 12. H. Niederreiter and C. P. Xing. The algebraic-geometry approach to lowdiscrepancy sequences. In H. Niederreiter et al., editors, Monte Carlo and Quasi-Monte Carlo Methods 1996, volume 127 of Lecture Notes Statistics, pages 139–160. Springer-Verlag, 1998. 13. A. B. Owen. Orthogonal arrays for computer experiments, integration and visualization. Statist. Sinica, 2:438–452, 1992. 14. W. Ch. Schmid and R. Wolf. Bounds for digital nets and sequences. Acta Arith., 78:377–399, 1997. 15. S. Tezuka and T. Tokuyama. A note on polynomial arithmetic analogue of Halton sequences. ACM Trans. Modeling and Computer Simulation, 4:279–284, 1994.

On Ergodic Measures for McKean–Vlasov Stochastic Equations A. Yu. Veretennikov1 School of Mathematics, University of Leeds (UK) & Institute of Information Transmission Problems (Russia) veretenn @ maths.leeds.ac.uk Summary. Conditions for existence and uniqueness of invariant measures and weak convergence to these measures for stochastic McKean-Vlasov equations have been established, along with similar approximation results and a new version of existence and uniqueness of strong solutions to these equations.

1 Introduction Let us consider the McKean-Vlasov equation in Rd , dXt = b[Xt , µt ] dt + dWt , X0 = x0 ∈ Rd , (1)
where b[x, µ] := b(x, y) µ(dy) for any measure µ (this is a notation convention), with locally bounded Borel functions b(·, ·) : Rd × Rd → Rd , and d-dimensional Wiener process Wt . Here µt is the marginal distribution of Xt . Strictly speaking, one should call a solution of the equation (1) the couple (Xt , µt ). However, with a slight abuse of notation we will call solution just the process Xt , having in mind that actually it is a couple. The equation was suggested by M. Kac [Kac56] as a stochastic toy model for the Vlasov kinetic equation of plasma (cf. [LLLP81]). It is a particular case of a mean field type equation which is rather popular in theoretical physics and applied mathematics. The study of the equation (1) was initiated by McKean [McK66]. A general introduction to the topic can be found in [Szn91]. By ergodic measures we mean here the existence of a stationary distribution, its uniqueness, and a convergence to this distribution in some topology. The (1) relates to the following nonlinear equation for measures, ∂t µt = L∗ (µt )µt ,

with

L(µ) = ∆/2 + b[x, µ]∂x ,

(2)

in the sense that the distribution of Xt solves the (2) provided X0 is distributed with respect to µ0 and the process W is independent from X0 . Initial data in

472

A. Yu. Veretennikov

this paper will be always fixed, although generalizations to any initial measure with appropriate finite moments are straightforward. An important method of solving the (1) approximately is a use of the N -particle equation with weak interaction, dXti,N =

N 1 ( b(Xti,N , Xtj,N ) dt + dWti , X0i,N = x0 , N j=1

1≤i≤N ,

(3)

with d-dimensional independent Wiener processes Wti . It is known that under reasonable assumptions the process X i,N converges weakly to the solution of the McKean-Vlasov equation with the same W i , see [Szn91, BRTV98, Mal03], et al., ¯ ti , µit ] dt + dWti , X ¯ 0 = x0 ∈ Rd , ¯ ti = b[X (4) dX ¯ i (given initial data). This result is called where µit stands for the law of X t propagation of chaos for the McKean-Vlasov equation. Here the law µit actually does not depend on i, if the solution of the (1) is unique in law (e.g., see conditions for that in [Fun84, BRTV98], and also in the Theorem 2 below). The measure µit satisfies in the weak sense a non-linear PDE (2). It would also make sense to consider different initial data for different particles, but we will not use this here. Large deviation results can be found in [DG87], approximation results in [BT97] et al.; the approach from [KV04] can be applied here, too. The paper [Mal03] is based on log-Sobolev inequalities and Dirichlet forms technique. In [Kot95, Sko88] one can find other propagation of chaos results. The topic of the paper will be ergodic properties of the (4) and some relations to the first equation. The most close works are [BRTV98] and [Mal03] which contain condition for ergodicity and (exponential) convergence rate to equilibrium. In compare to [BRTV98], we allow any finite dimension, and do not impose assumptions of dissipation greater than linear; our approach is different. In compare to [Mal03], we do not require gradient form of the interaction drift term, although our conditions are still restrictive. Other close papers are [Tam84], – also with gradient type drift, – [BCCP98, Fun84]. In [Shi94] a different class of interacting diffusions is considered, results are also existence and uniqueness of invariant measures, however, even the setting can hardly be compared with ours. For more complete bibliography see references in [BRTV98, Nag93, Szn91] et al. We also investigate the Euler approximations for the (3), dXti,N,h =

N 1 ( b(Xκi,N,h , Xκj,N,h ) dt + dWti , X0i,N = x0 , 1 ≤ i ≤ N , (5) h (t) h (t) N j=1

with κh (t) := [t/h]h, and N any natural, where [a] denotes the integer part of the value a, that is, the maximal integer not exceeding a. For approximations we only consider the “case (2◦ )” for the sake of simplicity; its assumptions

On Ergodic Measures

473

seem to be more reasonable. The only change in assumptions is a more restrictive condition (13) on the drift b instead of the linear growth condition in the Theorem 2. Section 2 contains main results: a new version of an existence theorem, a uniqueness and ergodicity theorem under two sets of assumptions, and an ergodicity result for approximations. Section 3 is a collection of proofs.

2 Main Results Main assumptions for existence: we assume that the function b(x, ·) has a linear growth in the first variable, |b(x, y)| ≤ C(1 + |x|),

(6)

and is continuous with respect to the second variable y for any x. Fully continuous bounded version for more general equations, – i.e. with a non-constant diffusion matrix coefficient and more general dependence of both coefficients on distribution, – can be found in [Fun84] established by using the martingale problem method and tightness. Assumptions in [Fun84] allow non-bounded coefficients as well, but in addition require an appropriate Lyapunov function, which may be dropped in the bounded case. Another existence and uniqueness theorem for d = 1 under assumptions not reduced to that from [Fun84] (nor from the present paper), can be found in [BRTV98]. Theorem 1. Under assumption (6), there is a strong solution to the (1). Various uniqueness assumptions can be found in papers [BRTV98, Fun84] et al. The latter (i.e. [Fun84]) requires some kind of Lipschitz condition to this end. In the next theorem, we assume either Lipschitz condition (which, hence, implies uniqueness), or another set of conditions close to it although not exactly the same. Uniqueness in distribution not only for the limiting stationary measure but also for the distribution µ1t for any finite t follows from the auxiliary estimate (16). Notice that in [Fun84] uniqueness has been established for a more general class of equations under Lipschitz type conditions. Main assumptions for ergodicity and uniqueness are as follows: • Coefficient b is decomposed into two parts, b(x, y) = b0 (x) + b1 (x, y) , where the first part is responsible for the “environment”, while the second for the interaction itself. • b: recurrence (7) lim supb(x, y), x ≤ −r < 0 , |x|→∞

y

[Essential is that the value r is fixed, and should not change with N → ∞ when we use auxiliary estimates in the proof of the theorem 2, item E.]

474

A. Yu. Veretennikov

• b0 : attraction to zero which grows at least linearly with distance (= onesided Lipschitz condition), for any x, x , b0 (x) − b0 (x ), x − x  ≤ −c0 |x − x |2

(c0 > 0) .

(8)

The next two assumptions (9-10) are required if c0 is any positive (case (1◦ ) of the theorem 2 below). Instead, one can assume (11) and (12). • b1 : anti-symmetry of interactions (a hint to the 3rd Newton’s Law), b1 (x, x ) − b1 (x , x) = 0 .

(9)

• b1 : “attraction” between particles, which increases with distance, in a certain non-rigorous sense x−x ¯ ), b1 (x, x ) − b1 (¯ x, x ¯ ) ≤ 0 , (x − x ) − (¯

(10)

e.g., one might imagine a system of particles connected pairwise by elastic strings; the analogy, of course, is not exact, but just gives an example how interaction may not decrease with distance; needless to say that this is not a plasma, but on the other hand, strings might be interesting, too. The next assumptions (11-12) which replace (9-10) are more relaxing. • b0 : large attraction of any particle towards zero, b1 c0 > CLip ,

(11)

b1 < ∞ is the best constant satisfying for all x, x , y, where CLip b1 |x − x | . max (|b1 (x, y) − b1 (x , y)|, |b1 (y, x) − b1 (y, x )|) ≤ CLip

(12)

Theorem 2. Let either of the two sets of assumptions hold true: (1◦ ) (6-8) with r ≥ r(d) large enough and (9-10) with any c0 > 0; or (2◦ ) (6-8) with r ≥ r(d) large enough and (11-12). Then, in both cases, the distribution µt is unique, and there exists a weak limit µt =⇒ µ∞ ,

t→∞,

and, moreover, µ1,N =⇒ µ∞ , t

N, t → ∞ .

The measure µ∞ is a unique invariant one for the (1), in particular, it does not depend on X0 . For the approximation result, we now assume that the function b(x, ·) satisfies all assumptions of existence as above, and the function b is bounded, sup |b(x, y)| ≤ C < ∞.

(13)

x,y

the law of Xti,N,h . In the next Theorem we only consider the Denote µi,N,h t ◦ case (2 ) from Theorem 2, and a notation µ∞ is used. Under assumptions of this Theorem and according to the Theorem 2, there exists a unique stationary measure for the (1); µ∞ denotes this measure.

On Ergodic Measures

475

Theorem 3. Let conditions (6-8) and (11-12) be satisfied, with r ≥ r(d) large enough in the assumption (7). Then there is a weak limit µ1,N,h =⇒ µ1,N,h , ∞ h moreover, =⇒ µ∞ , µ1,N,h ∞

→∞,

N, h−1 → ∞ .

3 Proofs 3.1 Auxiliary Results Lemma 1. Let (6) and (7) hold true. Then for any m > 0 there exists rm > 0 such that for any r ≥ rm , sup sup E|Xti,N |m < ∞ . N

t

This Lemma will be only used in subsection 3.3. Even more, it could be avoided in that subsection, too. However, for a possible future progress we would like to keep all technical estimates to be as good as possible. Proof. The proof follows directly from the comparison theorem for the |X 1,N | and a corresponding one-dimensional markovian diffusion with reflection, and bounds for the invariant measure for this reflected markovian diffusion, cf. [Ver97] concerning a comparison of similar type. We now formulate the result from [VK03] concerning convergence for Euler schemes for SDEs without interactions. To this end, we temporarily, – for this subsection only, – assume that in the (1) the function b does not depend on the variable y. This assumption is used in Proposition 1 below without a reminder. Notice that it is not convenient here to use results from [KV04] for more general approximations, simply because in the present state they do require a certain smoothness which unfortunately depends on dimension; hence, when N → ∞, we actually need b ∈ C ∞ in order to use these results. On the contrary, Gaussian case does not require any smoothness. We formulate a rather special case of the Theorem 1, item 3, from [VK03], and only the part which concerns convergence to equilibrium, not mixing (which is the main point in the cited work). Proposition 1 ([VK03]). Under assumption (7) with r > (m − 1)d/2 and m > 4, for any h small enough there exists the invariant measure µh∞ , and h ) the following bound for β-mixing and for marginal distributions µhh = L(Xh holds true. Then µhh − µh∞ T V ≤ C(1 + |x|m )(1 + h)−k−1 , with some C > 0, and any k ∈ (0, (m − 2)/2).

476

A. Yu. Veretennikov

Here ·T V is a total variation norm. Underline that here all constants C, m, k may be chosen uniformly in h ≤ 1. Below we will apply this result to the ˆ N,h = (X 1,N,h , . . . , X N,N,h ). We will also need the following slightly process X kh kh kh different version of this result. Proposition 2. Let the process Xt be decomposed into K components of dimensions d1 , . . . , dK , Xt = (Xt1 , . . . , XtK ), and for each component X j the following recurrence condition holds true: bj (x), xj  ≤ −rj ,

|xj | ≥ M,

rj > (m − 1)dj /2 .

Then for any h small enough there exists the invariant measure µh∞ , and the h following bound for β-mixing and for marginal distributions µhh = L(Xh ) holds true. If m > 4 and r > (m − 1)d/2, then µhh − µh∞ T V ≤ C(1 + |x|m )(1 + h)−k−1 , with some C > 0, and any k ∈ (0, (m − 2)/2). Proposition 1 is proved in [VK03]. Proposition 2 follows from the same calculus in addition to the reasoning from Sect. 3.3.E below. 3.2 Proof of Theorem 1 This is based on Krylov’s successive approximations due to tightness and Krylov’s bounds for stochastic integrals, cf. [Kry80] for the ordinary Itˆ o SDE. The advantage is that Krylov’s technique does not require continuity of coefficients. The approximations read, (n)

dXtn = bn [Xtn , µnt ] dt + dWt

,

X0 = x0 ,

where bn (·, y) is a smooth approximation of the function b(·, y) in the function space Lp,loc (Rd ) which is also bounded uniformly with respect to y for any fixed n, and, moreover, satisfies a uniform linear growth condition (6) for all n’s. Due to [Fun84], we have a solution X n with a corresponding measure µn and a corresponding d-dimensional Wiener process denoted by W (n) (just notice that it is not at all W n ). Next, due to tightness of the couple (X.n , W.(n) ) in C([0, t]; Rd ) for any t, – which follows from standard stochastic integral inequalities, – one can find  a sub-sequence n → ∞ such that µn has a weak limit in C([0, t]; Rd ) for any t. Due to Skorokhod’s embedding theorem (see [Sko65] or [Kry80]), one can change probability space and find another sub-sequence n → ∞ such   that, moreover, (X.n , W.(n ) ) → (X.0 , W.(0) ) in C([0, t]; Rd ) almost surely.   (n ) Here all W.(n ) are d-dimensional Wiener processes, and all Xtn are FtW  measurable (since all X.n are strong solutions). In the limit, from

On Ergodic Measures 

 t 

Xtn − x0 =

 (n ) n n n b (Xs , y)µs (dy) dt + Wt

477

(14)

0

we get

 t  Xt0

− x0 =

 b(Xs0 , y)µ0s (dy)

(0)

dt + Wt

,

(15)

0

by the Lebesgue dominated convergence theorem. Details of this convergence based on Krylov’s bounds for Itˆ o processes and on the approximation bn (·, y) − b(·, y)Lp,loc → 0, n → ∞, may be found in [Kry80] and [Sko65] for  ordinary SDEs: they include freezing of bn . For the (14) one has, in addition, to take into account the continuity with respect to the variable y and weak  convergence of marginal distributions µns =⇒ µ0s . Concerning strong solutions for all these equations see [Ver81]. 3.3 Proof of Theorem 2 A. We are going to show that ¯ ti |2 ≤ C/N . sup E|Xti,N − X

(16)

t

This part in case (1◦ ) follows closely [BRTV98] and [Mal03]. Since we do not use directly any uniqueness result for the (1), we shall say precisely what is ¯ i : this is any solution of the (4). However, we take a solution with meant by X t the same distribution µt = µit for any i. The latter is certainly possible due to the Theorem 1 which asserts, in particular, that any solution Xt for (1) is strong. Hence, for any W i we can take the same functional of the corresponding Wiener path (W i ) for all i’s, which implies the same distribution, too. We add that since any solution of the (1) is strong, then the estimate (16) relates to any such solution, even if it is not unique. After having established this 1 N precaution we get, with µ ˆN t := Law(Xt , . . . , Xt ), ¯ ti )2 = 2(Xti,N − X ¯ ti )(b[Xti,N , µ ¯i 1 d(Xti,N − X ˆN t ] − b[Xt , µt ]) dt ¯ ti )(b1 [Xti,N , µ ¯i 1 = 2(Xti,N − X ˆN t ] − b1 [Xt , µt ]) dt ¯ i )(b0 (Xti,N ) − b0 (X ¯ i )) dt +2(Xti,N − X t t   i,N N i 1 i 2 ¯ i )(b1 [Xti,N , µ ¯ ¯ ≤ 2(Xti,N − X ˆ ] − b [ X , µ ]) − 2c |X − X | dt . 1 0 t t t t t t Therefore,

478

A. Yu. Veretennikov

E

N (

¯ i )2 − E (Xti,N − X t

i=1

= 2E

 t( N

N (

¯ i )2 (Xsi,N − X s

i=1

  ¯ ri ) b[Xri,N , µ ¯i 1 (Xri,N − X ˆN r ] − b[Xr , µr ] dr

s i=1

≤ −2c0 E

 t( N

¯ ri |2 dr |Xri,N − X

s i=1

+2E

 t( N

  ¯ i ) b1 [X i,N , µ ¯i 1 (Xri,N − X ˆN r r r ] − b1 [Xr , µr ] dr .

s i=1

We have, A := E

N (

  ¯ ri ) b1 [Xri,N , µ ¯i 1 (Xri,N − X ˆN r ] − b1 [Xr , µr ] (17)

i=1

=E

N ( i=1

+E

N ( i=1

⎛

(Xri,N

⎞ N (   1 ¯ ri ) ⎝ ¯ ri , X ¯ rj ) ⎠ b1 (Xri,N , Xrj,N ) − b1 (X −X N j=1 ⎛

(Xri,N

⎞ N (   1 ¯ i) ⎝ ¯ i, X ¯ j ) − b1 [X ¯ i , µ1 ] ⎠ =: A1 + A2 . b1 (X −X r r r r r N j=1

Case (1◦ ). Using the anti-symmetry and the conditions on b1 , we get,   ¯ i ) b1 (X i,N , X j,N ) − b1 (X ¯ i, X ¯ j) (Xri,N − X r r r r r   ¯ rj ) b1 (Xrj,N , Xri,N ) − b1 (X ¯ rj , X ¯ ri ) +(Xrj,N − X    ¯ ri − X ¯ rj ) b1 (Xri,N , Xrj,N ) − b1 (X ¯ ri , X ¯ rj ) ≤ 0 . = (Xri,N − Xrj,N ) − (X Hence, the first term is not positive, while the second possesses the bound,    1N ¯ ri ) 1 1N b1 (X ¯ ri , X ¯ rj ) − b1 [X ¯ ri , µ1r ] | |A2 | = |E i=1 (Xri,N − X j=1 N ≤

1 N

    1/2 2 1/2 1N   i,N 1N  i 2 i ¯j i 1 ¯ ¯ ¯ E X b − X ( X , X ) − b [ X , µ ] E 1 1 r r r r r r i=1 j=1   ¯ r1 |2 1/2 , ≤ CN 1/2 E|Xr1,N − X

(18)

¯ i, X ¯ j ) − b1 [X ¯ i , µ1 ] are non-correlated for difbecause random variables b1 (X r r r t ¯ 1 )2 , then (t > s) ferent j’s. In the other words, if α(t) := E(Xt1,N − X t

On Ergodic Measures





t

N α(t) − N α(s) ≤ −2c0

N α(r) dr + CN 1/2 s

479

t

α1/2 (r) dr .

(19)

s

This implies α(t) ≤

C2 . 4c20 N

Case (2◦ ). The value A from (17) can be considered as follows. The term A2 possesses the bound (18). Let us estimate A1 : ⎛ ⎞ ( N ( N   1 i,N i ⎝ i,N j,N i,N ¯ j ⎠ ¯ |A1 | ≤ E b1 (Xr , Xr ) − b1 (Xr , Xr ) (Xr − Xr ) N i=1 j=1 ⎛ ⎞ N N ( ( i,N   1 i ⎝ i,N ¯ j i ¯ j ⎠ ¯ ¯ + E b1 (Xr , Xr ) − b1 (Xr , Xr ) (Xr − Xr ) N i=1 j=1 b1 ≤ CLip

N 1 ( ¯ ri )(Xrj,N − X ¯ rj )| E |(Xri,N − X N i,j=1

b1 + CLip

N 1 ( ¯ ri |2 ≤ 2C b1 N α(r) . E |Xri,N − X Lip N i,j=1

Hence, we get 



t

b1 N α(t) − N α(s) ≤ −(2c0 − 2CLip )

N α(r) dr + CN 1/2 s

t

α1/2 (r) dr , (20) s

which implies, α(t) ≤

C2 . b1 2 4(c0 − CLip ) N

B. Uniqueness of distribution µt now follows directly from the bound (16). 1 N i Indeed, since distribution µ ˆN t ≡ Law(Xt , . . . , Xt ) is unique, and µt has been chosen the same for any i, that is, it does not depend on N at all, then we =⇒ µit as N → ∞. But clearly µi,N = µ1,N for any i, due to the have, µi,N t t t N uniqueness of µ ˆt and a symmetry (remind that the initial data X0i is the same for each i). Hence, the limit µit is indeed unique. C. The following statement also follows directly from the bound (16), although it will not be used in the sequel. Corollary 1. Under assumptions of the Theorem 2, for any finite number of indices i1 < i2 . . . < ik ,     i1 ¯ tik , N → ∞ , ¯t , . . . , X (21) Xti1 ,N , . . . , Xtik ,N =⇒ X

480

A. Yu. Veretennikov

uniformly with respect to t ≥ 0, where the random variables in the right hand side are independent. ¯ i are independent because W i are, while finite dimenIndeed, all processes X sional convergence (21) follows from (16). This is a propagation of chaos type result, – the term suggested by M. Kac, – saying that different particles behave nearly independently if their total number is large. ˆ N = (X 1,N , . . . , X N,N ) is ergodic, possesses mixing, and D. Show that X t ˆ tN tends in TV topology to some limiting measure, µN hence, the law of X ∞. Then this measure is stationary for the equation system (3). Naturally, this − µ1,N implies the convergence for projections, too, µ1,N t ∞ TV → 0 as t → ∞. E. We will use the method from [Ver97, Ver99] for couples of independent recurrent processes. Firstly, we need r to be large enough so that for some m greater than 2 the assertion of the Lemma 1 holds true. Although in this lemma the values rm are not explicit, however, certain bounds which are linear in m are actually available for them, cf. [Ver99]. The foundation of the approach to establishing beta-mixing as well as convergence rate to a (unique) equilibrium measure in total variation in [Ver99] consists of two estimates, ˆ t |m I(t < τ ) ≤ C(1 + |X ˆ 0 |m ) , sup EXˆ 0 |X t≥0

and ˆ 0 |m ) , EXˆ 0 τ k+1 ≤ C(1 + |X ˆ t | ≤ R), for R large enough, with the appropriate k. with τ := inf(s : |X In turn, for both bounds the main technical tool is the inequality ˆ x, ˆb(ˆ x) ≤ −(r − ε) < 0 for any fixed ε > 0 and for |ˆ x| > R, where one can choose R to be arbitrary large. This inequality is inappropriate, however, in our case; instead we can apply the method used also in [Ver97, Ver99] for couples of independent recurrent processes. The basic estimate for this in the present setting is N ( ˆ xi , ˆbi (ˆ x) |ˆ xi |m−2 = −∞. (22) lim |ˆ x|→∞

i=1

The latter follows from two remarks. (1◦ ) We notice that for each d-tuple of ˆ(i+1)d ), the value the form (ˆ xid+1 , . . . , x d (

x ˆid+j ˆbid+j (ˆ x)

j=1

is bounded from above; moreover, it is negative once |ˆ xi | = |(ˆ xid+1 , . . . , x ˆ(i+1)d )| is greater than some constant, R0 ; finally, it approaches the value −r or less if |ˆ xi | is large enough. (2◦ ) Since we actually compare the values

On Ergodic Measures

481

ˆ xi , ˆbi (ˆ x) |ˆ xi |m−2 , and m is greater than two (see above), it remains to notice that as |ˆ x| tends i ˆi i m−2 x) |ˆ x| either remains to infinity, for each 1 ≤ i ≤ N the value ˆ x , b (ˆ bounded, or tends to −∞; and at least one of them does tend to −∞. Hence, (22) holds true. The rest of the proof follows the calculus and arguments from [Ver99], as we mentioned above, for the couples of independent processes and stopping time denoted in [Ver99] by γn . , and the weakest among F. Now we use the double limit theorem for µ1,N t the two topologies, both being stronger than weak one. One limit is uniform due to the bound (16). All assertions of the theorem follow from the double limit theorem. Indeed, both convergence assertions are straightforward, and ˆN uniqueness of µ∞ follows from uniqueness of µ t . The limiting measure cannot N ˆ∞ does not depend on it. As long as depend on X0 because the measure µ the measure µ∞ is invariant for the (1), it is unique as well, because we have convergence to this measure, while prelimiting distributions µ1,N ∞ , N = 2, 3, . . . , are unique. Finally, to show that µ∞ is indeed invariant, it suffices to pass to the limit as t → ∞ in the integral equality (t, s ≥ 0)  t+s b[Xr , µr ] dr + Ws{t} , Ws{t} := Wt+s − Wt , (23) Xt+s = Xt + t

using Skorokhod’s technique (see [Sko65] or [Kry80]), quite similarly to the passage from (14) to (15) indeed. We denote Yst = Xs+t , then (23) reads,  s b[Yrt , νrt ] dr + Ws{t} , (24) Yst = Y0t + 0

νrt

with notation = µt+r . Due to tightness, exactly as in the proof of the Theo  rem 1, we can choose a subsequence t → ∞ such that the couple (Y.t , W.{t } ) weakly converges. Changing the probability space by the Skorokhod method,   we can assume that the couple (Y.t , W.{t } ) converges along some new sub  sequence t → ∞ almost surely, (Y.t , W.{t } ) → (Y.0 , W.{0} ) in the weak  topology in C([0, ∞); Rd ). In particular, νrt =⇒ νr0 . But we already know  that νrt = µt +r =⇒ µ∞ . Hence, from (24) rewritten on the new probability space, we get in the limit  s b[Yr0 , νr0 ] dr + Ws{0} , Ys0 = Y00 + 0 {0}

is a new Wiener process in Rd , and νr0 ≡ µ∞ is the distribution where W· 0 of Yr for any r. Thus, µ∞ is an invariant measure indeed. The Theorem is proved. 3.4 Proof of Theorem 3 A. We are going to show that

482

A. Yu. Veretennikov

¯ i |2 ≤ C(N −1 + h1/2 ) . sup E|Xti,N,h − X t

(25)

t

The statement of the Theorem then follows from this inequality either directly, – i.e., after analysing ergodic properties of the process Xti,N,h , – or from the Theorem 2 above. We prefer the latter reasoning because the last statement has been already established. However, ergodic properties of Xti,N,h will be needed anyway. Throughout the calculus, whenever we compare the values like Xti,N,h and i,N,h under expectation, we always have in mind the following bound: Xs   E|Xti,N,h − Xsi,N,h |2 ≤ 2 E|Wti − Wsi |2 + b2L∞ |t − s|2 . Similarly to the calculus in the Theorem 2 we get by the Itˆ o formula,

E

N (

¯ ti )2 − E (Xti,N,h − X

i=1

= 2E

 t( N s i=1

N (

¯ si )2 (Xsi,N,h − X

i=1

N,h ¯ ri )(b[X i,N,h , µ ¯ ri , µ1r ]) dr (Xri,N,h − X ] − b[X κh (r) ˆ r

≤ −2c0 E

 t( N

¯ i |2 dr |Xri,N,h − X r

s i=1

+2E

 t( N s i=1

N,h ¯ i )(b1 [X i,N,h , µ ¯ i , µ1 ]) dr. (Xri,N,h − X ] − b1 [X r r r κh (r) ˆ r

We have, E

N (

N,h ¯ i )(b1 [X i,N,h , µ ¯ i , µ1 ]) (Xri,N,h − X ] − b1 [X r r r κh (r) ˆ r

i=1

⎛

⎞ N ( ¯ ri ) ⎝ 1 ¯ ri , X ¯ rj ))⎠ =E (Xri,N,h − X (b1 (Xκi,N,h , Xrj,N,h ) − b1 (X h (r) N i=1 j=1 N (

⎞ N ( ¯ ri ) ⎝ 1 ¯ ri , X ¯ rj ) − b1 [X ¯ ri , µ1r ])⎠ =: A1 + A2 . (26) +E (Xri,N,h − X (b1 (X N i=1 j=1 N (

⎛

The second term here possesses the bound,

On Ergodic Measures

|A2 | = |E

N 1

¯ i) (Xri,N,h − X r

i,j=1

≤

1 N

1

¯i ¯j N (b1 (Xr , Xr )

483

 ¯ i , µ1 ]) | − b1 [X r r

⎛  2 ⎞1/2 1/2 N N    1 1 ¯i 2 ¯ i, X ¯ j ) − b1 [X ¯ i , µ1 ] ⎠ ⎝E E Xri,N,h − X b1 (X r r r r r i=1

j=1

  ¯ 1 |2 1/2 . ≤ CN 1/2 E|Xr1,N,h − X r

(27)

The first term may be estimated as follows, b1 |A1 | ≤ 2CLip αh (r) + CN h1/4 α1/2 (r) ,

¯ 1 )2 . So, one gets (t > s), where αh (t) := E(Xt1,N,h − X t  N α (t) − N α (s) ≤ −2c0 h

h



t h

N α (r) dr + C(N

1/2

+ Nh

1/4

t

(αh (r))1/2 dr .

)

s

s

or 

t

αh (t) − αh (s) ≤ −2c0 s

This implies αh (t) ≤

αh (r) dr + C(N −1/2 + h1/4 )



t

(αh (r))1/2 dr . (28) s

 2 C2 N −1/2 + h1/4 . b1 2 4(c0 − CLip )

B. The following statement follows directly from the bound (25), and it is natural to formulate it here, although it will not be used in the sequel. Corollary 2. Under assumptions of the Theorem 3, for any finite number of indices i1 < i2 . . . < ik ,    i1  ¯t , . . . , X ¯ tik , N, h−1 → ∞ , Xti1 ,N,h , . . . , Xtik ,N,h =⇒ X (29) uniformly with respect to t ≥ 0, where the random variables in the right hand side are independent. ˆ N,h = C. We are going to show that the homogeneous Markov process X kh 1,N,h N,N,h N,h (Xkh , . . . , Xkh ) converges to an equilibrium measure µ ˆ∞ in total variation as kh → ∞, with at least a polynomial rate. This convergence is uniform in s < h and h ≤ 1 (see Proposition 1). Then the measure µ ˆN,h ∞ is stationary N,h ˆ for the “big” Markov process X , k = 0, 1, 2, . . . This implies a convergence kh 1,N,h 1,N,h for projections, too, namely, µkh − µ∞ TV → 0 as k → ∞. ˆ N,h tends in TV topology to some other limOf course, the law of X kh+T iting measure which depends also on T . However, as T < h, the difference

484

A. Yu. Veretennikov

of all these limits become negligible when h becomes small, just because ˆ tN,h − X ˆ sN,h |2 ≤ CN h. sup|t−s|≤h E|X D. We will show the ergodicity and mixing under assumption (7). The approach to establishing uniform beta-mixing bounds as well as convergence rate to a (unique) equilibrium measure in total variation for approximation schemes consists of two estimates, ˆ tN,h |m I(t < τ ) ≤ C(1 + |X ˆ N,h |m ) , sup sup EXˆ h |X 0 0

h≤1 t≥0

and ˆ 0 |m ) , sup EXˆ 0 τ k+1 ≤ C(1 + |X h≤1

ˆ tN,h | |X

with τ := inf(s : ≤ R), for R large enough, with the appropriate k. Due to the Proposition 2, these two bounds follow both from the assertion lim

|ˆ x|→∞

N (

ˆ xi , ˆbi (ˆ x) |ˆ xi |m−2 = −∞.

(30)

i=1

The latter follows from (7). This completes the proof.

Acknowledgements The author is grateful to Clare Hall and Isaac Newton Institute for Mathematical Sciences, University of Cambridge, for the hospitality during his stay in Michaelmas term 2003 and a very stimulating environment, and grants INTAS-99-0590 and RFBR 02-01-0444 for the financial support.

References [BRTV98] Benachour, S., Roynette, B., Talay, D., Vallois, P.: Nonlinear selfstabilizing processes. I: Existence, invariant probability, propagation of chaos. Stochastic Processes Appl. 75(2), 173–201 (1998) [BRV98] Benachour, S., Roynette, B., Vallois, P.: Nonlinear self-stabilizing processes. II: Convergence to invariant probability, Stochastic Processes Appl. 75(2), 203–224, (1998) [BCCP98] Benedetto, D., Caglioti, E., Carrillo, J.A., Pulvirenti, M.: A nonMaxwellian steady distribution for one-dimensional granular media. J. Stat. Phys. 91(5-6), 979–990 (1998) [BT97] Bossy, M., Talay, D.: A stochastic particle method for the McKeanVlasov and the Burgers equation. Math. Comput. 66, No. 217, 157–192 (1997)

On Ergodic Measures [DG87] [Fun84]

[Kac56] [Klo05]

[KV04]

[Kot95]

[Kry80] [LLLP81]

[Mal03] [McK66] [Nag93] [Shi94]

[Sko65] [Sko88] [Szn91] [Tam84] [Ver81]

[Ver97] [Ver99]

485

Dawson, D.A., G¨ artner, J.: Large deviations from the McKean-Vlasov limit for weakly interacting diffusions. Stochastics 20, 247–308 (1987) Funaki, T.: A certain class of diffusion processes associated with nonlinear parabolic equations. Z.Wahrscheinlichkeitstheor. Verw. Geb., 67, 331–348 (1984) Kac, M.: Foundations of kinetic theory. Proc. 3rd Berkeley Sympos. Math. Statist. Probability, Vol.3, 171–197 (1956) Klokov, S.A.: On lower bounds for mixing rates for a class of Markov Processes (Russian). Teor. Veroyatnost. i Primenen, submitted; preprint version at http://www.mathpreprints.com/math/Preprint/ klokov/20020920/1/ Klokov, S.A., Veretennikov, A.Yu.: Polynomial mixing and convergence rate for a class of Markov processes, (Russian) Teor. Veroyatnost. i Primenen., 49(1), 21–35, (2004); Engl. translation to appear Kotelenez, P.: A class of quasilinear stochastic partial differential equations of McKean-Vlasov type with mass conservation. Probab. Theory Relat. Fields, 102(2), 159–188 (1995) Krylov, N.V.: Controlled diffusion processes. NY et al., Springer (1980) Landau, L.D., Lifshitz, E.M. Course of theoretical physics, Vol. 10, Physical kinetics/by E. M. Lifshitz and L. P. Pitaevskii, Oxford, Pergamon (1981) Malrieu, F.: Logarithmic Sobolev inequalities for some nonlinear PDE’s. Stochastic Process. Appl., 95(1), 109–132 (2001) McKean, H.P.: A class of Markov processes associated with nonlinear parabolic equations. Proc. Natl. Acad. Sci. USA 56, 1907-1911 (1966) Nagasawa, M.: Schr¨ odinger equations and diffusion theory. Basel: Birkh¨ auser (1993) Shiga, T.: Stationary distribution problem for interacting diffusion systems. In: Dawson, D. A. (ed.) Measure-valued processes, stochastic partial differential equations, and interacting systems. Providence, RI, American Mathematical Society, CRM Proc. Lect. Notes, 5, 199-211 (1994) Skorokhod, A.V.: Studies in the theory of random processes. Reading, Mass., Addison-Wesley (1965) Skorokhod, A.V.: Stochastic equations for complex systems. Dordrecht (Netherlands) etc., D. Reidel Publishing Company (1988) Sznitman, A.-S.: Topics in propagation of chaos. Calcul des probabilit´es, Ec. d’´et´e, Saint-Flour/Fr. 1989, Lect. Notes Math. 1464, 165–251 (1991) Tamura, Y.: On asymptotic behavior of the solution of a non-linear diffusion equation. J. Fac. Sci., Univ. Tokyo, Sect. I A 31, 195-221 (1984) Veretennikov, A.Yu.: On strong solutions and explicit formulas for solutions of stochastic integral equations. Math. USSR Sb., 39, 387–403 (1981) Veretennikov, A.Yu.: On polynomial mixing bounds for stochastic differential equations. Stochastic Processes Appl. 70(1), 115–127 (1997) Veretennikov, A. Yu. On polynomial mixing and the rate of convergence for stochastic differential and difference equations. (Russian) Teor. Veroyatnost. i Primenen. 44, 1999, no. 2, 312–327; English translation: Theory Probab. Appl. 44, no. 2, 361–374 (1999)

486 [VK03]

A. Yu. Veretennikov Veretennikov, A.Yu., Klokov, S.A.: On the mixing rates for the Euler scheme for stochastic difference equations. (Russian), Dokl. Akad. Nauk (Doklady Acad. Sci. of Russia), 395(6), 738–739 (2004); Engl. translation to appear

On the Distribution of Some New Explicit Inversive Pseudorandom Numbers and Vectors Arne Winterhof Johann Radon Institute for Computational and Applied Mathematics, Austrian Academy of Sciences, Altenbergerstr. 69, 4040 Linz, Austria [email protected] Summary. Inversive methods are attractive alternatives to the linear method for pseudorandom number generation. A particularly attractive method is the new explicit inversive method recently introduced by Meidl and the author. We present nontrivial results on the distribution of pseudorandom numbers and vectors generated by this method over the full period and in parts of the period. Moreover, we establish some new results on the statistical properties of parallel streams of such pseudorandom numbers. These results demonstrate that the new method is eminently suitable for the generation of parallel streams of pseudorandom numbers with desirable properties. The proofs are based on new bounds on certain exponential sums over finite fields.

Key words: Pseudorandom numbers, Inversive method, Discrepancy, Random numbers for parallel processors

1 Introduction Let IFq be the finite field of order q = pk with a prime p and an integer k ≥ 1. Further let η ∈ IF∗q be an element of order T ≥ 2. Moreover, let  −1 γ if γ ∈ IF∗q , γ= 0 if γ = 0. For given α, β ∈ IF∗q we generate a sequence γ0 , γ1 , . . . of elements of IFq by γn = αη n + β

for n = 0, 1, . . .

(1)

(see [10]). We study exponential sums over IFq which in the simplest case are of the form N −1 ( χ(γn ) for 1 ≤ N ≤ T , n=0

488

A. Winterhof

where χ is a nontrivial additive character of IFq . Upper bounds for these exponential sums are then applied to the analysis of three new inversive methods for pseudorandom number and vector generation. These new methods are defined as follows. Let {ρ1 , ρ2 , . . . , ρk } be an ordered basis of IFq over IFp . If γn = cn,1 ρ1 + cn,2 ρ2 + . . . + cn,k ρk

with all cn,i ∈ IFp ,

then we derive digital explicit inversive pseudorandom numbers of period T in the interval [0, 1) by putting yn =

k (

cn,j p−j

j=1

and explicit inversive pseudorandom vectors of period T by un =

1 (cn,1 , cn,2 , . . . , cn,k ) ∈ [0, 1)k p

for n = 0, 1, . . .. Moreover, for an integer s with 1 ≤ s ≤ q − 1, choose α1 , . . . , αs , β1 , . . . , βs ∈ IF∗q such that the elements α1−1 β1 , . . . , αs−1 βs of IF∗q are distinct. Then we put γn(i) = αi η n + βi

for i = 1, 2, . . . , s and n = 0, 1, . . . .

(2)

If (i)

(i)

(i)

γn(i) = cn,1 ρ1 + cn,2 ρ2 + . . . + cn,k ρk

(i)

with all cn,j ∈ IFp ,

(3)

then we consider the digital explicit inversive pseudorandom numbers of period T k ( (i) (i) yn = cn,j p−j ∈ [0, 1) j=1

and define explicit inversive parallel pseudorandom vectors of period T by   yn = yn(1) , . . . , yn(s) ∈ [0, 1)s for n = 0, 1, . . . . After some auxiliary results in Sect. 2 we prove some new bounds for incomplete exponential sums over finite fields in Sect. 3 which allow us to give nontrivial results on the distribution of sequences of digital explicit inversive pseudorandom numbers, explicit inversive pseudorandom vectors, and explicit inversive parallel pseudorandom vectors of period T . The application to digital explicit inversive pseudorandom numbers and explicit inversive parallel pseudorandom vectors is presented in Sect. 4 and to explicit inversive pseudorandom vectors in Sect. 5. Similar results on a different family of explicit inversive generators of period q can be found in [4, 5, 15, 19–21].

Distribution of Inversive Pseudorandom Numbers

489

On the one hand the sequence (γn ) defined by (1) has excellent structural properties (see [3, 4, 10]) and desirable distribution properties. On the other hand its generation, in particular the inversion, is expensive. However, using Montgomery’s trick [11, 12, 23] several inverses can be computed simultaneously by one inversion and some extra multiplications. An alternative is parallelization. Moreover, for example in the case that q is a power of two, using (optimal) normal basis representation and the Itoh-Tsujii inversion algorithm [7], the generation of the sequence (γn ) can be essentially sped up.

2 Auxiliary Results The following bound for exponential sums can be found in [13, Theorem 2]. Lemma 1. Let χ be a nontrivial additive character of IFq and let f /g be a rational function over IFq . Let v be the number of distinct roots of the polynomial g in the algebraic closure IFq of IFq . Suppose that f /g is not of the form Ap − A, where A is a rational function over IFq . Then (  f (ξ)  χ ≤ (max(deg(f ), deg(g)) + v ∗ − 2)q 1/2 + δ , ξ∈IF g(ξ) g(ξ)=q0 where v ∗ = v and δ = 1 if deg(f ) ≤ deg(g), and v ∗ = v + 1 and δ = 0 otherwise. The following result in [19, Lemma 2] provides a condition when Lemma 1 is applicable. Lemma 2. Let f /g be a rational function over IFq such that g is not divisible by the pth power of a nonconstant polynomial over IFq , f = 0, and deg(f ) − deg(g) ≡ 0 mod p or deg(f ) < deg(g). Then f /g is not of the form Ap − A, where A is a rational function over IFq . We also need a bound on mixed exponential sums (see [22]). Lemma 3. Let χ be a nontrivial additive character and ψ a nontrivial multiplicative character of IFq and let f /g be a rational function over IFq . Let v be the number of distinct roots of the polynomial g in the algebraic closure IFq of IFq . Then (   f (ξ) χ ψ(ξ) ≤ (max(deg(f ), deg(g)) + v ∗ − 1)q 1/2 , g(ξ) ξ∈IF∗q g(ξ)=0 where v ∗ = v if deg(f ) ≤ deg(g), and v ∗ = v + 1 otherwise.

490

A. Winterhof

Put eT (z) = exp(2πiz/T ). Lemma 4. For an integer 1 ≤ N ≤ T we have −1   T −1 N ( 4 ( eT (un) ≤ T log T + 0.8 . π2 u=1 n=0

Proof. We have T −1 N −1 −1 T( ( sin πN u/T ( eT (un) = sin πu/T u=1 n=0

u=1

≤

4 gcd(N, T )2 T log T + 0.38T + 0.608 + 0.116 2 π T

by [1, Theorem 1]. Here the constant 4/π 2 is best possible. For improvements of the other constants see [2] and references therein.  Let C(p) denote the set of integers h with −p/2 < h ≤ p/2 and let Ck (p) be the set of k-dimensional points (h1 , . . . , hk ) with hj ∈ C(p) for 1 ≤ j ≤ k. For (h1 , . . . , hk ) ∈ Ck (p) we put Qp (h1 , . . . , hk ) = 1 if (h1 , . . . , hk ) = 0 and π Qp (h1 , . . . , hk ) = p−d csc |hd | if (h1 , . . . , hk ) = 0 , p ∗ (p) be the set where d = d(h1 , . . . , hk ) is the largest j with hj = 0. Let Cs×k ∗ (p) of all nonzero s × k matrices with entries in C(p). For H = (hij ) ∈ Cs×k we define s  Wp (H) = Qp (hi1 , . . . , hik ) . i=1

The following lemma is obtained by using [14, Lemma 3.13] for p = 2 and an inequality in the proof of [17, Theorem 2] for p > 2. Lemma 5. For any s ≥ 1 and k ≥ 1 we have s  ( k +1 W2 (H) < , 2 ∗ H∈Cs×k (2)

(

 Wp (H) <

∗ H∈Cs×k (p)

s 2 2 k log p + k + 1 if p > 2 . π 5

3 Bounds for Exponential Sums  For n = 0, 1, . . . let

(1)

(2)

(s)

γn , γn , . . . , γn

 be the vector of elements of IFq

generated by (2). For a nontrivial additive character χ of IFq , for µ1 , . . . , µs ∈ IFq , and for an integer N with 1 ≤ N ≤ T we consider the exponential sums

Distribution of Inversive Pseudorandom Numbers

SN =

N −1 (

χ

 s (

n=0

491

 µi γn(i)

.

i=1

Theorem 1. If µ1 , µ2 , . . . , µs are not all 0, then we have   |ST | = O sq 1/2 . Proof. We can assume that s < T since otherwise the result is trivial. Then we have   s T −1 ( q−1 q − 1 ( n |ST | = χ µi αi η + βi T T n=0 i=1   s ( ( = χ µi αi ξ (q−1)/T + βi ξ∈IF∗q i=1 (  f (ξ)  (q − 1)s ≤ +1+ χ , ξ∈IF T g(ξ) g(ξ)=q0 where f (x) =

s (

µi

i=1

and g(x) =

s 

s 

(αj x(q−1)/T + βj )

j=1 j=i

(αj x(q−1)/T + βj ) .

j=1

α1−1 β1 , . . . , αs−1 βs

Since are distinct, g is squarefree, and since at least one µi is nonzero, the uniqueness of the partial fraction decomposition for rational functions implies that f = 0. Since deg(f ) < deg(g), Lemma 1 and Lemma 2 yield the result.  Theorem 2. If µ1 , µ2 , . . . , µs are not all 0, then we have   |SN | = O sq 1/2 log T for 1 ≤ N < T . Proof. We can again assume that s < T . With σn = χ(σn )

N −1 (

i=1

(i)

µi γn we have

T −1 1 ( eT (u(n − t)) T u=0 n=0 t=0  T −1   T −1 N −1 ( 1 ( ( eT (−ut) χ(σn )eT (un) = T u=0 t=0 n=0  T −1   T −1 T −1 N −1 ( 1 ( ( N ( χ(σn ) + eT (−ut) χ(σn )eT (un) = T n=0 T u=1 t=0 n=0

SN =

T −1 (

1s

492

A. Winterhof

and so T −1 T −1 N −1 ( N 1 ( ( |SN | ≤ |ST | + eT (ut) χ(σn )eT (un) . T T u=1 t=0 n=0 For 1 ≤ u ≤ T − 1 we define the nontrivial multiplicative character ψu of IFq by ψu (ϑn ) = eT (un), 0 ≤ n ≤ q − 2 , with a primitive element ϑ of IFq . Then we have  s  T −1 ( ( ( q−1 χ(σn )eT (un) = χ µi αi ξ (q−1)/T + βi ψu (ξ) T n=0 i=1 ξ∈IF∗ q   (q − 1)s ( f (ξ) ≤ + χ ψu (ξ) , T g(ξ) ξ∈IF∗q g(ξ)=0 where f (x) =

s (

µi

i=1

and g(x) =

s 

s 

(αj x(q−1)/T + βj )

j=1 j=i

(αj x(q−1)/T + βj ) .

j=1

Lemmas 3 and 4 yield T −1 N −1 −1 −1 N −1 T(   T( ( ( ( 1/2 eT (ut) χ(σn )eT (un) ≤ s 2q +1 eT (ut) u=1 t=0 n=0 u=1 t=0    4 1/2 ≤ s 2q +1 T log T + 0.8 . π2 Hence we obtain by Theorem 1 the theorem.



Theorem 2 is nontrivial only if N is at least of the order of magnitude sq 1/2 log T . Now we prove a bound which is nontrivial for N at least of the order of magnitude sq 1/2 using a method introduced in [18]. Theorem 3. If µ1 , µ2 , . . . , µs are not all 0, then we have   |SN | = O s1/2 N 1/2 q 1/4 for 1 ≤ N < T .

Distribution of Inversive Pseudorandom Numbers

493

T Proof. We can assume that s+1 ≤ q−1 q 1/2 since otherwise the result is trivial. 1s (i) With σn = i=1 µi γn and any integer m with 0 ≤ m < T we have N −1 ( (4) χ (σn+m ) ≤ 2m . S N − n=0

For an integer M with 1 ≤ M ≤ T we use the above inequality for m = 0, 1, . . . , M − 1 and we get M |SN | ≤ W + (M − 1)M ,

(5)

−1 M −1 M −1 −1 N( N( ( ( W = χ(σn+m ) ≤ χ (σn+m ) .

where

n=0 m=0

n=0 m=0

By the Cauchy-Schwarz inequality we obtain

2 W ≤N χ (σn+m ) n=0 m=0 M −1  s  2 ( NT ( ( m ξ (q−1)/T + β ≤ χ µ α η i i i q−1 ∗ N −1 M −1 ( (

2

ξ∈IFq

NT = q−1

m=0

M −1 (

i=1

(



χ

m1 ,m2 =0 ξ∈IF∗ q

s (



µi αi η m1 ξ (q−1)/T + βi − αi η m2 ξ (q−1)/T + βi



 .

i=1

If m1 = m2 , then the sum over ξ is equal to q − 1. For m1 = m2 let f (x) = (η m2 − η m1 )x(q−1)/T

s ( i=1

and g(x) =

s 

µi αi

s 

(αj η m1 x(q−1)/T + βj )(αj η m2 x(q−1)/T + βj )

j=1 j=i

(αj η m1 x(q−1)/T + βj )(αj η m2 x(q−1)/T + βj ) .

j=1

Then

 ( ( s   χ µi αη i+m1 ξ (q−1)/T + β − αη i+m2 ξ (q−1)/T + β ξ∈IF∗q i=1 q−1 ≤ 2s + 1 + T

( ξ∈IFq ,g ∗ (ξ) =0

 χ

f ∗ (ξ) g ∗ (ξ)

 ,

f g where f ∗ = (f,g) and g ∗ = (f,g) . For the application of Lemmas 1 and 2 it is ∗ sufficient that g is squarefree (p = 2!) and f ∗ = 0.

494

A. Winterhof

In g(x) we can have repetition of factors only if there exist 1 ≤ i, j ≤ s with i = j such that (6) αi βj = αj βi η m2 −m1 . Then αi η m1 x(q−1)/T +βi is a common factor of f and g. Hence g ∗ is squarefree. Suppose we have f ∗ = 0. Let i be an index with µi = 0. Then for each zero ξ of αi η m1 x(q−1)/T + βi we have 0 = f ∗ (ξ) = f (ξ) = (1 − η m2 −m1 )µi αi−2 βi

s 

(αi βj − αj βi )(αi βj − αj η m2 −m1 βi ) .

j=1 j=i

This yields the existence of 0 ≤ j ≤ s − 1, i = j, satisfying (6). There are at most s − 1 possible indices m2 = m1 satisfying (6) for given m1 and i. For these m2 we estimate trivially. By Lemmas 1 and 2 we obtain     4(q − 1)s NT q−1 2 2 1/2 −2 q +1 + 1 + 2s W ≤ M s(q − 1) + M q−1 T T ≤ N (M sT + 4M 2 sq 1/2 ) . G F T we get Choosing M = q1/2 W2 ≤ 5sN q 1/2 M2 and thus |SN | ≤

√ 1/2 1/2 1/4 T 5s N q + 1/2 q

by (5). Note that the constant can be improved if we use shifts in both directions in (4). 

4 Digital Explicit Inversive Pseudorandom Numbers and Explicit Inversive Parallel Pseudorandom Vectors We use the bounds for exponential sums of Theorems 1, 2, and 3 to derive results on the distribution of sequences of explicit inversive parallel pseudorandom vectors over the full period and in parts of the period. (i) (i) Given s sequences y0 , y1 , . . . (1 ≤ i ≤ s) of digital explicit inversive pseudorandom numbers with the conditions in Sect. 1, we consider the explicit inversive parallel pseudorandom vectors   yn = yn(1) , yn(2) , . . . , yn(s) ∈ [0, 1)s for n = 0, 1, . . . .

Distribution of Inversive Pseudorandom Numbers

495

Given a single sequence y0 , y1 , . . . of digital explicit inversive pseudorandom numbers of period T and a dimension s ≥ 1, we may also consider the points yn = (yn , yn+1 , . . . , yn+s−1 ) ∈ [0, 1)s for n = 0, 1, . . . . (i)

By choosing αi = αη i−1 and βi = β we have yn+i−1 = yn for i = 1, . . . , s and may regard these vectors as special explicit inversive parallel vectors. From now on we focus on the general case of parallel vectors. Then for any integer N with 1 ≤ N ≤ T we define the star discrepancy ∗(s)

DN

= sup |FN (J) − V (J)| , J

where the supremum is extended over all subintervals J of [0, 1)s containing the origin, FN (J) is N −1 times the number of points among y0 , y1 , . . . , yN −1 falling into J, and V (J) denotes the s-dimensional volume of J. In the follow∗(s) ing we establish an upper bound for DN . Theorem 4. For any sequence of s-dimensional explicit inversive parallel ∗(s) pseudorandom vectors and for any 1 ≤ N < T the star discrepancy DN satisfies ∗(s) DN = O(min(N −1 q 1/2 log T, N −1/2 q 1/4 )(log q)s ) . ∗ Proof. For H = (hij ) ∈ Cs×k (p) we define the exponential sum

SN (H) =

N −1 ( n=0

⎛ ep ⎝

s ( k (

⎞ hij cn,j ⎠ , (i)

i=1 j=1

(i)

where the cn,j ∈ IFp are as in (3). Then by a general discrepancy bound in [6, Theorem 1(ii) and Lemma 3(iii)] (see also [14, Theorem 3.12] for a slightly weaker version) we obtain  s ( 1 1 ∗(s) + Wp (H)|SN (H)|. (7) DN ≤ 1 − 1 − q N ∗ H∈Cs×k (p)

Let {δ1 , . . . , δk } be the dual basis of the given ordered basis {ρ1 , . . . , ρk } of IFq over IFp . Then by a well-known principle (see [8, p. 55]) we have (i)

cn,j = Tr(δj γn(i) ) for 1 ≤ j ≤ k, 1 ≤ i ≤ s, and n ≥ 0 , where Tr denotes the trace function from IFq to IFp . Therefore

496

A. Winterhof

SN (H) =

N −1 ( n=0

=

N −1 (

⎛ ep ⎝ ⎛

=

⎞ hij Tr(δj γn(i) )⎠

i=1 j=1

⎛ ⎞⎞ s ( k ( ep ⎝Tr ⎝ hij δj γn(i) ⎠⎠

n=0 N −1 (

s ( k (

χ

 s (

n=0

i=1 j=1



µi γn(i)

,

i=1

where χ is the canonical additive character of IFq and µi =

k (

hij δj ∈ IFq for 1 ≤ i ≤ s .

j=1

Since H is not the zero matrix and {δ1 , . . . , δk } is a basis of IFq over IFp , it follows that µ1 , . . . , µs are not all 0. Hence we may apply Theorems 2 and 3. The result follows by (7), Theorems 2 and 3, and Lemma 5.  Theorem 5. For any sequence of s-dimensional explicit inversive parallel ∗(s) pseudorandom vectors the star discrepancy DT satisfies ∗(s)

DT

= O(q 1/2 T −1 (log q)s ) .

Proof. The theorem follows by (7), Theorem 1, and Lemma 5 with the same arguments as in the proof of the previous theorem. 

5 Explicit Inversive Pseudorandom Vectors Statistical independence properties of pseudorandom vectors are customarily assessed by the discrete discrepancy (see [14, Section 10.2]). Given a sequence u0 , u1 , . . . of explicit inversive pseudorandom vectors and an integer s ≥ 1, we consider the ks-dimensional points vn = (un , un+1 , . . . , un+s−1 ) ∈ [0, 1)ks for n = 0, 1, . . . . Then for any integer N with 1 ≤ N ≤ T we define the discrete discrepancy (s)

EN,p = max |FN (J) − V (J)| , J

where the maximum is over all subintervals J of [0, 1)ks of the form J=

 ks   ai bi , p p i=1

Distribution of Inversive Pseudorandom Numbers

497

with integers ai , bi for 1 ≤ i ≤ ks, where FN (J) is N −1 times the number of points v0 , v1 , . . . , vN −1 falling into J and V (J) denotes the ks-dimensional volume of J. Theorem 6. For any sequence of k-dimensional inversive pseudorandom vec(s) tors, for any s ≥ 1, and for any 1 ≤ N < T the discrete discrepancy EN,p satisfies (s) EN,p = O(min(N −1 q 1/2 log T, N −1/2 q 1/4 )(log p)ks ) . ∗ ∗ (p) be the set of nonzero vectors in Cks (p). For h ∈ Cks (p) we Proof. Let Cks define the exponential sum

SN (h) =

N −1 (

ep (p(h · vn )) ,

n=0

where the dot denotes the standard inner product. By [16, Corollary 3] we get (s) EN,p

1 ≤ N

 max |SN (h)|

∗ (p) h∈Cks

4 0.61 log p + 1.41 + π2 p

ks .

∗ (p) we write For a fixed h ∈ Cks

h = (h0 , h1 , . . . , hs−1 ) with hi ∈ Ck (p) for 0 ≤ i ≤ s − 1, where not all hi are 0. Then we have ⎞  s  N −1 ⎛ s k N −1 ( ( ( (( SN (h) = ep p hi · un+i = ep ⎝ hij cn+i−1,j ⎠ , n=0

n=0

i=1

i=1 j=1

where hi = (hi1 , . . . , hik ) for 0 ≤ i ≤ s − 1 and all hij ∈ C(p). As in the proof of Theorem 4 we get   s N −1 ( ( χ µi γn+i−1 SN (h) = n=0

i=1



and thus the result.

Theorem 7. For any sequence of k-dimensional inversive pseudorandom vec(s) tors and for any s ≥ 1 the discrete discrepancy ET,p with q = pk satisfies ET,p = O(q 1/2 T −1 (log p)ks ) . (s)

Proof. The theorem follows with the same arguments as in the proof of the previous theorem by Theorem 1. 

498

A. Winterhof

6 Structural Properties The following extension of Marsaglia’s lattice test (cf. [9]) was introduced in [3]. For given s ≥ 1 and N ≥ 2 we say that a generator η0 , η1 , . . . over IFq passes the s-dimensional N -lattice test if the vectors yn −y0 , n = 1, . . . , N −s, span IFsq , where yn = (ηn , ηn+1 , . . . , ηn+s−1 ) ∈ IFsq

for n = 0, 1, . . . , N − s .

In [4, Section 4.1] it was shown that an explicit inversive generator of period T passes the s-dimensional N -lattice test if   N −4 T −3 , s ≤ min . 3 2 Moreover, the explicit inversive parallel pseudorandom vectors of period T possess an optimal nonlinearity property expressed in the next theorem. Theorem 8. Every hyperplane in IFsq contains at most s of the points 

(1)

 γn(1) , . . . , γn(s) ,

n = 0, 1, . . . , T − 1 ,

(s)

with γn · · · γn = 0. If the hyperplane passes through the origin of IFsq , then (1)

(s)

it contains at most s − 1 of these points (γn , . . . , γn ). Proof. The proof is a straightforward adaptation of the proof of [15, Theorem 1]. 

Acknowledgment The author is supported by the Austrian Academy of Sciences and by the grant S8313 of the Austrian Science Fund (FWF).

References 1. Cochrane, T.: On a trigonometric inequality of Vinogradov. J. Number Theory, 27, 9–16 (1987) 2. Cochrane, T., Peral, J.C.: An asymptotic formula for a trigonometric sum of Vinogradov. J. Number Theory, 91, 1–19 (2002) 3. Dorfer, G., Winterhof, A.: Lattice structure and linear complexity profile of nonlinear pseudorandom number generators. Appl. Algebra Engrg. Comm. Comp., 13, 499–508 (2003)

Distribution of Inversive Pseudorandom Numbers

499

4. Dorfer, G., Winterhof, A.: Lattice structure of nonlinear pseudorandom number generators in parts of the period. In: Niederreiter, H. (ed) Monte Carlo and Quasi-Monte Carlo Methods 2002. Springer, New York, 199–211 (2004) 5. Eichenauer-Herrmann, J.: Statistical independence of a new class of inversive congruential pseudorandom numbers. Math. Comp., 60, 375–384 (1993) 6. Hellekalek, P.: General discrepancy estimates: the Walsh function system. Acta Arith., 67, 209–218 (1994) 7. Itoh, T., Tsujii, S.: A fast algorithm for computing multiplicative inverses in GF(2m ) using normal bases. Inform. and Comput., 78, 171–177 (1988) 8. Lidl, R., Niederreiter, H.: Introduction to Finite Fields and Their Applications, revised ed. Cambridge University Press, Cambridge (1994) 9. Marsaglia, G.: The structure of linear congruential sequences. In: Zaremba, S.K. (ed) Applications of Number Theory to Numerical Analysis. Academic Press, New York, 249–285 (1972) 10. Meidl, W., Winterhof, A.: On the linear complexity profile of some new explicit inversive pseudorandom numbers. J. Complexity, 20, 350–355 (2004) 11. Mishra, P.K., Sarkar, P.: Application of Montgomery’s trick to scalar multiplication for elliptic and hyperelliptic curves using a fixed base point. In: Bao, F., Deng, R., Zhou, J. (eds) Public Key Cryptography—PKC 2004. Lecture Notes in Comput. Sci., 2947, Springer, Berlin, 41–54 (2004) 12. Montgomery, P.L.: Speeding the Pollard and elliptic curve methods of factorization. Math. Comp., 48, 243–264 (1987) 13. Moreno, C.J., Moreno, O.: Exponential sums and Goppa codes: I. Proc. Amer. Math. Soc., 111, 523–531 (1991) 14. Niederreiter, H.: Random Number Generation and Quasi-Monte Carlo Methods. SIAM, Philadelphia (1992) 15. Niederreiter, H.: On a new class of pseudorandom numbers for simulation methods. J. Comp. Appl. Math., 56, 159–167 (1994) 16. Niederreiter, H.: Pseudorandom vector generation by the inversive method. ACM Trans. Modeling and Computer Simulation, 4, 191–212 (1994) 17. Niederreiter, H.: Improved bounds in the multiple-recursive matrix method for pseudorandom number and vector generation. Finite Fields Appl., 2, 225–240 (1996) 18. Niederreiter, H., Shparlinski, I.E.: On the distribution of inversive congruential pseudorandom numbers in parts of the period. Math. Comp., 70, 1569–1574 (2001) 19. Niederreiter, H., Winterhof, A.: Incomplete exponential sums over finite fields and their applications to new inversive pseudorandom number generators. Acta Arith., 93, 387–399 (2000) 20. Niederreiter, H., Winterhof, A.: On the lattice structure of pseudorandom numbers generated over arbitrary finite fields. Appl. Algebra Engrg. Comm. Comp., 12, 265–272 (2001) 21. Niederreiter, H., Winterhof, A.: On a new class of inversive pseudorandom numbers for parallelized simulation methods. Periodica Mathematica Hungarica, 42, 77–87 (2001) 22. Perel’muter, G.I.: Estimate of a sum along an algebraic curve (Russian). Mat. Zametki, 5, 373–380 (1969); Engl. transl. Math. Notes, 5, 223–227 (1969) 23. Shacham, H., Boneh, D.: Improving SSL handshake performance via batching. In: Topics in Cryptology—CT-RSA 2001 (San Francisco, CA). Lecture Notes in Comput. Sci., 2020, Springer, Berlin, 28–43 (2001)

Error Analysis of Splines for Periodic Problems Using Lattice Designs Xiaoyan Zeng1 , King-Tai Leung2 , and Fred J. Hickernell3 1

2

3

Department of Mathematics, Hong Kong Baptist University, Kowloon Tong, Hong Kong SAR, China, [email protected], presently at Department of Applied Mathematics, Illinois Institute of Technology, Chicago, IL, 60616, USA [email protected] Department of Mathematics, Hong Kong Baptist University, Kowloon Tong, Hong Kong SAR, China [email protected], Department of Mathematics, Hong Kong Baptist University, Kowloon Tong, Hong Kong SAR, China, [email protected], presently at Department of Applied Mathematics, Illinois Institute of Technology, Chicago, IL, 60616, USA [email protected]

Summary. Splines are minimum-norm approximations to functions that interpolate the given data, (xi , f (xi )). Examples of multidimensional splines are those based on radial basis functions. This paper considers splines of a similar form but where the kernels are not necessarily radially symmetric. The designs, {xi }, considered here are node-sets of integration lattices. Choosing the kernels to be periodic facilitates the computation of the spline and the error analysis. The error of the spline is shown to depend on the smoothness of the function being approximated and the quality of the lattice design. The quality measure for the lattice design is similar, but not equivalent to, traditional quality measures for lattice integration rules.

Key words: spline, lattice designs, circulant matrix

1 Introduction The study of approximation methods is an important area of numerical analysis. During the last three decades there have been many important contributions including those of [3,9,11,12,17]. The spline method is one of the approximation methods that can be traced to a seminal article of [13] and is widely used in physics [1], signal processing and imaging [15], and statistics [18]. For more theoretical results, see [2] and [5]. 

This research was supported by a Hong Kong RGC grant HKBU2007/03P.

502

X. Zeng et al.

It is well-known that the number of nodes required to obtain a given accuracy using product algorithms increases exponentially with increasing dimension. How to avoid the “curse of dimensionality” is an important question. [6] provides some results for interpolation. But for the spline method, there are few results. This article mainly discusses the high dimensional function approximation using the spline method. The theoretical frame is described in the following paragraph. It is known that for Hilbert spaces the spline method is the optimal linear algorithm for function approximation. This result is derived below. Assume that H is a Hilbert space of real-valued functions on a domain X with a reproducing kernel K(x, y), the representer of function evaluation , f (y) = f (·), K(·, y),

∀ y ∈ X,

f ∈H.

Here K(x, y) is positive definite and symmetric, that is, n−1 (

K(x, y) = K(y, x), ∀x, y ∈ X ,

ai aj K(xi , xj ) ≥ 0, ∀xi , ai .

(1)

i,j=0

Let B be some Banach space of functions on X , and I denote the injection operator for function recovery from H to B. Let U be a linear algorithm, U(f ) = ω T f , where ω := (ω0 , · · · , ωn−1 )T ∈ Bn , and f := (f (x0 ), · · · , f (xn−1 )) is a vector of the function values on the design, {x0 , · · · , xn−1 }. Furthermore, we define the linear functionals Iξ (f ) := f (ξ) and Uξ (f ) := U(f )(ξ). Instead of the standard definition of the error sup f − U(f )B , we will use the average of the worst case error over f H ≤1

the domain to describe the discrepancy between f and U (f ). Let now eξ (f ) denote the worst case error of approximation at ξ, eξ (f ) =

T

sup |Iξ (f ) − Uξ (f )| = K(·, ξ) − ω(ξ) a(·)H ,

(2)

f H ≤1

where a(x) = (K(x, x0 ), · · · , K(x, xn−1 ))T . The root mean square error for the space of functions averaged over the domain X is defined as err(n, H) :=

 



X

2 1/2 eξ (f ) dξ 1/2 T

= X

T

K(·, ξ) − ω(ξ) a(·), K(·, ξ) − ω(ξ) a(·) dξ

.

This implies the following lemma. Lemma 1. Let H be a Hilbert space with reproducing kernel K(x, y) which is positive definite and symmetric. The error for approximation is

Error Analysis of Splines for Periodic Problems Using Lattice Designs

err2 (n, H) = C1 − 2C2 ({xi }) + C3 ({xi }),

503

(3)

where the quantities C1 , C2 ({xi }), C3 ({xi }) are defined as follows:  2 C1 = K(ξ, ξ)2 , C2 ({xi }) = ω T (ξ)a(ξ) dξ , X

 C3 ({xi }) = 

ω T (ξ)Kω(ξ) dξ = trace(KW) , X

ω(ξ)ω T (ξ) dξ,

W= X

K = (K(xj , xi ))i,j .

For a given design, the choice of weights, ω(ξ) that minimizes eξ (f ) in an algorithm gives C2 ({xi }) = C3 ({xi }) = (2) is ω(ξ) = K−1 a(ξ). Such
trace(K−1 A), where A = X a(ξ)aT (ξ)dξ. Consequently, we have following lemma. Lemma 2. Let H be a Hilbert space with reproducing kernel K(x, y) which is positive definite and symmetric. The choice ω(ξ) = K−1 a(ξ) is the optimal weight for a given design. The approximation error for the algorithm is given by (4) err2 (n, H) = K(ξ, ξ)22 − trace(K−1 A). The error for approximation by a general linear algorithm has been derived in Lemma 1 above, and the specific case for an optimal choice of weight is given in Lemma 2. This is a spline algorithm as explained below. Given n different design points x0 , · · · , xn−1 ∈ X , and n information data f (x0 ), · · · , f (xn−1 ), the spline s(x) ∈ H is defined to interpolate the data and have minimum norm: s(x) = argmin{gH : g ∈ H, g(xi ) = f (xi ), i = 0, · · · , n − 1}.

(5)

From [18], it is known that a spline s(x) is a linear combination of K(x, xi ), that is n−1 ( ci K(x, xi ), (6) s(x) = i=0

where the coefficients ci are determined by the interpolation condition, s(xi ) = f (xi ). This makes the spline the same as the choice of ω as in Lemma 2. The remainder of this article is organized as follows. Some basic concepts of integration lattices are introduced in Sect. 2. In Sect. 3 the error of the spline using an integration lattice design is shown to depend on the smoothness of the function being approximated and the quality of the lattice design. In the final section some numerical results are provided.

504

X. Zeng et al.

2 Integration Lattices Lattice designs are widely used in multiple integration because of their simplicity and practicality for high dimensional problems. A d-dimensional integration lattice, L, is a set satisfying the following conditions Zd ⊆ L ⊂ Rd ,

x, y ∈ L ⇒ x + y ∈ L ,

where Zd is the set of d-dimensional integer vectors. What interests us most is the node set of integration lattice, P := L ∩ [0, 1)d , because we often restrict the domain to the half open unit cube without losing generality. The number of nodes is denoted by n. The generators of a lattice L are the linearly independent vectors, h1 , · · · , ht , such that any x ∈ L is an integer linear combination of h1 , · · · , ht , plus some integer vector. A lattice generated only by one generator h/n, where h has no common factor with the cardinality n, is called a rank-1 lattice. We mainly discuss the case of rank-1 lattice points and simply take h as the generator for convenience. Another important concept is the dual lattice, L⊥ = {z ∈ Zd : zT x ∈ Z, ∀x ∈ L}, which for a rank-1 lattice is reduced to L⊥ = {z ∈ Zd : zT h = 0 mod n}. The reproducing kernel is chosen as K({x − y}) from now on, where the braces around x − y mean that the fractional part of each component is to be taken. The domain may usually be assumed, after some appropriate transformation, to be the unit cube and the design points are chosen to be the node set of rank-1 lattice. Utilizing the Fourier expansion of the kernel, we obtain the elements of the matrix K in terms of the Fourier coefficients, 3 K(ν), as follows: ( 2πıν T h(j−i)/n 3 K(ν)e ki,j = K({xj − xi }) = , (7) ν∈Zd

√ where ı = −1. To simplify the notation, afterwards, sums involving wave numbers are assumed to be over the whole set Zd . As for matrix A, we have    T a(ξ)a(ξ) d ξ = K({ξ − xi })K({ξ − xj })d ξ . A= [0,1]d

[0,1]d

i,j

Therefore, the elements are ai,j =

(

3 2 (ν)e2πıν T h(i−j)/n . K

(8)

ν

3 Error Analysis for Spline Using Lattice Designs In this section, as in the previous one, the kernel is chosen as K({x − y}) and a rank-1 lattice design is employed.

Error Analysis of Splines for Periodic Problems Using Lattice Designs

505

Definition 1. Given n numbers ci , i = 0, . . . , n − 1, a circulant matrix C is a matrix whose rows are the following form ⎛ ⎞ c0 cn−1 cn−2 . . . c1 ⎜ c1 c0 cn−1 . . . c2 ⎟ ⎜ ⎟ ⎜ ⎟ (9) C = ⎜ c2 c1 c0 . . . c3 ⎟ . ⎜ .. .. .. . . .. ⎟ ⎝ . . . ⎠ . . cn−1 cn−2 cn−3 . . . c0 From formulas (7) and (8), it is clear that the matrices K and A are circulant, 1 3 2πıν T hj/n i.e. their rows are cyclically shifted copies of kj = ν K(ν)e , and 1 32 2πıν T hj/n , j = 0, · · · , n − 1, respectively. aj = ν K (ν)e From [16] it is known that circulant matrices have a very important property, which is shown in the following lemma. Lemma  If a matrix C is circulant, there exists a unitary matrix F =  −2πıij/n3. such that e i,j C = F−1 diag(ˆ c0 , cˆ1 , · · · , cˆn−1 )F . Here, cˆi =

n−1 1

e−2πıij/n cj , i = 0, · · · , n − 1.

j=0

For convenience, in the remainder of the article, the notation L⊥ (l) is used to denote the set {ν : hT ν = l mod n} and L⊥ = L⊥ (0). Because A and K are circulant matrices, the approximation error of Lemma 2 may be written as follows. Theorem 1. If {xi } is the node set of a rank-1 lattice, then ⎛



⎜n−1 ⎜( ⎜ err(n, H) = ⎜ ⎜ ⎝ l=0

1 ν∈L⊥ (l)

2 3 K(ν)

−

1

1 ν∈L⊥ (l)

3 K(ν)

ν∈L⊥ (l)

⎞1/2 3 2 (ν) ⎟ K ⎟ ⎟ ⎟ ⎟ ⎠

.

(10)

Proof The first item C1 is independent of the design and of the following form,  ( 3 K(ν). K(ξ, ξ)dξ = K(0) = C1 = [0,1]d

ν

The sum is divided into n terms, that is, C1 =

n−1 (

(

l=0

ν∈L⊥ (l)

3 K(ν) .

(11)

506

X. Zeng et al.

The following step is to get the second term in (4), trace(K−1 A). Matrix K and A are circulant matrices, so it is easy to obtain the following results from Lemma 3: −1 )F , K−1 = F−1 diag(kˆ0−1 , kˆ1−1 , · · · , kˆn−1 and

A = F−1 diag(ˆ a0 , a ˆ1 , · · · , a ˆn−1 )F .

In the remainder of the proof, let Ω = e2πı/n . It doesn’t change the value of the trace to alter the order of the matrices, so, ⎛ n−1 ⎞ 1 −lj Ω a j⎟ n−1 (⎜ ⎜ j=0 ⎟ trace(K−1 A) = ⎜ n−1 ⎟ . ⎝ 1 −lj ⎠ l=0 Ω kj j=0

For l = 0, · · · , n − 1, n−1 1

Ω −lj aj =

j=0

=

n−1 1

Ω −lj

j=0 n−1 1 n−1 1

n−1 1

1

3 2 (ν)e2πıkj/n K

k=0 ν∈L⊥ (k)

1

3 2 (ν)Ω j(k−l) K

(12)

j=0 k=0 ν∈L⊥ (k) 1 32

K (ν) .

=

ν∈L⊥ (l)

Similarly, n−1 ( j=0

Ω −lj kj =

(

3 K(ν).

(13)

ν∈L⊥ (l)

Combining (11), (12) and (13), the proof is completed.  Next, the upper bound of err(n, H) in (10) will be discussed. The reproducing kernel K(·, y) is now chosen as a function whose smoothness is characterized by the parameter α > 1 that controls the decay of the Fourier coefficient in the L2 norm. This indicates that the reproducing Hilbert space H consists of periodic functions whose rth derivatives are square integrable in each variable for all r ≤ α/2. For convenience sake, let the Fourier coefficient 3 of the kernel be K(ν) = (¯ ν1 ν¯2 · · · ν¯d )−α , where ν¯i = max(1, νi ). The simplest case of d = 1 is studied first. Lemma 4. If dimension d = 1, generator h = 1,−& n2 ' + 1 ≤ l ≤ # n2 $, then there exists a constant Cα , depending only on α, for which ( ¯l−α + n−α < 3 K(ν) ≤ ¯l−α + Cα n−α , ν=l mod n where ¯l = max(1, l).

Error Analysis of Splines for Periodic Problems Using Lattice Designs

507

Proof If l = 0, (

+∞ (

3 K(ν) =

ν∈L⊥

(mn)−α = 1 + 2

m=−∞

(

(mn)−α = 1 + 2n−α ζ(α) .

m>0

If l > 0, ( ν∈L⊥ (l)

=l

+∞ (

3 K(ν) =

−α

(mn + l)−α

m=−∞

−α

+n

 +∞ (

−α

(m + l/n)

m=1


−α

+

 n −α 2

+∞ (

+

 −α

(m − l/n)

m=1 −α

+ 2n

+∞ (

m−α < l−α + n−α (2α + 2ζ(α)).

m=1

When l < 0, the sum is also less than |l|−α + n−α (2α + 2ζ(α)) in similar manner. On the other hand, it is easy to prove, for any l, −& n2 ' + 1 ≤ l ≤ # n2 $,  the sum is larger than ¯l−α + n−α ζ(α). Lemma 5. If dimension d = 1 and the generator is h = 1, we have err(n, H) = Θ(n−α/2+1/2 ) . Proof From Lemma 4 above, it follows that Therefore,

1

3 K(ν) = ¯l−α + Θ(n−α ).

ν=l mod n 3 2 (ν) = l−2α + Θ(n−2α ), and the following formula is nonK

1 ν∈L⊥ (l)

negative and of order of n−α . 

1 ν∈L⊥ (l)

2 3 K(ν)

−

1

1 ν∈L⊥ (l)

3 K(ν)

3 2 (ν) K

 −α    ¯l + Θ(n−α ) 2 − ¯l−2α + Θ(n−2α ) = ¯l−α + Θ(n−α )

ν∈L⊥ (l)

=

¯l−2α + 2¯l−α Θ(n−α ) + Θ(n−2α ) − (¯l−2α + Θ(n−2α )) = Θ(n−α ). ¯l−α + Θ(n−α )

Since err(n, H) is the sum of n formulas like that above, thus,  1/2 err(n, H) = n · Θ(n−α ) = Θ(n−α/2+1/2 ) .



Theorem 2. For dimension d = 1 and generator h ∈ Z, with h < n and h having no common factor with n there exists a unique integer k in the sense of modulo n, such that

508

X. Zeng et al.

( ν∈L⊥ (l)

(

3 K(ν) = ν=k

mod

3 K(ν) . n

Therefore, err(n, H) is of order of −α/2 + 1/2 for arbitrarily chosen h. Proof If hν0 = m0 n + l, hν = mn + l we get h(ν0 − ν) = n(m0 − m). Because h has no common factor with n, m0 − m should be multiple of h. Consequently, if we know ν0 , we can get all other wave numbers satisfying ν ∈ L⊥ (l), by  choosing different m to compute ν = mn + ν0 . Therefore, k = ν0 . Next, a series of lemmas is given for deducing an upper bound on (n, H), for d > 1, as given in Theorem 1. It should be mentioned that the parameters C(d, α) in the following results may be different. From [14], it follows that Lemma 6. There exists a rank-1 lattice generator h and a constant C(d, α) depending on d and α, such that ( ˆ K(ν) ≤ C(d, α)n−α+ε , 0 =ν∈L⊥

where ε is an arbitrarily small positive number. Examples of the above lemma are Fibonacci lattices (2-dimensional rank-1 lattices): N = Fi , h = [1, Fi−1 ] , where F1 = 1, F2 = 1, Fi = Fi−1 + Fi−2 , ∀i > 2. From [8], it follows that ˆ ˆ ˆ + µ) ≤ 2αd K(µ). Lemma 7. For dimension d ≥ 1, K(ν) K(ν Lemma 8. Let ε1 > 0, γ1 = (1 + ε1 )/α, γ2 = (α − 1 − ε1 )/2α, ε be an arbiˆ trarily small positive number, and ν l ∈ argmax {K(ν) : ν ∈ L⊥ (l)}. If α > 2, ν there exists a generator h and a constant C(d, α, ε1 ), such that   γ1 ( ˆ l + µ) < C(d, α, ε1 ) K(ν ˆ l) K(ν n−γ2 α+ε . 0 =µ∈L⊥

Proof By Lemma 7 it follows that   γ γ ˆ ˆ l) ˆ ˆ l) K(µ) K(ν K(µ) K(ν αd ˆ l + µ) ≤ 2αd  K(ν γ+1 ≤ 2  γ+1 , ˆ ˆ K(ν l ) K(ν l + µ) where γ = (2 + 2ε1 )/(α − 1 − ε1 ). Therefore, γ2   γ1 αd 2 ˆ l + µ) ≤ K(ν ˆ l) K(ν . ˆ K(µ)

Error Analysis of Splines for Periodic Problems Using Lattice Designs

509

Thus, by Lemma 6, there exists a generator h, such that   γ1 ( ˆ l + µ) < C(d, α, ε1 ) K(ν ˆ l) K(ν n−γ2 α+ε . 0 =µ∈L⊥

 Theorem 3. If dimension d > 1, α > 2, ε > 0, then there exists a generator h and a constant C(d, α, ε) only depending on d, α and ε, such that err(n, H) < C(d, α, ε)n(−α+1)/4+ε . ˆ Proof As in Lemma 8, let ν l ∈ argmax {K(ν) : ν ∈ L⊥ (l)}. It easy to see ν that ⎛ ⎞1/2 ( ( ( ˆ ˆ ˆ 2 (ν)⎠ K(ν) > K(ν) +⎝ K , 2 ν∈L⊥ (l)

ν∈L⊥ (l)

and

⎛

( ν∈L⊥ (l)

Consequently,  1 ν∈L⊥ (l)

⎞1/2

(

ˆ K(ν) −⎝

ν∈L⊥ (l)

ˆ 2 (ν)⎠ K

ν∈L⊥ (l)

2 ˆ K(ν)

−

1

1

(

<

ˆ + νl) . K(ν

0 =ν∈L⊥

ˆ 2 (ν) K

ν∈L⊥ (l)

(

<2

ˆ K(ν)

ˆ + νl) . K(ν

0 =ν∈L⊥

ν∈L⊥ (l)

By Lemma 8, it follows that err2 (n, H) = C(d, α, ε1 )n−γ2 α+ε

n−1 (

γ1 ˆ l) K(ν .

l=0

Moreover, it is clear that n−1 (

 γ1  γ1 (  ˆ l) ˆ K(ν K(µ) < .

l=0

µ

Since γ1 α > 1+ε1 , the sum in the right side is equal to the value (1+2ζ(γ1 α))d which is dependent on d, α, ε1 . Thus, n−1 (

 γ1 ˆ l) K(ν < C(d, α, ε1 ) .

l=0

With inequality above, Lemma 8 and 6, the proof is complete.



510

X. Zeng et al.

Based on the numerical experiment in the next section, we believe that the upper bound in Theorem 3 is not sharp. The lower bound of err(n, H) is also an interesting topic. We suspect it is err(n, H) ≥ Cα,d n−α/4 , but a more thorough investigation will be the subject of future work. [7] use trigonometric polynomials to approximate multivariate functions in weighted Korobov spaces. In their algorithms, the lattice generators are constructed component by component. They proved that the convergence rate is about O(n−α/4+ε ) under a looser error criterion than the one used here. The following lemma provides a method to simulate the err(n, H). Lemma 9. Let d ≥ 1, h = (1, h1 , . . . , hd−1 ) ∈ Zd , h is a good rank-1 lattice point, and hi has no common factor with n, then, ( ( ˆ ˆ K(ν) = K(ν) + O(n−α ) . ν∈L⊥ (l)

ν∈L⊥ (l), ν∈[−n,n]d ∩Zd

Proof Let us consider the case 0 < l < #n/2$ first. We have (

ˆ K(ν) =

ν∈L⊥ (l)

( 

¯2 · · · λ ¯d (λ1 + l) λ

−α

.

λ∈L⊥

If νi > n, because l < n/2, λ1 > 0 and λ1 > n/2, λi = νi > n, i = 2, · · · , d, ( ( ¯1λ ¯ d )−α = T1 . ¯2 . . . λ ˆ K(ν) < (λ ν ∈L⊥ (l), νi >n

λ∈L⊥ , λi >n/2

If νi < −n, λ1 < −n, therefore λi < −2l, λi = νi < −n, i = 2, · · · , d, ( ( ( ¯1λ ¯ d )−α < 2 ¯1λ ¯ d )−α = T2 . ¯2 . . . λ ¯2 . . . λ ˆ K(ν) <2 (λ (λ ν ∈L⊥ (l), νi <−n

λ∈L⊥ , λi <−n

λ∈L⊥ , λi <−n/2

The first inequality is due to the inequality |a/(a + l)| < 2, if |a| > 2|l|. From [10], we know that T1 + T2 = O(n−α ). We can obtain the same result for −&n/2' + 1 < l < 0 in a similar way. 

4 Experimental Results 1 3 To compute the error, we have to compute two sums, ν∈L⊥ (l) K(ν) and 1 32 ν∈L⊥ (l) K (ν). There are two methods. One is to get the exact value of the sum by approximating the integral of the kernel using a rank-1 lattice rule. The other is to approximate the sums by adding every term in which the absolute value of each coordinate is less than n and is based on Lemma 9. The first method is easy, however it will cause disastrous round-off errors when the

Error Analysis of Splines for Periodic Problems Using Lattice Designs

511

Dimension d=1

100 10–1

err

10–2 10–3 10–4 10–5 10–6 2 10

103

n

104

Dimension d=2 100

err

10–1

10–2

10–3 102

103

104

Dimension d=3

100

err

n

10–1

10–2 101

102

n

103

104

Fig. 1. solid : α = 2, dashed : α = 4.

dimension is larger than 2 and α is bigger than 2, so it is not practical. We use the second method. The results are shown in Figure 1 using a log-log scale. The solid lines represent the results of the Hilbert space of functions whose first derivatives are square integrable in each variable, while the dashed ones are of Hilbert space of functions whose first and second derivatives are square integrable in each variable. For dimension 1, the generator is chosen as h = 1, the convergence rate is O(n−(α−1)/2 ), α = 2, 4 respectively in this case. This

512

X. Zeng et al.

is the same as the conclusion of the Theorem 2. For dimension 2 we choose Fibonacci lattices, and the convergence rate is about O(n−α/4 ), somewhat better than the upper bound derived in Theorem 3. Since for higher dimension there are no analytic constructions for good generators as described in Lemma 6, we use a fixed generator h with components hj = 17797j−1 , j = 1, · · · , d, an example of an extensible lattice introduced in [4]. From multidimensional results, we can see that for even α = 2, the error converges to zero. Consequently, the convergence rate in Theorem 3 can be improved. We will use lattice designs to approximate the following functions as test functions, which are infinitely differentiable and continuous respectively, f1 (x) =

d 

ecos(2πxi ) ,

(14)

xi (xi − 1) .

(15)

i=1

f2 (x) =

d  i=1

The results are shown in Figure 2. 100

Error

10−2 10−4

¬ N−1

10−6

¬

x(x−1)

¬

ecos(2

10−8 10−10 10−12 10−14 10−1610

0

1

10

2

10 n

3

x)

4

10

10

100 ¬

x(x−1)y(y−1)

¬ N−1

Error

10−5

10−10

10−15 100

¬

101

102

103

ecos(2 x )ecos(2 104

n

Fig. 2. Convergence rate for dimension 1 and 2.

y )

Error Analysis of Splines for Periodic Problems Using Lattice Designs

513

In the numerical experiment, the kernel functions we use are following: K({x − y}) =

d 

1 . 1 − β cos(2π{x i − yi }) i=1

(16)

The kernel function is symmetric and periodic between [0, 1)d . The parameter β can control the flatness of the kernel function. For the two test functions chosen, we plot the root mean square error against the number of nodes with a log-log scale. From Figure 2 it is found that the convergence rate is related to the smoothness of the test function. The test function which is continuous has a low convergence rate of about O(n−1 ). The test function which is infinitely differentiable approximates exactly, except for round-off errors for a moderate value of n.

References 1. O. Botella. On a collocation B-spline method for the solution of the NavierStokes equations. Comp. & Fluids, 31:397–420, 2002. 2. C. de Boor. Practical guide to splines. Springer-Verlag, New York, 1978. 3. R. Franke. Scattered data interpolation: tests of some methods. Math. Comp., 38:181–200, 1982. 4. F. J. Hickernell, H. S. Hong, P. L’Ecuyer, and C. Lemieux. Extensible lattice sequences for quasi-Monte Carlo quadrature. SIAM J. Sci. Comput., 22:1117– 1138, 2000. 5. R. Holmes. R-splines in Banach spaces: I. Interpolation of linear manifolds. J. Math. Anal. Appl., 40:574–593, 1972. 6. L. K. Hua and Y. Wang. Application of number theory to numerical analysis. Springer-Verlag, Berlin, 1981. 7. F. Y. Kuo, I. H. Sloan, and H. Wozniakowski. Lattice rules for multivariate approximation in the worst case setting. Monte Carlo and quasi-Monte Carlo methods 2004 (H. Niederreiter, D. Talay, eds.), Springer-Verlag, New York, 2005. 8. D. Li and F. J. Hickernell. Trigonometric spectral collocation methods on lattices. Recent Advances in Scientific Computing and Partial Differential Equations (S. Y. Cheng, C. W. Shu, T. Tang, eds.), AMS Series on Contemporary Mathematics, American Mathematical Society, Providence, RI, 2003. 9. C. A. Micchelli. Interpolation of scattered data: distance matrices and conditionally positive definite functions. Constr. Approx., 1:11–22, 1986. 10. H. Niederreiter. Random number generation and quasi-Monte Carlo methods. Society for Industrial and Applied Mathematics, Philadelphia, 1992. 11. M. J. D. Powell. Radial basis functions for multivariable interpolation: a review. Algorithms for Approximation (J.C. Mason, M.G.Cox, eds.), Oxford University Press, Oxford, 1987. 12. T. J. Rivlin. Introduction to the approximation of functions. Dover Publications, New York, 1981. 13. I. J Schoenberg. Contribution to the problem of approximation of equidistant data by analytic functions. Quar. Appl. Math., 4:45–99, 112–141, 1946.

514

X. Zeng et al.

14. I. H. Sloan and S. Joe. Lattice methods for multiple integration. The Clarendon Press Oxford University Press, New York, 1994. 15. M. Unser. Splines: A perfect fit for signal and image processing. IEEE Signal Processing Magazine, 16:22–38, 1999. 16. C. Van Loan. Computational framework for the fast Fourier transform. SIAM, Philadelphia, 1992. 17. R. S. Varga. Functional and approximation theory in numerical analysis. CBMN Regional Conference Series in Applied Mathematics, SIAM, Philadelphia, 1971. 18. G. Wahba. Spline models for observational data. SIAM, Philadelphia, 1990.